### Install PANINIpy from source

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/installation.md

Clone the repository and install PANINIpy and its dependencies using the requirements.txt file.

```console
git clone https://github.com/HKU-Complex-Networks-Lab/PANINIpy.git
cd PANINIpy
pip install -r requirements.txt
```

--------------------------------

### Minimal Working Example for PANINIpy

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/installation.md

This example demonstrates how to use the MDL_backboning module to compute global and local backbones from a weighted edge list. Ensure Python version is >=3.9 and <3.12.

```python
from paninipy.mdl_backboning import MDL_backboning

# Define a weighted edge list
elist = [
    (0, 1, 12), (0, 3, 20), (0, 4, 8),
    (1, 2, 1), (1, 4, 3),
    (2, 0, 1), (2, 1, 3),
    (3, 2, 3), (3, 4, 1),
    (4, 3, 1)
]

# Compute backbones using out-edges
backbone_global, backbone_local, compression_global, compression_local = MDL_backboning(
    elist, directed=True, out_edges=True
)

# Display results
print('Global Backbone:', backbone_global)
print('Inverse Compression Ratio (Global):', compression_global)
print('Local Backbone:', backbone_local)
print('Inverse Compression Ratio (Local):', compression_local)
```

--------------------------------

### Install PANINIpy using pip

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/installation.md

Install PANINIpy and its dependencies directly from PyPI using pip.

```console
pip install paninipy
```

--------------------------------

### Install Dependencies and Pytest for Testing

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/installation.md

Install all necessary dependencies, including pytest, for running PANINIpy tests manually.

```console
pip install -r requirements.txt
pip install paninipy
pip install pytest
```

--------------------------------

### Importing Paninipy Libraries

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hypergraph_binning.md

Initial setup required to access the MDL_hypergraph_binning function and plotting utilities.

```python
import time
from paninipy.hypergraph_binning import MDL_hypergraph_binning
import matplotlib.pyplot as plt
```

--------------------------------

### Run MDL Hypergraph Binning (Exact vs. Greedy)

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/paninipy/hypergraph_binning/demo.ipynb

This code snippet demonstrates how to use the MDL_hypergraph_binning function with both exact and greedy algorithms. It initializes an example event dataset and time step width, then measures and prints the results and runtime for each method.

```python
from functions import MDL_hypergraph_binning
import time

#example event dataset X and time step width dt
X = [('a1','b2',1,1.1),('a1','b3',1,1.5),('a1','b2',1,1.6),('a2','b2',1,1.7),('a2','b3',1,1.9),\
    ('a4','b5',1,5.5),('a1','b3',1,150),('a1','b3',1,160),('a4','b6',1,170),('a2','b3',1,190)]
dt = 1

start_exact = time.time()
results_exact = MDL_hypergraph_binning(X,dt,exact=True)
runtime_exact = time.time() - start_exact

start_greedy = time.time()
results_greedy = MDL_hypergraph_binning(X,dt,exact=False)
runtime_greedy = time.time() - start_greedy

print('Exact Dynamic Program Results: ')
print('     compression ratio =',round(results_exact[0],4))
print('     MDL-optimal event partition =',results_exact[1])
print('     number of time steps =',results_exact[2])
print('     runtime =',round(runtime_exact,4))

print('Greedy Algorithm Results: ')
print('     compression ratio =',round(results_greedy[0],4))
print('     MDL-optimal event partition =',results_greedy[1])
print('     number of time steps =',results_greedy[2])
print('     runtime =',round(runtime_greedy,4))
```

--------------------------------

### Initialize Past Merges Dictionary

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/community_regionalization.md

Initializes the 'ddl_c' dictionary, which stores information about past merges and is indexed by spatially adjacent neighbors. This is a setup step for network analysis.

```python
ddl_c = {}
for e in spatial_elist:
```

--------------------------------

### Import Libraries and Configure Matplotlib

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/community_regionalization.md

Imports necessary libraries for data manipulation and visualization, and configures Matplotlib for LaTeX rendering. Ensure these libraries are installed before use.

```python
import pickle
import pandas as pd
import geopandas as gpd
import matplotlib
import matplotlib.pyplot as plt
from paninipy.community_regionalization import greedy_opt

# LaTeX preamble
matplotlib.rcParams.update({'text.usetex': True})
matplotlib.rcParams.update({'text.latex.preamble': r"",
                            "font.serif": "Times"})
matplotlib.rcParams.update({'font.family':'serif'})
```

--------------------------------

### Initialize Environment and Imports

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/paninipy/community_regionalization/example.ipynb

Sets up the necessary libraries and configures Matplotlib for LaTeX rendering.

```python
import pickle
import pandas as pd
import geopandas as gpd
import matplotlib
import matplotlib.pyplot as plt

from greedy_algorithm import greedy_opt

# LaTeX preamble
matplotlib.rcParams.update({'text.usetex': True})
matplotlib.rcParams.update({'text.latex.preamble': r"",
                            "font.serif": "Times"})
matplotlib.rcParams.update({'font.family':'serif'})
```

--------------------------------

### Initialize Network_hubs Class

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/hub_identification.md

Constructor for the Network_hubs class.

```python
class Network_hubs:
    def __init__(self,name):
        self.name = name
```

--------------------------------

### Network_hubs.__init__

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hub_identification.md

Initializes the Network_hubs class instance.

```APIDOC
## CLASS Network_hubs.__init__

### Description
Initialize the Network_hubs class.

### Parameters
- **name** (string) - Required - The name identifier for the network hubs instance.
```

--------------------------------

### Define weighted edge list

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/mdl_backboning.md

Example structure for a weighted edge list represented as a list of tuples.

```python
# Weighted edge list for the example
elist = [
    (0, 1, 12), (0, 3, 20), (0, 4, 8),
    (1, 2, 1), (1, 4, 3),
    (2, 0, 1), (2, 1, 3),
    (3, 2, 3), (3, 4, 1),
    (4, 3, 1)
]
```

--------------------------------

### get_M - Get Number of Matches

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Calculates the total number of matches or unique matches from a match list.

```APIDOC
## GET /api/matches/count

### Description
Retrieves the count of matches from a given match list. Can optionally return the count of unique matches.

### Method
GET

### Endpoint
/api/matches/count

### Parameters
#### Query Parameters
- **match_list** (ndarray) - Required - Array of matches, where each match is represented as `[player_i, player_j]` or `[player_i, player_j, wins_ij]`.
- **return_unique** (bool) - Optional - If true, returns both the total match count and the unique match count.

### Request Example
```json
{
  "match_list": [[1, 2], [2, 3], [1, 3]],
  "return_unique": true
}
```

### Response
#### Success Response (200)
- **M** (int) - The total number of matches (sum of wins if applicable).
- **E** (int) - The number of unique matches (only returned if `return_unique` is true).

#### Response Example
```json
{
  "M": 3,
  "E": 3
}
```
```

--------------------------------

### get_N - Get Number of Unique Players

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Calculates the total number of unique players involved in a series of matches.

```APIDOC
## GET /api/players/count

### Description
Retrieves the count of unique players from a given match list.

### Method
GET

### Endpoint
/api/players/count

### Parameters
#### Query Parameters
- **match_list** (ndarray) - Required - Array of matches, where each match is represented as `[player_i, player_j]` or `[player_i, player_j, wins_ij]`.

### Request Example
```json
{
  "match_list": [[1, 2], [2, 3], [1, 3]]
}
```

### Response
#### Success Response (200)
- **N** (int) - The total number of unique players.

#### Response Example
```json
{
  "N": 3
}
```
```

--------------------------------

### Get number of matches

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Calculates the total number of matches or unique matches from a match list. Handles match lists with or without win counts.

```python
def get_M(match_list: np.ndarray, return_unique: bool = False) -> int:
    """
    Get the number of matches in a match list

    Parameters
    ----------
    match_list : ndarray
        Array of matches of the form [[i, j],...] or [[i, j, w_ij],...] where w_ij is the number of times i beats j
    return_unique : bool
        If True, return the number of unique matches


    Returns
    -------
    M : int
        Number of matches in the match list
    """

    # Get the number of columns in the match list
    num_cols = match_list.shape[1]

    # Get the number of matches
    M = np.sum([int(el[2]) for el in match_list]) if num_cols == 3 else len(match_list)

    if return_unique:
        # Get the number of unique matches
        if num_cols == 3:
            E = len(match_list)
        elif num_cols == 2:
            E = len(np.unique(match_list, axis=0))
        return M, E

    return M
```

--------------------------------

### Get number of unique players

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Calculates the total number of unique players involved in a series of matches. Assumes match data is in a NumPy array.

```python
def get_N(match_list: np.ndarray) -> int:
    """
    Get the number of unique players in a match list

    Parameters
    ----------
    match_list : ndarray
        Array of matches of the form [[i, j],...] or [[i, j, w_ij],...] where w_ij is the number of times i beats j

    Returns
    -------
    N : int
        Number of unique players in the match list
    """

    # Get the number of unique players
    N = len(np.unique(match_list[:, :2]))
    return N
```

--------------------------------

### Initialize and Assign Vertex Property Maps

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/paninipy/partial_rankings/example.ipynb

Create and populate vertex property maps for cluster labels, BT labels, and node positions based on player strengths.

```python
# Create a vertex property map for the cluster labels
cluster_labels = g.new_vertex_property("int")

# Create a vertex property map for the BT labels
bt_labels = g.new_vertex_property("int")

# Assign the player strengths to the vertex property map
for i, v in enumerate(g.vertices()):
    rating[v] = player_strengths[g.vp.id[v]]
    bt_rating[v] = bt_sigmas[g.vp.id[v]]
    pos_pr[v] = [-i, -np.log(rating[v])]  # Position nodes based on PR strengths
    pos_bt[v] = [-i, -np.log(bt_rating[v])]  # Position nodes based on BT strengths
    for j, cluster in enumerate(clusters):
        if g.vp.id[v] in clusters[cluster]:
            cluster_labels[v] = int(j)
            bt_labels[v] = int(i)

# Assign the vertex property maps to the graph
g.vp.rating = rating
g.vp.bt_rating = bt_rating
g.vp.pos_bt = pos_bt
g.vp.pos_pr = pos_pr
g.vp.cluster_labels = cluster_labels
g.vp.bt_labels = bt_labels
```

--------------------------------

### Calculate MDL Configuration (Exact DP)

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/hypergraph_binning.md

Implements the exact dynamic programming solution to identify the Minimum Description Length (MDL) configuration. It calculates the best MDL value and boundaries for a given time range.

```python
dl = np.log((N-1)*(T-1)) + logchoose(nk + tauk - 1, tauk - 1) + logchoose(nk + S - 1, S - 1) + logchoose(nk + D - 1, D - 1) \
                + logOmega(num_ss,num_ds) + logOmega(num_sds,num_kts)
        past_dls[(i,j)] = dl
        past_nks[(i,j)] = nk

        return dl

    if exact:
        """
        exact dynamic programming solution to identify MDL configuration
        """
        LMDL,LMDL_int = {0:0},{}
        for j in range(1,T+1):
            best_MDL = np.inf
            for i in range(1,j+1):
                L = LMDL[i-1] + DL(i,j)
                if L < best_MDL:
                    best_MDL = L
                    LMDL_int[j] = i
            LMDL[j] = best_MDL

        j = T
        i = np.inf
        MDL_boundaries = []
        while i > 1:
            i = LMDL_int[j]
            MDL_boundaries.append((i,j))
            j = i-1
        MDL_boundaries = sorted(MDL_boundaries,key = lambda I:I[0])
        best_MDL = LMDL[T]
```

--------------------------------

### Print summary and return results

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Outputs convergence statistics to stderr and returns the final network configuration.

```python
# Print summary
print(f"Converged in {iter_count} iterations", file=stderr)
print(f"Partial Rankings: {min_R}", file=stderr)
print(f"Initial DL: {initial_dl}", file=stderr)
print(f"Min DL: {min_dl}", file=stderr)
print(f"BT DL: {bt_dl}", file=stderr)
print(f"LPOR: {bt_dl - min_dl}", file=stderr)
print(f"CR: {min_dl / initial_dl}", file=stderr)

if full_trace:
    return trace_list

return {
    "N": N,
    "M": M,
    "<k>": M / N,
    "R": min_R,
    "BT_R": BT_R,
    "DL": min_dl,
    "BT_DL": bt_dl,
    "LPOR": bt_dl - min_dl,
    "CR": min_dl / initial_dl,
    "Strengths": min_sigmas,
    "Clusters": min_clusters,
}
```

--------------------------------

### get_edges - Get In and Out Edges

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Parses a match list to generate dictionaries representing out-edges (player i beats player j) and in-edges (player j is beaten by player i).

```APIDOC
## GET /api/matches/edges

### Description
Processes a match list to determine the directed edges representing wins between players.

### Method
GET

### Endpoint
/api/matches/edges

### Parameters
#### Query Parameters
- **match_list** (ndarray) - Required - Array of matches, where each match is represented as `[player_i, player_j]` or `[player_i, player_j, wins_ij]`.

### Request Example
```json
{
  "match_list": [[1, 2, 5], [2, 3, 2], [1, 3, 1]]
}
```

### Response
#### Success Response (200)
- **e_out** (dict) - A dictionary where keys are player IDs and values are dictionaries mapping opponent IDs to the number of times the key player beat the opponent.
- **e_in** (dict) - A dictionary where keys are player IDs and values are dictionaries mapping opponent IDs to the number of times the key player was beaten by the opponent.

#### Response Example
```json
{
  "e_out": {
    "1": {"2": 5, "3": 1},
    "2": {"3": 2}
  },
  "e_in": {
    "2": {"1": 5},
    "3": {"2": 2, "1": 1}
  }
}
```
```

--------------------------------

### Run MDL Network Population Clustering

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/population_clustering.md

Initializes and runs the MDL_populations algorithm to cluster networks. Requires edge sets, node count, and parameters like K0 and n_fails.

```python
mdl_pop = MDL_populations(edgesets=nets, N=node_num, K0=1, n_fails=100, directed=False, max_runs=np.inf)
mdl_pop.initialize_clusters()
clusters, modes, L = mdl_pop.run_sims()
```

--------------------------------

### DefaultDict Class Methods

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/community_regionalization.md

Documentation for the DefaultDict class, including initialization and item retrieval.

```APIDOC
## DefaultDict._init_

### Description
Initialize the DefaultDict class.

### Method
__init__

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **default_factory** (function) - Required - The default factory function.
- **kwargs** (any) - Optional - Additional keyword arguments.

### Request Example
```json
{
  "default_factory": "some_function",
  "key1": "value1"
}
```

### Response
#### Success Response (200)
None (initialization)

#### Response Example
None
```

```APIDOC
## DefaultDict._getitem_

### Description
Get an item from the DefaultDict.

### Method
_getitem_

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **key** (any) - Required - The key to get the item for.

### Request Example
```json
{
  "key": "some_key"
}
```

### Response
#### Success Response (200)
- **value** (any) - The value associated with the key.

#### Response Example
```json
{
  "value": "some_value"
}
```
```

--------------------------------

### Initialize Graph Property Maps

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/paninipy/partial_rankings/example.ipynb

Create vertex property maps for ratings and positions in the graph.

```python
# Create a vertex property map for the player strengths
rating = g.new_vertex_property("double")

# Create a vertex property map for the player BT strengths
bt_rating = g.new_vertex_property("double")

# Create a vertex property map for the nodes positions
pos_bt = g.new_vertex_property("vector<double>")
pos_pr = g.new_vertex_property("vector<double>")
```

--------------------------------

### Initialize Clusters for MDL Populations

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/population_clustering.md

Initializes K0 random clusters and calculates their modes and the total description length. Assigns networks to clusters randomly.

```python
def initialize_clusters(self):
    """
    initialize K0 random clusters and find their modes as well as the total description length of this configuration
    """
    for i,s in enumerate(np.random.permutation(range(self.S))):
        k = str(i % self.K0)
        if k in self.C:
            self.C[k].add(s)
        else:
            self.C[k] = set()
            self.C[k].add(s)
    for k in range(self.K0):
        k = str(k)
        self.E[k] = self.generate_Ek(self.C[k])
        self.A[k] = self.update_mode(self.E[k],len(self.C[k]))
    self.L = sum([self.Lk(self.A[k],self.E[k],len(self.C[k])) for k in self.C])
```

--------------------------------

### Run MDL Regionalization Algorithm

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/distributional_regionalization.md

Initializes the MDL_regionalization class and runs the core algorithm with the prepared adjacency list, distance data, and population data. Prints the inverse compression ratio and cluster labels.

```python
mdl_instance = MDL_regionalization("example")
inverse_compression_ratio, cluster_labels, dists = mdl_instance.MDL_regionalization(adjlist, dists, pops)

print(inverse_compression_ratio)
print(cluster_labels)
```

--------------------------------

### Printing Binning Results

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hypergraph_binning.md

Outputs the compression ratio, partition labels, time steps, and runtime for the executed algorithms.

```python
print('Exact Dynamic Program Results: ')
print('     compression ratio =', round(results_exact[0], 4))
print('     MDL-optimal event partition =', results_exact[1])
print('     number of time steps =', results_exact[2])
print('     runtime =', round(runtime_exact, 4))

print('Greedy Algorithm Results: ')
print('     compression ratio =', round(results_greedy[0], 4))
print('     MDL-optimal event partition =', results_greedy[1])
print('     number of time steps =', results_greedy[2])
print('     runtime =', round(runtime_greedy, 4))
```

--------------------------------

### Initialize Network Analysis Parameters

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/partial_rankings.md

Initializes parameters for network analysis, including updating sigmas using the BT model, calculating initial description length (DL), and setting up variables for tracking minimum DL, ranks, and clusters. This is typically done before the main iterative refinement loop.

```python
# Compute initial BT scores
# print("Computing initial BT scores", file=stderr)
update_sigmas_bt(sigmas, e_out, e_in)

# Compute initial DL
min_dl = dl = initial_dl = total_dl()
bt_dl = initial_dl - loggamma(N + 1) - np.log(N)
min_R = N
min_sigmas = sigmas
min_clusters = clusters
print(f"Initial DL: {initial_dl}", file=stderr)
print(f"Initial Ranks: {R}", file=stderr)
# Print the number of workers
if sync:
    cluster_resources = ray.cluster_resources()
    num_workers = int(cluster_resources.get("CPU", 0))
    print(f"Number of workers in the Pool: {num_workers}", file=stderr)
print(f"Tolerance: {TARGET}", file=stderr)

# Compute number of unique ranks inferred by BT
BT_R = len(set(sigmas.values()))

# If full trace, append resulst dictioanry to trace_list
if full_trace:
    results_dict = {
        "N": N,  # Number of nodes
        "M": M,  # Number of edges
        "<k>": M / N,  # Average degree
        "R": R,  # Number of unique ranks
        "BT_R": BT_R,  # Number of unique ranks inferred by BT model
        "DL": dl,  # Description length
        "BT_DL": bt_dl,  # Description length of BT model
        "LPOR": bt_dl - dl,  # Log posterior odds ratio
        "CR": 1,  # Compression ratio
        "Strengths": sigmas,  # Strengths of each cluster
        "Clusters": deepcopy(clusters),  # Clusters
    }
    trace_list = [results_dict]

iter_count = 0  # Initialise iteration counter
```

--------------------------------

### Read GML and Prepare Data

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hub_identification.md

Loads the converted GML file and initializes the Network_hubs object.

```python
G = nx.read_gml(f'{output_file}.gml', label='id')
pos_x = nx.get_node_attributes(G,'x')
pos_y = nx.get_node_attributes(G,'y')
pos = {node: (pos_x[node], pos_y[node]) for node in G.nodes()}
print(G.nodes(data=True))
print("\nEdges with attributes:")
print(G.edges(data=True))
net = set((u, v, 1) for u, v in G.edges())
num_node = len(G.nodes(data=True))
nh = Network_hubs("Example Network")
```

--------------------------------

### Clone PANINIpy Repository

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/contributing.md

Clone your forked repository to your local machine to begin making changes.

```console
git clone https://github.com/[username]/PANINIpy.git
```

--------------------------------

### Initialize and Run Discontiguous Clustering

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/paninipy/population_clustering/README.md

Use this for discontiguous clustering. Initialize the MDL_populations object with network data and parameters, then run the simulations to obtain cluster assignments and network modes.

```python
MDLobj = MDL_populations(edgesets,N,K0,n_fails,bipartite,directed,max_runs)
MDLobj.initialize_clusters()
C,A,L = MDLobj.run_sims()
```

--------------------------------

### MDL_populations.__init__

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/population_clustering.md

Initializes the MDL_populations class for clustering network populations.

```APIDOC
## MDL_populations.__init__(edgesets, N, K0=1, n_fails=100, bipartite=None, directed=False, max_runs=np.inf)

### Description
Initialize the MDL_populations class.

### Parameters
#### Path Parameters
- **edgesets** (list) - Required - List of sets containing edges for each network.
- **N** (int) - Required - Number of nodes in each network.
- **K0** (int) - Optional - Initial number of clusters.
- **n_fails** (int) - Optional - Number of failed moves before termination.
- **bipartite** (array/None) - Optional - Bipartite node counts or None.
- **directed** (bool) - Optional - Whether edges are directed.
- **max_runs** (float) - Optional - Maximum number of allowed moves.
```

--------------------------------

### Executing Binning Algorithms

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hypergraph_binning.md

Runs the hypergraph binning algorithm using either the exact dynamic programming method or the greedy approximation.

```python
start_exact = time.time()
results_exact = MDL_hypergraph_binning(X, dt, exact=True)
runtime_exact = time.time() - start_exact
```

```python
start_greedy = time.time()
results_greedy = MDL_hypergraph_binning(X, dt, exact=False)
runtime_greedy = time.time() - start_greedy
```

--------------------------------

### Import Libraries

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hub_identification.md

Initial imports required for network analysis and visualization.

```python
import matplotlib.pyplot as plt
from paninipy.hub_identification import Network_hubs
import networkx as nx
```

--------------------------------

### Import paninipy libraries

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/mdl_backboning.md

Required imports for utilizing MDL backboning and visualization tools.

```python
import networkx as nx
import matplotlib.pyplot as plt
from paninipy.mdl_backboning import MDL_backboning
```

--------------------------------

### MDL Backboning Algorithm

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/mdl_backboning.md

Implements the Minimum Description Length (MDL) backboning algorithm for both global and local methods. It can handle directed and undirected graphs and offers options for edge tracking and empty backbone allowance.

```python
def MDL_backboning(elist,directed=True,out_edges=True,allow_empty=True,CR_type='Max'):

    """
    input: elist consisting of directed tuples [(i,j,w_ij)] for edges i --> j with weight w_ij
           'directed' arg tells us whether input edge list is directed or undirected
           'out_edges' arg tells us whether to track out-edges or in-edges attached to each node in the local pruning method
           'allow_empty' arg tells us whether or not we allow empty backbones (empty and full are equivalent by symmetry)
           'CR_type' arg allows adjusting the type of compression ratio.
               'Relative': divides MDL by DL of corresponding (global or local) naive model
               'Max': divides MDL by max(DL of naive global model,DL of naive local model)
    output: edge lists for MDL backbones for global and local methods, corresponding inverse compression ratios
    """

    def fglobal(W,E,Wb,Eb):
        """
        global description length objective
        """
        initial_cost = np.log(E+1) + np.log(W-E+1)
        return initial_cost + logchoose(E,Eb) + logchoose(Wb-1,Eb-1) + logchoose(W-Wb-1,E-Eb-1)

    def flocal(si,ki,sbi,kbi):
        """
        local description length objective at node-level
        """
        initial_cost = np.log(ki+1) + np.log(si-ki+1)
        return initial_cost + logchoose(ki,kbi) + logchoose(sbi-1,kbi-1) + logchoose(si-sbi-1,ki-kbi-1)

    def naiveglobal(W,E):
        """
        naive global description length objective
        """
        return fglobal(W,E,0,0)

    def naivelocal(si,ki):
        """
        naive local description length objective at node-level
        """
        return flocal(si,ki,0,0)

    #add two directed edges for each undirected edge if input is undirected. don't duplicate self-edges.
    if not(directed):
        self_edge_indices = set([i for i,e in enumerate(elist) if e[0] == e[1]])
        elist = list(elist) + [(e[1],e[0],e[2]) for i,e in enumerate(elist) if not(i in self_edge_indices)]

    #reverse edge order if we want the local pruning method to focus on in-degrees and in-strengths
    #does not make any difference for undirected networks
    if not(out_edges):
        elist = [(e[1],e[0],e[2]) for e in elist]

    #computational complexity bottleneck: sort edge list by decreasing weight in O(ElogE) time
    elist = sorted(elist,key = lambda e:e[-1],reverse=True)

    #initialize variables for input network
    W = sum([e[-1] for e in elist])
    E = len(elist)
    adj_edges,adj_weights = {},{}
    for e in elist:
        i,j,w_ij = e
        if not(i in adj_edges): adj_edges[i] = []
        if not(i in adj_weights): adj_weights[i] = []
        adj_edges[i].append(j)
        adj_weights[i].append(w_ij)
    nodes = set([e[0] for e in elist]+[e[1] for e in elist])
    N = len(nodes)

    #greedily add edges to global backbone and track total description length
    Lglobal0 = naiveglobal(W,E)
    Lglobal = fglobal(W,E,0,0)
    min_DL_global = Lglobal
    backbone_Eb = 0
    Wb,Eb = 0,0
    for e in elist:

        i,j,w_ij = e
        Eb += 1
        Wb += w_ij
        Lglobal += fglobal(W,E,Wb,Eb) - fglobal(W,E,Wb-w_ij,Eb-1)

        if Lglobal < min_DL_global:
            min_DL_global = Lglobal
            backbone_Eb = Eb

    if (backbone_Eb == 0) and not(allow_empty): backbone_Eb = E #by symmetry, DL is equivalent, so can choose to keep all edges

    #greedily add edges to local backbone and track description length at each node
    Llocal0,min_DL_local = logchoose(N+W-E-1,W-E),logchoose(N+W-E-1,W-E)
    backbone_degrees = {}
    for i in adj_edges:

```

--------------------------------

### Process and visualize rankings

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/partial_rankings.md

Execution script to initialize the generator and calculate node positions for visualization.

```python
# Initialize and process the match list
generator = RankingPlotGenerator("wolf.txt")
generator.load_and_preprocess()
generator.create_graph()

# Retrieve and process the graph for both BT and Partial Rankings
g_pr = generator.graph.copy()
g_bt = generator.graph.copy()

# Get sigmas
pr_sigmas = generator.sigmas
bt_sigmas = generator.bt_sigmas

# Dctionary mapping node labels to pr sigmas
label_to_pr_sigma = {}
for node in g_pr.nodes():
    for key, cluster in generator.clusters.items():
        if node in cluster:
            label_to_pr_sigma[node] = pr_sigmas[key]

# Initialise node coordinates
for i, node in enumerate(g_pr.nodes()):
    x = -i
    y_pr = np.log(label_to_pr_sigma[node])
    y_bt = np.log(bt_sigmas[node])
    g_pr.nodes[node]['pos'] = (x, y_pr)
    g_bt.nodes[node]['pos'] = (x, y_bt)

# Adjust positions of nodes in the same cluster
for key, cluster in generator.clusters.items():
    if len(cluster) == 1:
        # Set x-coordinate to zero
        g_pr.nodes[list(cluster)[0]]['pos'] = (0, g_pr.nodes[list(cluster)[0]]['pos'][1])
        g_bt.nodes[list(cluster)[0]]['pos'] = (0, g_bt.nodes[list(cluster)[0]]['pos'][1])
    elif len(cluster) == 2:
        # Evenly space in [-a, a] interval
        a = 2
        x_coords = np.linspace(-a, a, len(cluster))
        for i, node in enumerate(cluster):
            g_pr.nodes[node]['pos'] = (x_coords[i], g_pr.nodes[node]['pos'][1])
            g_bt.nodes[node]['pos'] = (x_coords[i], g_bt.nodes[node]['pos'][1])
    else:
        # Evenly space in [-a, a] interval
        a = 6
        x_coords = np.linspace(-a, a, len(cluster))
        for i, node in enumerate(cluster):
            g_pr.nodes[node]['pos'] = (x_coords[i], g_pr.nodes[node]['pos'][1])
            g_bt.nodes[node]['pos'] = (x_coords[i], g_bt.nodes[node]['pos'][1])
```

--------------------------------

### Calculate MDL Configuration (Approximate Greedy)

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/hypergraph_binning.md

Implements an approximate greedy agglomerative solution to identify the MDL configuration. It iteratively merges segments to minimize the description length.

```python
    else:
        """
        approximate greedy agglomerative solution to identify MDL configuration
        """
        MDL_boundaries = [(i,i) for i in range(1,T+1)]
        current_boundaries = [(i,i) for i in range(1,T+1)]
        for K in np.arange(T,1,-1).astype('int'):

            best_k,best_delta_dl = -100,np.inf
            for k in range(K-1):
                i1,i2 = current_boundaries[k]
                i3,i4 = current_boundaries[k+1]
                delta_dl = DL(i1,i4) - DL(i1,i2) - DL(i3,i4)
                if delta_dl < best_delta_dl:
                    best_delta_dl = delta_dl
                    best_k = k

            i1,i2 = current_boundaries[best_k]
            i3,i4 = current_boundaries[best_k+1]
            current_boundaries = current_boundaries[:best_k] + [(i1,i4)] + current_boundaries[best_k+2:]
            if best_delta_dl < 0:
                MDL_boundaries = [b for b in current_boundaries]

        best_MDL = sum([DL(b[0],b[1]) for b in MDL_boundaries])

        MDLK1 = DL(1,T)
        if MDLK1 < best_MDL:
            best_MDL = MDLK1
            MDL_boundaries = [(1,T)]
```

--------------------------------

### Compute Backbones using Out-Edges

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/mdl_backboning.md

Computes global and local network backbones using the MDL backboning algorithm with directed edges and out-edges enabled. Requires an edge list as input.

```python
backbone_global, backbone_local, compression_global, compression_local = MDL_backboning(
    elist, directed=True, out_edges=True
)
```

--------------------------------

### Visualize Network Backbones

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/mdl_backboning.md

Visualizes the original network, global backbone, and local backbone. It calculates and displays summary statistics for each network, including total weight and number of edges. Saves the visualization to 'mdl_network_backbones.png'.

```python
def visualize_backbones(elist, backbone_global, backbone_local, compression_global, compression_local):
    G_original = nx.DiGraph()
    G_global = nx.DiGraph()
    G_local = nx.DiGraph()

    for i, j, w in elist:
        G_original.add_edge(i, j, weight=w)
    for i, j, w in backbone_global:
        G_global.add_edge(i, j, weight=w)
    for i, j, w in backbone_local:
        G_local.add_edge(i, j, weight=w)

    pos = nx.spring_layout(G_original, seed=42)
    W_original = sum([d['weight'] for u, v, d in G_original.edges(data=True)])
    E_original = G_original.number_of_edges()

    W_global = sum([d['weight'] for u, v, d in G_global.edges(data=True)])
    E_global = G_global.number_of_edges()

    W_local = sum([d['weight'] for u, v, d in G_local.edges(data=True)])
    E_local = G_local.number_of_edges()

    plt.figure(figsize=(18, 6))

    plt.subplot(1, 3, 1)
    nx.draw_networkx_nodes(G_original, pos, node_color='lightblue', node_size=500)
    nx.draw_networkx_edges(G_original, pos, arrowstyle='->', arrowsize=15)
    nx.draw_networkx_labels(G_original, pos)
    plt.title('Original Network')
    plt.axis('off')
    plt.text(0.5, -0.1, f'Total weight of the network = {W_original}\nTotal number of edges = {E_original}', ha='center', transform=plt.gca().transAxes)

    plt.subplot(1, 3, 2)
    nx.draw_networkx_nodes(G_global, pos, node_color='red', node_size=500)
    nx.draw_networkx_edges(G_global, pos, arrowstyle='->', arrowsize=15)
    nx.draw_networkx_labels(G_global, pos)
    plt.title('Global Backbone')
    plt.axis('off')
    plt.text(0.5, -0.1, f'Total weight of the network = {W_global}\nTotal number of edges = {E_global}\nInverse compression ratio = {compression_global:.4f}', ha='center', transform=plt.gca().transAxes)

    plt.subplot(1, 3, 3)
    nx.draw_networkx_nodes(G_local, pos, node_color='lightgreen', node_size=500)
    nx.draw_networkx_edges(G_local, pos, arrowstyle='->', arrowsize=15)
    nx.draw_networkx_labels(G_local, pos)
    plt.title('Local Backbone')
    plt.axis('off')
    plt.text(0.5, -0.1, f'Total weight of the network = {W_local}\nTotal number of edges = {E_local}\nInverse compression ratio = {compression_local:.4f}', ha='center', transform=plt.gca().transAxes)
    plt.tight_layout()
    plt.savefig("mdl_network_backbones.png", bbox_inches='tight', dpi=200)
    plt.show()

visualize_backbones(elist, backbone_global, backbone_local, compression_global, compression_local)
```

--------------------------------

### MDL_backboning(elist, directed, out_edges, allow_empty)

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/mdl_backboning.md

Compute the MDL-optimal global and local network backbones from the given edge list.

```APIDOC
## MDL_backboning(elist, directed=True, out_edges=True, allow_empty=True)

### Description
Compute the MDL-optimal global and local network backbones from the given edge list.

### Parameters
- **elist** (list) - Required - List of edges as tuples (i, j, w_ij).
- **directed** (bool) - Optional - Boolean indicating if the network is directed, defaults to True.
- **out_edges** (bool) - Optional - Boolean indicating whether to track out-edges (True) or in-edges (False), defaults to True.
- **allow_empty** (bool) - Optional - Allows empty backbones if True, defaults to True.

### Response
- **backbone_global** (list) - Edge list of the global MDL-optimal backbone.
- **backbone_local** (list) - Edge list of the local MDL-optimal backbone.
- **compression_global** (float) - Inverse compression ratio for the global backbone.
- **compression_local** (float) - Inverse compression ratio for the local backbone.
```

--------------------------------

### Execute Greedy Optimization

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/paninipy/community_regionalization/example.ipynb

Unpacks the loaded data and runs the greedy optimization algorithm.

```python
# Unpack the data
name = data["Name"]
N = data["N"]
spatial_elist = data["Spatial Edgelist"]
flow_elist = data["Flow Edgelist"]

# Run the greedy algorithm
print(f"Running greedy algorithm on {name}...")
DLs, partitions = greedy_opt(N, spatial_elist, flow_elist)
```

--------------------------------

### Utility Functions

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/mdl_backboning.md

Helper functions for network conversion and mathematical calculations used in MDL backboning.

```APIDOC
## Utility Functions

### logchoose(n, k)
Compute the logarithm of the binomial coefficient.

### logmultiset(n, k)
Compute the logarithm of the multiset coefficient.

### to_undirected(edge_list, policy="sum")
Convert a directed edge list to an undirected edge list by merging edges based on the specified policy.
```

--------------------------------

### Construct Network Backbone and Compute Compression Ratios

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Code/mdl_backboning.md

This function constructs the MDL-optimal backbone edgelists and computes inverse compression ratios. It handles directed and undirected graphs, and different compression ratio types.

```python
si,ki,sbi,kbi = sum(adj_weights[i]),len(adj_edges[i]),0,0
        Llocali = flocal(si,ki,0,0)
        Llocal0 += naivelocal(si,ki)
        best_Llocali,best_kbi,best_sbi = Llocali,kbi,sbi
        for w_ij in adj_weights[i]:

            kbi += 1
            sbi += w_ij
            Llocali += flocal(si,ki,sbi,kbi) - flocal(si,ki,sbi-w_ij,kbi-1)

            if Llocali < best_Llocali:
                best_Llocali = Llocali
                best_kbi = kbi
                best_sbi = sbi

        if (best_kbi == 0) and not(allow_empty): #by symmetry, DL is equivalent, so can choose to keep all edges
            best_kbi = ki

        min_DL_local += best_Llocali
        backbone_degrees[i] = best_kbi

    #construct MDL-optimal backbone edgelists based on identified description lengths
    backbone_global = elist[:backbone_Eb]

    backbone_local = []
    for i in adj_edges:
        MDL_kbi = backbone_degrees[i]
        for index,j in enumerate(adj_edges[i][:MDL_kbi]):
            backbone_local.append((i,j,adj_weights[i][index]))

    if out_edges == False: #if out_edges == False, reverse edge order for local method back to format of input
        backbone_local = [(e[1],e[0],e[2]) for e in backbone_local]

    if not(directed): #convert backbone to undirected edge tuples if input was undirected
        backbone_global = np.unique([tuple([sorted([e[0],e[1]])+[e[2]]]) for e in backbone_global])
        backbone_local = np.unique([tuple([sorted([e[0],e[1]])+[e[2]]]) for e in backbone_local])

    #compute inverse compression ratios
    if CR_type == 'Relative':
        compression_global,compression_local = min_DL_global/Lglobal0,min_DL_local/Llocal0

    elif CR_type == 'Max':
        compression_global,compression_local = min_DL_global/max(Lglobal0,Llocal0),min_DL_local/max(Lglobal0,Llocal0)

    elif CR_type == 'Min':
        compression_global,compression_local = min_DL_global/min(Lglobal0,Llocal0),min_DL_local/min(Lglobal0,Llocal0)

    return backbone_global,backbone_local,compression_global,compression_local
```

--------------------------------

### Visualizing Binning Results

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/hypergraph_binning.md

Defines a function to plot the event partitions and executes it for both algorithm variants.

```python
def visualize_binning(X, result, method):
    labels = result[1]
    times = [t for _, _, _, t in X]
    pairs = [(src, dest) for src, dest, _, _ in X]
    times_transformed = times
    unique_nodes = sorted(list(set([src for src, dest in pairs] + [dest for src, dest in pairs])))
    node_pos = {node: i for i, node in enumerate(unique_nodes)}

    colors = ['skyblue' if label == 0 else 'lightcoral' for label in labels]

    fig, ax = plt.subplots(figsize=(12, 4))
    for (src, dest), time, color in zip(pairs, times_transformed, colors):
        src_pos = node_pos[src]
        dest_pos = node_pos[dest]
        ax.plot([time, time], [src_pos, dest_pos], color=color, marker='o', markersize=11, linestyle='-', zorder=1)
        ax.text(time, src_pos, f"{src}", ha='center', va='center', fontsize=9, fontweight='bold', zorder=2, color='black')
        ax.text(time, dest_pos, f"{dest}", ha='center', va='center', fontsize=9, fontweight='bold', zorder=2, color='black')

    ax.set_xlim(min(times_transformed) - 0.5, max(times_transformed) + 0.5)
    ax.set_ylim(min(node_pos.values()) - 1, max(node_pos.values()) + 1)
    ax.set_yticks([])
    ax.set_xlabel('Time')
    ax.set_title(f'Event Partition in Time Scale with {method} Dynamic Solution')
    plt.savefig(f"timeline_plot_with_log_transform_{method}.png", bbox_inches='tight', dpi=200)
    plt.show()
```

```python
visualize_binning(X, results_exact, 'exact dynamic programming')
```

```python
visualize_binning(X, results_greedy, 'fast greedy agglomerative')
```

--------------------------------

### Clone PANINIpy Repository for Testing

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/installation.md

Clone the PANINIpy GitHub repository to access the Tests folder for manual testing.

```console
git clone https://github.com/HKU-Complex-Networks-Lab/PANINIpy.git
cd PANINIpy
```

--------------------------------

### Compute Total Description Length

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/community_regionalization.md

Calculates the total description length for the current configuration. This function does not require any parameters.

```python
greedy_opt.total_dl()
```

--------------------------------

### greedy_opt.total_dl()

Source: https://github.com/hku-complex-networks-lab/paninipy/blob/main/Documentation/Papers/community_regionalization.md

Compute the total description length.

```APIDOC
## greedy_opt.total_dl()

### Description
Compute the total description length.

### Response
#### Success Response (200)
- **float** - Total description length.
```