### Install Mandala in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/01_hello.ipynb

This Python snippet checks if the execution environment is Google Colab and, if so, installs the `mandala` library directly from its GitHub repository. This ensures the necessary dependencies are available for running `mandala` examples within a Colab notebook.

```python
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass# Run this if in a Google Colab notebook
```

--------------------------------

### Install Mandala in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This snippet provides a conditional installation command for the `mandala` library, specifically tailored for Google Colab environments. It checks for the presence of `google.colab` and installs the library from its GitHub repository if detected, ensuring necessary dependencies are met for interactive notebooks.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass# Run this if in a Google Colab notebook
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/03_cf.ipynb

Installs the `mandala` library from its GitHub repository, specifically for use within a Google Colab environment, ensuring the necessary dependencies are available for the examples.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Initializing a ComputationFrame from Custom Variable Groups and Forward Expansion in Python

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/03_cf.ipynb

Demonstrates how to manually initialize a `ComputationFrame` by providing a dictionary of `Ref` iterables, which serve as the CF's variables. It includes a setup code block to generate the `Ref`s. The example then shows how to visualize the initial CF and subsequently expand it forward selectively for specific variables (e.g., 'model') to add related computational history.

```Python
with storage: 
    models, test_accs = [], []
    X_train, X_test, y_train, y_test = generate_dataset()
    for n_estimators in [10, 20, 40, 80]:
        model, train_acc = train_model(X_train, y_train, n_estimators=n_estimators)
        models.append(model)
        if storage.unwrap(train_acc) > 0.8: # conditional execution
            test_acc = eval_model(model, X_test, y_test)
            test_accs.append(test_acc)
```

```Python
cf = storage.cf({'model': models, 'test_acc': test_accs})
pprint(cf)
cf.draw(verbose=True)
```

```Python
cf.expand_forward(varnames=['model'], inplace=True)
cf.draw(verbose=True)
```

--------------------------------

### Install Mandala in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/02_retracing.ipynb

Installs the `mandala` library from its GitHub repository, specifically for use within Google Colab environments. This ensures the necessary dependencies are available for running the examples.

```python
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Define and Track Dependencies with Mandala @op and @track

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This Python example demonstrates how to define an operation (`@op`) and track its dependencies (`@track`) using the `mandala` library. The `add` function is an operation that depends on the `square` function, which is marked for dependency tracking. Changes to `square` will invalidate relevant past calls of `add`.

```python
from unittest.mock import patch
from mandala.utils import mock_input # to simulate user input non-interactively

@op # define a new @op to compose with `inc`
def add(x, y):
    print("Hello from add!")
    return x + square(y)

@track # dependency tracking decorator
def square(num):
    return num**2
```

--------------------------------

### Install Mandala Python Library

Source: https://github.com/amakelov/mandala/blob/master/README.md

Instructions for installing the `pymandala` library using pip, including both the stable release and the latest development version directly from GitHub.

```Bash
pip install pymandala
```

```Bash
pip install git+https://github.com/amakelov/mandala
```

--------------------------------

### Minimal ComputationFrame Example in Python

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/blog/01_cf.md

This example demonstrates how to define Python functions decorated with `@op` from the `mandala` library, run computations, and then create and expand a `ComputationFrame` from the results. It illustrates the core workflow of running computations, creating a CF, adding context, and converting it to a pandas DataFrame for analysis. The example includes conditional execution to show how partial computations are handled, with missing values represented by nulls.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass

from mandala.imports import *

@op(output_names=['y'])
def increment(x): return x + 1

@op(output_names=['w'])
def add(y, z): return y + z

# compose the operations
with Storage() as storage: # the `storage` automatically stores calls to `@op`s
    for x in range(5):
        y = increment(x)
        if x % 2 == 0:
            w = add(y=y, z=x)

# get a CF for just the `increment` operation 
cf = storage.cf(increment)
# expand the CF to include the `add` operation
cf = cf.expand_forward()
# draw the CF
cf.draw(verbose=True, orientation='LR')
# convert the CF to a dataframe
print(cf.df().to_markdown())
```

--------------------------------

### Define and Use @op Decorated Functions for Memoization

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This example demonstrates the core functionality of `mandala`'s `@op` decorator. It shows how to define a function (`inc`) that automatically tracks inputs, outputs, and code, and how to use a `Storage` object to persist and reuse computation results, preventing redundant executions of the same call.

```python
from mandala.imports import *
import time

storage = Storage( # stores all `@op` calls
    # where to look for dependencies; use `None` to prevent versioning altogether
    deps_path='__main__' 
    ) 

@op
def inc(x):
    print("Hello from inc!")
    time.sleep(1) # simulate a long operation
    return x + 1

with storage: # all `@op` calls inside this block will be stored in `storage`
    start = time.time()
    a = inc(1)
    b = inc(1) # this will not be executed, but reused
    end = time.time()
    print(f'Took {round(end - start)} seconds')
```

--------------------------------

### APIDOC: Output of storage.versions() for Function 'add'

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This output provides detailed information about the versions of the `add` function, including its content and semantic version IDs. It serves as documentation for the `storage.versions` method's output structure, showing how dependencies are listed and version metadata is presented.

```APIDOC
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ### Dependencies for version of function add from module __main__                                              │
│ ### content_version_id=7cd06a0178abc60d137bb47bceafa5f9                                                        │
│ ### semantic_version_id=455b6b8789fb67940e41dbbb135292f7                                                       │
│                                                                                                                │
│ ################################################################################                               │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

--------------------------------

### Track and Manage Function Versions with @track

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/01_hello.ipynb

This example showcases `mandala`'s automatic per-call versioning and dependency tracking. It defines an `add` function that depends on a `@track`-decorated `square` function. When `add` is modified, `mandala` detects the semantic change, creating a new version. The snippet simulates user input to confirm the change and demonstrates how `mandala` manages function versions.

```python
from unittest.mock import patch
from mandala.utils import mock_input # to simulate user input non-interactively

@op # define a new @op to compose with `inc`
def add(x, y):
    print("Hello from add!")
    return x + square(y)

@track # dependency tracking decorator
def square(num):
    return num**2

# same computations as before, change to `add` will be detected
with patch('builtins.input', mock_input(['y'])):
    with storage:
        for i in range(5):
            j = inc(i)
            if i % 2 == 0:
                k = add(i, j)
```

--------------------------------

### Minimal Example of ComputationFrame Usage

Source: https://github.com/amakelov/mandala/blob/master/docs_source/blog/01_cf.ipynb

This example defines two Python functions decorated with `@op` from `mandala`, runs computations using them, and then creates and expands a `ComputationFrame` from the results. It demonstrates how to draw the CF and convert it to a pandas DataFrame for analysis.

```Python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass

from mandala.imports import *

@op(output_names=['y'])
def increment(x): return x + 1

@op(output_names=['w'])
def add(y, z): return y + z

# compose the operations
with Storage() as storage: # the `storage` automatically stores calls to `@op`s
    for x in range(5):
        y = increment(x)
        if x % 2 == 0:
            w = add(y=y, z=x)

# get a CF for just the `increment` operation 
cf = storage.cf(increment)
# expand the CF to include the `add` operation
cf = cf.expand_forward()
# draw the CF
cf.draw(verbose=True, orientation='LR')
# convert the CF to a dataframe
print(cf.df().to_markdown())
```

--------------------------------

### Initialize and Visualize ComputationFrame from an @op

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/03_cf.md

This snippet demonstrates how to create a `ComputationFrame` (CF) object by collecting all recorded calls to a specified `@op` function (e.g., `train_model`) from `storage`. It then uses `pprint` to display a summary of the CF, including its variables, operations, and computational graph. Finally, `cf.draw(verbose=True)` is called to generate a visual representation of the graph.

Example `Call` object representation:
```
'train_model': {Call(train_model, hid=e60...)}
}
```

Example `pprint(cf)` output:
```
ComputationFrame with:
    5 variable(s) (14 unique refs)
    1 operation(s) (4 unique calls)
Computational graph:
    var_0@output_0, var_1@output_1 = train_model(X_train=X_train, n_estimators=n_estimators, y_train=y_train)
```

```Python
cf = storage.cf(train_model)
pprint(cf)
cf.draw(verbose=True)
```

--------------------------------

### Install Mandala in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/06_advanced_cf.ipynb

This snippet checks if the environment is Google Colab and, if so, installs the Mandala library directly from its GitHub repository. This ensures the necessary dependencies are available for running the examples.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Python Inspecting Function Versions and Dependencies

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This snippet shows how to programmatically inspect the versions and dependencies of a Python function using `storage.versions(add)`. This method provides details about the content and semantic version IDs of the function, along with its dependencies.

```Python
storage.versions(add)
```

--------------------------------

### Example Output of cProfile.print_callees()

Source: https://github.com/amakelov/mandala/blob/master/docs_source/blog/02_deps.ipynb

This snippet shows a simplified example of the output generated by `pstats.Stats.print_callees()`. It illustrates how `cProfile` aggregates call data, showing `ncalls`, `tottime`, and `cumtime` for functions and their callees. This aggregated view highlights the limitation of tracking per-call dependencies.

```text
Function                                  called...
                ncalls  tottime  cumtime
(__init__)  ->       1    0.000    0.000  (f)
(m)         -> 
(g)         ->       1    0.000    0.000  (__init__)
                     1    0.000    0.000  (m)
                     1    0.000    0.000  (__init__)
                     1    0.000    0.000  (m)
(m)         -> 
(f)         -> 
(__init__)  -> 
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/01_storage_and_ops.ipynb

This snippet provides a conditional installation command for the Mandala library, specifically tailored for Google Colab environments. It checks for the presence of `google.colab` and installs the library from its GitHub repository if detected.

```Python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/gotchas.md

Installs the `mandala` library from GitHub specifically for use within a Google Colab environment, handling potential errors during installation.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Query and Visualize Computations with ComputationFrame

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/01_hello.ipynb

This snippet demonstrates how to use `ComputationFrame`s to query and visualize relationships between `@op`-decorated functions. It defines an `add` `@op` function, performs several calls to `inc` and `add` within a storage context, then retrieves a `ComputationFrame` for `inc` calls. It shows how to draw the computation frame and expand it to include connected calls, providing a high-level view of computational graphs.

```python
@op # define a new @op to compose with `inc`
def add(x, y):
    print("Hello from add!")
    return x + y

with storage:
    for i in range(5):
        j = inc(i)
        if i % 2 == 0:
            k = add(i, j)

# get & visualize the computation frame for all calls to `inc`
cf = storage.cf(inc)
print('Computation frame for `inc`:')
cf.draw(verbose=True, orientation='LR') # visualize the computation frame

# expand the computation frame to include all calls connected to the calls of
# `inc` through shared inputs/outputs
cf.expand_all(inplace=True)
print('Expanded computation frame for `inc`:')
cf.draw(verbose=True, orientation='LR', path='test.jpg') # visualize the computation frame
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/01_storage_and_ops.md

This snippet provides a conditional installation command for the Mandala library, specifically tailored for Google Colab environments. It checks for the presence of `google.colab` and installs the library if detected, ensuring the necessary dependencies are available for execution.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Python: Example of Global and Nested Function Tracking

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/blog/02_deps.md

Demonstrates the global variable and function tracking mechanism in action with a comprehensive example. It includes global variables, a simple function, a class with methods, and a nested class with its own methods, all decorated with `@track` to show how their interactions with global state are monitored.

```Python
A = 23
B = 42

@track
def f(x):
    return x + A

class C:
    @track
    def __init__(self, x):
        self.x = x + B

    @track
    def m(self, y):
        return self.x + y

    class D:
        @track
        def __init__(self, x):
            self.x = x + f(x)

        @track
        def m(self, y):
            return y + A

@track
def g(x):
    if x % 2 == 0:
        return C(x).m(x)
    else:
        return C.D(x).m(x)
```

--------------------------------

### Convert ComputationFrame to Pandas DataFrame

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This example demonstrates how to convert a `ComputationFrame` into a standard pandas DataFrame for easier analysis. It shows that the DataFrame columns represent nodes in the computational graph (functions and variables), and each row represents a computation trace, potentially padded with `NaN`s.

```python
print(cf.df().to_markdown())
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/03_cf.md

This snippet provides a conditional installation of the `mandala` library from its GitHub repository, specifically designed for execution within a Google Colab environment. It checks for the `google.colab` module and installs `mandala` if detected.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Install Mandala in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/05_collections.md

Provides a Python snippet to install the `mandala` library from its GitHub repository, specifically tailored for execution within a Google Colab environment.

```python
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Demonstrate Python sys.settrace Tracer Context Manager

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/blog/02_deps.md

This example shows how to use the `Tracer` context manager to observe function calls in a simple Python program. It defines two functions, `f` and `g`, and then uses `Tracer` to log their execution flow, illustrating the output of the tracing mechanism.

```python
### in funcs.py
def f(x):
    return x + 1

### in IPython session
In [1]: from funcs import *

In [2]: def g(x):
   ...:     return f(x) + 1
   ...: 

In [3]: with Tracer():
   ...:     g(23)
   ...: 
Calling __main__.g
Calling funcs.f
Returning from funcs.f
Returning from __main__.g
Calling funcs.__exit__ # you'd have to manually remove this one
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/02_retracing.md

This snippet provides a conditional installation command for the `mandala` library, specifically tailored for Google Colab environments. It checks for the presence of `google.colab` and then uses `pip` to install `mandala` directly from its GitHub repository, ensuring the necessary dependencies are met for subsequent code execution.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Demonstrate Automatic Memoization with @op Decorator

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/01_hello.ipynb

This example illustrates `mandala`'s automatic memoization using the `@op` decorator. It defines an `inc` function that simulates a long operation and decorates it with `@op`. When `inc(1)` is called twice within a `Storage` context, the second call is reused from storage, demonstrating that the function body is executed only once, saving computation time.

```python
from mandala.imports import *
import time

storage = Storage( # stores all `@op` calls
    # where to look for dependencies; use `None` to prevent versioning altogether
    deps_path='__main__'
    )

@op
def inc(x):
    print("Hello from inc!")
    time.sleep(1) # simulate a long operation
    return x + 1

with storage: # all `@op` calls inside this block will be stored in `storage`
    start = time.time()
    a = inc(1)
    b = inc(1) # this will not be executed, but reused
    end = time.time()
    print(f'Took {round(end - start)} seconds')
```

--------------------------------

### Inspect Stored Function Versions

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/01_hello.ipynb

This simple command demonstrates how to inspect the different versions of an `@op`-decorated function that have been stored in the `mandala` storage. It provides insight into how `mandala` tracks changes and creates new versions based on semantic modifications or dependency changes.

```python
storage.versions(add)
```

--------------------------------

### Minimal Example of Python Function Call Tracking

Source: https://github.com/amakelov/mandala/blob/master/docs_source/blog/02_deps.ipynb

This example demonstrates how to apply the `@track` decorator to functions `f` and `g`, and then use the `Tracer` class as a context manager to observe the call graph generated during execution. It shows how the `t.graph` attribute captures the sequence of calls.

```Python
@track
def f(x):
    return x + 1

@track
def g(x):
    return f(x) + 1

with Tracer() as t:
    g(23)

# Access the generated graph:
# print(t.graph) # This would output [('__main__', 'g', '__main__', 'f')]
```

--------------------------------

### Install Mandala Library in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/gotchas.ipynb

This snippet provides a robust way to install the `mandala` library specifically within a Google Colab environment. It attempts to import `google.colab` and, if successful, uses `pip` to install `mandala` directly from its GitHub repository, handling potential import errors gracefully.

```Python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Compose @op Functions and Visualize Computation Frames

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This snippet illustrates how `@op`-decorated functions can be composed and how `ComputationFrame`s provide a high-level view of these relationships. It defines a new `@op` function (`add`), executes a series of composed calls, and then demonstrates how to retrieve, draw, and expand a `ComputationFrame` to visualize the computational graph.

```python
@op # define a new @op to compose with `inc`
def add(x, y):
    print("Hello from add!")
    return x + y

with storage:
    for i in range(5):
        j = inc(i)
        if i % 2 == 0:
            k = add(i, j)

# get & visualize the computation frame for all calls to `inc`
cf = storage.cf(inc) 
print('Computation frame for `inc`:')
cf.draw(verbose=True, orientation='LR') # visualize the computation frame

# expand the computation frame to include all calls connected to the calls of
# `inc` through shared inputs/outputs
cf.expand_all(inplace=True) 
print('Expanded computation frame for `inc`:')
cf.draw(verbose=True, orientation='LR', path='test.jpg')
```

--------------------------------

### Run Python Script to Generate Documentation

Source: https://github.com/amakelov/mandala/blob/master/docs_source/readme.md

This command executes the `make_docs.py` Python script, which is responsible for converting Jupyter notebooks into markdown documentation files. Ensure that the script is located in the current working directory and that Python is installed and accessible in your environment.

```bash
python make_docs.py
```

--------------------------------

### Install Mandala Library in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/04_versions.ipynb

This snippet provides a robust way to install the `mandala` library from its GitHub repository, specifically tailored for Google Colab environments. It uses a `try-except` block to handle potential import errors, ensuring the installation command is executed only when necessary.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Install Mandala Library in Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/06_advanced_cf.md

This snippet provides a conditional installation command for the `mandala` library, specifically designed for Google Colab environments. It checks for the `google.colab` module and installs the library from its GitHub repository if detected, ensuring necessary dependencies are met for subsequent code execution.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Run a Machine Learning Experiment Workflow with Mandala Storage

Source: https://github.com/amakelov/mandala/blob/master/_demos/cf_vid.ipynb

This snippet demonstrates how to execute the previously defined Mandala operations within a `storage` context. It orchestrates a series of machine learning experiments, including conditional data scaling, training SVC models with different kernels, training RandomForest models with varying estimators, and evaluating an ensemble of all trained models. Mandala automatically tracks the dependencies and results of each step.

```python
with storage:
    for scale in (True, False):
        X, y = get_data()
        if scale:
            X = scale_data(X=X)
        X_train, X_test, y_train, y_test = get_train_test_split(X=X, y=y)

        svc_models = []
        for kernel in ('linear', 'rbf', 'poly'):
            svc_model = train_svc(X_train=X_train, y_train=y_train, kernel=kernel)
            svc_acc = eval_model(model=svc_model, X_test=X_test, y_test=y_test)
            svc_models.append(svc_model)
        
        rf_models = []
        for n_estimators in (5, 10, 20):
            rf_model = train_random_forest(X_train=X_train, y_train=y_train, n_estimators=n_estimators)
            rf_acc = eval_model(model=rf_model, X_test=X_test, y_test=y_test)
            rf_models.append(rf_model)
        
        ensemble_acc = eval_ensemble(models=svc_models + rf_models, X_test=X_test, y_test=y_test)
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/04_versions.md

This Python snippet provides a conditional installation of the `mandala` library, specifically tailored for Google Colab environments. It attempts to import `google.colab` and, if successful, installs `mandala` directly from its GitHub repository using pip.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Example Usage of the Python `Tracer` Context Manager

Source: https://github.com/amakelov/mandala/blob/master/docs_source/blog/02_deps.ipynb

This snippet demonstrates how to use the `Tracer` context manager to observe function calls within a Python program. It shows a simple `f` function in `funcs.py` and a `g` function in an IPython session, illustrating the output generated by the `Tracer` when `g(23)` is called.

```Python
### in funcs.py
def f(x):
    return x + 1

### in IPython session
In [1]: from funcs import *

In [2]: def g(x):
   ...:     return f(x) + 1
   ...: 

In [3]: with Tracer():
   ...:     g(23)
   ...: 
Calling __main__.g
Calling funcs.f
Returning from funcs.f
Returning from __main__.g
Calling funcs.__exit__ # you'd have to manually remove this one
```

--------------------------------

### Example Output of cProfile print_callees Method

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/blog/02_deps.md

This snippet shows a simplified example of the output generated by `pstats.Stats.print_callees()`. It lists functions and their direct callees, along with call counts (`ncalls`), total time spent in the function itself (`tottime`), and cumulative time (`cumtime`). This aggregated data helps identify call relationships but lacks per-call granularity for dependency tracking.

```text
Function                                  called...
                ncalls  tottime  cumtime
(__init__)  ->       1    0.000    0.000  (f)
(m)         -> 
(g)         ->       1    0.000    0.000  (__init__)
                     1    0.000    0.000  (m)
                     1    0.000    0.000  (__init__)
                     1    0.000    0.000  (m)
(m)         -> 
(f)         -> 
(__init__)  -> 
```

--------------------------------

### Install Mandala for Google Colab Environment

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/02_ml.ipynb

This Python snippet checks if the execution environment is Google Colab. If it is, it installs the `mandala` library directly from its GitHub repository, ensuring all necessary dependencies are met for running the tutorial.

```python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Install Mandala for Google Colab

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/05_collections.ipynb

This snippet provides a conditional installation command for the `mandala` library, specifically tailored for Google Colab environments to ensure the library is available before use.

```Python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass
```

--------------------------------

### Install Mandala and Import ML Libraries in Python

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/02_ml.md

This code block first attempts to install the `mandala` library, specifically targeting Google Colab environments. Following the installation, it imports essential Python libraries for machine learning tasks, including `numpy`, `pandas`, `sklearn` components, and the recommended `mandala` functionalities, setting a random seed for reproducibility.

```Python
# for Google Colab
try:
    import google.colab
    !pip install git+https://github.com/amakelov/mandala
except:
    pass

from typing import Tuple
import numpy as np
import pandas as pd
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

# recommended way to import mandala functionality
from mandala.imports import *

np.random.seed(0)
```

--------------------------------

### Demonstrate Global Variable and Nested Call Tracking

Source: https://github.com/amakelov/mandala/blob/master/docs_source/blog/02_deps.ipynb

This comprehensive example showcases the global tracking mechanism in action. It defines global variables `A` and `B`, functions `f` and `g`, and nested classes `C` and `D` with methods, all decorated with `@track`. The example illustrates how global variables and calls to functions and methods (including nested ones) are tracked and recorded in the `Tracer`'s graph.

```python
A = 23
B = 42

@track
def f(x):
    return x + A

class C:
    @track
    def __init__(self, x):
        self.x = x + B

    @track
    def m(self, y):
        return self.x + y

    class D:
        @track
        def __init__(self, x):
            self.x = x + f(x)

        @track
        def m(self, y):
            return y + A

@track
def g(x):
    if x % 2 == 0:
        return C(x).m(x)
    else:
        return C.D(x).m(x)
```

--------------------------------

### Define and Execute ML Pipeline with Mandala Versioning

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/04_versions.ipynb

This comprehensive example illustrates a machine learning pipeline integrated with `mandala`'s versioning capabilities. It defines functions for data loading (`@op`), data scaling (`@track` for non-memoized dependency), model training (`@op`), and model evaluation (`@op`), then executes the pipeline to demonstrate how `mandala` tracks and manages different versions of `op` calls based on their dependencies.

```python
from sklearn.datasets import load_digits
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

N_CLASS = 10

@track # to track a non-memoized function as a dependency
def scale_data(X):
    return StandardScaler(with_mean=True, with_std=False).fit_transform(X)

@op
def load_data():
    X, y = load_digits(n_class=N_CLASS, return_X_y=True)
    return X, y

@op
def train_model(X, y, scale=False):
    if scale:
        X = scale_data(X)
    return LogisticRegression(max_iter=1000, solver='liblinear').fit(X, y)

@op
def eval_model(model, X, y, scale=False):
    if scale:
        X = scale_data(X)
    return model.score(X, y)

with storage:
    X, y = load_data()
    for scale in [False, True]:
        model = train_model(X, y, scale=scale)
        acc = eval_model(model, X, y, scale=scale)
```

--------------------------------

### Python Workflow for Detecting and Recomputing Function Changes

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This snippet demonstrates a Python workflow where changes to a function (`add`) are detected within a `storage` context. It shows how the system prompts the user to decide on recomputation for dependent calls, simulating user input with `patch('builtins.input', mock_input(['y']))`. The `inc` and `add` functions are called repeatedly to trigger the change detection.

```Python
with patch('builtins.input', mock_input(['y'])):
    with storage:
        for i in range(5):
            j = inc(i)
            if i % 2 == 0:
                k = add(i, j)
```

--------------------------------

### Define and Execute a Memoized ML Pipeline

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/02_retracing.md

This example demonstrates building a simple machine learning pipeline using `mandala`'s `@op` decorator. It defines functions for data loading, model training (RandomForestClassifier), and accuracy calculation, all of which are memoized. The pipeline is executed within a `mandala.Storage` context, showcasing how `@op`s compose and automatically manage computational history for efficient reuse.

```python
from mandala.imports import *
from sklearn.datasets import load_digits
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

@op
def load_data(n_class=2):
    print("Loading data")
    return load_digits(n_class=n_class, return_X_y=True)

@op
def train_model(X, y, n_estimators=5):
    print("Training model")
    return RandomForestClassifier(n_estimators=n_estimators,
                                  max_depth=2).fit(X, y)

@op
def get_acc(model, X, y):
    print("Getting accuracy")
    return round(accuracy_score(y_pred=model.predict(X), y_true=y), 2)

storage = Storage()

with storage:
    X, y = load_data() 
    model = train_model(X, y)
    acc = get_acc(model, X, y)
    print(acc)
```

--------------------------------

### Example Output: Mandala Function Version Details

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/04_versions.md

This block presents a formatted output from `mandala`'s `storage.versions()` call, specifically for the `train_model` function. It illustrates the detailed information `mandala` provides for each function version, including `content_version_id`, `semantic_version_id`, and the tracked dependencies within the module, confirming how different function versions are distinguished.

```APIDOC
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ ### Dependencies for version of function train_model from module __main__                                       │
│ ### content_version_id=db93a1e9c60fb37868575845a7afe47d                                                        │
│ ### semantic_version_id=2acaa8919ddd4b5d8846f1f2d15bc971                                                       │
│                                                                                                               │
│ ################################################################################                                │
│ ### IN MODULE "__main__"                                                                                       │
│ ################################################################################                                │
│                                                                                                               │
│ @op                                                                                                            │
│ def train_model(X, y, scale=False):                                                                            │
```

--------------------------------

### Define and Execute ComputationFrame Operations

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/06_advanced_cf.ipynb

This example initializes a `Storage` object and defines several atomic operations (`inc`, `add`, `square`, `divmod_`) using the `@op` decorator. It then executes a series of computations within the storage context, building a complex computation graph involving these operations and their dependencies.

```python
from mandala.imports import *
storage = Storage()

@op
def inc(x): return x + 1

@op
def add(y, z): return y + z

@op
def square(w): return w ** 2

@op
def divmod_(u, v): return divmod(u, v)

with storage:
    xs = [inc(i) for i in range(5)]
    ys = [add(x, z=42) for x in xs] + [square(x) for x in range(5, 10)]
    zs = [divmod_(x, y) for x, y in zip(xs, ys[3:8])]
```

--------------------------------

### Converting ComputationFrame to Pandas DataFrame with `.df()`

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/03_cf.md

Demonstrates how to convert a `ComputationFrame` (CF) into a `pandas.DataFrame` using the `.df()` method for easier analysis. The example shows dropping specific columns and formatting the output as markdown, revealing that some computations may only partially follow the full graph.

```python
cf = storage.cf(train_model).expand_all()
print(cf.df().drop(columns=['X_train', 'y_train']).to_markdown())
```

--------------------------------

### Initialize Mandala Storage and Define Exit Hook for Visualization

Source: https://github.com/amakelov/mandala/blob/master/_demos/cf_vid.ipynb

This snippet sets up the Mandala `Storage` object, which manages the computational graph and data provenance. It also defines and registers an `exit_hook` function. This hook automatically visualizes the computation graph (`demo.svg`) and displays a DataFrame of the pipeline's results, masking sensitive data columns, upon exiting the `storage` context.

```python
from mandala.imports import *
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from typing import Any
import numpy as np
np.random.seed(42)

storage = Storage(deps_path='__main__')

def exit_hook(storage: Storage):
    ops = storage.ops.cache
    if "scale_data" not in ops:
        cf = storage.cf(ops['get_data'])
    elif "get_train_test_split" not in ops:
        cf = storage.cf(ops['scale_data']) | storage.cf(ops['get_data'])
    elif "train_svc" not in ops:
        cf = storage.cf(ops['get_train_test_split'])
    elif "eval_model" not in ops:
        cf = storage.cf(ops['train_svc']) | storage.cf(ops['train_random_forest'])
    elif "eval_ensemble" not in ops:
        cf = storage.cf(ops['eval_model'])
    else:
        cf = storage.cf(ops['eval_ensemble']) | storage.cf(ops['eval_model'])
    cf = cf.expand_all()
    cf.draw(path='demo.svg', verbose=True, show_how="none")
    df = cf.df(values='objs', include_calls=False)
    bad_cols = ['X', 'y', 'X_train', 'X_test', 'y_train', 'y_test', 'X_scaled']
    for col in bad_cols:
        if col in df:
            # replace the values with "..."
            df[col] = df[col].apply(lambda x: "...")
    display(df)
storage._exit_hooks.append(exit_hook)
```

--------------------------------

### Convert ComputationFrame to Pandas DataFrame

Source: https://github.com/amakelov/mandala/blob/master/docs_source/tutorials/01_hello.ipynb

This code demonstrates how to extract a standard pandas DataFrame from a `ComputationFrame`. The DataFrame represents computation traces, with columns corresponding to nodes in the computational graph (functions and variables) and rows representing individual traces, potentially padded with `NaN`s, facilitating easier analysis.

```python
print(cf.df().to_markdown())
```

--------------------------------

### Performing Full Computation Frame Expansion with expand_all

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/02_ml.md

This example demonstrates the `cf.expand_all()` method, which fully expands the computation frame to include all reachable variables and operations, providing a comprehensive view of the entire computational graph. The output shows the structure of the `ComputationFrame`.

```python
expanded_cf = cf.expand_all()
expanded_cf
```

--------------------------------

### Sequentially Expanding ComputationFrame Backwards with `expand_back`

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/03_cf.md

Demonstrates how to precisely expand a `ComputationFrame` (CF) by adding calls related to specific variables one by one using `expand_back`. The example shows a sequence of expansions for 'v', 'X_test', 'model', and 'X_train', illustrating how the CF detects and reuses calls when histories converge.

```python
cf = storage.cf(test_acc)
cf = cf.expand_back('v')
cf.draw(verbose=True)

cf = cf.expand_back('X_test')
cf.draw(verbose=True)

cf = cf.expand_back('model')
cf.draw(verbose=True)

cf = cf.expand_back('X_train')
cf.draw(verbose=True)
```

--------------------------------

### Python Function Semantic Diff for 'add' Function

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/tutorials/01_hello.md

This diff output highlights the semantic change made to the `add` function. The original `return x + y` is replaced with `return x + square(y)`, indicating a new dependency on a `square` function. This change triggers the recomputation prompt from the system.

```Diff
╭───────────────────────────────────────────────────── Diff ──────────────────────────────────────────────────────╮
│   1  def add(x, y):                                                                                            │
│   2      print("Hello from add!")                                                                              │
│   3 -    return x + y                                                                                          │
│   4 +    return x + square(y)                                                                                  │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

--------------------------------

### Accessing Memoized Results as an Imperative Storage Interface in Python

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/02_retracing.ipynb

This example illustrates how a composition of memoized `@op`s acts as an imperative storage interface. It shows how to iterate through specific parameters and directly access the unwrapped results of `acc` and `model` from storage, allowing users to focus on particular outputs of interest without recomputing.

```python
with storage:
    for n_class in (5,):
        X, y = load_data(n_class) 
        for n_estimators in (5,):
            model = train_model(X, y, n_estimators=n_estimators)
            acc = get_acc(model, X, y)
            print(storage.unwrap(acc), storage.unwrap(model))
```

--------------------------------

### Define Data Preparation, Model Training, and Evaluation Operations in Mandala

Source: https://github.com/amakelov/mandala/blob/master/_demos/cf_vid.ipynb

This snippet defines a set of reusable machine learning operations using Mandala's `@op` decorator. These operations cover the entire ML pipeline: generating synthetic data, splitting it, scaling features, training Support Vector Classifiers and RandomForest models, and evaluating their accuracy, including ensemble predictions. Each operation's inputs and outputs are tracked by Mandala.

```python
@op(output_names=["X", "y"])
def get_data():
    return make_moons(n_samples=1000, noise=0.3, random_state=42)

@op(output_names=["X_train", "X_test", "y_train", "y_test"])
def get_train_test_split(X, y):
    return tuple(train_test_split(X, y, test_size=0.2, random_state=42))

@op(output_names=["X_scaled"])
def scale_data(X):
    scaler = StandardScaler()
    X = scaler.fit_transform(X)
    return X

from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

@op(output_names=["svc_model"])
def train_svc(X_train, y_train, C: float = 1.0, kernel: str = "linear"):
    model = SVC(C=C, kernel=kernel)
    model.fit(X_train, y_train)
    return model

@op(output_names=["rf_model"])
def train_random_forest(X_train, y_train, n_estimators: int = 5, max_depth: int = 5):
    model = RandomForestClassifier(n_estimators=n_estimators, max_depth=max_depth)
    model.fit(X_train, y_train)
    return model

@op(output_names=["accuracy",])
def eval_model(model, X_test, y_test):
    y_pred = model.predict(X_test)
    acc = accuracy_score(y_test, y_pred)
    return acc

@op(output_names=["accuracy"])
def eval_ensemble(models: MList[Any], X_test, y_test):
    y_preds = [model.predict(X_test) for model in models]
    y_pred = np.mean(y_preds, axis=0) > 0.5
    acc = accuracy_score(y_test, y_pred)
    return acc
```

--------------------------------

### Creating and Inspecting a ComputationFrame from a Single Ref in Python

Source: https://github.com/amakelov/mandala/blob/master/docs_source/topics/03_cf.ipynb

Demonstrates the simplest way to initialize a `ComputationFrame` (CF) from a single `Ref` object. It shows how to print a text description, draw a pictorial representation, examine references by variable, and calls by function. It also illustrates how to expand the CF backward to include the full computational history of its values and then inspect the expanded variables and calls.

```Python
cf = storage.cf(test_acc)
pprint(cf) # text description of the CF 
cf.draw(verbose=True) # pictorial representation of the CF
```

```Python
pprint(f'Refs by variable:\n{cf.refs_by_var()}')
pprint(f'Calls by operation:\n{cf.calls_by_func()}')
```

```Python
cf.expand_back(inplace=True, recursive=True)
pprint(cf)
cf.draw(verbose=True)
```

```Python
pprint({vname: storage.unwrap(refs) 
 for vname, refs in cf.refs_by_var().items()
 if vname not in ['X_train', 'X_test', 'y_train', 'y_test'] # to save space
 })

pprint(cf.calls_by_func())
```

--------------------------------

### Visualize Computation Graph with In-Place Updates (Python)

Source: https://github.com/amakelov/mandala/blob/master/docs_source/blog/01_cf.ipynb

This code expands the `ComputationFrame` starting from `get_train_test_split` to visualize the high-level computation graph. It demonstrates how Mandala represents in-place updates (like data scaling) as cycles in this graph, while internally maintaining an acyclic low-level call graph.

```python
cf = storage.cf(get_train_test_split).expand_back(recursive=True)
cf.draw(verbose=True, orientation='TB')
```

--------------------------------

### Create and Visualize a ComputationFrame in Python

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/03_cf.md

Shows how to create a `ComputationFrame` from a dictionary of variables (models, test_accs) and then visualize its structure using `pprint` and `draw` methods.

```python
cf = storage.cf({'model': models, 'test_acc': test_accs})
pprint(cf)
cf.draw(verbose=True)
```

--------------------------------

### Memoized Code as Imperative Storage Interface in Python

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/02_retracing.md

This example demonstrates how a memoized composition of `@op`s effectively acts as an imperative storage interface. By modifying the code to focus on specific results of interest, you can directly access and print the unwrapped values of `Ref` objects, such as accuracy and the trained model, showcasing the flexibility of the memoization system.

```Python
with storage:
    for n_class in (5,):
        X, y = load_data(n_class) 
        for n_estimators in (5,):
            model = train_model(X, y, n_estimators=n_estimators)
            acc = get_acc(model, X, y)
            print(storage.unwrap(acc), storage.unwrap(model))
```

--------------------------------

### Create and Visualize ComputationFrame from Ref (Python)

Source: https://github.com/amakelov/mandala/blob/master/docs/docs/topics/03_cf.md

Initializes a ComputationFrame (CF) from a `Ref` object using `storage.cf()`. It then prints a text description of the CF and generates a pictorial representation using `cf.draw(verbose=True)` to visualize its structure.

```python
cf = storage.cf(test_acc)
pprint(cf) # text description of the CF 
cf.draw(verbose=True) # pictorial representation of the CF
```