Nested Pandas

https://nested-pandas.readthedocs.io/en/latest
Admin
Nested Pandas is an extension of pandas for efficient representation and manipulation of nested...
Tokens:205,928
Snippets:1,805
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### Setup Development Environment

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/contributing.rst

Use these bash commands to create a conda environment, clone the repository, install development dependencies, and run unit tests.

```bash
conda create -n nested_pandas_env python=3.11
conda activate nested_pandas_env

git clone https://github.com/lincc-frameworks/nested-pandas.git
cd nested-pandas/
bash ./.setup_dev.sh

pip install pytest
pytest
```

--------------------------------

### Setup Development Environment and Run Tests

Source: https://nested-pandas.readthedocs.io/en/latest/gettingstarted/contributing.html

Use these commands to create a conda environment, activate it, clone the repository, install development dependencies, and run unit tests for verification.

```bash
conda create -n nested_pandas_env python=3.11
conda activate nested_pandas_env

git clone https://github.com/lincc-frameworks/nested-pandas.git
cd nested-pandas/
bash ./.setup_dev.sh

pip install pytest
pytest

```

--------------------------------

### Import necessary libraries and generate example data

Source: https://nested-pandas.readthedocs.io/en/latest/pre_executed/njit_map_rows.html

Imports the required libraries and generates a sample nested frame for demonstration purposes. Ensure these libraries are installed.

```python
from nested_pandas.datasets import generate_data
import numpy as np
from numba import njit

# example frame
nf = generate_data(10_000, 1000, seed=1)
```

--------------------------------

### Install and Run Pytest

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/installation.rst

Install the pytest framework and run the unit tests to verify the integrity of your local nested-pandas installation.

```bash
pip install pytest
pytest
```

--------------------------------

### Install Development Dependencies

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/installation.rst

Install optional dependencies required for running unit tests and building documentation. Note that the exact command might require single quotes around '.[dev]' depending on your shell.

```bash
pip install .[dev]
```

--------------------------------

### Generate Example Data with Nested-Pandas

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/tutorials/data_manipulation.ipynb

Imports the nested_pandas library and generates an example dataset for demonstration purposes.

```python
import nested_pandas as npd
from nested_pandas.datasets import generate_data

# Begin by generating an example dataset
ndf = generate_data(5, 20, seed=1)
ndf
```

--------------------------------

### Install nested-pandas from Source

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/installation.rst

Clone the repository and install the latest development version of nested-pandas. This is useful for contributing to the project or testing new features.

```bash
git clone https://github.com/lincc-frameworks/nested-pandas.git
cd nested-pandas
pip install .
```

--------------------------------

### Install nested-pandas from source

Source: https://nested-pandas.readthedocs.io/en/latest/gettingstarted/installation.html

Downloads the nested-pandas source code, installs it, and any necessary dependencies within the activated conda environment. Use `pip install .[dev]` to also install development dependencies.

```bash
git clone https://github.com/lincc-frameworks/nested-pandas.git
cd nested-pandas
pip install .

```

```bash
pip install .[dev]  # it may be necessary to use `pip install .'[dev]'` (with single quotes) depending on your machine.

```

--------------------------------

### Generate Toy Dataset with Nested Pandas

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.datasets.generation.generate_data.html

Use generate_data to create a NestedFrame. The first example shows generating a dataset with a single nested layer. The second example demonstrates creating multiple nested columns with custom sizing using a dictionary.

```python
from nested_pandas.datasets import generate_data
nf1 = generate_data(10,100)
```

```python
nf2 = generate_data(10, {"nested_a": 100, "nested_b": 200})
```

--------------------------------

### Benchmarking Setup and Plotting Function

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/pre_executed/njit_map_rows.ipynb

Sets up functions for timing code execution and plotting performance comparisons between Python and Numba JIT.

```python
# benchmarking setup
import timeit
import matplotlib.pyplot as plt

n_nested_list = [100, 200, 300, 500, 750, 1000, 1250, 1500]
n_base_list = [7500, 10_000, 12_500, 15_000]


def bench(func, nf):
    timings = timeit.repeat(
        lambda: func(nf),
        number=10,
        repeat=5,
    )
    timings = np.array(timings)
    return timings.mean(), timings.std()


def plot_bench(py_func, njit_func, title, n_base_list=n_base_list, n_nested_list=n_nested_list):
    plt.figure()
    for n_base in n_base_list:
        speedups = []
        speedup_errs = []

        for n_nested in n_nested_list:
            nf = generate_data(n_base, n_nested, seed=1)  # build frame for each size

            mean_py, std_py = bench(py_func, nf)  # python bench
            mean_njit, std_njit = bench(njit_func, nf)  # njit bench

            speedup = mean_py / mean_njit  # mean speedup ratio


```

--------------------------------

### Install nested-pandas with Pip

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/installation.rst

Use this command to install the latest stable release of nested-pandas from PyPI.

```bash
% pip install nested-pandas
```

--------------------------------

### Install nested-pandas

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/quickstart.ipynb

Install the nested-pandas library and its dependencies using pip. This command should be run in a valid Python environment.

```python
# % pip install nested-pandas
```

--------------------------------

### Install nested-pandas using pip

Source: https://nested-pandas.readthedocs.io/en/latest/gettingstarted/installation.html

Installs the latest release version of nested-pandas from the Python Package Index (PyPI).

```bash
% pip install nested-pandas

```

--------------------------------

### Initialize NestedFrame and join nested DataFrame

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.fillna.html

Demonstrates the initialization of a NestedFrame with sample data and the subsequent joining of another DataFrame as a nested column. This setup is a prerequisite for applying fillna.

```python
import nested_pandas as npd
import pandas as pd
import numpy as np
nf = npd.NestedFrame(
    data={"a": [np.nan, 20, np.nan], "b": [np.nan, np.nan, 30], "c": [10, np.nan, np.nan]},
    index=[0, 1, 2]
)
nested = pd.DataFrame(
    data={"d": [np.nan, np.nan, np.nan], "e": [np.nan, 1, np.nan]},
    index=[0, 1, 2]
)
nf = nf.join_nested(nested, "nested")
```

--------------------------------

### keys

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get the 'info axis' (see Indexing for more).

```APIDOC
## keys

### Description
Get the 'info axis' (see Indexing for more).

### Method
`keys()`
```

--------------------------------

### Set up conda environment for nested-pandas

Source: https://nested-pandas.readthedocs.io/en/latest/gettingstarted/installation.html

Creates and activates a new conda environment for installing nested-pandas from source. Ensures Python 3.11 is used.

```bash
conda create -n nested_pandas_env python=3.11
conda activate nested_pandas_env

```

--------------------------------

### Example: Nesting List Columns

Source: https://nested-pandas.readthedocs.io/en/latest/_modules/nested_pandas/nestedframe/core.html

Illustrates using `nest_lists` to pack specified list-valued columns into a single new nested column within the NestedFrame.

```python
>>> import nested_pandas as npd
>>> nf = npd.NestedFrame({"c":[1,2,3], "d":[2,4,6],
...                   "e":[[1,2,3], [4,5,6], [7,8,9]]},
...                   index=[0,1,2])

>>> nf.nest_lists(columns=["e"], name="nested")
   c  d                nested
0  1  2  [{e: 1}; …] (3 rows)
1  2  4  [{e: 4}; …] (3 rows)
2  3  6  [{e: 7}; …] (3 rows)
```

--------------------------------

### NJIT Explode Graph Example

Source: https://nested-pandas.readthedocs.io/en/latest/pre_executed/njit_map_rows.html

Demonstrates using njit with map_rows for an 'explode' operation. This is typically run once for JIT compilation before benchmarking.

```python
def run_scaled_max_flux_njit_explode(nf):
    nf["nested.a"] = nf["a"]
    nf.map_rows(
        scaled_max_flux_njit_explode,
        columns=["nested.flux", "nested.a"],
        row_container="args",
        output_names="scaled_max_flux",
        njit=True,
    )


run_scaled_max_flux_njit_explode(nf.copy())  # run once for jit compilation before benchmark

plot_bench(
    run_scaled_max_flux_py,
    run_scaled_max_flux_njit_explode,
    title="njit over python execution - scaled_max_flux (explode)",
)
```

--------------------------------

### Example: Creating NestedFrame from Flat DataFrame

Source: https://nested-pandas.readthedocs.io/en/latest/_modules/nested_pandas/nestedframe/core.html

Demonstrates creating a NestedFrame from a flat DataFrame, specifying base columns and allowing the remaining columns to be automatically identified as nested.

```python
>>> import nested_pandas as npd
>>> nf = npd.NestedFrame({"a":[1,1,1,2,2], "b":[2,2,2,4,4],
...                   "c":[1,2,3,4,5], "d":[2,4,6,8,10]},
...                   index=[0,0,0,1,1])

>>> npd.NestedFrame.from_flat(nf, base_columns=["a","b"])
   a  b                      nested
0  1  2  [{c: 1, d: 2}; …] (3 rows)
1  2  4  [{c: 4, d: 8}; …] (2 rows)
```

--------------------------------

### get

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedSeries.html

Retrieves an item from the NestedSeries using a key.

```APIDOC
## get(key[, default])

### Description
Get item from object for given key (ex: DataFrame column).

### Parameters
- **key** - object. The key to retrieve.
- **default** - object, default None. The value to return if the key is not found.
```

--------------------------------

### Create a NestedFrame from a flat DataFrame

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.from_flat.html

This example demonstrates how to initialize a NestedFrame from a flat pandas DataFrame. It shows the creation of the initial DataFrame and then its conversion using `from_flat`, specifying the base columns.

```python
import nested_pandas as npd

nf = npd.NestedFrame({"a":[1,1,1,2,2], "b":[2,2,2,4,4],
                  "c":[1,2,3,4,5], "d":[2,4,6,8,10]},
                  index=[0,0,0,1,1])
```

```python
npd.NestedFrame.from_flat(nf, base_columns=["a","b"])
```

--------------------------------

### Utility Methods

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Methods for viewing and manipulating data, including getting the tail, squeezing dimensions, and setting flags.

```APIDOC
## squeeze

### Description
Squeeze 1 dimensional axis objects into scalars.

### Parameters
- **axis** (int or str, optional) - Select a specific axis to squeeze.

## set_axis

### Description
Assign desired index to given axis.

### Parameters
- **labels** (list or Index) - The new labels.
- **axis** (int or str, optional) - The axis to set.
- **copy** (bool, optional) - Whether to copy the object.

## set_flags

### Description
Return a new object with updated flags.

### Parameters
- **copy** (bool, optional) - Whether to copy the object.
- **allows_duplicate_labels** (bool, optional) - Whether to allow duplicate labels.

## stack

### Description
Stack the prescribed level(s) from columns to index.

### Parameters
- **level** (int or str or list, optional) - Level(s) to stack.
- **dropna** (bool, optional) - Whether to drop NA/null values.
- **sort** (bool, optional) - Sort the resulting index.
- **future_stack** (bool, optional) - Use future stack implementation.

## tail

### Description
Return the last n rows.

### Parameters
- **n** (int, optional) - Number of rows to return.

## take

### Description
Return the elements in the given _positional_ indices along an axis.

### Parameters
- **indices** (int or array-like) - Indices to take.
- **axis** (int or str, optional) - Axis to take from.

```

--------------------------------

### get

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get item from object for given key (ex: DataFrame column).

```APIDOC
## get

### Description
Get item from object for given key (ex: DataFrame column).

### Method
`get(key[, default])`

### Parameters
- **key**: The key to retrieve.
- **default**: Value to return if key is not found.
```

--------------------------------

### Basic njit_map_rows Usage

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/pre_executed/njit_map_rows.ipynb

Demonstrates the fundamental application of njit_map_rows for iterating and processing rows of a NumPy array using Numba's njit decorator. Ensure Numba is installed.

```python
from numba import njit
import numpy as np

@njit
def njit_map_rows(func, arr):
    """Applies a function to each row of a NumPy array using Numba.

    Args:
        func: The function to apply to each row.
        arr: The input NumPy array.

    Returns:
        A new NumPy array with the results of applying the function to each row.
    """
    return np.array([func(row) for row in arr])

@njit
def add_one(row):
    return row + 1

# Example usage:
my_array = np.array([[1, 2], [3, 4]])
result = njit_map_rows(add_one, my_array)
print(result)
```

--------------------------------

### Handling Different Row Lengths (Caution)

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/pre_executed/njit_map_rows.ipynb

This example demonstrates a potential issue when applying njit_map_rows to arrays where rows might have different lengths or the function expects a fixed size. Numba might raise errors or produce unexpected results. It's generally recommended to have uniform row lengths.

```python
import numba
import numpy as np

@numba.njit
def safe_row_op(row):
    # Safely access elements, assuming row might be shorter than expected
    val = 0.0
    if row.shape[0] > 0:
        val += row[0]
    if row.shape[0] > 1:
        val += row[1]
    return val

# Example with potentially ragged data (Numba prefers contiguous arrays)
data = np.array([
    [1, 2, 3],
    [4, 5],
    [6, 7, 8, 9]
], dtype=object) # Using dtype=object to allow raggedness, but this is often not JIT-friendly

# Note: njit_map_rows typically expects a homogeneous NumPy array.
# Applying it directly to a dtype=object array with varying lengths might not work as expected or efficiently.
# For truly ragged arrays, consider alternative approaches or padding.

# Example with a homogeneous array for demonstration:
data_uniform = np.arange(12.0).reshape(4, 3)
result_uniform = numba.njit_map_rows(safe_row_op, data_uniform)
print("Result for uniform data:", result_uniform)
```

--------------------------------

### Import generate_data from nested_pandas.datasets

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.get_subcolumns.html

Import the necessary function to generate sample data for demonstration.

```python
from nested_pandas.datasets import generate_data
```

--------------------------------

### Numba-compiled function for NumPy array processing

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/pre_executed/njit_map_rows.ipynb

This example shows a typical Numba-optimized function that operates on NumPy arrays, which is often the underlying data structure for pandas DataFrames. This is a more direct way to leverage njit for high-performance numerical computations.

```python
from numba import njit
import numpy as np

@njit
def process_numpy_array(arr):
    result = np.empty(arr.shape[0], dtype=np.float64)
    for i in range(arr.shape[0]):
        # Example computation on each row (represented as a 1D slice of the array)
        result[i] = arr[i, 0] * arr[i, 1] + arr[i, 2]
    return result

# Example usage with a NumPy array
data = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
processed_data = process_numpy_array(data)
print(processed_data)

```

--------------------------------

### Generate Sample Data

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestSeriesAccessor.query.html

Generates a sample NestedSeries for demonstration purposes. This is a prerequisite for using the query method.

```python
from nested_pandas.datasets.generation import generate_data
nf = generate_data(5, 5, seed=1)
```

--------------------------------

### Benchmarking njit vs Python - Weighted Mean Slope (Asymptotic)

Source: https://nested-pandas.readthedocs.io/en/latest/pre_executed/njit_map_rows.html

Analyzes the asymptotic behavior of njit versus Python for the weighted_mean_slope function, showing how performance changes with increasing nested column width. Useful for identifying performance bottlenecks.

```python
n_base_shrink = [7500, 10_000]
n_nested_list_asy = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000]

plot_bench(
    run_weighted_mean_slope_py,
    run_weighted_mean_slope_njit,
    title="njit over python execution - weighted_mean_slope (asymptotic behavior)",
    n_base_list=n_base_shrink,
    n_nested_list=n_nested_list_asy,
)
```

--------------------------------

### Generate Sample Nested DataFrame

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.utils.count_nested.html

Sets up pandas display options and generates a sample nested DataFrame for demonstration purposes. This is a prerequisite for using other utility functions.

```python
import pandas as pd
# Show all columns
pd.set_option("display.width", 200)
pd.set_option("display.max_columns", None)
from nested_pandas.datasets.generation import generate_data
nf = generate_data(5, 10, seed=1)
```

--------------------------------

### mode

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get the mode(s) of each element along the selected axis.

```APIDOC
## mode

### Description
Get the mode(s) of each element along the selected axis.

### Parameters
- `axis` ({0 or 'index', 1 or 'columns', None}): Axis to direct the mode calculation.
- `numeric_only` (bool): If True, only consider numeric columns.
- `dropna` (bool): Whether to exclude NA values.

### Example
```python
nested_frame.mode(axis=0, numeric_only=False, dropna=True)
```
```

--------------------------------

### Generate Sample Data

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.describe.html

Generates a sample NestedFrame for demonstration purposes. This is a prerequisite for using the `describe()` method.

```python
from nested_pandas.datasets.generation import generate_data
nf = generate_data(5,5, seed=1)
```

--------------------------------

### Get Base Columns

Source: https://nested-pandas.readthedocs.io/en/latest/_modules/nested_pandas/nestedframe/core.html

Returns a list of column names that are not nested.

```python
@property
def base_columns(self) -> list[str]:
        """Returns the list of base (non-nested) column names"""
        nested_mask = self.dtypes.apply(lambda dtype: not isinstance(dtype, NestedDtype))
        return self.columns[nested_mask].tolist()
```

--------------------------------

### Initialize Arrays for Flat Spectrum Dataframe

Source: https://nested-pandas.readthedocs.io/en/latest/pre_executed/nested_spectra.html

Initializes empty NumPy arrays to hold the wavelength, flux, error, and index data for building a flat spectrum dataframe.

```python
import numpy as np

# Build a flat spectrum dataframe

# Initialize some empty arrays to hold the flat data
wave = np.array([])
flux = np.array([])
err = np.array([])
index = np.array([])
```

--------------------------------

### rsub

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Subtraction of dataframe and other, element-wise (binary operator rsub).

```APIDOC
## rsub

### Description
Get Subtraction of dataframe and other, element-wise (binary operator rsub).

### Parameters
- `other`: The object to subtract with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rsub(5)
```
```

--------------------------------

### Numba JIT vs. Python: Weighted Mean Slope (Asymptotic Behavior)

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/pre_executed/njit_map_rows.ipynb

Compares the performance of a Numba njit-compiled loop against a pure Python implementation for calculating the weighted mean slope. This snippet visualizes the asymptotic behavior as the nested column width increases, showing a slight advantage for njit even at large scales.

```python
n_base_shrink = [7500, 10_000]
n_nested_list_asy = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000]

plot_bench(
    run_weighted_mean_slope_py,
    run_weighted_mean_slope_njit_loop,
    title="njit over python execution - weighted_mean_slope (asymptotic behavior for loop)",
    n_base_list=n_base_shrink,
    n_nested_list=n_nested_list_asy,
)
```

--------------------------------

### rmul

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Multiplication of dataframe and other, element-wise (binary operator rmul).

```APIDOC
## rmul

### Description
Get Multiplication of dataframe and other, element-wise (binary operator rmul).

### Parameters
- `other`: The object to multiply with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rmul(2)
```
```

--------------------------------

### rmod

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Modulo of dataframe and other, element-wise (binary operator rmod).

```APIDOC
## rmod

### Description
Get Modulo of dataframe and other, element-wise (binary operator rmod).

### Parameters
- `other`: The object to take the modulo with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rmod(5)
```
```

--------------------------------

### .nest Accessor

Source: https://nested-pandas.readthedocs.io/en/latest/reference.html

Documentation for the .nest accessor, including its constructor and functions.

```APIDOC
## .nest Accessor

### Constructor
Details on how to use the .nest accessor.

### Functions
List and description of functions available through the .nest accessor.
```

--------------------------------

### radd

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Addition of dataframe and other, element-wise (binary operator radd).

```APIDOC
## radd

### Description
Get Addition of dataframe and other, element-wise (binary operator radd).

### Parameters
- `other`: The object to add with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.radd(5)
```
```

--------------------------------

### ne

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Not equal to of dataframe and other, element-wise (binary operator ne).

```APIDOC
## ne

### Description
Get Not equal to of dataframe and other, element-wise (binary operator ne).

### Parameters
- `other`: The object to compare with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.

### Example
```python
nested_frame.ne(10)
```
```

--------------------------------

### Benchmarking njit vs Python - Max Slope (Asymptotic)

Source: https://nested-pandas.readthedocs.io/en/latest/pre_executed/njit_map_rows.html

Compares the performance of njit and Python execution for the max_slope function as nested column width increases. Useful for understanding the limits of njit optimization.

```python
n_base_shrink = [7500, 10_000]
n_nested_list_asy = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000]

plot_bench(
    run_max_slope_py,
    run_max_slope_njit,
    title="njit over python execution - max_slope (asymptotic behavior)",
    n_base_list=n_base_shrink,
    n_nested_list=n_nested_list_asy,
)
```

--------------------------------

### multiply

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Multiplication of dataframe and other, element-wise (binary operator mul).

```APIDOC
## multiply

### Description
Get Multiplication of dataframe and other, element-wise (binary operator mul).

### Parameters
- `other`: The object to multiply with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.multiply(2)
```
```

--------------------------------

### mul

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Multiplication of dataframe and other, element-wise (binary operator mul).

```APIDOC
## mul

### Description
Get Multiplication of dataframe and other, element-wise (binary operator mul).

### Parameters
- `other`: The object to multiply with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.mul(2)
```
```

--------------------------------

### Generate Sample Data

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestSeriesAccessor.set_filled_column.html

Generates sample data using the `generate_data` function. This is a prerequisite for demonstrating the `set_filled_column` method.

```python
from nested_pandas.datasets.generation import generate_data
nf = generate_data(3, 2, seed=1)
```

--------------------------------

### mod

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Modulo of dataframe and other, element-wise (binary operator mod).

```APIDOC
## mod

### Description
Get Modulo of dataframe and other, element-wise (binary operator mod).

### Parameters
- `other`: The object to take the modulo with.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.mod(5)
```
```

--------------------------------

### Asymptotic Behavior Plotting for Njit Loop

Source: https://nested-pandas.readthedocs.io/en/latest/pre_executed/njit_map_rows.html

Visualizes the performance scaling of an njit-compiled function with manual loops across different data sizes.

```python
n_base_shrink = [7500, 10_000]
nested_list_asy = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000]

plot_bench(
    run_weighted_mean_slope_py,
    run_weighted_mean_slope_njit_loop,
    title="njit over python execution - weighted_mean_slope (asymptotic behavior for loop)",
    n_base_list=n_base_shrink,
    n_nested_list=n_nested_list_asy,
)
```

--------------------------------

### expanding

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Provide expanding window calculations.

```APIDOC
## expanding

### Description
Provide expanding window calculations.

### Method
`expanding(min_periods, axis, , ...)`

### Parameters
- **min_periods**: Minimum number of observations in window required for a value; otherwise, NA.
- **axis**: Axis to compute the expanding window over (0 or 'index', 1 or 'columns').
- **...**: Additional parameters.
```

--------------------------------

### lt

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Less than of dataframe and other, element-wise (binary operator lt).

```APIDOC
## lt

### Description
Get Less than of dataframe and other, element-wise (binary operator lt).

### Method
`lt(other[, axis, level])`

### Parameters
- **other**: The other operand.
- **axis**: Axis to use for the binary operation.
- **level**: Broadcast across level, matching on index values on given level(s).
```

--------------------------------

### gt

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Greater than of dataframe and other, element-wise (binary operator gt).

```APIDOC
## gt

### Description
Get Greater than of dataframe and other, element-wise (binary operator gt).

### Method
`gt(other[, axis, level])`

### Parameters
- **other**: The other operand.
- **axis**: Axis to use for the binary operation.
- **level**: Broadcast across level, matching on index values on given level(s).
```

--------------------------------

### rtruediv

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Floating division of dataframe and other, element-wise (binary operator rtruediv).

```APIDOC
## rtruediv

### Description
Get Floating division of dataframe and other, element-wise (binary operator rtruediv).

### Parameters
- `other`: The object to divide by.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rtruediv(2)
```
```

--------------------------------

### head

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Return the first n rows.

```APIDOC
## head

### Description
Return the first n rows.

### Method
`head([n])`

### Parameters
- **n**: Number of rows to return. Defaults to 5.
```

--------------------------------

### rpow

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Exponential power of dataframe and other, element-wise (binary operator rpow).

```APIDOC
## rpow

### Description
Get Exponential power of dataframe and other, element-wise (binary operator rpow).

### Parameters
- `other`: The object to raise the power to.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rpow(2)
```
```

--------------------------------

### Create and Activate Conda Environment

Source: https://nested-pandas.readthedocs.io/en/latest/_sources/gettingstarted/installation.rst

Set up a dedicated conda environment for nested-pandas development to avoid dependency conflicts.

```bash
conda create -n nested_pandas_env python=3.11
conda activate nested_pandas_env
```

--------------------------------

### NestedSeries.__init__()

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedSeries.html

Initializes a NestedSeries object. This is the constructor for the class.

```APIDOC
## NestedSeries.__init__()

### Description
Initializes a NestedSeries object.

### Method
__init__
```

--------------------------------

### rfloordiv

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Integer division of dataframe and other, element-wise (binary operator rfloordiv).

```APIDOC
## rfloordiv

### Description
Get Integer division of dataframe and other, element-wise (binary operator rfloordiv).

### Parameters
- `other`: The object to divide by.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rfloordiv(2)
```
```

--------------------------------

### rdiv

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Floating division of dataframe and other, element-wise (binary operator rtruediv).

```APIDOC
## rdiv

### Description
Get Floating division of dataframe and other, element-wise (binary operator rtruediv).

### Parameters
- `other`: The object to divide by.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.rdiv(2)
```
```

--------------------------------

### pow

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Exponential power of dataframe and other, element-wise (binary operator pow).

```APIDOC
## pow

### Description
Get Exponential power of dataframe and other, element-wise (binary operator pow).

### Parameters
- `other`: The object to raise the power to.
- `axis` ({0 or 'index', 1 or 'columns', None}): Select axis of DataFrame to broadcast over.
- `level` (int or label): Broadcast across this level of the index.
- `fill_value`: Fill existing NaN values, and use this value to replace all too.

### Example
```python
nested_frame.pow(2)
```
```

--------------------------------

### Generate Sample Data

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestSeriesAccessor.drop.html

Generates sample nested data for demonstration purposes. This is a prerequisite for using the drop method.

```python
from nested_pandas.datasets.generation import generate_data
nf = generate_data(5, 2, seed=1)
```

--------------------------------

### sample

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Return a random sample of items from an axis of object.

```APIDOC
## sample

### Description
Return a random sample of items from an axis of object.

### Parameters
- `n` (int): Number of items to return.
- `frac` (float): Fraction of axis items to return.
- `replace` (bool): Allow sampling of the same row multiple times.
- `weights` (str or ndarray or Series, optional): Default 'None' results in equal probability weighting.
- `random_state` (int, array-like, BitGenerator, np.random.RandomState, np.random.Generator, optional): If int, array-like, or BitGenerator, seed for random number generator.
- `axis` ({0 or 'index', 1 or 'columns', None}): Axis to sample from.
- `ignore_index` (bool, default False): If True, result index will be labeled 0, 1, ..., n - 1.

### Example
```python
nested_frame.sample(n=10, random_state=42)
```
```

--------------------------------

### le

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Less than or equal to of dataframe and other, element-wise (binary operator le).

```APIDOC
## le

### Description
Get Less than or equal to of dataframe and other, element-wise (binary operator le).

### Method
`le(other[, axis, level])`

### Parameters
- **other**: The other operand.
- **axis**: Axis to use for the binary operation.
- **level**: Broadcast across level, matching on index values on given level(s).
```

--------------------------------

### Initialize NestedExtensionArray

Source: https://nested-pandas.readthedocs.io/en/latest/_modules/nested_pandas/series/ext_array.html

Constructs a NestedExtensionArray from pyarrow.Array or pyarrow.ChunkedArray. It handles both list-struct and struct-list input formats, ensuring internal storage is consistent.

```python
def __init__(self, values: pa.Array | pa.ChunkedArray, *, validate: bool = True) -> None:
        if isinstance(values, pa.Array):
            values = pa.chunked_array([values])

        # If list-struct
        if is_pa_type_a_list(values.type):
            list_struct_storage = ListStructStorage(values)
        # If struct-list
        else:
            struct_list_storage = StructListStorage(values, validate=validate)
            list_struct_storage = ListStructStorage.from_struct_list_storage(struct_list_storage)

        self._storage = list_struct_storage
        self._dtype = NestedDtype(self._storage.type)
```

--------------------------------

### ge

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Greater than or equal to of dataframe and other, element-wise (binary operator ge).

```APIDOC
## ge

### Description
Get Greater than or equal to of dataframe and other, element-wise (binary operator ge).

### Method
`ge(other[, axis, level])`

### Parameters
- **other**: The other operand.
- **axis**: Axis to use for the binary operation.
- **level**: Broadcast across level, matching on index values on given level(s).
```

--------------------------------

### floordiv

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.html

Get Integer division of dataframe and other, element-wise (binary operator floordiv).

```APIDOC
## floordiv

### Description
Get Integer division of dataframe and other, element-wise (binary operator floordiv).

### Method
`floordiv(other[, axis, level, fill_value])`

### Parameters
- **other**: The other operand.
- **axis**: Axis to use for the binary operation.
- **level**: Broadcast across level, matching on index values on given level(s).
- **fill_value**: Fill existing NA/null values with the given value.
```

--------------------------------

### head

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedSeries.html

Returns the first n rows of the NestedSeries.

```APIDOC
## head([n])

### Description
Return the first n rows.

### Parameters
- **n** - int, default 5. The number of rows to return.
```

--------------------------------

### Initialize Nested Series

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestSeriesAccessor.set_list_column.html

Generates sample data for use with nested-pandas. This is a prerequisite for demonstrating other methods.

```python
from nested_pandas.datasets.generation import generate_data
nf = generate_data(2, 2, seed=1)
```

--------------------------------

### NestedFrame.all_columns

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedFrame.all_columns.html

Access the all_columns property to get a dictionary of columns for each base and nested dataframe.

```APIDOC
## property NestedFrame.all_columns: dict

### Description
Returns a dictionary of columns for each base/nested dataframe.

### Type
dict
```

--------------------------------

### Generate Sample Data

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestSeriesAccessor.to_flatten_inner.html

Generates sample data for testing nested structures. Imports `NestedFrame` and `generate_data`.

```python
from nested_pandas import NestedFrame
from nested_pandas.datasets import generate_data
nf = generate_data(5, 2, seed=1).rename(columns={"nested": "inner"})
nf["b"] = "b"  # Shorten width of example output
```

--------------------------------

### Get Nested Columns

Source: https://nested-pandas.readthedocs.io/en/latest/_modules/nested_pandas/nestedframe/core.html

Returns a list of base column names that contain nested data structures.

```python
@property
def nested_columns(self) -> list:
        """retrieves the base column names for all nested dataframes"""
        nested_mask = self.dtypes.apply(lambda dtype: isinstance(dtype, NestedDtype))
        return self.columns[nested_mask].tolist()
```

--------------------------------

### compare(other, align_axis=1, keep_shape=False, keep_equal=False, ...) 

Source: https://nested-pandas.readthedocs.io/en/latest/reference/api/nested_pandas.NestedSeries.html

Compares the Series with another Series and highlights the differences.

```APIDOC
## compare(other, align_axis=1, keep_shape=False, keep_equal=False, ...)

### Description
Compare to another Series and show the differences.

### Method
`compare(other[, align_axis, keep_shape, ...])`
```

--------------------------------

### Get Flat Series Field

Source: https://nested-pandas.readthedocs.io/en/latest/_modules/nested_pandas/series/accessor.html

Extracts a specific field from the flattened nested arrays and returns it as a pandas Series.

```APIDOC
## .nest.get_flat_series(field: str)

### Description
Get the flat-array field as a pd.Series.

### Parameters
#### Path Parameters
- **field** (str) - Required - Name of the field to get.

### Returns
- **pd.Series** - The flat-array field.

### Examples
```python
>>> nf["nested"].nest.get_flat_series("flux")
```
```
Tokens:
Raw