### Installation

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/user_guide.md

Install the pandarallel library using pip. The --upgrade and --user flags are optional.

```APIDOC
## Installation

```bash
pip install pandarallel [--upgrade] [--user]
```
```

--------------------------------

### Install Pandaral·lel via pip

Source: https://github.com/nalepae/pandarallel/blob/master/README.md

Command to install the pandarallel package using the Python package manager. This is the standard method for setting up the library in a local environment.

```bash
pip install pandarallel
```

--------------------------------

### Python: Ensure Imports Inside Function for Windows

Source: https://context7.com/nalepae/pandarallel/llms.txt

Demonstrates the correct way to define a function for use with pandarallel on Windows. All imports, such as the 'math' module in this example, must be included within the function definition itself. This is crucial because Windows uses a 'spawn' method for multiprocessing, which requires each process to be self-contained with its own imports.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize()

df = pd.DataFrame({'a': np.random.rand(100000), 'b': np.random.rand(100000)})

# CORRECT on Windows - all imports inside the function
def good_func(row):
    import math
    return math.sin(row['a']) + math.cos(row['b'])

result = df.parallel_apply(good_func, axis=1)
print(result.head())
```

--------------------------------

### Self-contained function requirement for Windows

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/index.md

This example highlights the requirement for self-contained functions when using Pandarallel on Windows. Functions must include their own dependencies (like importing modules) to ensure proper execution in spawned processes. Functions relying on external definitions may fail on Windows.

```python
# ✅ Valid everywhere
def func_self_contained(x):
    import math
    return math.sin(x.a**2) + math.sin(x.b**2)

# ❌ Forbidden On Windows (math is defined outside func)
def func_not_self_contained(x):
    import pandas as pd # Assume pandas is imported globally
    return math.sin(x.a**2) + math.sin(x.b**2)

# To use func_not_self_contained on Windows, you would need to:
# 1. Import math inside the function.
# 2. Or, ensure the function is defined within the scope where math is available when the process is spawned.
```

--------------------------------

### Enabling Jupyter Progress Bars

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/troubleshooting.md

If progress bars appear as raw VBox text in Jupyter, the necessary widget extensions must be installed and enabled. This snippet provides the shell commands to configure the environment.

```bash
pip install ipywidgets
jupyter nbextension enable --py widgetsnbextension
jupyter labextension install @jupyter-widgets/jupyterlab-manager
```

--------------------------------

### Parallelize Pandas groupby apply with Pandarallel

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/index.md

This example shows how to parallelize the `apply` function after a `groupby` operation in Pandas using Pandarallel. It converts `df.groupby(args).apply(func)` to `df.groupby(args).parallel_apply(func)`, enabling faster group-wise computations. Ensure the applied function is self-contained for cross-platform compatibility.

```python
import pandas as pd
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

def sum_col1(group):
    return group['col1'].sum()

df = pd.DataFrame({'group_col': ['A', 'B', 'A', 'B'], 'col1': [1, 2, 3, 4]})

# Standard groupby apply
# result_standard = df.groupby('group_col').apply(sum_col1)

# Parallel groupby apply
result_parallel = df.groupby('group_col').parallel_apply(sum_col1)
print(result_parallel)
```

--------------------------------

### Initialize and Use Pandaral·lel

Source: https://github.com/nalepae/pandarallel/blob/master/README.md

Demonstrates how to initialize the library and replace a standard pandas apply method with its parallel counterpart. The initialization enables progress bars for monitoring execution.

```python
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Replace df.apply(func) with:
df.parallel_apply(func)
```

--------------------------------

### Initialization and Parallel Execution

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_windows.ipynb

Initializes the pandarallel environment and demonstrates how to replace standard Pandas apply methods with parallel versions.

```APIDOC
## Initialization

### Description
Initializes the pandarallel multiprocessing environment.

### Method
N/A

### Endpoint
pandarallel.initialize()

### Parameters
- **nb_workers** (int) - Optional - Number of workers to use.
- **progress_bar** (bool) - Optional - Whether to show a progress bar.

## Parallel Apply

### Description
Executes a function in parallel across DataFrame rows or columns.

### Method
N/A

### Endpoint
df.parallel_apply(func, axis=1)

### Parameters
- **func** (callable) - Required - The function to apply. Must be self-contained (imports inside the function).
- **axis** (int) - Required - 0 for columns, 1 for rows.

### Request Example
```python
def func(x):
    import math
    return math.sin(x.a**2) + math.sin(x.b**2)

res_parallel = df.parallel_apply(func, axis=1)
```
```

--------------------------------

### Initialize Pandarallel

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/user_guide.md

Importing and initializing the pandarallel library. The initialization method allows configuration of worker counts, progress bars, and verbosity levels to optimize parallel processing.

```python
from pandarallel import pandarallel
pandarallel.initialize(nb_workers=4, progress_bar=True, verbose=2)
```

--------------------------------

### Parallel Rolling and Expanding Windows

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_windows.ipynb

Demonstrates parallelizing rolling and expanding window calculations, which are common in time-series or sequential data analysis.

```python
res_parallel = df.groupby('a').b.rolling(4).parallel_apply(func, raw=False)
res_parallel = df.groupby('a').b.expanding(4).parallel_apply(func, raw=False)
```

--------------------------------

### Initialize Pandarallel

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_mac_linux.ipynb

Initializes the pandarallel environment to enable parallel processing for pandas operations.

```python
from pandarallel import pandarallel
pandarallel.initialize()
```

--------------------------------

### Initialization

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/user_guide.md

Initialize pandarallel for parallel processing. This function takes several optional parameters to configure the parallel execution environment.

```APIDOC
## Initialization

### Description
Initialize pandarallel to enable parallel processing. This function configures the number of workers, progress bar display, verbosity level, and memory file system usage.

### Method
```python
pandaraallel.initialize()
```

### Parameters
#### Optional Parameters
- **nb_workers** (int) - Number of workers for parallelization. Defaults to the number of available cores.
- **progress_bar** (bool) - If `True`, displays progress bars. Defaults to `False`.
- **verbose** (int) - Verbosity level for logs (0: none, 1: warnings, 2: all). Defaults to `2`.
- **use_memory_fs** (bool | None) - Determines whether to use memory file system for data transfer. If `None`, it defaults based on availability. If `True`, it enforces memory file system usage. If `False`, it uses multiprocessing data transfer (pipe).

*Note: `shm_size_mb` is deprecated and should not be used.*

### Request Example
```python
from pandarallel import pandarallel

# Initialize with default settings
pandaraallel.initialize()

# Initialize with custom settings
pandaraallel.initialize(nb_workers=4, progress_bar=True, verbose=1, use_memory_fs=True)
```

### Response
This function does not return a value but configures the environment for subsequent parallel operations.
```

--------------------------------

### Parallel Mapping and Grouping

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_windows.ipynb

Demonstrates parallel mapping and grouped operations for efficient data processing.

```APIDOC
## Parallel Map

### Description
Applies a function to every element of a Series in parallel.

### Endpoint
series.parallel_map(func)

## Parallel Groupby Apply

### Description
Applies a function to each group of a DataFrame in parallel.

### Endpoint
df.groupby(by).parallel_apply(func)

### Request Example
```python
# Groupby apply example
res_parallel = df.groupby("a").parallel_apply(func)

# Map example
res_parallel = df.a.parallel_map(func)
```
```

--------------------------------

### Parallelize Rolling and Expanding Windows

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_mac_linux.ipynb

Illustrates the use of parallel_apply for rolling and expanding window calculations on grouped or ungrouped data.

```python
def func(x):
    return x.iloc[0] + x.iloc[1] ** 2 + x.iloc[2] ** 3 + x.iloc[3] ** 4

res = df.groupby('a').b.rolling(4).apply(func, raw=False)
res_parallel = df.groupby('a').b.rolling(4).parallel_apply(func, raw=False)
res.equals(res_parallel)
```

--------------------------------

### Detecting Physical CPU Cores

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/troubleshooting.md

To optimize pandarallel performance, it is recommended to limit workers to the number of physical CPU cores rather than logical cores. This snippet uses the psutil library to retrieve the physical core count.

```python
import psutil
psutil.cpu_count(logical=False)
```

--------------------------------

### Parallelize Series Map and Apply

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_mac_linux.ipynb

Demonstrates parallelizing map and apply operations on individual pandas Series objects.

```python
def func(x, power, bias=0):
    return math.log10(math.sqrt(math.exp(x**power))) + bias

res = df.a.apply(func, args=(2,), bias=3)
res_parallel = df.a.parallel_apply(func, args=(2,), bias=3)
res.equals(res_parallel)
```

--------------------------------

### Initialize Pandarallel

Source: https://context7.com/nalepae/pandarallel/llms.txt

Initialize pandarallel to enable parallel operations. This function must be called before using any parallel methods. It can be configured with options for the number of workers, progress bar display, verbosity, and memory file system usage.

```python
from pandarallel import pandarallel

# Basic initialization with default settings (uses all physical CPU cores)
pandarallel.initialize()

# Enable progress bars to track computation progress
pandarallel.initialize(progress_bar=True)

# Full configuration with all options
pandarallel.initialize(
    nb_workers=4,           # Number of workers (default: number of physical cores)
    progress_bar=True,      # Display progress bars (default: False)
    verbose=2,              # Verbosity: 0=none, 1=warnings, 2=all (default: 2)
    use_memory_fs=None      # Use /dev/shm for data transfer (default: auto-detect)
)
# Output: INFO: Pandarallel will run on 4 workers.
# Output: INFO: Pandarallel will use Memory file system to transfer data...
```

--------------------------------

### Parallel Series Apply

Source: https://context7.com/nalepae/pandarallel/llms.txt

Apply a function to each element of a Series in parallel. This is the parallel equivalent of Series.apply(), distributing the computation across multiple CPU cores for faster execution.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create a large Series
series = pd.Series(np.random.rand(1000000))

# Define a computationally expensive function
def complex_calculation(x):
    import math
    result = 0
    for i in range(100):
        result += math.sin(x * i) * math.cos(x * i)
    return result

# Standard apply
# result = series.apply(complex_calculation)

# Parallel apply - distributes work across all CPU cores
result = series.parallel_apply(complex_calculation)
print(result.head())
# 0    0.123456
# 1    0.234567
# 2    0.345678
# ...
```

--------------------------------

### Parallelize Groupby Apply

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_mac_linux.ipynb

Shows how to parallelize custom function application on grouped DataFrame objects.

```python
def func(df):
    dum = 0
    for item in df.b:
        dum += math.log10(math.sqrt(math.exp(item**2)))
    return dum / len(df.b)

res = df.groupby("a").apply(func)
res_parallel = df.groupby("a").parallel_apply(func)
res.equals(res_parallel)
```

--------------------------------

### Parallel Apply Operation

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_windows.ipynb

Demonstrates how to use parallel_apply to speed up row-wise operations on a DataFrame. The function must be self-contained to ensure compatibility across different operating systems.

```python
def func(x):
    import math
    return math.sin(x.a**2) + math.sin(x.b**2)

res_parallel = df.parallel_apply(func, axis=1)
```

--------------------------------

### Parallelize DataFrame Applymap

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_mac_linux.ipynb

Demonstrates the use of parallel_applymap to perform element-wise operations across a DataFrame.

```python
def func(x):
    return math.sin(x**2) - math.cos(x**2)

res = df.applymap(func)
res_parallel = df.parallel_applymap(func)
res.equals(res_parallel)
```

--------------------------------

### Parallelize Pandas applymap with Pandarallel

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/index.md

This code illustrates how to parallelize the `applymap` function on a Pandas DataFrame using Pandarallel. It transforms `df.applymap(func)` into `df.parallel_applymap(func)`, speeding up element-wise operations across all CPU cores. Functions used with `parallel_applymap` must be self-contained, particularly on Windows.

```python
import pandas as pd
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

def square(x):
    return x**2

df = pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})

# Standard applymap
# result_standard = df.applymap(square)

# Parallel applymap
result_parallel = df.parallel_applymap(square)
print(result_parallel)
```

--------------------------------

### Parallelize Pandas apply with Pandarallel

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/index.md

This snippet demonstrates how to parallelize the `apply` function on a Pandas DataFrame using Pandarallel. It replaces the standard `df.apply(func)` with `df.parallel_apply(func)` for faster execution on multi-core systems. Ensure the function `func` is self-contained, especially on Windows.

```python
import pandas as pd
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

def my_function(row):
    # Perform some computation
    return row['col1'] * row['col2']

df = pd.DataFrame({'col1': [1, 2, 3], 'col2': [4, 5, 6]})

# Standard apply
# result_standard = df.apply(my_function, axis=1)

# Parallel apply
result_parallel = df.parallel_apply(my_function, axis=1)
print(result_parallel)
```

--------------------------------

### Parallel Apply for Pandas Expanding Windows within Groups

Source: https://context7.com/nalepae/pandarallel/llms.txt

Applies a function over expanding windows for each group in a Pandas DataFrame in parallel. This is useful for cumulative calculations within groups, such as a growing Sharpe ratio. It leverages pandarallel for performance gains on large datasets. Pandarallel must be initialized.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create grouped data
df = pd.DataFrame({
    'group': np.repeat(['X', 'Y', 'Z'], 50000),
    'value': np.random.rand(150000)
})

# Define expanding function (cumulative calculation)
def expanding_sharpe_ratio(window):
    import numpy as np
    if len(window) < 2:
        return np.nan
    returns = np.diff(window)
    return np.mean(returns) / (np.std(returns) + 1e-10)

# Parallel groupby expanding apply
result = df.groupby('group')['value'].expanding().parallel_apply(
    expanding_sharpe_ratio, raw=True
)
print(result.tail(10))
```

--------------------------------

### Parallel Apply for Pandas Rolling Windows (Series)

Source: https://context7.com/nalepae/pandarallel/llms.txt

Applies a custom function to a rolling window of a Pandas Series in parallel. This is the parallel equivalent of Series.rolling().apply(), enabling faster computation of window-based statistics on large time series or sequential data. Ensure pandarallel is initialized before use.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create time series data
series = pd.Series(np.random.rand(500000))

# Define custom rolling function
def rolling_zscore(window):
    import numpy as np
    return (window[-1] - np.mean(window)) / np.std(window)

# Parallel rolling apply
result = series.rolling(window=20).parallel_apply(rolling_zscore, raw=True)
print(result.head(25))
```

--------------------------------

### Defining Self-Contained Functions for Windows Multiprocessing

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/troubleshooting.md

When using pandarallel on Windows, functions must be self-contained to avoid serialization issues. This snippet demonstrates the difference between a forbidden approach that relies on global imports and a valid approach where imports are localized within the function scope.

```python
# Forbidden
import math
def func(x):
    return math.sin(x.a**2) + math.sin(x.b**2)

# Valid
def func(x):
    import math
    return math.sin(x.a**2) + math.sin(x.b**2)
```

--------------------------------

### Parallel Apply for Pandas Rolling Windows within Groups

Source: https://context7.com/nalepae/pandarallel/llms.txt

Enables parallel computation of functions over rolling windows applied to groups within a Pandas DataFrame. This combines the power of groupby, rolling operations, and parallel processing for efficient analysis of grouped time-series or sequential data. Requires pandarallel initialization.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create grouped time series data
df = pd.DataFrame({
    'group': np.repeat(['A', 'B', 'C'], 100000),
    'timestamp': np.tile(pd.date_range('2020-01-01', periods=100000), 3),
    'value': np.random.rand(300000)
}).set_index('timestamp')

# Define rolling function for each group
def rolling_max_min_ratio(window):
    import numpy as np
    return np.max(window) / (np.min(window) + 1e-10)

# Parallel groupby rolling apply
result = df.groupby('group')['value'].rolling(window=10).parallel_apply(
    rolling_max_min_ratio, raw=True
)
print(result.head(15))
```

--------------------------------

### Parallel DataFrame Applymap

Source: https://context7.com/nalepae/pandarallel/llms.txt

Apply a function to every element of a DataFrame in parallel. This is the parallel equivalent of DataFrame.applymap(), suitable for element-wise operations that benefit from parallel processing.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

df = pd.DataFrame({
    'x': np.random.rand(500000),
    'y': np.random.rand(500000),
    'z': np.random.rand(500000)
})

# Define element-wise transformation
def expensive_transform(x):
    import math
    return math.exp(math.sin(x * math.pi))

# Standard applymap
# result = df.applymap(expensive_transform)

# Parallel applymap - applies function to each cell in parallel
result = df.parallel_applymap(expensive_transform)
print(result.head())
#           x         y         z
# 0  1.234567  2.345678  1.456789
# 1  2.718281  1.567890  2.234567
# ...
```

--------------------------------

### Parallel Map Operation

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_windows.ipynb

Shows the use of parallel_map for element-wise operations on a Series, providing a significant performance boost for large datasets.

```python
def func(x):
    import math
    return math.log10(math.sqrt(math.exp(x**2)))

res_parallel = df.a.parallel_map(func)
```

--------------------------------

### Parallelize DataFrame Apply

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_mac_linux.ipynb

Compares standard pandas apply with parallel_apply for row-wise operations on a DataFrame.

```python
def func(x):
    return math.sin(x.a**2) + math.sin(x.b**2)

res = df.apply(func, axis=1)
res_parallel = df.parallel_apply(func, axis=1)
res.equals(res_parallel)
```

--------------------------------

### Parallel GroupBy Apply

Source: https://github.com/nalepae/pandarallel/blob/master/docs/examples_windows.ipynb

Applies a function in parallel to groups within a DataFrame, which is useful for complex aggregations on partitioned data.

```python
def func(df):
    import math
    dum = 0
    for item in df.b:
        dum += math.log10(math.sqrt(math.exp(item**2)))
    return dum / len(df.b)

res_parallel = df.groupby("a").parallel_apply(func)
```

--------------------------------

### Parallelize Pandas series map with Pandarallel

Source: https://github.com/nalepae/pandarallel/blob/master/docs/docs/index.md

This snippet demonstrates parallelizing the `map` function on a Pandas Series using Pandarallel. It replaces `series.map(func)` with `series.parallel_map(func)` for efficient execution on multi-core processors. The function passed to `parallel_map` should be self-contained, especially for Windows compatibility.

```python
import pandas as pd
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

def multiply_by_two(x):
    return x * 2

s = pd.Series([1, 2, 3, 4])

# Standard map
# result_standard = s.map(multiply_by_two)

# Parallel map
result_parallel = s.parallel_map(multiply_by_two)
print(result_parallel)
```

--------------------------------

### Parallel Map for Pandas Series

Source: https://context7.com/nalepae/pandarallel/llms.txt

Applies a function or mapping to each element of a Pandas Series in parallel. This is an optimized version of Series.map() for faster execution on large Series, suitable for simple element-wise transformations. It requires the pandarallel library to be initialized.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create a Series with text data
series = pd.Series(['hello world'] * 500000)

# Define mapping function
def process_text(text):
    return text.upper().replace(' ', '_')

# Parallel map
result = series.parallel_map(process_text)
print(result.head())
```

--------------------------------

### Parallel DataFrame Apply

Source: https://context7.com/nalepae/pandarallel/llms.txt

Apply a function to rows or columns of a DataFrame in parallel. This is the parallel equivalent of DataFrame.apply(), offering significant speedups for computationally intensive functions.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create a sample DataFrame
df = pd.DataFrame({
    'a': np.random.randint(0, 100, 1000000),
    'b': np.random.randint(0, 100, 1000000),
    'c': np.random.randint(0, 100, 1000000)
})

# Define a computationally intensive function
def compute_row(row):
    import math
    return math.sin(row['a']**2) + math.cos(row['b']**2) + math.tan(row['c'])

# Standard pandas apply (uses 1 core)
# result = df.apply(compute_row, axis=1)

# Parallel apply (uses all cores) - ~4x faster on 4-core CPU
result = df.parallel_apply(compute_row, axis=1)

# Column-wise parallel apply
def normalize_column(col):
    return (col - col.mean()) / col.std()

normalized = df.parallel_apply(normalize_column, axis=0)
print(normalized.head())
```

--------------------------------

### Parallel Apply for Pandas GroupBy

Source: https://context7.com/nalepae/pandarallel/llms.txt

Performs a parallel apply operation on grouped Pandas DataFrame objects. This method is the parallel counterpart to DataFrameGroupBy.apply() and is highly effective when dealing with a large number of groups, distributing the computation across multiple cores. It requires pandarallel initialization.

```python
import pandas as pd
import numpy as np
from pandarallel import pandarallel

pandarallel.initialize(progress_bar=True)

# Create DataFrame with groups
df = pd.DataFrame({
    'category': np.random.choice(['A', 'B', 'C', 'D', 'E'], 1000000),
    'value1': np.random.rand(1000000),
    'value2': np.random.rand(1000000)
})

# Define function to apply to each group
def group_statistics(group):
    import numpy as np
    return pd.Series({
        'mean_v1': group['value1'].mean(),
        'std_v1': group['value1'].std(),
        'mean_v2': group['value2'].mean(),
        'correlation': group['value1'].corr(group['value2'])
    })

# Parallel groupby apply - each group processed on separate core
result = df.groupby('category').parallel_apply(group_statistics)
print(result)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.