Staircase

https://github.com/staircase-dev/staircase
Admin
Staircase is a Python package built on NumPy and Pandas that enables data analysis through...
Tokens:26,581
Snippets:315
Trust Score:5.3
License:MIT
Update:2 months ago
Show doc for...
Tokens:
Raw
### Setup imports

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/cookbook.rst

Standard imports required for the cookbook examples.

```python
>>> import staircase as sc
>>> import pandas as pd
>>> import numpy as np
```

--------------------------------

### Install dependencies with Poetry

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/environment.rst

Run this command in the project root to install dependencies defined in poetry.lock.

```bash
poetry install
```

--------------------------------

### Install pre-commit hooks

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/environment.rst

Execute this command from within the virtual environment to install the hooks defined in .pre-commit-config.yaml.

```bash
pre-commit install
```

--------------------------------

### Create and configure documentation environment

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/documentation.rst

Commands to set up a virtual environment and install dependencies for building documentation locally.

```bash
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
```

--------------------------------

### Install staircase via package managers

Source: https://github.com/staircase-dev/staircase/blob/master/README.md

Use these commands to install the staircase library from PyPI or conda-forge.

```bash
python -m pip install staircase
```

```bash
conda install -c conda-forge staircase
```

--------------------------------

### Install staircase via PyPI

Source: https://github.com/staircase-dev/staircase/blob/master/docs/getting_started/install.rst

Use this command to install the latest version of staircase from the Python Package Index.

```bash
python -m pip install staircase
```

--------------------------------

### Layering with null start and end parameters

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/layering.rst

Demonstrates how null values in start or end parameters represent negative infinity and infinity respectively.

```python
>>> fig, axes = plt.subplots(nrows=1, ncols=2,  figsize=(7,3), tight_layout=True)
>>> sc.Stairs().layer(start=2).plot(ax=axes[0], arrows=True)
>>> axes[0].set_title("sc.Stairs().layer(start=2)")
>>> sc.Stairs().layer(end=2).plot(ax=axes[1], arrows=True)
>>> axes[1].set_title("sc.Stairs().layer(end=2)")
```

--------------------------------

### Define data generation setup

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Defines the make_data function used to generate synthetic interval data for benchmarking.

```python
setup = """
import staircase_one as sc1
import staircase as sc2
import numpy as np
import pandas as pd
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)

def make_data(use_dates, intervals=4000):
    starts = np.random.randint(-24*60, 24*365*60, intervals)
    ends = starts + np.round(np.random.triangular(2*60, 8*60, 24*60, intervals))
    values = np.random.randint(1, 20, intervals)
    
    df = pd.DataFrame({"start":starts, "end":ends, "value":values}).query("end > 0")
    df.loc[df["start"] < 0, "start"] = np.nan
    df.loc[df["end"] > 365*24*60, "end"] = np.nan

    if use_dates:
        df["start"] = pd.to_datetime(df["start"], unit="m", origin=pd.Timestamp("2021"))
        df["end"] = pd.to_datetime(df["end"], unit="m", origin=pd.Timestamp("2021"))
    
    return df.sort_values('end', na_position='last').sort_values('start', na_position='first')
    
"""
```

--------------------------------

### Layering with scalar and vector parameters

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/layering.rst

Shows that scalar values for start or end are equivalent to single-element lists.

```python
>>> fig, axes = plt.subplots(nrows=1, ncols=3,  figsize=(8,3), sharey=True, sharex=True, tight_layout=True)
>>> sc.Stairs().layer(1, 2).plot(ax=axes[0], arrows=True)
>>> axes[0].set_title("sc.Stairs().layer(1, 2)")
>>> sc.Stairs().layer([1], 2).plot(ax=axes[1], arrows=True)
>>> axes[1].set_title("sc.Stairs().layer([1], 2)")
>>> sc.Stairs().layer(1, [2]).plot(ax=axes[2], arrows=True)
>>> axes[2].set_title("sc.Stairs().layer(1, [2])")
```

--------------------------------

### Install staircase via Anaconda

Source: https://github.com/staircase-dev/staircase/blob/master/docs/getting_started/install.rst

Use this command to install staircase from the conda-forge channel on Anaconda Cloud.

```bash
conda install -c conda-forge staircase
```

--------------------------------

### Inspect Stairs object internals

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/internals.rst

Examples of inspecting various Stairs object configurations using the internals helper function.

```python
@savefig internals_1.png
internals(sc.Stairs(initial_value = -0.5, start=0, end=1, value=1.5))
```

```python
@savefig internals_2.png
internals(sc.Stairs(start=0.5))
```

```python
@savefig internals_3.png
internals(sc.Stairs(end=0.5))
```

```python
@savefig internals_4.png
internals(sc.Stairs(initial_value=1).clip(0,1))
```

```python
@savefig internals_5.png
internals(sc.Stairs(initial_value=1).clip(None,1))
```

```python
@savefig internals_6.png
internals(sc.Stairs(initial_value=1).mask((0,1)))
```

```python
@savefig internals_7.png
internals(sc.Stairs(initial_value=-0.5, start=-0.5, end=1.5).mask((0,1)))
```

--------------------------------

### Count initial queue size

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/queues.rst

Verifies the number of vessels in the queue at the start of the period by counting NaT values in the entry column.

```python
data["enter"].isna().sum()
```

--------------------------------

### Creating a Stairs object

Source: https://github.com/staircase-dev/staircase/blob/master/README.md

Initializes a Stairs object for a specific country using start and end time columns.

```python
>>> import staircase as sc

>>> views_AU = sc.Stairs(data.query("country == 'AU'"), "start", "end")
>>> views_AU
<staircase.Stairs, id=1609972469384>
```

--------------------------------

### Displaying site view data

Source: https://github.com/staircase-dev/staircase/blob/master/README.md

Displays the structure of the site view data containing start times, end times, and country codes.

```python
>>> data
                       start                   end   country
0                        NaT   2021-10-01 00:00:50        AU
1                        NaT   2021-10-01 00:07:45        AU
2                        NaT   2021-10-01 00:05:58        AU
3                        NaT   2021-10-01 00:08:48        AU
4                        NaT   2021-10-01 00:05:26        AU
...                      ...                   ...       ...
425728   2021-10-31 23:57:16                   NaT        US
425729   2021-10-31 23:57:25                   NaT        US
425730   2021-10-31 23:58:59                   NaT        US
425731   2021-10-31 23:59:45                   NaT        US
425732   2021-10-31 23:59:59                   NaT        US
```

--------------------------------

### Build documentation with make.bat

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/documentation.rst

Commands to generate static HTML documentation using the provided make.bat script.

```bash
make.bat html
```

```bash
make.bat clean html
```

--------------------------------

### Run code formatting and linting tools

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/code.rst

Execute Black and Flake8 from the project root to ensure code style compliance.

```bash
black staircase
git diff upstream/master -u -- "*.py" | flake8 --diff
```

--------------------------------

### Initialize Stairs with closed parameter

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/endpoints.rst

Demonstrates creating Stairs instances with left-closed and right-closed intervals and plotting them.

```python
import staircase as sc
import matplotlib.pyplot as plt

sf1 = sc.Stairs(closed="left").layer(0, 1)
sf2 = sc.Stairs(closed="right").layer(0, 1)

fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharex=True, sharey=True)
sf1.plot(ax=axes[0], arrows=True);
axes[0].set_title("sf1 (left-closed)");
sf2.plot(ax=axes[1], arrows=True);
@savefig endpoints.png
axes[1].set_title("sf2 (right-closed)");
```

--------------------------------

### Initialize staircase environment

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/index.rst

Standard imports required for working with staircase and its dependencies.

```python
>>> import staircase as sc
>>> import pandas as pd
>>> import numpy as numpy
>>> import matplotlib.pyplot as plt
```

--------------------------------

### Import data and initialize DataFrame

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/dataleakage.rst

Loads the Melbourne housing dataset and parses the Date column for time-series analysis.

```python
import pandas as pd
import staircase as sc
import matplotlib.pyplot as plt
from matplotlib.dates import MonthLocator, DateFormatter

data = pd.read_csv(house_data, parse_dates=['Date'], dayfirst=True)
data
```

--------------------------------

### Layering with null values

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/gotchas.rst

Demonstrates how null values in start and end columns of a dataframe affect the layering of step functions.

```python
import staircase as sc
import pandas as pd

df = pd.DataFrame({"start":[5, None, None], "end":[None, 2, None]})
```

```python
df
fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharex=True, sharey=True)
sc.Stairs().layer("start", "end", frame=df[:-1]).plot(axes[0], arrows=True);
axes[0].set_title("frame=df[:-1]")
sc.Stairs().layer("start", "end", frame=df).plot(axes[1], arrows=True);
@savefig gotchas_layering.png
axes[1].set_title("frame=df")
```

--------------------------------

### Initialize and Plot Stairs Object

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/slicing.rst

Create a Stairs object from test data and visualize it.

```python
import staircase as sc
import pandas as pd

df = sc.make_test_data(seed=42)
sf = sc.Stairs(df, "start", "end")
@savefig slicing1.png
sf.plot()
```

--------------------------------

### Import dependencies

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Initial imports for numerical and timing operations.

```python
import numpy as np
import timeit
```

--------------------------------

### Plotting with Arrows for Infinite Intervals

Source: https://github.com/staircase-dev/staircase/blob/master/docs/getting_started/intro_tutorials/03_plotting.rst

Use the arrows parameter to indicate the presence of infinite intervals at the start and end of the function.

```python
@savefig intro_tutes_plot_arrows.png
sf.plot(arrows=True);
```

--------------------------------

### Using Staircase Operators

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/versiontwo.rst

Demonstrates the use of standard operators with Staircase objects, highlighting the difference between standard and reflected operator calls.

```python
sf = sc.Stairs().layer([1,4,2], [3,6,5], [1,1,2])
sf + 3  # staircase.Stairs.add
3 + sf  # staircase.Stairs.radd
```

--------------------------------

### Analyze datetime-based step functions

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/distributions.rst

Demonstrates handling step functions with datetime domains and converting Timedelta results to numerical units.

```python
df = sc.make_test_data(dates=True, positive_only=True, seed=42)
sf = sc.Stairs(df, "start", "end")
@savefig distribution_sf.png
sf.plot();
```

```python
sf.value_sums()
```

```python
hist_days = sf.value_sums()/pd.Timedelta(1, "day")
hist_days
@savefig distribution_sf_bar.png
hist_days.plot.bar();
```

--------------------------------

### Calculate time-based rolling average

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/cookbook.rst

Computes a rolling average over a fixed duration window by adding a Timedelta to the event start times.

```python
>>> # test data
>>> rng = np.random.default_rng(seed=0)  # seed random number generator
>>> def gen_values():
...    arr = np.array([])
...    for i in range(10):
...        bound = rng.integers(0,100)
...        bounds = (bound, 100) if bound < 50 else (0, bound)
...        arr = np.append(
...            arr,
...            rng.integers(*bounds, 100)
...        )
...    return arr
...
>>> times = (
...     pd.Timestamp("2021") + 
...     pd.Series(rng.integers(0,365*24, 1000)).apply(pd.Timedelta, unit="h")
... )
...
>>> events = pd.DataFrame(
...     {
...         "time": np.sort(times),
...         "value": gen_values(),
...     }
... )

>>> # recipe
>>> end = events["time"] + pd.Timedelta(28, "D")
>>> rolling_sum_over_time = sc.Stairs(events, start="time", end=end, value="value")
>>> rolling_count_over_time = sc.Stairs(events, start="time", end=end)
>>> rolling_average_over_time = rolling_sum_over_time/rolling_count_over_time
>>> rolling_average_over_time.plot()
```

--------------------------------

### Compare V1 and V2 performance for statistical methods

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Benchmarks comparing ECDF, percentile, and histogram methods between V1 and V2.

```python
stmt1 = "s1A.ecdf_stairs()"
stmt2 = "s2A.ecdf"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.percentile_stairs()"
stmt2 = "s2A.percentile"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.hist(bin_edges=bin_edges)"
stmt2 = "s2A.hist(bins=bins)"

hist_setup = """
low, high = s2A.min(), s2A.max()+0.0001
bins = pd.interval_range(low, high, 10, closed='left')
bin_edges=np.linspace(low, high, 11)
"""

compare(stmt1, stmt2, 1000, 1, create_stairs+hist_setup)
```

--------------------------------

### Create and plot a staircase object with datetime domain

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/dates.rst

Demonstrates initializing a Stairs object with pandas.Timestamp values and plotting the result.

```python
import pandas as pd
import staircase as sc
sf = sc.Stairs().layer(pd.Timestamp("2021"), pd.Timestamp("2022"), 3)
@savefig dates_simple.png
sf.plot(arrows=True)
```

--------------------------------

### Activate Poetry virtual environment

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/environment.rst

Use this command to activate the virtual environment created by Poetry.

```bash
poetry shell
```

--------------------------------

### Modify step function with layers

Source: https://github.com/staircase-dev/staircase/blob/master/docs/getting_started/intro_tutorials/02_creating.rst

Use the layer method to add values to specific intervals. These examples demonstrate sequential updates to the step function.

```python
sf.layer(1,3)
sf.to_frame()
```

```python
sf.layer(4,6)
sf.to_frame()
```

```python
sf.layer(2,5,2)
sf.to_frame()
```

--------------------------------

### Compare V1 and V2 performance for arithmetic and logical operations

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Benchmarks comparing standard arithmetic and logical operators between V1 and V2 implementations.

```python
stmt1 = "s1A - s1B"
stmt2 = "s2A - s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A * s1B"
stmt2 = "s2A * s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A / (s1B+0.0000001)"
stmt2 = "s2A / s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A < s1B"
stmt2 = "s2A < s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A <= s1B"
stmt2 = "s2A <= s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A > s1B"
stmt2 = "s2A > s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A >= s1B"
stmt2 = "s2A >= s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A == s1B"
stmt2 = "s2A == s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A != s1B"
stmt2 = "s2A != s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.make_boolean()"
stmt2 = "s2A.make_boolean()"
compare(stmt1, stmt2, 900, 1, create_stairs)
```

```python
stmt1 = "~s1A"
stmt2 = "~s2A"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A & s1B"
stmt2 = "s2A & s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A | s1B"
stmt2 = "s2A | s2B"
compare(stmt1, stmt2, 900, 1, create_stairs)
```

--------------------------------

### Filling Null Values in Step Functions

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/masking.rst

Demonstrating various fillna strategies including constant values, forward fill, backward fill, and using another step function.

```python
>>> sf = sc.Stairs(start=[2,3,4], end=[5,4,6], value=[1,2,-1])
>>> masker = sc.Stairs().layer(None,3,2).layer(5,6);
>>> fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharex=True, sharey=True)
>>> sf.where(masker).plot(ax=axes[0], arrows=True);
>>> axes[0].set_title("sf.where(masker)");
>>> sf.where(masker).fillna(0).plot(ax=axes[1], arrows=True);
>>> axes[1].set_title("sf.where(masker).fillna(0)")
```

```python
>>> sf = sc.Stairs(start=[2,3,4], end=[5,4,6], value=[1,2,-1])
>>> masker = sc.Stairs().layer(None,3,2).layer(5,6);
>>> fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharex=True, sharey=True)
>>> sf.where(masker).plot(ax=axes[0], arrows=True);
>>> axes[0].set_title("sf.where(masker)");
>>> sf.where(masker).fillna("ffill").plot(ax=axes[1], arrows=True);
>>> axes[1].set_title('sf.where(masker).fillna("ffill")')
```

```python
>>> masker = sc.Stairs().layer(None,3,2).layer(5,6);
>>> fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharex=True, sharey=True)
>>> sf.where(masker).plot(ax=axes[0], arrows=True);
>>> axes[0].set_title("sf.where(masker)");
>>> sf.where(masker).fillna("bfill").plot(ax=axes[1], arrows=True);
>>> axes[1].set_title('sf.where(masker).fillna("bfill")')
```

```python
>>> masker = sc.Stairs().layer(None,3,2).layer(5,6);
>>> fig, axes = plt.subplots(ncols=3, figsize=(9,3), sharex=True, sharey=True)
>>> sf.where(masker).plot(ax=axes[0], arrows=True);
>>> axes[0].set_title("sf.where(masker)");
>>> sf.negate().plot(ax=axes[1], arrows=True);
>>> axes[1].set_title("-sf");
>>> sf.where(masker).fillna(-sf).plot(ax=axes[2], arrows=True);
>>> axes[2].set_title('sf.where(masker).fillna(-sf)')
```

--------------------------------

### Run pre-commit hooks

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/environment.rst

Use this command to manually trigger all pre-commit hooks against all files in the repository.

```bash
pre-commit run --all-files
```

--------------------------------

### Run compatibility tests with Tox

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/code.rst

Use Tox to verify compatibility against specific versions of dependencies like pandas and numpy.

```bash
tox --recreate -e py37-pandas10-numpy114
```

--------------------------------

### Calculate business hours using logical operations

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/cookbook.rst

Demonstrates four equivalent ways to combine a 9-to-5 step function with a weekend mask to isolate business hours.

```python
business_hours_stairs = nine_five_stairs.mask(weekends)

business_hours_stairs = nine_five_stairs.where(~weekends)

business_hours_stairs = nine_five_stairs * ~weekends

business_hours_stairs = nine_five_stairs & ~weekends
```

--------------------------------

### Initialize staircase step functions from intervals

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/intervals.rst

Creates Stairs objects representing intervals [3,6) and [5,7) with left-closed boundaries.

```python
import pandas as pd
import staircase as sc

a = sc.Stairs(start=3, end=6, closed="left")
b = sc.Stairs(start=5, end=7, closed="left")
```

--------------------------------

### Restrict domain using clip

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/masking.rst

Demonstrates restricting a step function's domain to a specific interval using the clip method.

```python
sf = sc.Stairs(start=[2,3,4], end=[5,4,6], value=[1,2,-1])
fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharex=True, sharey=True)
sf.plot(ax=axes[0], arrows=True);
axes[0].set_title("sf");
sf.clip(3,5).plot(ax=axes[1], arrows=True);
@savefig masking_clip.png
axes[1].set_title("sf.clip(3,5)");
```

--------------------------------

### Compare V1 and V2 performance for array-based operations

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Benchmarks for operations performed on arrays of stairs objects.

```python
create_arrays = """
use_dates=True
dfs = [make_data(use_dates) for i in range(10)]
s1array = [sc1.Stairs(use_dates=use_dates).layer(df['start'], df['end'], df['value']) for df in dfs]
s2array = [sc2.Stairs().layer(df['start'], df['end'], df['value']) for df in dfs]
"""
```

```python
stmt1 = "sc1.mean(s1array)"
stmt2 = "sc2.mean(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.median(s1array)"
stmt2 = "sc2.median(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.min(s1array)"
stmt2 = "sc2.min(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.max(s1array)"
stmt2 = "sc2.max(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.sample(s1array, x)"
stmt2 = "sc2.sample(s2array, x)"

sample_setup = """
x = pd.date_range('2021', '2022')
s1array = pd.Series(s1array)
s2array = pd.Series(s2array)
"""

compare(stmt1, stmt2, 100, 1, create_arrays+sample_setup)
```

--------------------------------

### Specify domain bounds with infinity

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/faq.rst

Demonstrates using staircase.inf as a substitute for None when defining domain bounds for clipping.

```python
my_step_function.clip(-sc.inf, 10)
```

```python
my_step_function.clip(None, 10)
```

--------------------------------

### Initialize a StairsArray

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/arraymethods.rst

Create a StairsArray by passing a collection of staircase.Stairs instances to the constructor.

```python
import staircase as sc

df = sc.make_test_data(groups=["a", "b", "c"])
a = sc.Stairs(df.query("group == 'a'"), "start", "end");
b = sc.Stairs(df.query("group == 'b'"), "start", "end");
c = sc.Stairs(df.query("group == 'c'"), "start", "end");

sc.StairsArray([a,b,c])
```

--------------------------------

### Alternative set difference implementations

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/intervals.rst

Shows how to calculate set difference using masking or multiplication.

```python
result = a.mask(b).fillna(0)
```

```python
result = a * ~b
```

--------------------------------

### Sample Step Function Values

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/dataleakage.rst

Retrieves values from a staircase step function at specific time points.

```python
av_price_samples = av_house_prices(sample_times)
av_price_samples
```

--------------------------------

### Perform statistics on clipped functions

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/masking.rst

Shows how to perform statistical calculations on a clipped step function, including an efficient approach for multiple operations.

```python
sf.clip(3,5).mean()
sf.clip(3,5).std()
sf.clip(3,5).min()
```

```python
sf_clip = sf.clip(3,5)
sf_clip.mean()
sf_clip.std()
sf_clip.min()
```

--------------------------------

### Perform set operations on intervals

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/intervals.rst

Demonstrates union, intersection, and set difference using logical operators on Stairs objects.

```python
result = a | b  # result will be a Stairs instance
stairs_to_intervals(result)
```

```python
result = a & b  # result will be a Stairs instance
stairs_to_intervals(result)
```

```python
result = a & ~b  # result will be a Stairs instance
stairs_to_intervals(result)
```

--------------------------------

### Sample step functions at endpoints

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/endpoints.rst

Shows how the closed parameter influences the values returned when sampling at interval boundaries.

```python
sf1([0, 0.5, 1])
sf2([0, 0.5, 1])
```

--------------------------------

### Calculate and plot queue size distribution

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/queues.rst

Generates a probability distribution of queue sizes and plots it as a bar chart.

```python
queue_hist = queue.hist(stat="probability")
queue_hist
```

```python
queue_hist.index = queue_hist.index.left
ax = queue_hist.plot.bar()
ax.set_xlabel("queue size", fontsize="12")
@savefig case_study_queue_hist_bar.png
ax.set_ylabel("probability", fontsize="12")
```

--------------------------------

### Create Stairs objects

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/utilisation.rst

Group data by asset and construct a staircase.Stairs object for each asset.

```python
asset_use = data.groupby("asset")[["start", "end"]].apply(sc.Stairs, start="start", end="end")
asset_use
```

--------------------------------

### Create step functions for house sales

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/dataleakage.rst

Constructs step functions representing the cumulative count of houses sold and the cumulative sum of house prices over time.

```python
houses_sold = sc.Stairs(data, start="Date")
```

```python
sum_houses_prices = sc.Stairs(data, start="Date", value="Price")
```

--------------------------------

### Plot individual asset usage

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/utilisation.rst

Visualize the step function for a specific asset.

```python
ax = asset_use['Z'].plot()
@savefig case_study_asset_z.png
ax.set_xlim('2020-1-1', '2020-1-2');
```

--------------------------------

### Create and plot queue step function

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/queues.rst

Constructs a staircase step function representing the queue size and clips it to the year 2020.

```python
queue = sc.Stairs(frame=data, start="enter", end="leave")
queue = queue.clip(pd.Timestamp("2020"), pd.Timestamp("2021"))
@savefig case_study_queue.png
queue.plot();
```

--------------------------------

### Aggregate to Daily Mean

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/queues.rst

Slices the step function into daily intervals and calculates the mean for each day.

```python
day_range = pd.date_range("2020", freq="D", periods=366)
queue_tonnes.slice(day_range).mean()
```

--------------------------------

### Compare histogram performance

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_floats.ipynb

Benchmarks histogram operations with specific binning configurations.

```python
stmt1 = "s1A.hist(bin_edges=bin_edges)"
stmt2 = "s2A.hist(bins=bins)"

hist_setup = """
low, high = s2A.min(), s2A.max()+0.0001
bins = pd.interval_range(low, high, 10, closed='left')
bin_edges=np.linspace(low, high, 11)
"""

compare(stmt1, stmt2, 1000, 1, create_stairs+hist_setup)
```

--------------------------------

### Instantiate a Stairs object

Source: https://github.com/staircase-dev/staircase/blob/master/docs/getting_started/intro_tutorials/02_creating.rst

Create a new instance of the Stairs class, which defaults to an initial value of zero across the entire domain.

```python
import staircase as sc

sf = sc.Stairs()
```

--------------------------------

### Apply masks to step functions

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/masking.rst

Illustrates creating a masker object and applying it to a step function using the mask method.

```python
masker = sc.Stairs().layer(None,3,2).layer(5,6);
@savefig masking_masker.png
masker.plot(arrows=True)
```

```python
fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharey=True, sharex=True)
sf.plot(ax=axes[0], arrows=True);
axes[0].set_title("sf");
sf.mask(masker).plot(ax=axes[1], arrows=True);
@savefig masking_mask.png
axes[1].set_title("sf.mask(masker)");
```

--------------------------------

### Statistical calculations on unclipped functions

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/gotchas.rst

Illustrates how staircase handles infinite intervals during statistical operations like mean calculations.

```python
sf1 = sc.make_test_data(seed =1).pipe(sc.Stairs, "start", "end")
sf2 = sc.make_test_data(seed =2).pipe(sc.Stairs, "start", "end")

ax = sf1.plot(color="red", label="sf1")
sf2.plot(ax, color="blue", label="sf2");
@savefig gotchas_integral.png
ax.legend()
```

```python
(sf1 > sf2).mean()
```

--------------------------------

### Comparing multiple distributions with ECDFs

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/distributions.rst

Compares step functions and their corresponding ECDFs across multiple groups.

```python
df = sc.make_test_data(dates=False, groups=["A","B","C","D"], seed=60)
series = df.groupby("group")[["start", "end", "value"]].apply(sc.Stairs, "start", "end", "value")
fig, axes = plt.subplots(ncols=2, figsize=(7,3))
for i, stairs in series.items():
    stairs.plot(axes[0], label=i)
axes[0].set_title("step functions");
for i, stairs in series.items():
    stairs.ecdf.plot(axes[1], label=i)
axes[1].legend();
axes[1].set_title("ECDFs");
```

--------------------------------

### Run pytest test suite

Source: https://github.com/staircase-dev/staircase/blob/master/docs/development/guide/code.rst

Execute the full test suite or specific test files using the pytest framework.

```bash
pytest tests
```

```bash
pytest path\to\test.py
```

--------------------------------

### Create and plot queue tonnes step function

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/queues.rst

Models the total cargo tonnage in the queue over time using the 'value' parameter to weight entries and exits.

```python
queue_tonnes = sc.Stairs(frame=data, start="enter", end="leave", value="tonnes")
queue_tonnes = queue_tonnes.clip(pd.Timestamp("2020"), pd.Timestamp("2021"))
@savefig case_study_queue_tonnes.png
queue_tonnes.plot();
```

--------------------------------

### Calculating fractiles and percentiles

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/distributions.rst

Retrieves specific distribution values using fractile and percentile methods.

```python
print(f"The median value of the step function is {sf.fractile(0.5)}")
print(f"The 80th percentile of the step function values is {sf.percentile(80)}")
```

--------------------------------

### Plotting with custom styles and arrows

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/versiontwo.rst

Demonstrates the use of the style and arrows parameters in the Stairs.plot method to customize visualization.

```python
sf = sc.Stairs().layer([1,4,2], [3,6,5], [1,1,2])
fig, axes = plt.subplots(ncols=2, figsize=(7,3), sharey=True, tight_layout=True)
sf.plot(ax=axes[0], arrows=False, style="step");
axes[0].set_title('arrows=False, style="step"');
sf.plot(ax=axes[1], arrows=True, style="hlines");
@savefig version_two_plotting.png
axes[1].set_title('arrows=True, style="hlines"');
```

--------------------------------

### Calculating Integrals with Masking

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/masking.rst

Using where with comparison operators to calculate integrals over specific function domains.

```python
sf.where(sf > 0).integral() + (-sf).where(sf < 0).integral()
```

--------------------------------

### Define comparison utility

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Utility function to measure and display the speedup ratio between two statements.

```python
from IPython.display import display, Markdown

def compare(stmt1, stmt2, repeat, number, extra_setup):
    
    def time(stmt):
        return np.divide(
            timeit.repeat(
                stmt,
                setup=setup+extra_setup,
                repeat=repeat,
                number=number,
            ),
            repeat
        ).min()

    time1 = time(stmt1)
    time2 = time(stmt2)
    ratio = time1/time2
    if ratio < 1:
        speedup = f"{time1/time2:.2f}"
    else:
        speedup = str(float(f"{time1/time2:.2g}")).removesuffix(".0")
    display(Markdown(f"## Speed up ~ {speedup}x"))
```

--------------------------------

### Compare V1 and V2 performance

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_floats.ipynb

Executes performance comparisons for various operations using the compare function.

```python
stmt1 = "s1A - s1B"
stmt2 = "s2A - s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A * s1B"
stmt2 = "s2A * s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A / (s1B+0.0000001)"
stmt2 = "s2A / s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A < s1B"
stmt2 = "s2A < s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A <= s1B"
stmt2 = "s2A <= s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A > s1B"
stmt2 = "s2A > s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A >= s1B"
stmt2 = "s2A >= s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A == s1B"
stmt2 = "s2A == s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A != s1B"
stmt2 = "s2A != s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.make_boolean()"
stmt2 = "s2A.make_boolean()"
compare(stmt1, stmt2, 900, 1, create_stairs)
```

```python
stmt1 = "~s1A"
stmt2 = "~s2A"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A & s1B"
stmt2 = "s2A & s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A | s1B"
stmt2 = "s2A | s2B"
compare(stmt1, stmt2, 900, 1, create_stairs)
```

```python
stmt1 = "s1A.ecdf_stairs()"
stmt2 = "s2A.ecdf"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.percentile_stairs()"
stmt2 = "s2A.percentile"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

--------------------------------

### Plot cumulative sales and price sums

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/dataleakage.rst

Visualizes the step functions for the number of houses sold and the total sum of prices.

```python
fig, axes = plt.subplots(nrows=2, figsize=(10,5), sharex=True)

houses_sold.plot(axes[0]);
axes[0].set_title("Number of houses sold over time");

sum_houses_prices.plot(axes[1]);
@savefig case_study_data_leakage_quotients.png
axes[1].set_title("Sum of houses prices sold over time");
```

--------------------------------

### Import asset data

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/utilisation.rst

Load asset usage data from a CSV file into a pandas DataFrame.

```python
import pandas as pd
import staircase as sc

data = pd.read_csv(asset_data, parse_dates=['start', 'end'])
data
```

--------------------------------

### Benchmark staircase operations

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Executes performance comparisons for various staircase methods.

```python
stmt1 = 'sc1.Stairs(use_dates=True).layer(dfA["start"], dfA["end"], dfA["value"])'
stmt2 = 'sc2.Stairs(dfA, start="start", end="end", value="value")'
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = 'sc1.Stairs(use_dates=True).layer(dfA["start"], dfA["end"], dfA["value"]).layer(dfB["start"], dfB["end"], dfB["value"])'
stmt2 = 'sc2.Stairs(dfA, start="start", end="end", value="value").layer("start", "end", "value", frame = dfB)'
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.max()"
stmt2 = "s2A.max()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.min()"
stmt2 = "s2A.min()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.integrate()"
stmt2 = "s2A.integral()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.mean()"
stmt2 = "s2A.mean()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.median()"
stmt2 = "s2A.median()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.percentile(20)"
stmt2 = "s2A.percentile(20)"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.percentile(0.2*100)"
stmt2 = "s2A.fractile(0.2)"
compare(stmt1, stmt2, 900, 1, create_stairs)
```

```python
stmt1 = "s1A.std()"
stmt2 = "s2A.std()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.var()"
stmt2 = "s2A.var()"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.cov(s1B)"
stmt2 = "s2A.cov(s2B)"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A.corr(s1B)"
stmt2 = "s2A.corr(s2B)"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

```python
stmt1 = "s1A + s1B"
stmt2 = "s2A + s2B"
compare(stmt1, stmt2, 1000, 1, create_stairs)
```

--------------------------------

### Compare array-based operations

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_floats.ipynb

Benchmarks statistical and sampling operations on arrays of staircase objects.

```python
create_arrays = """
use_dates=False
dfs = [make_data(use_dates) for i in range(10)]
s1array = [sc1.Stairs(use_dates=use_dates).layer(df['start'], df['end'], df['value']) for df in dfs]
s2array = [sc2.Stairs().layer(df['start'], df['end'], df['value']) for df in dfs]
"""
```

```python
stmt1 = "sc1.mean(s1array)"
stmt2 = "sc2.mean(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.median(s1array)"
stmt2 = "sc2.median(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.min(s1array)"
stmt2 = "sc2.min(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.max(s1array)"
stmt2 = "sc2.max(s2array)"
compare(stmt1, stmt2, 100, 1, create_arrays)
```

```python
stmt1 = "sc1.sample(s1array, x)"
stmt2 = "sc2.sample(s2array, x)"

sample_setup = """
x = np.linspace(0,24*365,366)
s1array = pd.Series(s1array)
s2array = pd.Series(s2array)
"""

compare(stmt1, stmt2, 100, 1, create_arrays+sample_setup)
```

--------------------------------

### Define staircase object creation

Source: https://github.com/staircase-dev/staircase/blob/master/docs/release_notes/v1_v2_dates.ipynb

Initializes Stairs objects for both v1 and v2 using generated data.

```python
create_stairs = """
use_dates = True

dfA = make_data(use_dates)
dfB = make_data(use_dates)

s1A = sc1.Stairs(use_dates=use_dates).layer(dfA['start'], dfA['end'])
s1B = sc1.Stairs(use_dates=use_dates).layer(dfB['start'], dfB['end'])

s2A = sc2.Stairs().layer(dfA['start'], dfA['end'])
s2B = sc2.Stairs().layer(dfB['start'], dfB['end'])
"""
```

--------------------------------

### Concise Feature Creation

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/dataleakage.rst

Combines step function creation and sampling into a single operation.

```python
sample_times = data["Date"] - pd.Timedelta(0.5, "day")
data["average_price"] = (
    sc.Stairs(data, start="Date", value="Price") /
    sc.Stairs(data, start="Date")
)(sample_times)
```

--------------------------------

### Calculate and plot average house price over time

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/dataleakage.rst

Computes the quotient of the two step functions to derive the average house price over time.

```python
fig, ax = plt.subplots(figsize=(10,3))

av_house_prices = sum_houses_prices/houses_sold
av_house_prices.plot(ax);
@savefig case_study_data_leakage_average.png
ax.set_title("Average houses price over time");
```

--------------------------------

### Inspect Stairs constructor signature

Source: https://github.com/staircase-dev/staircase/blob/master/docs/getting_started/intro_tutorials/02_creating.rst

Use Python's inspect module to view the parameters available for the Stairs class constructor.

```python
import inspect
inspect.signature(sc.Stairs)
```

--------------------------------

### make_test_data

Source: https://github.com/staircase-dev/staircase/blob/master/docs/reference/misc.rst

Generates sample data for testing purposes within the staircase framework.

```APIDOC
## make_test_data

### Description
Generates a set of test data for use with staircase objects.

### Signature
`staircase.make_test_data()`
```

--------------------------------

### Generate and Plot Histogram

Source: https://github.com/staircase-dev/staircase/blob/master/docs/case_studies/queues.rst

Creates a probability histogram for queue tonnes with custom bins and plots it as a bar chart.

```python
queue_tonnes_hist = queue_tonnes.hist(bins = range(0, 2100000, 100000), stat="probability")
queue_tonnes_hist
```

```python
ax = queue_tonnes_hist.plot.bar()
ax.set_xlabel("queue size in tonnes", fontsize="12")
ax.set_ylabel("probability", fontsize="12")
@savefig case_study_queue_tonnes_hist_bar.png
plt.tight_layout()
```

--------------------------------

### Create a 9am to 5pm daily step function

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/cookbook.rst

Creates a boolean step function representing business hours between 9am and 5pm for the year 2021.

```python
nine_am = pd.date_range("2021-1-1 09:00", "2022", inclusive="left")
five_pm = pd.date_range("2021-1-1 17:00", "2022", inclusive="left")
nine_five_stairs = sc.Stairs(start=nine_am, end=five_pm)
```

--------------------------------

### Plotting and calculating ECDF values

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/distributions.rst

Demonstrates plotting an ECDF and calculating cumulative probabilities at specific points.

```python
sf.ecdf.plot()
```

```python
print(f"{sf.ecdf(16) :.0%} of the time sf <= 16")
print(f"{sf.ecdf(20) :.0%} of the time sf <= 20")
print(f"{sf.ecdf(20) - sf.ecdf(16):.0%} of the time 16 < sf <= 20")
```

--------------------------------

### Aggregate Slices with IntervalIndex

Source: https://github.com/staircase-dev/staircase/blob/master/docs/user_guide/slicing.rst

Create a Stairs object from non-date data and perform multiple aggregations on sliced intervals.

```python
df = sc.make_test_data(dates=False, seed=42)
sf = sc.Stairs(df, "start", "end")
@savefig slicing2.png
sf.plot()
```

```python
ii = pd.IntervalIndex.from_breaks(range(0, 101, 5))
sf.slice(ii).agg(["min", "max"])
```
Tokens:
Raw