### Load Sample Tick Data from mlfinpy Datasets Source: https://github.com/baobach/mlfinpy/blob/main/docs/UserGuide.rst Loads sample tick data provided by the mlfinpy library using the `load_tick_sample` function. This is useful for testing and demonstration purposes. ```python from mlfinpy.datasets import ( load_tick_sample, load_stock_prices, load_dollar_bar_sample ) # Load sample tick data tick_df = load_tick_sample() ``` -------------------------------- ### Example Usage of Imbalance Bars Functions Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Demonstrates how to use the get_ema_dollar_imbalance_bars and get_const_dollar_imbalance_bars functions from the mlfinpy.data_structure module. ```python from mlfinpy.data_structure import get_ema_dollar_imbalance_bars, get_const_dollar_imbalance_bars # EMA, Const Dollar Imbalance Bars dollar_imbalance_ema = get_ema_dollar_imbalance_bars('FILE_PATH', num_prev_bars=3, exp_num_ticks_init=100000, exp_num_ticks_constraints=[100, 1000], expected_imbalance_window=10000) dollar_imbalance_const = get_const_dollar_imbalance_bars('FILE_PATH', exp_num_ticks_init=100000, expected_imbalance_window=10000) ``` -------------------------------- ### Fractional Differentiation with ADF Test - Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/FractionalDifferentiated.rst Demonstrates how to import price data, compute fractionally differentiated features using the frac_diff_ffd function with a specified d parameter (0.5 in this example), and visualize the minimum d value that achieves stationarity using plot_min_ffd. The function analyzes the trade-off between stationarity and memory retention. ```python import numpy as np import pandas as pd from mlfinpy.util.frac_diff import frac_diff_ffd, plot_min_ffd # Import price data data = pd.read_csv('FILE_PATH') # Deriving the fractionally differentiated features frac_diff_series = frac_diff_ffd(data['close'], 0.5) # Plotting the graph to find the minimum d # Make sure the input dataframe has a 'close' column plot_min_ffd(data) ``` -------------------------------- ### Apply Fix-width Window Fractional Differentiation (FFD) Source: https://github.com/baobach/mlfinpy/blob/main/docs/UserGuide.rst Applies the Fractionally Differentiated Features (FFD) method to a time series to achieve stationarity while preserving memory, crucial for predictive power. Requires a DataFrame with a 'close' column. ```python from mlfinpy.util.frac_diff import frac_diff_ffd, plot_min_ffd # Deriving the fractionally differentiated features dollar_ffd = frac_diff_ffd(dollar.close, 0.5) # Plotting the graph to find the minimum d # Make sure the input dataframe has a 'close' column plot_min_ffd(dollar) ``` -------------------------------- ### Format Tick Data for mlfinpy Source: https://github.com/baobach/mlfinpy/blob/main/docs/UserGuide.rst Formats raw tick data by combining 'Date' and 'Time' columns and selecting 'date', 'price', and 'volume'. This structured data can then be saved to a CSV file or used as a pandas DataFrame input for mlfinpy. ```python # Don't convert to datetime here, it will take forever to convert # on account of the sheer size of tick data files. date_time = data['Date'] + ' ' + data['Time'] new_data = pd.concat([date_time, data['Price'], data['Volume']], axis=1) new_data.columns = ['date', 'price', 'volume'] ``` -------------------------------- ### Label Events with Vertical Barriers and Get Event Properties in MLFinPy Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/2_sample_weights.ipynb Creates vertical time barriers and labels events with profit-taking and stop-loss levels. The get_events function computes event metadata including exit times (t1), target returns, and profit-taking/stop-loss flags. Returns a DataFrame with 1474 labeled events spanning from 2010 to 2019, each with specified minimum return thresholds and vertical barrier constraints. ```python vertical_barriers = mlpy.labeling.add_vertical_barrier(t_events, dollar_bars['close'], num_days=1) events = mlpy.labeling.get_events( close = dollar_bars['close'], t_events = t_events, pt_sl = [1,1], target = daily_vol, min_ret = 0.007, num_threads = 1, vertical_barrier_times = vertical_barriers ) events ``` -------------------------------- ### Save Formatted Data to CSV Source: https://github.com/baobach/mlfinpy/blob/main/docs/UserGuide.rst Saves the pre-processed and newly formatted tick data to a CSV file. This is the recommended method for handling large datasets, as mlfinpy can read directly from disk paths. ```python # Save to csv new_data.to_csv('FILE_PATH', index=False) ``` -------------------------------- ### Generate Dollar Bars using mlfinpy Source: https://github.com/baobach/mlfinpy/blob/main/docs/UserGuide.rst Transforms raw tick data into a structured dollar bar format using the `standard_bars.get_dollar_bars` function. This method aggregates trades based on a cumulative dollar value threshold, helping to normalize return distributions. ```python from mlfinpy.data_structure import standard_bars # Dollar Bars with threshold $50,000 per bar dollar = standard_bars.get_dollar_bars(tick_df, threshold=50_000) ``` -------------------------------- ### Load MLfin.py Financial Datasets (Python) Source: https://github.com/baobach/mlfinpy/blob/main/docs/index.rst Demonstrates how to load sample financial datasets provided by the MLfin.py package. This includes loading tick data, dollar bar data, and stock prices for E-Mini S&P 500 futures and various ETFs. ```python from mlfinpy.datasets import ( load_tick_sample, load_stock_prices, load_dollar_bar_sample ) # Load sample tick data tick_df = load_tick_sample() # Load sample dollar bar data dollar_bars_df = load_dollar_bar_sample() # Load sample stock prices data stock_prices_df = load_stock_prices() ``` -------------------------------- ### Get Constant Tick Imbalance Bars Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Calculates Imbalance Bars using a Constant method for tick data. This function is part of the imbalance_bars module in mlfinpy.data_structure. ```python from mlfinpy.data_structure.imbalance_bars import get_const_tick_imbalance_bars # Example usage (assuming FILE_PATH is defined) data = get_const_tick_imbalance_bars('FILE_PATH', exp_num_ticks_init=100000, expected_imbalance_window=10000) ``` -------------------------------- ### Get Constant Volume Imbalance Bars Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Calculates Imbalance Bars using a Constant method for volume data. This function is part of the imbalance_bars module in mlfinpy.data_structure. ```python from mlfinpy.data_structure.imbalance_bars import get_const_volume_imbalance_bars # Example usage (assuming FILE_PATH is defined) data = get_const_volume_imbalance_bars('FILE_PATH', exp_num_ticks_init=100000, expected_imbalance_window=10000) ``` -------------------------------- ### End-to-End ML Pipeline for Financial Data Source: https://context7.com/baobach/mlfinpy/llms.txt Demonstrates a complete machine learning workflow from raw tick data to a trained classification model. It covers data loading, bar creation, event filtering using CUSUM, and label generation with triple barriers. ```python import pandas as pd import numpy as np from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import classification_report, f1_score from mlfinpy.data_structure.standard_bars import get_dollar_bars from mlfinpy.filters.filters import cusum_filter from mlfinpy.util.volatility import get_daily_vol from mlfinpy.labeling.labeling import add_vertical_barrier, get_events, get_bins # Step 1: Load and prepare data tick_data = pd.read_csv('tick_data.csv', parse_dates=['date_time']) bars = get_dollar_bars(tick_data, threshold=70000000, verbose=True) close_prices = bars.set_index('date_time')['close'] # Step 2: Filter events daily_vol = get_daily_vol(close_prices, lookback=100) events = cusum_filter(close_prices, threshold=daily_vol * 2) # Step 3: Create labels with triple barrier vertical_barriers = add_vertical_barrier(events, close_prices, num_days=5) triple_barrier_events = get_events( close=close_prices, t_events=events, pt_sl=[2, 2], target=daily_vol, min_ret=0.005, num_threads=4, vertical_barrier_times=vertical_barriers ) labels = get_bins(triple_barrier_events, close_prices) ``` -------------------------------- ### Sequential Bootstrapping Fourth Iteration in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Demonstrates the fourth iteration with three previously selected samples. Calculates final probability weights for remaining candidates, showing how the algorithm continues to minimize repeated sample selection. ```python phi = [1, 2, 0] uniqueness_array = np.array([None, None, None]) for i in range(0, 3): ind_mat_reduced = ind_mat[:, phi + [i]] label_uniqueness = get_ind_mat_average_uniqueness(ind_mat_reduced)[-1] uniqueness_array[i] = (label_uniqueness[label_uniqueness > 0].mean()) prob_array = uniqueness_array / sum(uniqueness_array) prob_array >> array([0.32653061224489793, 0.3061224489795918, 0.36734693877551017], dtype=object) ``` -------------------------------- ### Get Constant Dollar Imbalance Bars Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Calculates Imbalance Bars using a Constant method for dollar volume data. This function is part of the imbalance_bars module in mlfinpy.data_structure. ```python from mlfinpy.data_structure.imbalance_bars import get_const_dollar_imbalance_bars # Example usage (assuming FILE_PATH is defined) data = get_const_dollar_imbalance_bars('FILE_PATH', exp_num_ticks_init=100000, expected_imbalance_window=10000) ``` -------------------------------- ### Generate Meta-Labels using Triple Barrier Method (Python) Source: https://github.com/baobach/mlfinpy/blob/main/docs/Labelling.rst This code demonstrates the process of generating meta-labels using the triple-barrier method. It involves computing daily volatility, applying a CUSUM filter, defining vertical barriers, and then using the get_events and get_bins functions to compute the final meta-labels. Dependencies include numpy, pandas, and mlfinpy. ```python import numpy as np import pandas as pd import mlfinpy as ml # Read in data data = pd.read_csv('FILE_PATH') # Compute daily volatility daily_vol = ml.util.get_daily_vol(close=data['close'], lookback=50) # Apply Symmetric CUSUM Filter and get timestamps for events # Note: Only the CUSUM filter needs a point estimate for volatility cusum_events = ml.filters.cusum_filter(data['close'], threshold=daily_vol['2011-09-01':'2018-01-01'].mean()) # Compute vertical barrier vertical_barriers = ml.labeling.add_vertical_barrier(t_events=cusum_events, close=data['close'], num_days=1) pt_sl = [1, 2] min_ret = 0.005 triple_barrier_events = ml.labeling.get_events(close=data['close'], t_events=cusum_events, pt_sl=pt_sl, target=daily_vol, min_ret=min_ret, num_threads=3, vertical_barrier_times=vertical_barriers, side_prediction=data['side']) meta_labels = ml.labeling.get_bins(triple_barrier_events, data['close']) ``` -------------------------------- ### Create Indicator Matrix for Sample Concurrency Analysis Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Generates an indicator matrix used to analyze sample concurrency. Columns represent samples, and rows represent timestamps of price returns used in labeling. This is a prerequisite for sequential bootstrapping. ```python import pandas as pd ind_mat = pd.DataFrame(index = range(0,6), columns=range(0,3)) ind_mat.loc[:, 0] = [1, 1, 1, 0, 0, 0] ind_mat.loc[:, 1] = [0, 0, 1, 1, 0, 0] ind_mat.loc[:, 2] = [0, 0, 0, 0, 1, 1] ``` -------------------------------- ### Get EMA Tick Imbalance Bars Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Calculates Imbalance Bars using an Exponential Moving Average (EMA) method for tick data. This function is part of the imbalance_bars module in mlfinpy.data_structure. ```python from mlfinpy.data_structure.imbalance_bars import get_ema_tick_imbalance_bars # Example usage (assuming FILE_PATH is defined) data = get_ema_tick_imbalance_bars('FILE_PATH', num_prev_bars=3, exp_num_ticks_init=100000, exp_num_ticks_constraints=[100, 1000], expected_imbalance_window=10000) ``` -------------------------------- ### Get EMA Volume Imbalance Bars Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Calculates Imbalance Bars using an Exponential Moving Average (EMA) method for volume data. This function is part of the imbalance_bars module in mlfinpy.data_structure. ```python from mlfinpy.data_structure.imbalance_bars import get_ema_volume_imbalance_bars # Example usage (assuming FILE_PATH is defined) data = get_ema_volume_imbalance_bars('FILE_PATH', num_prev_bars=3, exp_num_ticks_init=100000, exp_num_ticks_constraints=[100, 1000], expected_imbalance_window=10000) ``` -------------------------------- ### Execute Sequential Bootstrap with MLFinPy in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Calls the seq_bootstrap function from MLFinPy with specified sample length, warmup samples, and verbose output. Returns the bootstrapped sample array generated through sequential selection based on indicator matrix uniqueness. ```python samples = seq_bootstrap(ind_mat, sample_length=4, warmup_samples=[1], verbose=True) >> [0.33333333 0.33333333 0.33333333] >> [0.35714286 0.21428571 0.42857143] >> [0.45454545 0.27272727 0.27272727] >> [0.32653061 0.30612245 0.36734694] samples >> [1, 2, 0, 2] ``` -------------------------------- ### Construct Dollar Bars and Calculate Volatility in Python Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/2_sample_weights.ipynb Creates dollar bars with a $50,000 threshold to aggregate tick data into meaningful trading events, removes duplicate index entries, and computes average daily volatility as a return threshold metric. This workflow implements concurrent labeling by generating time-indexed events and standardizing volatility measures for threshold-based triggering of T Events in machine learning label generation. ```python # Create dollar bars with $50,000 threshold dollar_bars = mlpy.data_structure.get_dollar_bars(data, 50_000) dollar_bars.set_index('date_time', inplace=True) # Keep first entry if duplicate index dollar_bars = dollar_bars[~dollar_bars.index.duplicated(keep='first')] # Average daily volatility to trigger T Events daily_vol = mlpy.util.get_daily_vol(dollar_bars['close']) mean_std = daily_vol.mean() ``` -------------------------------- ### Get EMA Dollar Imbalance Bars Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Calculates Imbalance Bars using an Exponential Moving Average (EMA) method for dollar volume data. This function is part of the imbalance_bars module in mlfinpy.data_structure. ```python from mlfinpy.data_structure.imbalance_bars import get_ema_dollar_imbalance_bars # Example usage (assuming FILE_PATH is defined) data = get_ema_dollar_imbalance_bars('FILE_PATH', num_prev_bars=3, exp_num_ticks_init=100000, exp_num_ticks_constraints=[100, 1000], expected_imbalance_window=10000) ``` -------------------------------- ### Generate Trading Signals and Position Summary Source: https://context7.com/baobach/mlfinpy/llms.txt This snippet demonstrates generating trading signals by converting calculated bet sizes into actual positions based on capital and current prices. It then prints a descriptive summary of these final positions, providing insights into the potential scale and distribution of trades. ```python import pandas as pd import numpy as np # Assuming 'bet_sizes', 'capital', and 'close_prices' are defined positions = bet_sizes * capital / close_prices.loc[bet_sizes.index] print(f"\nFinal positions:\n{positions.describe()}") ``` -------------------------------- ### Sequential Bootstrapping Algorithm in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Core pseudocode for the Sequential Bootstrapping algorithm that iteratively builds a sample array by selecting samples based on probability weights derived from average uniqueness of reduced indicator matrices. Each iteration adds a new sample and recalculates probabilities for remaining samples. ```python phi = [] while length(phi) < number of samples to bootstrap: average_uniqueness_array = [] for sample in samples: previous_columns = phi ind_mat_reduced = ind_mat[previous_columns + i] average_uniqueness_array[sample] = get_ind_mat_average_uniqueness(ind_mat_reduced) # Normalise so that probabilities sum up to 1 probability_array = average_uniqueness_array / sum(average_uniqueness_array) chosen_sample = random_choice(samples, probability = probability_array) phi.append(chosen_sample) ``` -------------------------------- ### Sequential Bootstrapping Second Iteration in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Demonstrates the second iteration of sequential bootstrapping where phi contains one previously selected sample. Calculates uniqueness for each candidate sample by creating reduced indicator matrices and computing probability weights. ```python phi = [1] # Sample chosen from the 2st step uniqueness_array = np.array([None, None, None]) for i in range(0, 3): ind_mat_reduced = ind_mat[:, phi + [i]] label_uniqueness = get_ind_mat_average_uniqueness(ind_mat_reduced)[-1] # The last value corresponds to appended i uniqueness_array[i] = (label_uniqueness[label_uniqueness > 0].mean()) prob_array = uniqueness_array / sum(uniqueness_array) prob_array >> array([0.35714285714285715, 0.21428571428571427, 0.42857142857142855], dtype=object) ``` -------------------------------- ### Run Monte-Carlo Experiment for Sequential Bootstrapping in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Performs a Monte-Carlo experiment to compare the average label uniqueness of sequential bootstrapping against standard random choice. It generates multiple samples, calculates uniqueness for each method, and records the results for analysis. Dependencies include numpy and mlfinpy's sequential bootstrapping and uniqueness calculation functions. ```python standard_unq_array = np.zeros(10000) * np.nan # Array of random sampling uniqueness seq_unq_array = np.zeros(10000) * np.nan # Array of Sequential Bootstapping uniqueness for i in range(0, 10000): bootstrapped_samples = seq_bootstrap(ind_mat, sample_length=3) random_samples = np.random.choice(ind_mat.shape[1], size=3) random_unq = get_ind_mat_average_uniqueness(ind_mat[:, random_samples]) random_unq_mean = random_unq[random_unq > 0].mean() sequential_unq = get_ind_mat_average_uniqueness(ind_mat[:, bootstrapped_samples]) sequential_unq_mean = sequential_unq[sequential_unq > 0].mean() standard_unq_array[i] = random_unq_mean seq_unq_array[i] = sequential_unq_mean ``` -------------------------------- ### Load Tick Data using Pandas Source: https://github.com/baobach/mlfinpy/blob/main/docs/UserGuide.rst Loads tick data from a CSV file into a pandas DataFrame. Assumes the CSV contains columns like 'Date', 'Time', 'Price', and 'Volume'. It's recommended not to convert to datetime immediately due to file size. ```python # Required Imports import numpy as np import pandas as pd data = pd.read_csv('data.csv') ``` -------------------------------- ### Import Financial Data Analysis Libraries in Python Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/2_sample_weights.ipynb Imports core libraries for financial data processing and machine learning: pandas for data manipulation, numpy for numerical computing, mlfinpy for financial machine learning utilities, and visualization libraries (plotly, matplotlib, seaborn). These dependencies enable tick data handling, statistical analysis, and financial metric calculations. ```python import pandas as pd import numpy as np import mlfinpy as mlpy from datetime import datetime, timedelta import statsmodels.api as sm # Plotting import plotly.graph_objects as go import matplotlib.pyplot as plt from pylab import rcParams import seaborn as sns ``` -------------------------------- ### Get Indicator Matrix Average Uniqueness in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Calculates the average label uniqueness for an indicator matrix using the get_ind_mat_average_uniqueness function from MLFinPy. This returns uniqueness values that can be filtered and analyzed to understand label overlap patterns in the dataset. ```python ind_mat_uniqueness = get_ind_mat_average_uniqueness(triple_barrier_ind_mat) ``` -------------------------------- ### Import Required Libraries for Financial ML Analysis Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/1_data_labelling.ipynb Imports essential libraries for financial data processing, machine learning, and visualization. Includes pandas/numpy for data manipulation, MLFinPy for financial feature engineering, scikit-learn for ML models, and Plotly for interactive charting. ```python import pandas as pd import numpy as np import mlfinpy as mlpy from datetime import datetime, timedelta import plotly.graph_objects as go # ML models from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score, precision_score, recall_score, RocCurveDisplay, roc_curve, auc ``` -------------------------------- ### Sequential Bootstrapping Third Iteration in Python Source: https://github.com/baobach/mlfinpy/blob/main/docs/Sampling.rst Shows the third iteration with two previously selected samples in phi. Recalculates probability array based on uniqueness of reduced indicator matrices containing the two prior selections plus each candidate sample. ```python phi = [1,2] uniqueness_array = np.array([None, None, None]) for i in range(0, 3): ind_mat_reduced = ind_mat[:, phi + [i]] label_uniqueness = get_ind_mat_average_uniqueness(ind_mat_reduced)[-1] uniqueness_array[i] = (label_uniqueness[label_uniqueness > 0].mean()) prob_array = uniqueness_array / sum(uniqueness_array) prob_array >> array([0.45454545454545453, 0.2727272727272727, 0.2727272727272727], dtype=object) ``` -------------------------------- ### Get Tick Bars from Financial Data Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Creates bars by sampling financial data after a fixed number of ticks have occurred. Returns Open, High, Low, Close data aggregated by tick count. Threshold parameter specifies the number of ticks required to trigger bar sampling. ```python from mlfinpy.data_structure import standard_bars # Tick Bars tick = standard_bars.get_tick_bars('FILE_PATH', threshold=5500, batch_size=1000000, verbose=False) ``` -------------------------------- ### Get Volume Bars from Financial Data Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Creates bars by sampling financial data after a fixed volume (number of contracts) has been traded. Returns Open, High, Low, Close data aggregated by cumulative volume. Threshold parameter specifies the volume amount required to trigger bar sampling. ```python from mlfinpy.data_structure import standard_bars # Volume Bars volume = standard_bars.get_volume_bars('FILE_PATH', threshold=28000, batch_size=1000000, verbose=False) ``` -------------------------------- ### Perform Sequential Bootstrap Sampling Source: https://context7.com/baobach/mlfinpy/llms.txt Implements sequential bootstrapping to generate robust sample indices while respecting label overlap and maximizing sample uniqueness. It involves building an indicator matrix, calculating average uniqueness, and performing the bootstrap. The output indices can be used to create bootstrap samples for model training and evaluation. ```python import numpy as np from mlfinpy.sampling.bootstrapping import get_ind_matrix, seq_bootstrap, get_ind_mat_average_uniqueness # Build indicator matrix showing which bars influence each label indicator_matrix = get_ind_matrix( samples_info_sets=triple_barrier_events['t1'], price_bars=bars ) # Calculate average uniqueness avg_uniqueness = get_ind_mat_average_uniqueness(indicator_matrix) print(f"Average sample uniqueness: {avg_uniqueness:.4f}") # Perform sequential bootstrap np.random.seed(42) bootstrap_indices = seq_bootstrap( ind_mat=indicator_matrix, sample_length=len(labels), compare=True, # Compare with standard bootstrap verbose=False, random_state=np.random.RandomState(42) ) # Output: # Standard uniqueness: 0.4231 # Sequential uniqueness: 0.7856 # Use bootstrap sample X_bootstrap = X.iloc[bootstrap_indices] y_bootstrap = y.iloc[bootstrap_indices] weights_bootstrap = weights.iloc[bootstrap_indices] # Train model on bootstrap sample rf_model.fit(X_bootstrap, y_bootstrap, sample_weight=weights_bootstrap) ``` -------------------------------- ### Load and Prepare Financial Tick Data Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/1_data_labelling.ipynb Loads sample tick data from MLFinPy dataset and converts the index to proper datetime format. The resulting DataFrame contains price and volume columns indexed by timestamp in nanoseconds. ```python # Load sample ticks data = mlpy.dataset.load_tick_sample() # Ensure date_time index in the correct format data.index = pd.to_datetime(data.index, unit='ns') data.head() ``` -------------------------------- ### Get Dollar Bars from Financial Data Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Creates bars by sampling financial data after a fixed monetary amount (price × volume) has been traded. Dollar bars are the most statistically stable of the four standard bar types. Suggested threshold is 1/50 of average daily dollar value for optimal statistical properties. ```python from mlfinpy.data_structure import standard_bars # Dollar Bars dollar = standard_bars.get_dollar_bars('FILE_PATH', threshold=50000, batch_size=1000000, verbose=False) ``` -------------------------------- ### Raw Returns Labeling with mlfinpy Source: https://github.com/baobach/mlfinpy/blob/main/docs/Labelling.rst Generates labels based on raw financial returns (simple or logarithmic). It allows for binary classification (sign of return) or numerical returns, with options for lagging and resampling data periods. Dependencies include pandas for data manipulation. ```python import pandas as pd from mlfinpy.labeling import raw_return # Import price data data = pd.read_csv('../Sample-Data/stock_prices.csv', index_col='Date', parse_dates=True) # Create labels numerically based on simple returns returns = raw_return.raw_returns(prices=data, lag=True) # Create labels categorically based on logarithmic returns returns = raw_return.raw_returns(prices=data, binary=True, logarithmic=True, lag=True) # Create labels categorically on weekly data with forward looking log returns. returns = raw_return.raw_returns(prices=data, binary=True, logarithmic=True, resample_by='W', lag=True) ``` -------------------------------- ### Get Labels from Triple Barriers Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/1_data_labelling.ipynb Extracts trading labels (bin classification) from triple barrier events using the labeling module. Returns a DataFrame containing actual returns (ret), target returns (trgt), and bin labels (-1, 0, or 1) representing sell, neutral, and buy signals respectively. ```python labels = mlpy.labeling.get_bins(tripple_barriers, dollar_bars['close']) labels ``` -------------------------------- ### Calculate Sample Weights using mlfinpy Source: https://context7.com/baobach/mlfinpy/llms.txt This snippet shows how to calculate sample weights for training machine learning models using mlfinpy's `attribution` module. It computes weights based on returns and time decay, combines them, and normalizes them. This is crucial for addressing potential biases in financial data. ```python from mlfinpy.sample_weights.attribution import get_weights_by_return, get_weights_by_time_decay # Assuming 'triple_barrier_events' and 'close_prices' are pre-defined pandas DataFrames # and 'common_idx' is the common index from the previous step return_weights = get_weights_by_return( triple_barrier_events.loc[common_idx], close_prices, num_threads=4 ) time_weights = get_weights_by_time_decay( triple_barrier_events.loc[common_idx], close_prices, decay=0.5, num_threads=4 ) sample_weights = return_weights * time_weights sample_weights = sample_weights / sample_weights.sum() * len(sample_weights) ``` -------------------------------- ### Get Time Bars from Financial Data Source: https://github.com/baobach/mlfinpy/blob/main/docs/FinancialDataStructure.rst Creates time-based bars by sampling financial data at fixed time intervals. Returns Open, High, Low, Close data sampled chronologically. Used for traditional OHLC analysis but has poor statistical properties compared to other sampling techniques since market information doesn't arrive on a fixed schedule. ```python from mlfinpy.data_structure import time_bars # Time bars time = time_bars.get_tick_bars('FILE_PATH', resolution = "MIN", num_units = 1, batch_size=1000000, verbose=False) ``` -------------------------------- ### Apply Fixed Time Horizon Labeling on Stock Prices Source: https://github.com/baobach/mlfinpy/blob/main/docs/Labelling.rst Demonstrates how to use the fixed_time_horizon function to generate labels for financial price data with various configurations. Supports static thresholds, dynamic thresholds as pandas Series, standardization with rolling windows, and resampling. The function requires matching indices between prices and threshold if threshold is a Series. ```python import pandas as pd import numpy as np from mlfinpy.labeling import fixed_time_horizon # Import price data. data = pd.read_csv('../Sample-Data/stock_prices.csv', index_col='Date', parse_dates=True) custom_threshold = pd.Series(np.random.random(len(data)), index=data.index) # Create labels with static threshold. labels = fixed_time_horizon(prices=data, threshold=0.01, lag=True) # Create labels with a dynamic threshold. labels = fixed_time_horizon(prices=data, threshold=custom_threshold, lag=True) # Create labels with standardization. labels = fixed_time_horizon(prices=data, threshold=1, lag=True, standardized=True, window=5) # Create labels after resampling weekly with standardization. labels = fixed_time_horizon(prices=data, threshold=1, resample_by='W', lag=True, standardized=True, window=4) ``` -------------------------------- ### Calculate Concurrent Events Count using MLFinPy Sampling Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/2_sample_weights.ipynb Computes the number of overlapping labeled events at each timestamp using the num_concurrent_events function. Takes the dollar bars index, event exit times, and event start indices as parameters. Returns a Series with concurrent event counts aligned to the price index, useful for analyzing event clustering and overlap patterns. ```python num_conc = mlpy.sampling.num_concurrent_events(dollar_bars.index, events['t1'], events.index) num_conc ``` -------------------------------- ### Generate Sample Weights by Return Attribution Source: https://context7.com/baobach/mlfinpy/llms.txt Calculates sample weights for machine learning models based on absolute returns, adjusted for label concurrency. This method uses triple barrier events and closing prices as input and can be parallelized for performance. The output weights can be directly used in model training frameworks like scikit-learn. ```python from mlfinpy.sample_weights.attribution import get_weights_by_return from sklearn.ensemble import RandomForestClassifier # Calculate sample weights based on returns sample_weights = get_weights_by_return( triple_barrier_events=triple_barrier_events, close_series=close_prices, num_threads=4, verbose=True ) # Use weights in ML model training X = features.loc[labels.index] y = labels['bin'] weights = sample_weights.loc[labels.index] rf_model = RandomForestClassifier(n_estimators=100, random_state=42) rf_model.fit(X, y, sample_weight=weights) print(f"Weighted accuracy: {rf_model.score(X, y, sample_weight=weights):.4f}") ``` -------------------------------- ### Generate Dollar Bars and Detect Market Events Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/1_data_labelling.ipynb Creates dollar bars with $50,000 threshold, removes duplicate timestamps, calculates daily volatility, and identifies significant market events using CUSUM filter. Adds vertical barriers to constrain event labeling window and applies triple barrier method for supervised classification labels. ```python # Create dollar bars with $50,000 threshold dollar_bars = mlpy.data_structure.get_dollar_bars(data, 50_000) dollar_bars.set_index('date_time', inplace=True) # Keep first entry if duplicate index dollar_bars = dollar_bars[~dollar_bars.index.duplicated(keep='first')] # Average daily volatility to trigger T Events mean_std = mlpy.util.get_daily_vol(dollar_bars['close']).mean() # Find timestamp of significant events using CUSUM filter t_events = mlpy.filters.cusum_filter(dollar_bars['close'], mean_std) # Add vertical barriers vertical_barriers = mlpy.labeling.add_vertical_barrier(t_events, dollar_bars['close'], num_days=1) # Apply tripple barriers tripple_barriers = mlpy.labeling.get_events( close = dollar_bars['close'], t_events = t_events, pt_sl = [1,1], target = mlpy.util.get_daily_vol(dollar_bars['close']), min_ret = 0.007, num_threads = 1, vertical_barrier_times = vertical_barriers ) ``` -------------------------------- ### Calculate Price Returns (Python) Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/1_data_labelling.ipynb This function calculates the daily returns from a given set of bars (presumably OHLC data). It extracts the closing prices, computes the percentage difference between consecutive closing prices, and returns the result as a float array. This is a common preprocessing step for financial time series analysis. ```python def get_returns(bars: np.ndarray) -> np.ndarray: close_prices = pd.Series(bars['close'], index=bars.index) return (close_prices.diff() / close_prices)[1:, ].astype(float) ``` -------------------------------- ### Create Scatter Plot of Concurrent Labels vs Daily Volatility Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/2_sample_weights.ipynb Generates a scatter plot comparing the number of concurrent labels against exponential weighted moving (EWM) standard deviation of returns. Uses matplotlib to visualize the relationship between two financial metrics, with set intersection to align indices between datasets. ```python fig, ax = plt.subplots(figsize=(11, 7)) ind = set(num_conc.index).intersection(set(daily_vol.index)) # Convert the set to a list for indexing ind = list(ind) ax.scatter(num_conc[ind], daily_vol[ind], alpha=0.3) ax.set_xlabel('#concurrent labels') ax.set_ylabel('EWM std of returns') plt.show() ``` -------------------------------- ### Derive Bins from Trading Events (Python) Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/1_data_labelling.ipynb This function takes the generated trading events and price data to compute 'bins'. The bins represent discrete categories (e.g., 0, 1) based on the relationship between return and target, along with the 'side' prediction. This step is crucial for preparing data for supervised machine learning. ```python bins_bb = mlpy.labeling.get_bins(events_bb, dollar_bars['close']) bins_bb ``` -------------------------------- ### Create Volume Bars from Tick Data using MLfin.py Source: https://context7.com/baobach/mlfinpy/llms.txt Generates volume bars by sampling data when a fixed volume threshold is reached. This provides a way to segment data based on trading activity. The input is tick data. ```python from mlfinpy.data_structure.standard_bars import get_volume_bars # Create volume bars with threshold of 1000 contracts volume_bars = get_volume_bars( file_path_or_df=tick_data, threshold=1000, batch_size=20000000, verbose=True ) # Use volume bars for further analysis print(f"Generated {len(volume_bars)} volume bars from {len(tick_data)} ticks") ``` -------------------------------- ### Calculate Reserve-Based Bet Sizes with Mixture Models Source: https://context7.com/baobach/mlfinpy/llms.txt Employs reserve-based bet sizing using fitted mixture of Gaussian distributions to model concurrent bets. This advanced method allows for the calculation of bet sizes considering fitted distribution parameters, with options to return these parameters. ```python from mlfinpy.bet_sizing.bet_sizing import bet_size_reserve # Calculate reserve-based bet sizes reserve_bets, fit_params = bet_size_reserve( events_t1=events_t1, sides=bet_sides, fit_runs=100, # Number of distribution fitting runs epsilon=1e-5, # Convergence tolerance factor=5, # Lambda factor variant=2, # Algorithm variant (1 or 2) max_iter=10000, num_workers=-1, # Use all CPU cores return_parameters=True ) print(reserve_bets[['t1', 'side', 'c_t', 'bet_size']].head()) print(f"Fitted mixture parameters: {fit_params}") ``` -------------------------------- ### Create Financial Features with Pandas Source: https://context7.com/baobach/mlfinpy/llms.txt This code snippet demonstrates how to create financial features such as returns and volatility using pandas DataFrames. It calculates percentage changes over different periods and rolling standard deviations. The features are then aligned with existing labels, and rows with missing values are dropped. ```python import pandas as pd # Assuming 'labels', 'close_prices', and 'bars' are pre-defined pandas DataFrames features = pd.DataFrame(index=labels.index) features['returns_5'] = close_prices.pct_change(5) features['returns_20'] = close_prices.pct_change(20) features['volatility'] = close_prices.rolling(20).std() features['volume_ratio'] = bars.set_index('date_time')['volume'] / bars.set_index('date_time')['volume'].rolling(20).mean() features = features.dropna() # Align labels and features common_idx = features.index.intersection(labels.index) X = features.loc[common_idx] y = labels.loc[common_idx, 'bin'] ``` -------------------------------- ### Extract and Display Timestamp Index from Daily Volatility Series Source: https://github.com/baobach/mlfinpy/blob/main/mlfinpy/researches/2_sample_weights.ipynb Converts the index of the daily volatility Series into a Python list of Timestamp objects. Useful for inspecting the temporal granularity and coverage of the volatility calculations, showing specific timestamps from January 2010 onwards at intraday frequency. ```python list(daily_vol.index) ``` -------------------------------- ### Calculate Yang-Zhang Volatility Source: https://context7.com/baobach/mlfinpy/llms.txt Estimates volatility using the Yang-Zhang method, which accounts for both overnight and intraday price movements. It requires open, high, low, and close price data for a specified lookback window. The output is a series of volatility estimates. ```python from mlfinpy.util.volatility import get_yang_zhang_vol yang_zhang_vol = get_yang_zhang_vol( open=bars['open'], high=bars['high'], low=bars['low'], close=bars['close'], window=20 ) # Yang-Zhang provides most comprehensive volatility estimate print(f"Yang-Zhang volatility: {yang_zhang_vol.tail()}") ```