Arbitrage Portfolio

https://github.com/yugoge/arbitrage_portfolio
Admin
This project provides tools or strategies for building and managing an arbitrage portfolio.
Tokens:3,407
Snippets:19
Trust Score:2.5
Update:1 year ago
Show doc for...
Tokens:
Raw
### Python Essential Libraries Import

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

Imports core Python libraries required for data manipulation, numerical computations, date and time operations, and advanced linear algebra. This setup is crucial for handling financial data, performing calculations, and implementing complex algorithms within the arbitrage portfolio project.

```python
import pandas as pd
import numpy as np
import datetime, calendar
from scipy.sparse.linalg import eigsh
```

--------------------------------

### Python Project Configuration Parameters

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

Sets up essential parameters for the arbitrage portfolio strategy. These include time horizons for forecasting, investment cycles, the number of factors to consider, the size of the stock universe, investment willingness, margin levels for long and short positions, industry keywords for filtering, and the method for alpha extraction. It also defines a list of financial metrics relevant for stock analysis.

```python
# Setting the basic parameters
T = 12 # Using 12 months to forecast the 13th month
I = 6 # Invetsment cycle
K = 6 # Optimal number of factors provided by the literature
N = 4000 # Using part of the valid stocks
S = 30 # Actual investment willingness
long_margin = 100
short_margin = 110
industry = ['能源', '石油', '发电'] # The key words of the industry selected (e.g. '能源', '石油', '发电'), [] if no industry is selected
method = 2 # Alpha extraction method used in the estimation, 1 for the first method provided by the literature, 2 for the second

financial_metrics_list = [
    "ROE",
    "ROA",
    "ROIC",
    "Gross profit to revenue",
    "Operating profit to revenue",
    "Operating cost to revenue",
    "EBITDA to revenue",
    "NI to revenue",
    "Net operating CF to revenue",
    "CapEx to depreciation",
    "Current asset ratio",
    "Asset liability ratio",
    "Current liability ratio",
    "Tangible asset ratio",
    "EPS basic",
    "EPS diluted",
    "Net asset per share",
    "Net operating CF per share",
    "Revenue per share",
    "Capital reserve per share",
    "Retained earnings per share",
    "FCF per share",
    "Equity multiplier"
]
```

--------------------------------

### Python: Extracting Explained R

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This snippet calculates the explained R (R_hat) using matrix operations and linear algebra. It involves estimating X and solving a system of linear equations. The result is a DataFrame indexed by Ticker and Date.

```python
# Extracting the explained R
R_hat = X_estimation @ np.linalg.solve(X_estimation.T @ X_estimation, X_estimation.T @ R_demean)
R_hat = R_hat.unstack(level=1).droplevel(0,axis=1)
R_hat
```

--------------------------------

### Data Preparation and Estimation with Pandas

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This code snippet prepares return data by unstacking, dropping levels, and selecting industries. It then proceeds with estimation using two different methods, calculating intermediate variables like X_E and F_previous, and deriving theta based on the chosen method.

```python
R_estimation = R_estimation.unstack(level=1).droplevel(0, axis=1).loc[selected_industry]
R_estimation

theta = pd.DataFrame()
Gb_hat = Gb_hat.loc[selected_industry]
X_holding = X_holding.loc[selected_industry]
for i in range(T-I+1):

    # Return in the previous month
    R_previous = R_estimation[[dates[i-T+I-1]]]

    # Characteristics in the previous month
    X_previous = X_estimation[X_estimation.index.get_level_values(1)==dates[i-T+I-1]].droplevel(1,axis=0).loc[selected_industry]
    
    if method == 1:
        # First method - theta
        X_E = X_previous.to_numpy() - Gb_hat @ np.linalg.solve(Gb_hat.T @ Gb_hat, Gb_hat.T @ X_previous)
        X_E.columns = characteristics
        theta[dates[i-T+I-1]] = pd.DataFrame(np.linalg.solve(X_E.T @ X_E, X_E.T @ R_previous), index=characteristics)
    else:
        # Second method - theta
        # F_previous is the coefficient of Gb_hat with respect to R_previous, regardless of the effect of intercept Ga
        F_previous = pd.DataFrame(np.linalg.solve(Gb_hat.T @ Gb_hat, Gb_hat.T @ R_previous), index=Gb_hat.columns, columns=[dates[i-T+I-1]])
        # Final regression to derive the linear estimation of Ga on X, which is denoted as theta
        Y = R_previous - Gb_hat @ F_previous
        theta[dates[i-T+I-1]] = pd.DataFrame(np.linalg.solve(X_previous.T @ X_previous, X_previous.T @ Y), index=characteristics)

# Forcast the effect of characteristics by regressing in the estimation period
time_labels = pd.DataFrame([list(range(T-I+1))], index=['Order'], columns=[dates[-T+I-1:]]).T
holding_time_label = [[T]]
theta = pd.DataFrame(holding_time_label @ np.linalg.solve(time_labels.T @ time_labels, time_labels.T @ theta.T.values), index=[dates[-1]], columns=characteristics).T

if method == 1:
    # First method - G_alpha
    X_E_holding = X_holding.to_numpy() - Gb_hat @ np.linalg.solve(Gb_hat.T @ Gb_hat, Gb_hat.T @ X_holding)
    X_E_holding.columns = characteristics
    Ga_hat = X_E_holding @ theta
else:
    # Second method - G_alpha
    # Weight of my portfolio
    Ga_hat = X_holding @ theta
    
Ga_hat
```

--------------------------------

### Prepare Characteristics Matrix for Arbitrage Portfolio (Python)

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code snippet prepares a characteristics matrix for an arbitrage portfolio. It filters data by date, shifts it to account for time lags, and then separates it into estimation and holding sets based on specified dates. The primary dependencies are the pandas library for data manipulation and pre-existing date and characteristic variables.

```python
X = period[period.index.get_level_values(1).isin(dates[-1-T:])][characteristics]
X_shifted = X.groupby(['Ticker']).shift(I).dropna()
X_estimation = X_shifted[X_shifted.index.get_level_values(1).isin(dates[-1-T+I:])]
X_holding = X_shifted[X_shifted.index.get_level_values(1)==dates[-1]].droplevel(1)
```

--------------------------------

### Python: Eigendecomposition for Factor Analysis

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This code performs eigendecomposition on the stock correlation matrix (RR_N) to identify principal components (factors). It then uses these to construct factor loadings (Gb_hat) and calculates projection coefficients (coef_beta) for the factors.

```python
# Eigendecomposition
RR_N = (R_hat @ R_hat.T) / N # Just like the expression of covariance between N stocks
eigen_values, eigen_vectors = eigsh(RR_N.to_numpy(), k=K, which='LM')
Gb_hat = np.sqrt(N) * pd.DataFrame(eigen_vectors, index=tickers, columns=[f'Factor {x}' for x in range(K, 0, -1)])
Gb_hat

# Calculation of the projection from X to G_beta
coef_beta = np.linalg.solve(X_estimation.T @ X_estimation, X_estimation.droplevel(1).T @ Gb_hat.loc[Gb_hat.index.repeat(T-I+1)])
coef_beta = pd.DataFrame(coef_beta @ eigen_values, index=characteristics, columns=[dates[-1]])
Gb_hat
```

--------------------------------

### Load and Prepare Financial Data with Pandas

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code loads financial data from a CSV file into a pandas DataFrame. It sets a MultiIndex from specified columns, converts all data to numeric types, coercing errors, and then converts the date level of the MultiIndex to datetime objects. This prepares the data for time-series analysis.

```python
panel = pd.read_csv('characteristics-2024-02-01.csv', index_col=[0,1], low_memory=False).apply(pd.to_numeric, errors='coerce')
panel.index = pd.MultiIndex.from_arrays([panel.index.get_level_values(0), pd.to_datetime(panel.index.get_level_values(1))])
panel
```

--------------------------------

### Select Stocks by Average Alpha (Python)

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code uses the pandas library to calculate the mean alpha for each ticker, identify the top N tickers based on this average, and filter the original dataset. It prepares the data for further analysis by selecting stocks with significant alpha.

```python
period['Alpha_Average'] = period['Alpha'].groupby('Ticker').transform('mean')
top_alpha_tickers = period.sort_values(by='Alpha_Average', ascending=False).index.get_level_values(0).unique()[:N]
period = period[period.index.get_level_values(0).isin(top_alpha_tickers)].drop('Alpha_Average', axis=1)
tickers = period.index.get_level_values(0).unique()
```

--------------------------------

### Extracting Column Names and Dates in Python

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code snippet extracts column names for financial characteristics from a panel dataset, excluding the 'Return' column. It also identifies and retrieves unique date values from the panel's index. This is useful for data preprocessing and analysis in financial modeling.

```python
# Column names of the characteristics and time
characteristics = panel.columns.drop('Return')
# Convert date values to Timestamp
dates = panel.index.get_level_values(1).unique()
```

--------------------------------

### Volatility Scaling with NumPy and Pandas

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This snippet calculates the annualized standard deviation of estimated returns and computes a scaling factor to adjust portfolio weights for target volatility. It uses NumPy for standard deviation calculation and pandas for data manipulation.

```python
# Scale towards a target vol
R_alpha = Ga_hat.T @ R_estimation
sd_scale_factor = 1 / (np.sqrt(T) * np.std(R_alpha, axis=1, ddof=1))
sd_scale_factor
```

--------------------------------

### Portfolio Weight Calculation with Pandas

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This code calculates the final portfolio weights (omega) by scaling the estimated factor exposures (Ga_hat) with the calculated volatility scaling factor (sd_scale_factor). The result is a DataFrame representing the weight of each asset in the portfolio.

```python
# Weight of my portfolio
omega = Ga_hat * sd_scale_factor
omega
```

--------------------------------

### Calculate and Sort Absolute Portfolio Weights (Python)

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code snippet calculates the absolute value of the portfolio weights, sorts the industries by these absolute weights in descending order, and then selects the top industries based on a minimum of S or the total number of selected industries. The absolute weight column is then removed.

```python
omega['Abs'] = omega.iloc[:,0].abs()
omega = omega.sort_values(by='Abs',ascending=False).head(min(S, len(selected_industry))).drop('Abs', axis=1)
omega
```

--------------------------------

### Rank Transform Characteristics to [0,1] in Python

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code snippet demonstrates how to rank transform financial characteristics to a [0,1] scale. It uses pandas for data manipulation and numpy for numerical operations. The transformation is applied by grouping data by 'Date' and then using `transform` with a lambda function to calculate the ranks within each group. Dependencies include pandas and numpy.

```python
period[characteristics] = period.groupby(['Date']).transform(lambda x: (1/(len(x)+1))*(np.argsort(np.argsort(np.array(x), kind='mergesort'), kind='mergesort')+1))[characteristics]
```

--------------------------------

### Filter Companies by Industry - Python

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code snippet filters a list of tickers to select companies belonging to a specified industry. It reads industry data from an Excel file and matches company tickers based on industry names. It requires the pandas library and an 'industry_companies.xlsx' file.

```python
import pandas as pd

# Assume 'tickers' is a list of all available company tickers
tickers = [...] # Example: ['ACU.A', 'ZYXI.O', ...]

# Assume 'industry' is a list of desired industries to filter by
industry = [] # Example: ['Technology', 'Healthcare']

# Choose the companies in the a specific industry
if not industry:
    selected_industry = tickers
else:
    industries = pd.read_excel('industry_companies.xlsx', index_col=0)
    selected_industry = [x for x in industries.columns if any(y for y in industry if y in str(industries[x][0])) and x in tickers]
```

--------------------------------

### De-mean Returns (Python)

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

De-means the estimated returns by subtracting the mean return for each ticker across the specified dates. This process helps in analyzing the relative performance of each ticker after accounting for overall market or period trends.

```python
# De-mean returns
R_demean = R_estimation.groupby(level=1).transform(lambda x: x-x.mean(axis=0))
R_demean.unstack(level=1).droplevel(0, axis=1)
```

--------------------------------

### Calculate Rolling Accumulated Returns (Python)

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

Calculates rolling accumulated returns for a given investment period without looking into future returns. It filters data for the specified dates, calculates cumulative product of returns, and drops any resulting NaN values. The estimation period is then selected based on the rolling window size.

```python
# Rolling accumulated returns in the given investment cycle without knowing the future returns
R = period[period.index.get_level_values(1).isin(dates[-T:])][['Return']]
R_rolling = R.groupby(level=0, group_keys=False).apply(lambda x: (x/100+1).rolling(window=I).apply(lambda y: y.prod())-1).dropna()
R_estimation = R_rolling[R_rolling.index.get_level_values(1).isin(dates[-T+I-1:])]
R_estimation.unstack(level=1).droplevel(0, axis=1)
```

--------------------------------

### Fill Missing Data in Financial Panel with Pandas

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python snippet uses the pandas library to fill missing values (N.A.) in a DataFrame. It specifically targets 'characteristics' columns and applies forward-fill (ffill) grouped by the first level of the index (likely tickers). This method is designed to handle delisting circumstances by imputing previous observations, excluding return-related data. Dependencies include the pandas library.

```python
panel_fill = panel.copy()
panel_fill[characteristics] = panel_fill[characteristics].groupby(level=0).transform(lambda group: group.ffill())
```

--------------------------------

### Clean Data by Removing Tickers with Missing Values

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code identifies and removes entire tickers (rows at the first level of a MultiIndex) from a pandas DataFrame if any of their data points are missing (NaN). This is a common data cleaning step to ensure data integrity.

```python
# Data cleaning
na_tickers = period[period.isna().any(axis=1)].index.get_level_values(0).unique()
period = period.drop(na_tickers, axis=0, level=0)
```

--------------------------------

### Filter Time-Series Data by Date Range

Source: https://github.com/yugoge/arbitrage_portfolio/blob/main/Arbitrage Portfolio Investment - Industry (energy) - Average Beta - Adjusted Cycle - Heatmap - 12 - 6 - Current - Feb.ipynb

This Python code filters a pandas DataFrame to keep only the rows within a specified recent date range. It assumes the DataFrame has a MultiIndex where the second level represents dates. NaNs are then filled linearly within the selected period.

```python
# Only keep the panel needed for current month and fill the NaNs linearly
period = panel_fill[panel_fill.index.get_level_values(1).isin(dates[-1-T:])]
```
Tokens:
Raw