### Get List of Downloaded Binance Coins

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

This Python code iterates through the 'Coin_Data/Hourly' directory to identify and collect the symbols of all successfully downloaded Binance cryptocurrency data files (ending in '.pkl'). It then prints the total count of downloaded coins and stores them in a DataFrame.

```python
bcoin = []
for filename in os.listdir("../../Coin_Data/Hourly"):
    if '.pkl'in filename and 'error' not in filename : 
        symbol = filename.split('_')[1]
        bcoin.append(symbol)
print("There are {} coins downloaded for Binance".format(len(bcoin)))
bcoin = pd.DataFrame({"coin_name":bcoin})
b_pump_coin = pump_bn.currency.drop_duplicates()
print("There are {} coins in our Binance pumped list".format(b_pump_coin.nunique()))
```

--------------------------------

### Import Project-Specific Utility Modules

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Imports necessary operating system and system path modules, along with custom utility functions from a parent directory. This setup is crucial for accessing shared functionalities and managing project structure.

```python
import os, sys
sys.path.append('../..') #Append the directory above to the path *verify levels*
import utils
```

--------------------------------

### Python Project Setup and Imports

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This snippet configures the Python environment for the project. It imports custom analysis utilities, the RandomForestClassifier from scikit-learn, sets pandas display options for better data visualization, and enables the autoreload extension for development efficiency along with inline plotting for matplotlib.

```python
from analysis_utils import *
from sklearn.ensemble import RandomForestClassifier as RF
pd.set_option('display.max_columns', 999)
%load_ext autoreload
%matplotlib inline
```

--------------------------------

### Display First Rows of DataFrame

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Shows the first few rows of the 'all_pumps' DataFrame. This is a common step to quickly inspect the data after selection and resetting the index.

```python
all_pumps.head()
```

--------------------------------

### Read Coin Info from Pickle File

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Loads the processed cryptocurrency information from a pickle file named 'all_coins_info.pkl'. This file is assumed to contain a pandas DataFrame with coin details.

```python
import pandas as pd
all_coins_info = pd.read_pickle('../../all_coins_info.pkl')
```

--------------------------------

### Import Random Forest Classifier in Python

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Imports the `RandomForestClassifier` class from the `sklearn.ensemble` module. This is a common machine learning algorithm used for classification tasks.

```python
from sklearn.ensemble import RandomForestClassifier as RF
```

--------------------------------

### Display First Few Rows of Training Features in Python

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Displays the first few rows of the `x_train` DataFrame. This is useful for inspecting the structure and content of the training features after data preparation and subsampling.

```python
x_train.head()
```

--------------------------------

### Print Test Confusion Matrix Components

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Prints the extracted true negatives, false positives, false negatives, and true positives from the test set confusion matrix.

```python
print(tn,fp,fn,tp)
```

--------------------------------

### Load and Display Hourly Social Data

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Loads hourly social media data from a pickle file and prints its shape. This is a preliminary step for data analysis and feature selection.

```python
h = pd.read_pickle("social_data_slide_scaled_h.pkl") 
social_feat_h = list(pd.read_pickle("social_features_h.pkl"))
print( "Hourly data",h.shape)
```

--------------------------------

### Inspect DataFrame Columns

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Displays the names of all columns present in the loaded Pandas DataFrame. This is useful for understanding the available data fields.

```python
dt.columns.values
```

--------------------------------

### Load Coin Information from JSON

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Loads cryptocurrency information from a JSON file named 'coinlist_022020.json'. This data is expected to contain details like coin IDs and symbols, likely obtained from an API.

```python
import json
with open("../../coinlist_022020.json","r") as j: 
    coin_info = json.load(j)
```

--------------------------------

### Display Head of Enriched Coin List

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Displays the first few rows of the 'coin_list' DataFrame after it has been merged with CryptoCompare data. This shows the currency, exchange, and the associated CryptoCompare 'Id'.

```python
coin_list.head()
```

--------------------------------

### Prepare Data for Subsampling in Python

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Loads a list of social features from 'social_features_d.pkl'. It then calls the `subsample` function to split and subsample the dataset, using the loaded social features as the input features (`x`) and the 'pumped_yn' column as the target variable (`y`).

```python
social_feat_d = list(pd.read_pickle("social_features_d.pkl"))
x_train, x_test, y_train, y_test = subsample(d[social_feat_d], d['pumped_yn'], num=2000)
```

--------------------------------

### Import Machine Learning Preprocessing and Model Selection Libraries (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Imports key modules from scikit-learn for data preprocessing (MinMaxScaler, StandardScaler), model selection (train_test_split, GridSearchCV), linear models (LogisticRegression), and performance metrics (confusion_matrix). These are crucial for building and evaluating machine learning models.

```python
from sklearn.preprocessing import MinMaxScaler, StandardScaler
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.linear_model import LogisticRegression as LR
from sklearn.metrics import confusion_matrix
```

--------------------------------

### Load Historical Data from JSON

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Loads historical cryptocurrency pump data from a JSON file into a Pandas DataFrame. Assumes the data is stored in 'allPumps_20200201.json'.

```python
dt = pd.read_json('../../allPumps_20200201.json')
```

--------------------------------

### Import Core Python Libraries for Data Analysis

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Imports essential Python libraries for data manipulation, API requests, time management, plotting, and visualization. These libraries form the foundation for data processing and analysis within the project.

```python
import pandas as pd 
import numpy as np
import json 
import requests 
import time  as sleeper
from datetime import datetime, date, time, timedelta
import pytz
import matplotlib.pyplot as plt
import matplotlib.dates as mdate
from mpl_finance import candlestick_ohlc
import seaborn as sns
import plotly.graph_objects as go
import plotly.express as px
import matplotlib.ticker as mticker
%load_ext autoreload
```

--------------------------------

### Extract Unique Currency-Exchange Pairs

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Creates a DataFrame containing unique combinations of currency and exchange from the 'all_pumps' data. This is useful for identifying which coins have been pumped on which exchanges.

```python
coin_list = all_pumps[['currency','exchange']].drop_duplicates()
coin_list.head(2)
```

--------------------------------

### Compare Downloaded and Pumped Binance Coins

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

This Python code performs two comparisons: first, it checks for coins that were downloaded but not found in the Binance pump list, and second, it identifies coins that were pumped on Binance but not successfully downloaded. This helps in understanding data discrepancies.

```python
#Test: do they overlap
print("There are {} coins downloaded but not pumped".format(len(list(set(bcoin) - set(b_pump_coin))) - 1))
# Thes are usually tokens, or forks of existing coins, not full cryptocurrencies

#Test: do they overlap

print("There are {} coins pumped but not downloable".format(len(set(b_pump_coin) - set(bcoin) )))
# set(b_pump_coin) - set(bcoin)
```

--------------------------------

### Save Enriched Coin List to Pickle

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Saves the 'coin_list' DataFrame, now enriched with CryptoCompare IDs, to a pickle file named 'coin_list.pkl' within the 'Coin_Data' directory. This file will be used for subsequent data downloads.

```python
coin_list.to_pickle('../../Coin_Data/coin_list.pkl')
```

--------------------------------

### Count Pumps by Exchange

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Groups the 'all_pumps' DataFrame by exchange and counts the number of pumps for each exchange. This provides a summary of pump activity per trading platform.

```python
all_pumps.groupby('exchange').exchange.count()
```

--------------------------------

### Save Coin Info DataFrame to Pickle

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Converts the 'Data' section of the loaded coin information (from JSON) into a pandas DataFrame and saves it to a pickle file. This allows for efficient loading of this processed data later.

```python
import pandas as pd
pd.DataFrame(coin_info['Data'].values()).to_pickle("../../all_coins_info.pkl")
```

--------------------------------

### Initialize Random Forest Classifier and Define Parameters (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Initializes a Random Forest classifier object and defines a dictionary of hyperparameters to be tuned. The parameters include the number of estimators, maximum features, maximum depth, and the criterion for splitting nodes.

```python
rf = RF()
params_rf = {'n_estimators':[50,100,200,500]
            ,'max_features': ['auto', 'sqrt', 'log2']
            ,'max_depth' : [10,15,25,50,75]
            ,'criterion' :['gini', 'entropy']}
```

--------------------------------

### Example Usage of getSocialDelta

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_2a_Format_Social_Data.ipynb

This code snippet demonstrates how to use the `getSocialDelta` function. It defines a list of columns to drop and then calls the function with a sample DataFrame `krl` and a delta of 1.

```python
drop_cols = ['date','date_utc']
krl_d1 = getSocialDelta(krl,1,drop_cols)
```

--------------------------------

### Import Libraries and Setup for Crypto Analysis (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5c_OHLCV_Limited.ipynb

Imports necessary libraries such as os, sys, and scikit-learn's RandomForestClassifier. It also sets pandas display options and loads the autoreload extension for interactive development.

```python
import os, sys
sys.path.insert(1,"../")
from analysis_utils import *

from sklearn.ensemble import RandomForestClassifier as RF

pd.set_option('display.max_columns', 999)
%load_ext autoreload
%matplotlib inline
```

--------------------------------

### Plot Bar Chart of Pumps by Weekday

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Generates a bar chart displaying the number of cryptocurrency pumps that occurred on each day of the week. This helps identify any weekly patterns in pump activity.

```python
all_pumps.groupby('pump_weekday').pump_weekday.count().plot(kind='bar')
```

--------------------------------

### Select and Reset DataFrame Columns

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Selects specific columns ('channelLink', 'channelTitle', 'currency', 'duration', 'exchange', 'priceBeforePump', 'signalTime') from the DataFrame and resets the index. This prepares the data for further analysis by focusing on relevant fields.

```python
all_pumps = dt[['channelLink','channelTitle','currency','duration','exchange','priceBeforePump','signalTime']]
all_pumps = all_pumps.reset_index(drop=True)
```

--------------------------------

### Initialize and Train Best RandomForestClassifier

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This code initializes a RandomForestClassifier with the best hyperparameters obtained from the grid search. It then trains this optimized model using the training data (x_train, y_train). The model is configured with 'entropy' criterion, 'max_depth' of 10, 'auto' max_features, and 100 estimators.

```python
rf_best = RF(criterion = 'entropy', max_depth = 10, max_features='auto', n_estimators=100)
rf_best.fit(x_train,y_train)
```

--------------------------------

### Plot Bar Chart of Pumps by Date

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Generates a bar chart showing the count of cryptocurrency pumps for each specific date. This visualization can reveal trends or spikes in pump activity over time.

```python
all_pumps.groupby('pump_date').pump_date.count().plot(kind='bar')
```

--------------------------------

### Display DataFrame Shape

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Prints the shape of the 'all_pumps' DataFrame, indicating the number of rows and columns. This provides a summary of the dataset's dimensions after data cleaning and transformation.

```python
all_pumps.shape
```

--------------------------------

### Subsample Data for Training

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Subsamples the loaded hourly social data and associated 'pumped_yn' labels for training and testing. It prints the shapes of the original and subsampled datasets, along with the number of positive samples in each.

```python
rf = RF()
x_train, x_test, y_train, y_test = subsample(h[social_feat_h], h['pumped_yn'], num=2000)

```

--------------------------------

### Load Scaled Social Data in Python

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Loads a pickled pandas DataFrame named 'social_data_slide_scaled_d.pkl' into the variable `d`. This DataFrame likely contains scaled social data relevant to the project. The shape of the loaded data is then printed.

```python
d = pd.read_pickle("social_data_slide_scaled_d.pkl")
print("Daily data",d.shape)
```

--------------------------------

### Python Project Initialization and Imports

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Model_5d_RF_Social.ipynb

Initializes the Python environment for the pump and dump crypto analysis project. It sets up the system path to include a parent directory for utility modules and loads essential libraries like pandas and matplotlib. The display options for pandas are also configured for better data visualization.

```python
import os, sys
sys.path.insert(1,"../")
from analysis_utils import *

pd.set_option('display.max_columns', 999)
%load_ext autoreload
%matplotlib inline
```

--------------------------------

### Load Scaled OHLCV Data (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This code snippet demonstrates how to load pre-processed OHLCV (Open, High, Low, Close, Volume) data and its corresponding features from pickle files. It prints the shape of the loaded hourly data, which is useful for understanding the dataset's dimensions before further analysis.

```python
h = pd.read_pickle("OHLCV_data_slide_scaled_h.pkl") ohlcv_feat_h = list(pd.read_pickle("ohlcv_features_h.pkl"))
print( "Hourly data",h.shape)
```

--------------------------------

### Save Processed Pump Data to Pickle File (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Serializes the 'final_pumps' DataFrame into a pickle file named 'all_pumps.pkl' in the parent directory. This allows for efficient storage and later retrieval of the processed data.

```python
final_pumps.to_pickle('../all_pumps.pkl')
```

--------------------------------

### Load Social Features and Get Count (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_4a_Normalize_Social_Daily.ipynb

Loads a list of social features from a pickle file and then prints the total number of features. This step identifies the features that will be used in the analysis.

```python
#create a list of features
social_feat = pd.read_pickle("social_features_d.pkl")
len(social_feat)
```

--------------------------------

### Count Unique Currencies in Dataset

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Prints the total number of unique currencies (coins) present in the 'all_pumps' dataset. This provides a high-level overview of the dataset's scope.

```python
print('There are {} unique currencies/coin in this dataset'.format(all_pumps.currency.drop_duplicates().count()))
```

--------------------------------

### Plot Bar Chart of Pumps by Exchange

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Generates a bar chart showing the count of cryptocurrency pumps for each exchange. This visualization helps compare the volume of pump events across different trading platforms.

```python
all_pumps.groupby('exchange').exchange.count().plot(kind='bar')
```

--------------------------------

### Create True Binance Pump Dataset

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

This Python code merges the filtered Binance pump data ('pump_bn') with the list of downloaded Binance coins ('bcoin') to create a 'true_pump' DataFrame. This ensures that the final dataset contains only valid, downloadable Binance pump events.

```python
## Crreat the new pump list as All Available Binance Coin x Pumped coin Data
true_pump = pump_bn.assign(key=1).merge(bcoin.assign(key=1))["currency","exchange","pumptime_edited","coin_name"]
```

--------------------------------

### Calculate Training Confusion Matrix

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Calculates and returns the confusion matrix for the training data using the trained Random Forest model. The result is raveled into a 1D array for easier interpretation.

```python
confusion_matrix(y_train,rf_best.predict(x_train)).ravel()
```

--------------------------------

### Define Random Forest Hyperparameter Grid

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Defines a parameter grid for hyperparameter tuning of a Random Forest classifier. This grid includes variations for n_estimators, max_features, max_depth, and criterion.

```python
rf = RF()
params_rf = {'n_estimators':[50,100,200]
            ,'max_features': ['auto', 'sqrt', 'log2']
            ,'max_depth' : [10,15,25,50,75]
            ,'criterion' :['gini', 'entropy']}
```

--------------------------------

### Plot Bar Chart of Pumps by Hour

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Generates a bar chart illustrating the number of cryptocurrency pumps occurring at each hour of the day (on a 24-hour scale). This helps identify peak times for pump events.

```python
all_pumps.groupby('pump_hour').pump_hour.count().plot(kind='bar')
```

--------------------------------

### Count Total Coins from API Data

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Prints the total number of unique coins available in the loaded CryptoCompare API data ('all_coins_info'). It counts the unique values in the 'Id' column.

```python
print("CrytoCompare API at this time contains",all_coins_info.Id.drop_duplicates().count(),"coins")
```

--------------------------------

### Load and Subsample Crypto Data (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

Loads OHLCV features from a pickle file and subsamples the dataset for training and testing. It uses the 'subsample' function to balance the 'pumped_yn' labels. The input 'd' is expected to be a pandas DataFrame containing the features and the target variable.

```python
ohlcv_feat_d = list(pd.read_pickle("ohlcv_features_d.pkl"))
x_train, x_test, y_train, y_test = subsample(d[ohlcv_feat_d], d['pumped_yn'], num=2000)
```

--------------------------------

### Calculate Day Start Timestamp (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_3a_Prepare_Social_Input_Daily.ipynb

Calculates the timestamp corresponding to the beginning of the day (12:00 AM) for a given timestamp. This is achieved by using the modulo operator with the number of seconds in a day.

```python
1562346032 - (1562346032 % (86400))
```

--------------------------------

### Merge Coin List with CryptoCompare IDs

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Merges the existing 'coin_list' (containing currency and exchange) with coin information from 'all_coins_info' based on the currency symbol. This process adds the 'Id' from CryptoCompare to the 'coin_list', using a left merge to keep all original entries.

```python
import pandas as pd
coin_list = pd.merge(left=coin_list,left_on='currency',right=all_coins_info[['Symbol','Id']],right_on='Symbol',how='left').drop('Symbol',axis=1)
```

--------------------------------

### Train Optimized Random Forest Model

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Trains a Random Forest classifier using the best hyperparameters identified by GridSearchCV. The model is trained on the training data (x_train, y_train).

```python
rf_best = RF(criterion = 'entropy', max_depth = 10, max_features='sqrt', n_estimators=100)
rf_best.fit(x_train,y_train)
```

--------------------------------

### Save DataFrame to Pickle File

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Saves a pandas DataFrame to a pickle file for later use. Pickle is a Python-specific binary format for serializing and de-serializing Python object structures.

```python
all_pumps.to_pickle('../all_pumps.pkl')
```

--------------------------------

### Concatenate and Deduplicate Pump Data (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Combines 'true_pump' and 'fake_pump' DataFrames into a single 'final_pumps' DataFrame and removes duplicate entries. This ensures a clean dataset for subsequent analysis.

```python
final_pumps = pd.concat([true_pump,fake_pump],axis=0).drop_duplicates()
```

--------------------------------

### Plot Bar Chart of Pump Frequency by Currency

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Generates a bar chart showing the frequency of cryptocurrency pumps for each unique currency, sorted in descending order. The plot is configured with a large figure size to accommodate many bars.

```python
all_pumps.groupby('currency').currency.count().sort_values(ascending=False).plot(kind='bar', figsize=(30,10))
```

--------------------------------

### Process Fake Binance Pump Events

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

This Python code generates a 'fake_pump' DataFrame by merging the extra generated dates ('extra_date') with the downloaded Binance coins ('bcoin'). It assigns a placeholder currency '***' and marks the exchange as 'Binance', creating synthetic pump events.

```python
## PROCESS FAKE PUMP
fake_pump = extra_date.assign(key=1).merge(bcoin.assign(key=1))[['pumptime_edited','coin_name']]
fake_pump["currency"] = '***'
fake_pump['exchange'] = 'Binance'
fake_pump['pumptime_edited'] = pd.to_datetime(fake_pump.pumptime_edited, utc=True)
fake_pump = fake_pump[['currency','exchange','pumptime_edited','coin_name']]
```

--------------------------------

### Extract Confusion Matrix Components for Test Set

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Calculates the confusion matrix for the test set and extracts the true negatives (tn), false positives (fp), false negatives (fn), and true positives (tp) into separate variables.

```python
tn, fp, fn, tp = confusion_matrix(y_test,rf_best.predict(x_test)).ravel()
```

--------------------------------

### Get Length of OHLCV Features

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This Python code snippet calculates and returns the number of features in the 'ohlcv_feat_d' dataset. It's a common operation for understanding the dimensionality of the data.

```python
len(ohlcv_feat_d)
```

--------------------------------

### Get Prediction Probabilities for Test Set

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This code generates the probability predictions for all classes for the entire test set using the trained `rf_best` model. The output `y_prob` is a 2D array where each row corresponds to a data point and each column represents the probability of belonging to a specific class.

```python
y_prob = rf_best.predict_proba(x_test)
```

--------------------------------

### Calculate Time-Series Differences in Crypto Data

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_1b_Social_Hourly_Download.ipynb

This snippet prepares data for time-series difference calculation by dropping date columns and creating two versions of the DataFrame: one starting from the second row ('a') and one excluding the last row ('b'). It then sets the 'time' column of 'b' to 0 and computes the element-wise difference ('a' - 'b') to get the changes between consecutive records. The shape of the resulting difference DataFrame and its first two rows are then displayed.

```python
a = krl.drop(['date','date_utc'],axis=1).loc[1:,:].reset_index(drop=True)
b = krl.drop(['date','date_utc'],axis=1).iloc[:-1,:].reset_index(drop=True)
b['time'] = 0
krl_final = a-b
print(krl_final.shape)
krl_final.head(2)
```

--------------------------------

### Get DataFrame Shape in Python

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_4c_Log_OHLCV_Hourly.ipynb

This snippet shows how to get the dimensions (number of rows and columns) of a pandas DataFrame. The output '(80940, 3528)' indicates the dataset size, which is crucial for understanding the scale of the data being processed.

```python
s.shape
```

--------------------------------

### Count Unique Currencies per Exchange

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Filters the 'all_pumps' DataFrame for a specific exchange (e.g., 'Binance') and then counts the number of unique currencies that were pumped on that exchange. This helps identify the diversity of coins affected by pumps on a given platform.

```python
#Investigating coins on Binance
b_coins = all_pumps[all_pumps.exchange == 'Binance']
b_coins.currency.drop_duplicates().count()
```

--------------------------------

### Get Hourly Historical Social Data

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_1_Download_pumped_coin_historical_data.ipynb

Retrieves hourly historical social data for a specified coin.

```APIDOC
## GET /data/social/coin/histo/hour

### Description
Retrieves hourly historical social data for a specified coin, including metrics like posts, comments, followers, and page views.

### Method
GET

### Endpoint
/data/social/coin/histo/hour

### Parameters
#### Query Parameters
- **api_key** (string) - Required - Your API key for authentication.
- **coinId** (string) - Required - The unique identifier of the coin.
- **toTs** (integer) - Required - The end Unix timestamp for the data retrieval.
- **limit** (integer) - Optional - The maximum number of records to return (default is 2000).

### Request Example
```
https://min-api.cryptocompare.com/data/social/coin/histo/hour?&coinId=877310&toTS=&toTs=1569888000&limit=2000&api_key=01a9f40086b2d1939be540e04f5ea2a3a0993c1288cc23fe1cd81fbee60ed842
```

### Response
#### Success Response (200)
- **time** (integer) - Unix timestamp for the data point.
- **comments** (integer) - Number of comments.
- **posts** (integer) - Number of posts.
- **followers** (integer) - Number of followers.
- **points** (integer) - Social points.
- **overview_page_views** (integer) - Overview page views.
- **analysis_page_views** (integer) - Analysis page views.
- **markets_page_views** (integer) - Markets page views.
- **charts_page_views** (integer) - Charts page views.
- **trades_page_views** (integer) - Trades page views.
- **forum_page_views** (integer) - Forum page views.
- **influence_score** (float) - Influence score.
- **twitter_followers** (integer) - Twitter followers.
- **twitter_following** (integer) - Twitter following.
- **twitter_tweet_count** (integer) - Twitter tweet count.
- **twitter_account_created** (integer) - Twitter account creation timestamp.
- **twitter_account_age** (integer) - Twitter account age in days.
- **twitter_account_verified** (integer) - Whether the Twitter account is verified (1 for true, 0 for false).
- **twitter_favourites_count** (integer) - Twitter favorites count.
- **twitter_listed_count** (integer) - Twitter listed count.
- **twitter_statuses_count** (integer) - Twitter statuses count.
- **twitter_url_count** (integer) - Twitter URL count.
- **twitter_description_length** (integer) - Twitter description length.
- **twitter_geo_enabled** (integer) - Whether Twitter geo is enabled (1 for true, 0 for false).
- **twitter_lang** (string) - Language of the Twitter account.
- **reddit_subscribers** (integer) - Reddit subscribers.
- **reddit_active_users (percentage)** (float) - Percentage of active Reddit users.
- **reddit_comment_total** (integer) - Total Reddit comments.
- **reddit_post_total** (integer) - Total Reddit posts.
- **reddit_score_total** (integer) - Total Reddit score.
- **reddit_comments_per_day** (float) - Reddit comments per day.
- **code_repo_stars** (integer) - Code repository stars.
- **code_repo_forks** (integer) - Code repository forks.
- **code_repo_subscribers** (integer) - Code repository subscribers.
- **code_repo_open_pull_issues** (integer) - Open pull issues in code repository.
- **code_repo_closed_pull_issues** (integer) - Closed pull issues in code repository.
- **code_repo_open_issues** (integer) - Open issues in code repository.
- **code_repo_closed_issues** (integer) - Closed issues in code repository.
- **code_repo_contributors** (integer) - Number of contributors to the code repository.

#### Response Example
```json
{
  "time": 1569888000,
  "comments": 111,
  "posts": 111,
  "followers": 111,
  "points": 111,
  "overview_page_views": 111,
  "analysis_page_views": 111,
  "markets_page_views": 111,
  "charts_page_views": 111,
  "trades_page_views": 111,
  "forum_page_views": 111,
  "influence_score": 1.0,
  "twitter_followers": 1000,
  "twitter_following": 500,
  "twitter_tweet_count": 10000,
  "twitter_account_created": 1420070400,
  "twitter_account_age": 3000,
  "twitter_account_verified": 1,
  "twitter_favourites_count": 5000,
  "twitter_listed_count": 100,
  "twitter_statuses_count": 10000,
  "twitter_url_count": 5,
  "twitter_description_length": 160,
  "twitter_geo_enabled": 0,
  "twitter_lang": "en",
  "reddit_subscribers": 50000,
  "reddit_active_users (percentage)": 10.5,
  "reddit_comment_total": 100000,
  "reddit_post_total": 50000,
  "reddit_score_total": 500000,
  "reddit_comments_per_day": 100.0,
  "code_repo_stars": 1000,
  "code_repo_forks": 200,
  "code_repo_subscribers": 50,
  "code_repo_open_pull_issues": 10,
  "code_repo_closed_pull_issues": 50,
  "code_repo_open_issues": 20,
  "code_repo_closed_issues": 100,
  "code_repo_contributors": 25
}
```
```

--------------------------------

### Get Daily Historical Social Data

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_1_Download_pumped_coin_historical_data.ipynb

Retrieves daily historical social data for a specified coin.

```APIDOC
## GET /data/social/coin/histo/day

### Description
Retrieves daily historical social data for a specified coin, including metrics like posts, comments, followers, and page views.

### Method
GET

### Endpoint
/data/social/coin/histo/day

### Parameters
#### Query Parameters
- **api_key** (string) - Required - Your API key for authentication.
- **coinId** (string) - Required - The unique identifier of the coin.
- **toTs** (integer) - Required - The end Unix timestamp for the data retrieval.
- **limit** (integer) - Optional - The maximum number of records to return (default is 2000).

### Request Example
```
https://min-api.cryptocompare.com/data/social/coin/histo/day?&coinId=26132&toTS=&toTs=1569888000&limit=2000&api_key=01a9f40086b2d1939be540e04f5ea2a3a0993c1288cc23fe1cd81fbee60ed842
```

### Response
#### Success Response (200)
- **time** (integer) - Unix timestamp for the data point.
- **comments** (integer) - Number of comments.
- **posts** (integer) - Number of posts.
- **followers** (integer) - Number of followers.
- **points** (integer) - Social points.
- **overview_page_views** (integer) - Overview page views.
- **analysis_page_views** (integer) - Analysis page views.
- **markets_page_views** (integer) - Markets page views.
- **charts_page_views** (integer) - Charts page views.
- **trades_page_views** (integer) - Trades page views.
- **forum_page_views** (integer) - Forum page views.
- **influence_score** (float) - Influence score.
- **twitter_followers** (integer) - Twitter followers.
- **twitter_following** (integer) - Twitter following.
- **twitter_tweet_count** (integer) - Twitter tweet count.
- **twitter_account_created** (integer) - Twitter account creation timestamp.
- **twitter_account_age** (integer) - Twitter account age in days.
- **twitter_account_verified** (integer) - Whether the Twitter account is verified (1 for true, 0 for false).
- **twitter_favourites_count** (integer) - Twitter favorites count.
- **twitter_listed_count** (integer) - Twitter listed count.
- **twitter_statuses_count** (integer) - Twitter statuses count.
- **twitter_url_count** (integer) - Twitter URL count.
- **twitter_description_length** (integer) - Twitter description length.
- **twitter_geo_enabled** (integer) - Whether Twitter geo is enabled (1 for true, 0 for false).
- **twitter_lang** (string) - Language of the Twitter account.
- **reddit_subscribers** (integer) - Reddit subscribers.
- **reddit_active_users (percentage)** (float) - Percentage of active Reddit users.
- **reddit_comment_total** (integer) - Total Reddit comments.
- **reddit_post_total** (integer) - Total Reddit posts.
- **reddit_score_total** (integer) - Total Reddit score.
- **reddit_comments_per_day** (float) - Reddit comments per day.
- **code_repo_stars** (integer) - Code repository stars.
- **code_repo_forks** (integer) - Code repository forks.
- **code_repo_subscribers** (integer) - Code repository subscribers.
- **code_repo_open_pull_issues** (integer) - Open pull issues in code repository.
- **code_repo_closed_pull_issues** (integer) - Closed pull issues in code repository.
- **code_repo_open_issues** (integer) - Open issues in code repository.
- **code_repo_closed_issues** (integer) - Closed issues in code repository.
- **code_repo_contributors** (integer) - Number of contributors to the code repository.

#### Response Example
```json
{
  "time": 1397088000,
  "comments": 0,
  "posts": 0,
  "followers": 0,
  "points": 0,
  "overview_page_views": 0,
  "analysis_page_views": 0,
  "markets_page_views": 0,
  "charts_page_views": 0,
  "trades_page_views": 0,
  "forum_page_views": 0,
  "influence_score": 0.0,
  "twitter_followers": 0,
  "twitter_following": 0,
  "twitter_tweet_count": 0,
  "twitter_account_created": 0,
  "twitter_account_age": 0,
  "twitter_account_verified": 0,
  "twitter_favourites_count": 0,
  "twitter_listed_count": 0,
  "twitter_statuses_count": 0,
  "twitter_url_count": 0,
  "twitter_description_length": 0,
  "twitter_geo_enabled": 0,
  "twitter_lang": "en",
  "reddit_subscribers": 0,
  "reddit_active_users (percentage)": 0.0,
  "reddit_comment_total": 0,
  "reddit_post_total": 0,
  "reddit_score_total": 0,
  "reddit_comments_per_day": 0.0,
  "code_repo_stars": 0,
  "code_repo_forks": 0,
  "code_repo_subscribers": 0,
  "code_repo_open_pull_issues": 0,
  "code_repo_closed_pull_issues": 0,
  "code_repo_open_issues": 0,
  "code_repo_closed_issues": 0,
  "code_repo_contributors": 0
}
```
```

--------------------------------

### Get Statistics for Log 3h Volume To

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_4c_Log_OHLCV_Hourly.ipynb

This code snippet calls the getStat function to perform a statistical comparison of 'log_3h_volumeto' between pumped and non-pumped events on Binance.

```python
getStat(d,"log_3h_volumeto",exchange)
```

--------------------------------

### Initialize Data Column Dictionary

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_3a_Prepare_Social_Input_Daily.ipynb

This Python code initializes a dictionary to store data columns, categorized by type (code, twitter, reddit, fb, market). It iterates through predefined lists of column names for each category and populates a nested dictionary structure. This is a setup step for further data processing.

```python
d = 32
col_dict = dict()
for f in ['code','twitter','reddit','fb','market']:
    dt = dict()
    for i in eval(f):
        # The rest of the loop body is missing in the provided snippet.
```

--------------------------------

### Get Statistics for Log Volume from H1

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_4c_Log_OHLCV_Hourly.ipynb

This code snippet calls the getStat function to perform a statistical comparison of 'log_volumefrom_h1' between pumped and non-pumped events on Binance.

```python
getStat(d,"log_volumefrom_h1",exchange)
```

--------------------------------

### Calculate Confusion Matrix on Training Set

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This code calculates and prints the confusion matrix for the predictions made on the training set. It uses the `confusion_matrix` function from scikit-learn to compute true negatives (tn), false positives (fp), false negatives (fn), and true positives (tp). The output shows the counts for each category.

```python
tn,fp,fn,tp = confusion_matrix(y_train,y_train_pred).ravel()
print(tn,fp,fn,tp)
```

--------------------------------

### Get Statistics for Log Volume from H0

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_4c_Log_OHLCV_Hourly.ipynb

This code snippet calls the getStat function to perform a statistical comparison of 'log_volumefrom_h0' between pumped and non-pumped events on Binance.

```python
exchange = "Binance"
getStat(d,"log_volumefrom_h0",exchange)
```

--------------------------------

### Train and Evaluate Random Forest Classifier (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This snippet shows the process of training a Random Forest Classifier with specific hyperparameters, fitting it to training data, and then evaluating its performance on both training and testing sets using a confusion matrix. It includes steps for initializing the model, fitting, predicting, and extracting true negatives, false positives, false negatives, and true positives.

```python
CV_rfc.best_params_

rf_best = RF(criterion = 'entropy', max_depth = 25, max_features='auto', n_estimators=50)
rf_best.fit(x_train,y_train)

confusion_matrix(y_train,rf_best.predict(x_train)).ravel()

tn, fp, fn, tp = confusion_matrix(y_test,rf_best.predict(x_test)).ravel()
print(tn,fp,fn,tp)

del tn, fp, fn, tp
```

--------------------------------

### Predict Probabilities and Binary Labels on Training Set

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This snippet demonstrates how to generate predictions from the trained RandomForestClassifier on the training data. It first calculates the probability of the positive class (class 1) using `predict_proba`. Then, it creates binary predictions (0 or 1) based on a probability threshold of 0.01.

```python
y_train_prob = rf_best.predict_proba(x_train)[:,1]
y_train_pred = np.zeros_like(y_train)
y_train_pred[y_train_prob > 0.01] = 1
```

--------------------------------

### Plot Pump Distribution by Exchange

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Generates a scatter plot showing cryptocurrency pumps over time, colored by exchange. It uses matplotlib for plotting and pandas for data manipulation. The plot visualizes pump events against their date and hour, with a legend indicating different exchanges.

```python
import matplotlib.patches as mpatches
color_dct = {"Binance":"orange","Cryptopia":"green","Bittrex":"blue","Yobit":"turquoise"}
exchange_fig = plt.figure(figsize=(10,5))
plt.scatter(all_pumps.event_date, all_pumps.pump_hour
            ,c=all_pumps.exchange.apply(lambda x:color_dct[x])
            )
plt.xlabel("Date of pumps")
plt.ylabel("Hour of pump on 24 hour scale")
#Legend
orange_patch = mpatches.Patch(color='orange', label='Binance')
green_patch = mpatches.Patch(color='green', label='Crytopia')
blue_patch = mpatches.Patch(color='blue', label='Bittrex')
turquoise_patch = mpatches.Patch(color='turquoise', label='Yobit')
plt.legend(handles=[orange_patch,green_patch,blue_patch,turquoise_patch])
plt.show()
```

--------------------------------

### Delete Confusion Matrix Variables

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5a_RF_Social_Models.ipynb

Deletes the variables tn, fp, fn, and tp that were used to store the components of the confusion matrix. This is a cleanup step.

```python
del tn, fp, fn, tp
```

--------------------------------

### Filter Binance Pumps by Criteria

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

This Python code snippet filters a DataFrame 'all_pumps' to select entries exclusively from Binance. It then extracts and deduplicates specific columns related to pump times and event dates. This is a preliminary step to isolate Binance-specific pump data.

```python
pt = all_pumps[all_pumps.exchange=="Binance"][["pumptime_edited","timestamp","event_date","event_time"]].drop_duplicates()
```

--------------------------------

### Remove Erroneous Data Entry

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Filters the 'all_pumps' DataFrame to remove rows where the 'pump_year' is 1999, addressing a specific data anomaly identified in the dataset.

```python
#There is 1 noise pump at 1999. Eliminate it. 
all_pumps = all_pumps[all_pumps.pump_year != 1999]
```

--------------------------------

### Get Column Names of ADX DataFrame in Python

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/General_Analysis/Mod_2_Acquire Social Data.ipynb

Retrieves and displays the names of all columns in the `adx` DataFrame. This is helpful for understanding the available data fields and for subsequent data selection or analysis.

```python
adx.columns
```

--------------------------------

### Get Current Working Directory (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/General_Analysis/Mod_0b_ General_Analysis.ipynb

Retrieves and prints the current working directory using the os module in Python. This is useful for understanding the execution context of the script.

```python
import os, sys
print(os.getcwd())
```

--------------------------------

### Get Current Working Directory

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_4e_Normalization_OHLCV_Hourly.ipynb

Prints the current working directory of the Python script. This is often used for debugging or to understand the file system context in which the script is operating.

```python
print(os.getcwd())
```

--------------------------------

### Count Coins Without CryptoCompare ID

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Calculates and prints the number of coins in the 'coin_list' that do not have a corresponding 'Id' from the CryptoCompare data. These are coins for which lookup data was not found.

```python
na_coin = coin_list[coin_list.Id.isnull()].currency.drop_duplicates().count()
print("There are",na_coin,"without ID.")
```

--------------------------------

### Load and Inspect Daily Scaled OHLCV Data (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5b_RF_Model_OHLCV.ipynb

This Python snippet loads a pickled pandas DataFrame containing scaled daily OHLCV data. It then prints the shape of the DataFrame to provide an overview of the dataset's dimensions. The `d.head()` command displays the first few rows of the DataFrame, allowing for a quick inspection of the data structure and content.

```python
d = pd.read_pickle("ohlcv_data_slide_scaled_d.pkl")
print("Daily data",d.shape)
d.head()
```

--------------------------------

### Setup KNN Pipeline and Cross-Validation

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/General_Analysis/Mod_3_DataAnalysis .ipynb

Configures a scikit-learn Pipeline for a KNeighborsClassifier, including data scaling and the classifier itself. It also sets up KFold cross-validation with 10 splits.

```python
k=10
knn_clf = KNeighborsClassifier(n_neighbors=20)
knn_pipeline = Pipeline([('transformer',minmax_scaler),('estimator',knn_clf)])
cv = KFold(n_splits=k)
```

--------------------------------

### Load and Prepare Coin List Data (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/Mod_1_Download_pumped_coin_historical_data.ipynb

Loads a list of cryptocurrencies and their IDs from a pickle file using pandas. It then cleans the data by removing duplicates and rows with missing IDs, preparing it for further processing.

```python
coinlist = pd.read_pickle('../../Coin_Data/coin_list.pkl')[['currency','Id']]
#We only select those with valid ID; 
coinlist = coinlist.drop_duplicates().dropna()
coinlist.head()
```

--------------------------------

### Initialize K-Nearest Neighbors Classifier

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/General_Analysis/Mod_3_DataAnalysis .ipynb

Creates an instance of the KNeighborsClassifier. By default, it uses the 'auto' algorithm for selecting the best algorithm to run, 'minkowski' distance metric, and 'uniform' weights.

```python
knn = KNeighborsClassifier()
```

--------------------------------

### Get Best Hyperparameters from Grid Search

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Before_Pump_Analysis/RF_Model/Mod_5c_OHLCV_and_Social.ipynb

Retrieves the best hyperparameters found by the `GridSearchCV` process. This allows for the selection of the optimal combination of parameters that yielded the highest 'roc_auc' score during cross-validation.

```python
CV_rfc.best_params_
```

--------------------------------

### Set up Pipeline for Cross-Validation (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/General_Analysis/Mod_4_Exclusive_Social_Media.ipynb

Creates a machine learning pipeline that combines feature scaling (MinMaxScaler) and a classifier (RandomForestClassifier). A KFold cross-validation strategy is also defined to evaluate the pipeline's performance robustly.

```python
from sklearn.pipeline import Pipeline
from sklearn.model_selection import KFold

minmax_scaler = MinMaxScaler()
clf = RandomForestClassifier(criterion='entropy',max_depth= 10, max_features='auto',n_estimators=750)

k = 10
pipeline = Pipeline([('transformer',minmax_scaler),('estimator',clf)])
cv = KFold(n_splits=k)
```

--------------------------------

### Display Shape of Processed Pump Data (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_0_Format_Pump_Data.ipynb

Retrieves and displays the dimensions (number of rows and columns) of the 'final_pumps' DataFrame after processing. This provides a quick overview of the dataset size.

```python
final_pumps.shape
```

--------------------------------

### Inspect Downloaded Pickle File (Python)

Source: https://github.com/hnghiem-nlp/pump_and_dump_crypto/blob/master/Final_Analysis/Final_Mod_1a_OHLCV_Hourly.ipynb

Reads a pickle file containing hourly cryptocurrency data from Binance, extracts the date from the 'date_utc' column, and displays the first two rows. This helps in understanding the format and content of individual exchange data files.

```python
## Check what's inside the downoaded file
dl = pd.read_pickle("../../Coin_Data/Hourly/Hourly_VIB_at_Binance.pkl")
dl['dt']= dl.date_utc.apply(lambda d:d.date)
dl.head(2)
```