### Warm Start and Parallel Processing in gplearn

Source: https://context7.com/trevorstephens/gplearn/llms.txt

Demonstrates how to use warm starting to resume training from a previous run and parallel processing for faster execution using multiple CPU cores. It also shows how to enable low memory mode for handling large populations.

```python
import numpy as np
from gplearn.genetic import SymbolicRegressor

# Generate training data
X = np.random.randn(500, 5)
y = X[:, 0] * X[:, 1] + np.sin(X[:, 2]) - X[:, 3] ** 2

# Initial training with 10 generations
est = SymbolicRegressor(
    population_size=2000,
    generations=10,
    warm_start=True,  # Enable warm start
    n_jobs=4,         # Use 4 parallel processes
    verbose=1,
    random_state=42
)
est.fit(X, y)
print(f"After 10 generations: {est._program.raw_fitness_}")

# Continue evolution for 10 more generations (total 20)
est.generations = 20
est.fit(X, y)
print(f"After 20 generations: {est._program.raw_fitness_}")

# Access evolution history
run_details = est.run_details_
print(f"Generations: {run_details['generation']}")
print(f"Best fitness per gen: {run_details['best_fitness']}")
print(f"Average program length: {run_details['average_length']}")

# Low memory mode for large populations
est_low_mem = SymbolicRegressor(
    population_size=10000,
    generations=50,
    low_memory=True,  # Discard old generations to save memory
    n_jobs=-1,        # Use all available cores
    random_state=42
)
est_low_mem.fit(X, y)
```

--------------------------------

### Import Classifiers and Utilities for Plotting

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Imports various classification algorithms from scikit-learn, along with utilities for data generation, preprocessing, and plotting. This setup is for comparing the performance of different classifiers.

```python
from gplearn.genetic import SymbolicClassifier
from matplotlib.colors import ListedColormap
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_moons, make_circles, make_classification
from sklearn.neural_network import MLPClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.gaussian_process.kernels import RBF
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
from sklearn.metrics import roc_auc_score
from sklearn.datasets import load_breast_cancer
```

--------------------------------

### Import make_function from gplearn

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Imports the necessary utility to create custom functions for genetic programming.

```python
from gplearn.functions import make_function
```

--------------------------------

### Define Classifier Names and Instances

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Sets up a list of names and corresponding classifier instances from scikit-learn and gplearn. This includes standard classifiers and the SymbolicClassifier for comparison.

```python
# Modified from https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html
# Code source: Gaël Varoquaux
#              Andreas Müller
# Modified for documentation by Jaques Grobler
# License: BSD 3 clause

h = .02  # step size in the mesh

names = ["Nearest Neighbors", "Linear SVM", "RBF SVM", "Gaussian Process",
         "Decision Tree", "Random Forest", "Neural Net", "AdaBoost",
         "Naive Bayes", "QDA", "SymbolicClassifier"]

classifiers = [
    KNeighborsClassifier(3),
    SVC(kernel="linear", C=0.025),
    SVC(gamma=2, C=1),
    GaussianProcessClassifier(1.0 * RBF(1.0)),
    DecisionTreeClassifier(max_depth=5),
    RandomForestClassifier(max_depth=5, n_estimators=10, max_features=1),
    MLPClassifier(alpha=1, tol=0.001),
    AdaBoostClassifier(),
    GaussianNB(),
    QuadraticDiscriminantAnalysis(),
    SymbolicClassifier(random_state=0)]
```

--------------------------------

### Generate and Prepare Datasets

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Creates synthetic datasets (moons, circles, and linearly separable) for evaluating the classifiers. It also adds noise to the linearly separable data to make it more challenging.

```python
X, y = make_classification(n_features=2, n_redundant=0, n_informative=2,
                           random_state=1, n_clusters_per_class=1)
rng = np.random.RandomState(2)
X += 2 * rng.uniform(size=X.shape)
linearly_separable = (X, y)

datasets = [make_moons(noise=0.3, random_state=0),
            make_circles(noise=0.2, factor=0.5, random_state=1),
            linearly_separable
            ]

figure = plt.figure(figsize=(27, 9))
i = 1
```

--------------------------------

### Export and Render Program as Graphviz

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Generates a Graphviz representation of the evolved program and saves it as a PNG image. This provides a visual structure of the symbolic expression, aiding in understanding the evolved logic.

```python
dot_data = gp._programs[0][3].export_graphviz()
graph = graphviz.Source(dot_data)
graph.render('images/ex3_fig1', format='png', cleanup=True)
graph
```

--------------------------------

### Create Custom Fitness Metrics with gplearn's make_fitness

Source: https://context7.com/trevorstephens/gplearn/llms.txt

This snippet illustrates how to define custom fitness functions for evaluating the quality of programs in gplearn. It provides examples of a weighted Mean Absolute Percentage Error (MAPE) and a custom R-squared metric. These custom metrics can be used to guide the genetic programming process based on specific evaluation criteria.

```python
import numpy as np
from gplearn.fitness import make_fitness
from gplearn.genetic import SymbolicRegressor

# Define a custom fitness metric: weighted MAPE
def _weighted_mape(y, y_pred, w):
    """Calculate weighted mean absolute percentage error."""
    eps = 1e-10
    ape = np.abs((y - y_pred) / (np.abs(y) + eps))
    return np.average(ape, weights=w)

# Create the fitness object (lower is better)
weighted_mape = make_fitness(
    function=_weighted_mape,
    greater_is_better=False,
    wrap=True
)

# Define a custom R-squared metric
def _custom_r2(y, y_pred, w):
    """Calculate weighted R-squared score."""
    ss_res = np.average((y - y_pred) ** 2, weights=w)
    ss_tot = np.average((y - np.average(y, weights=w)) ** 2, weights=w)
    return 1 - (ss_res / (ss_tot + 1e-10))

custom_r2 = make_fitness(
    function=_custom_r2,
    greater_is_better=True,
    wrap=True
)

# Use custom fitness in a regressor
est = SymbolicRegressor(
    metric=weighted_mape,  # or custom_r2
    population_size=1000,
    generations=15,
    random_state=0
)

X = np.random.randn(200, 3)
y = X[:, 0] ** 2 + 2 * X[:, 1] - X[:, 2]

est.fit(X, y)
print(f"Best fitness: {est._program.raw_fitness_}")
print(f"Program: {est._program}")
```

--------------------------------

### Print Evolved Program

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Accesses and prints the first program found in the hall of fame of the trained SymbolicTransformer. This displays the structure of the evolved symbolic expression.

```python
print(gp._programs[0][3])
```

--------------------------------

### Initialize and Train SymbolicTransformer

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Initializes and trains a SymbolicTransformer with specified parameters for genetic programming. It uses a custom function set and evolution parameters to generate new features.

```default
function_set = ['add', 'sub', 'mul', 'div',
                'sqrt', 'log', 'abs', 'neg', 'inv',
                'max', 'min']
gp = SymbolicTransformer(generations=20, population_size=2000,
                         hall_of_fame=100, n_components=10,
                         function_set=function_set,
                         parsimony_coefficient=0.0005,
                         max_samples=0.9, verbose=1,
                         random_state=0, n_jobs=3)
gp.fit(diabetes.data[:300, :], diabetes.target[:300])
```

--------------------------------

### Export SymbolicRegressor Program to Graphviz

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Exports the learned program from the SymbolicRegressor into a Graphviz format, which can then be used to visualize the structure of the symbolic expression as a tree.

```python
dot_data = est_gp._program.export_graphviz()
graph = graphviz.Source(dot_data)
graph
```

--------------------------------

### Import Libraries for Symbolic Regression (Python)

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Imports necessary libraries for symbolic regression using gplearn, including scikit-learn for regressors, numpy for numerical operations, and matplotlib for plotting.

```python
%matplotlib inline
from gplearn.genetic import SymbolicRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.tree import DecisionTreeRegressor
from sklearn.utils.random import check_random_state
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np
import graphviz
```

--------------------------------

### Load and Prepare Breast Cancer Dataset

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Loads the Wisconsin breast cancer dataset, shuffles it using a random state, and separates the data and target variables. This prepares the data for training and testing the classifier.

```python
rng = check_random_state(0)
cancer = load_breast_cancer()
perm = rng.permutation(cancer.target.size)
cancer.data = cancer.data[perm]
cancer.target = cancer.target[perm]
```

--------------------------------

### Visualize Program Tree with Graphviz

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/advanced.md

Generates a Graphviz representation of the program tree using `export_graphviz` and displays it as a PNG image using `pydotplus` and `IPython.display`. This requires Graphviz and pydotplus to be installed.

```python
from IPython.display import Image
import pydotplus
graph = est_gp._program.export_graphviz()
graph = pydotplus.graphviz.graph_from_dot_data(graph)
Image(graph.create_png())
```

--------------------------------

### Prepare Test Data

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Generates synthetic test data (X_test, y_test) for evaluating regression models. This data is used to assess how well the models generalize to unseen samples.

```python
X_test = rng.uniform(-1, 1, 100).reshape(50, 2)
y_test = X_test[:, 0]**2 - X_test[:, 1]**2 + X_test[:, 1] - 1
```

--------------------------------

### Visualize SymbolicClassifier Solution with Graphviz

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Exports the learned program from the SymbolicClassifier into a Graphviz format and creates a source object for visualization. This allows for a graphical representation of the decision tree found by the genetic programming algorithm.

```python
dot_data = est._program.export_graphviz()
graph = graphviz.Source(dot_data)
graph
```

--------------------------------

### Continue Evolution with Warm Start (Python)

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/advanced.md

Shows how to continue the evolution process of a `SymbolicRegressor` using the `warm_start` parameter. After initial training, the `generations` parameter is increased, and `warm_start` is set to `True` before fitting again to extend the evolution.

```python
from gplearn.genetic import SymbolicRegressor

# Initial evolution
est = SymbolicRegressor(generations=10)
est.fit(X, y)

# Continue evolution
est.set_params(generations=20, warm_start=True)
est.fit(X, y)
```

--------------------------------

### Print Program Parents and Details

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Prints the parent information of a genetic programming individual, including the evolution method, parent indices, and modified nodes. This helps understand how an individual was generated.

```default
print(est_gp._program.parents)

{'method': 'Crossover',
 'parent_idx': 1555,
 'parent_nodes': [1, 2, 3],
 'donor_idx': 78,
 'donor_nodes': []}
```

--------------------------------

### Print SymbolicRegressor Program

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Prints the symbolic expression found by the trained SymbolicRegressor. This allows for direct inspection of the mathematical formula that the model has learned.

```python
print(est_gp._program)
```

--------------------------------

### Iterate and Preprocess Datasets for Classification

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

This snippet demonstrates iterating through datasets, preprocessing them using StandardScaler, and splitting them into training and testing sets. It also includes plotting the input data and iterating through classifiers to plot decision boundaries and display classification scores.

```python
for ds_cnt, ds in enumerate(datasets):
    # preprocess dataset, split into training and test part
    X, y = ds
    X = StandardScaler().fit_transform(X)
    X_train, X_test, y_train, y_test = \
        train_test_split(X, y, test_size=.4, random_state=42)

    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                         np.arange(y_min, y_max, h))

    # just plot the dataset first
    cm = plt.cm.RdBu
    cm_bright = ListedColormap(['#FF0000', '#0000FF'])
    ax = plt.subplot(len(datasets), len(classifiers) + 1, i)
    if ds_cnt == 0:
        ax.set_title("Input data")
    # Plot the training points
    ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright,
               edgecolors='k')
    # Plot the testing points
    ax.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap=cm_bright, alpha=0.6,
               edgecolors='k')
    ax.set_xlim(xx.min(), xx.max())
    ax.set_ylim(yy.min(), yy.max())
    ax.set_xticks(())
    ax.set_yticks(())
    i += 1

    # iterate over classifiers
    for name, clf in zip(names, classifiers):
        ax = plt.subplot(len(datasets), len(classifiers) + 1, i)
        clf.fit(X_train, y_train)
        score = clf.score(X_test, y_test)

        # Plot the decision boundary. For that, we will assign a color to each
        # point in the mesh [x_min, x_max]x[y_min, y_max].
        if hasattr(clf, "decision_function"):
            Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()])
        else:
            Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]

        # Put the result into a color plot
        Z = Z.reshape(xx.shape)
        ax.contourf(xx, yy, Z, cmap=cm, alpha=.8)

        # Plot the training points
        ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright,
                   edgecolors='k')
        # Plot the testing points
        ax.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap=cm_bright,
                   edgecolors='k', alpha=0.6)

        ax.set_xlim(xx.min(), xx.max())
        ax.set_ylim(yy.min(), yy.max())
        ax.set_xticks(())
        ax.set_yticks(())
        if ds_cnt == 0:
            ax.set_title(name)
        ax.text(xx.max() - .3, yy.min() + .3, ('%.2f' % score).lstrip('0'),
                size=15, horizontalalignment='right')
        i += 1

plt.tight_layout()
plt.show()
```

--------------------------------

### Fit SymbolicTransformer to Data

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Trains the SymbolicTransformer model using the provided diabetes dataset. The model evolves programs based on the specified function set and parameters.

```python
gp.fit(diabetes.data[:300, :], diabetes.target[:300])
```

--------------------------------

### Configure SymbolicTransformer with Custom Function

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Initializes SymbolicTransformer with a custom function set that includes the previously defined 'logical' function, along with standard arithmetic operations. This allows the GP algorithm to utilize the custom logic during program evolution.

```python
function_set = ['add', 'sub', 'mul', 'div', logical]
gp = SymbolicTransformer(generations=2, population_size=2000,
                         hall_of_fame=100, n_components=10,
                         function_set=function_set,
                         parsimony_coefficient=0.0005,
                         max_samples=0.9, verbose=1,
                         random_state=0)
```

--------------------------------

### Visualize Main Parent Program in SymbolicRegressor

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Retrieves the main parent program from the SymbolicRegressor's history using the 'parent_idx' from the parent information. It then prints the parent program and its fitness, and visualizes it using Graphviz, highlighting the faded nodes.

```python
idx = est_gp._program.parents['parent_idx']
fade_nodes = est_gp._program.parents['parent_nodes']
print(est_gp._programs[-2][idx])
print('Fitness:', est_gp._programs[-2][idx].fitness_)
dot_data = est_gp._programs[-2][idx].export_graphviz(fade_nodes=fade_nodes)
graph = graphviz.Source(dot_data)
graph
```

--------------------------------

### Plot Program Parents using Graphviz

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Generates and displays a Graphviz plot of the parent programs used to create a new individual. It extracts parent indices and nodes to visualize the crossover or mutation process.

```default
idx = est_gp._program.parents['donor_idx']
fade_nodes = est_gp._program.parents['donor_nodes']
dot_data = est_gp._programs[-2][idx].export_graphviz(fade_nodes=fade_nodes)
graph = graphviz.Source(dot_data)
graph
```

--------------------------------

### Fit SymbolicClassifier to Dataset

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Initializes and fits a SymbolicClassifier with a specified parsimony coefficient and feature names to the first 400 samples of the prepared breast cancer dataset. A random state is set for reproducibility.

```python
est = SymbolicClassifier(parsimony_coefficient=.01,
                         feature_names=cancer.feature_names,
                         random_state=1)
est.fit(cancer.data[:400], cancer.target[:400])
```

--------------------------------

### Load Data and Initialize SymbolicTransformer

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

This snippet loads the diabetes dataset and initializes the SymbolicTransformer. It sets parameters like generations, population size, function set, and other genetic programming hyperparameters. The transformer is configured to evolve new features based on the input data.

```python
from gplearn.genetic import SymbolicTransformer
from sklearn.utils import check_random_state
from sklearn.datasets import load_diabetes
import numpy as np

rng = check_random_state(0)
diabetes = load_diabetes()
perm = rng.permutation(diabetes.target.size)
diabetes.data = diabetes.data[perm]
diabetes.target = diabetes.target[perm]

function_set = ['add', 'sub', 'mul', 'div', 'sqrt', 'log',
                'abs', 'neg', 'inv', 'max', 'min']
gp = SymbolicTransformer(generations=20, population_size=2000,
                         hall_of_fame=100, n_components=10,
                         function_set=function_set,
                         parsimony_coefficient=0.0005,
                         max_samples=0.9, verbose=1,
                         random_state=0)
```

--------------------------------

### Generate and Plot Ground Truth Data (Python)

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Generates ground truth data for a symbolic regression problem using numpy and visualizes it as a 3D surface plot using matplotlib. This helps in understanding the target function to be approximated.

```python
# Ground truth
x0 = np.arange(-1, 1, .1)
x1 = np.arange(-1, 1, .1)
x0, x1 = np.meshgrid(x0, x1)
y_truth = x0**2 - x1**2 + x1 - 1

ax = plt.figure().add_subplot(projection='3d')
ax.set_xlim(-1, 1)
ax.set_ylim(-1, 1)
ax.set_xticks(np.arange(-1, 1.01, .5))
ax.set_yticks(np.arange(-1, 1.01, .5))
surf = ax.plot_surface(x0, x1, y_truth, rstride=1, cstride=1, color='green', alpha=0.5)
plt.show()
```

--------------------------------

### Create Custom Fitness Measure (Python)

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/advanced.md

Uses the `make_fitness` factory function from gplearn to create a custom fitness measure based on a provided function. This example creates a MAPE fitness measure where a lower value is considered better.

```python
from gplearn.fitness import make_fitness

mape = make_fitness(function=_mape,
                    greater_is_better=False)
```

--------------------------------

### Predict and Score Models, Visualize Results

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Generates predictions using the trained SymbolicRegressor, DecisionTreeRegressor, and RandomForestRegressor. It then calculates the R^2 score for each model on the test data and visualizes the ground truth and the predictions of each model using 3D surface plots.

```python
y_gp = est_gp.predict(np.c_[x0.ravel(), x1.ravel()]).reshape(x0.shape)
score_gp = est_gp.score(X_test, y_test)
y_tree = est_tree.predict(np.c_[x0.ravel(), x1.ravel()]).reshape(x0.shape)
score_tree = est_tree.score(X_test, y_test)
y_rf = est_rf.predict(np.c_[x0.ravel(), x1.ravel()]).reshape(x0.shape)
score_rf = est_rf.score(X_test, y_test)

fig = plt.figure(figsize=(12, 10))

for i, (y, score, title) in enumerate([(y_truth, None, "Ground Truth"),
                                       (y_gp, score_gp, "SymbolicRegressor"),
                                       (y_tree, score_tree, "DecisionTreeRegressor"),
                                       (y_rf, score_rf, "RandomForestRegressor")]):

    ax = fig.add_subplot(2, 2, i+1, projection='3d')
    ax.set_xlim(-1, 1)
    ax.set_ylim(-1, 1)
    ax.set_xticks(np.arange(-1, 1.01, .5))
    ax.set_yticks(np.arange(-1, 1.01, .5))
    surf = ax.plot_surface(x0, x1, y, rstride=1, cstride=1, color='green', alpha=0.5)
    points = ax.scatter(X_train[:, 0], X_train[:, 1], y_train)
    if score is not None:
        score = ax.text(-.7, 1, .2, "$R^2 =\/ %.6f$" % score, 'x', fontsize=14)
    plt.title(title)

plt.show()
```

--------------------------------

### SymbolicTransformer - get_params

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/reference.md

Gets the parameters for this estimator, including those of contained subobjects.

```APIDOC
## GET /get_params

### Description
Get parameters for this estimator.

### Method
GET

### Endpoint
/get_params

### Parameters
#### Path Parameters
None

#### Query Parameters
- **deep** (boolean) - Optional - If True, will return the parameters for this estimator and contained subobjects that are estimators.

### Request Example
```json
{
  "deep": true
}
```

### Response
#### Success Response (200)
- **params** (object) - Parameter names mapped to their values.

#### Response Example
```json
{
  "params": {
    "p_subtree_mutation": 0.1,
    "max_samples": 0.9
  }
}
```
```

--------------------------------

### Get Metadata Routing in gplearn

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/reference.md

Retrieves the metadata routing configuration for the gplearn object. This allows for understanding how metadata is handled during the fitting process.

```python
get_metadata_routing()

  Get metadata routing of this object.

  Please check User Guide on how the routing
  mechanism works.

  * **Returns:**
    **routing**
    : A `MetadataRequest` encapsulating
      routing information.
```

--------------------------------

### Get Parameters for gplearn Estimator

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/reference.md

Retrieves the parameters of the gplearn estimator. If `deep` is True, it also returns parameters for contained sub-objects that are estimators.

```python
get_params(deep=True)

  Get parameters for this estimator.

  * **Parameters:**
    **deep**
    : If True, will return the parameters for this estimator and
      contained subobjects that are estimators.
  * **Returns:**
    **params**
    : Parameter names mapped to their values.
```

--------------------------------

### Transform Data and Concatenate Features

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Applies the trained SymbolicTransformer to the entire dataset to generate new features and then horizontally stacks these new features with the original data.

```default
gp_features = gp.transform(diabetes.data)
new_diabetes = np.hstack((diabetes.data, gp_features))
```

--------------------------------

### gplearn Configuration Parameters

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/reference.md

Configuration options for the gplearn genetic programming process. Includes parameters for sample selection, class weighting, feature naming, warm-starting, memory management, parallel processing, verbosity, and random state.

```python
max_samples
  : The fraction of samples to draw from X to evaluate each program on.

  class_weight
  : Weights associated with classes in the form `{class_label: weight}`.
    If not given, all classes are supposed to have weight one.
    The “balanced” mode uses the values of y to automatically adjust
    weights inversely proportional to class frequencies in the input data
    as `n_samples / (n_classes * np.bincount(y))`

  feature_names
  : Optional list of feature names, used purely for representations in
    the print operation or export_graphviz. If None, then X0, X1, etc
    will be used for representations.

  warm_start
  : When set to `True`, reuse the solution of the previous call to fit
    and add more generations to the evolution, otherwise, just fit a new
    evolution.

  low_memory
  : When set to `True`, only the current generation is retained. Parent
    information is discarded. For very large populations or runs with many
    generations, this can result in substantial memory use reduction.

  n_jobs
  : The number of jobs to run in parallel for fit. If -1, then the number
    of jobs is set to the number of cores.

  verbose
  : Controls the verbosity of the evolution building process.

  random_state
  : If int, random_state is the seed used by the random number generator;
    If RandomState instance, random_state is the random number generator;
    If None, the random number generator is the RandomState instance used
    by np.random. 
```

--------------------------------

### Export and Load gplearn Model with Pickle

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/advanced.md

This section shows how to save a trained gplearn model to a file using Python's `pickle` library and how to load it back later for continued use or prediction. It also includes an option to reduce file size by removing evolution history.

```python
import pickle

est = SymbolicRegressor()
est.fit(X_train, y_train)

delattr(est, '_programs')

with open('gp_model.pkl', 'wb') as f:
    pickle.dump(est, f)

with open('gp_model.pkl', 'rb') as f:
    est = pickle.load(f)
```

--------------------------------

### Train Tree-Based Regressors

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Initializes and trains a DecisionTreeRegressor and a RandomForestRegressor from scikit-learn. These models are trained on the same training data (X_train, y_train) to serve as benchmarks for comparison with the SymbolicRegressor.

```python
est_tree = DecisionTreeRegressor()
est_tree.fit(X_train, y_train)
est_rf = RandomForestRegressor()
est_rf.fit(X_train, y_train)
```

--------------------------------

### gplearn Function Set Definition

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/intro.md

Illustrates how to define and use custom functions within gplearn. The `make_function` factory allows for the creation of GP-compatible function nodes that can be incorporated into programs.

```python
from gplearn.functions import make_function

# Define a custom function (e.g., a simple power function)
def custom_power(x1, x2):
    return x1 ** x2

# Create a GP-compatible function node
# 'power' is the name, custom_power is the function, 2 is the arity
pow_func = make_function(name='power', func=custom_power, arity=2)

# Example of using the custom function in a program (conceptual)
# program = [pow_func, 2, 3] # Represents 2 ** 3

print(pow_func(2, 3)) # Example of direct function call

```

--------------------------------

### Load and Shuffle Diabetes Dataset

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Loads the Diabetes dataset and shuffles it using a provided random state. This is a common preprocessing step for machine learning tasks to ensure data randomness.

```default
rng = check_random_state(0)
diabetes = load_diabetes()
perm = rng.permutation(diabetes.target.size)
diabetes.data = diabetes.data[perm]
diabetes.target = diabetes.target[perm]
```

--------------------------------

### Train SymbolicRegressor

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Initializes and trains a SymbolicRegressor from gplearn. The regressor is configured with specific parameters like population size, generations, stopping criteria, crossover and mutation probabilities, sample subsampling, verbosity, parsimony coefficient, and a random state for reproducibility. It fits the regressor to the training data (X_train, y_train).

```python
est_gp = SymbolicRegressor(population_size=5000,
                           generations=20, stopping_criteria=0.01,
                           p_crossover=0.7, p_subtree_mutation=0.1,
                           p_hoist_mutation=0.05, p_point_mutation=0.1,
                           max_samples=0.9, verbose=1,
                           parsimony_coefficient=0.01, random_state=0)
est_gp.fit(X_train, y_train)
```

--------------------------------

### Train and Score Ridge Regressor

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Trains a Ridge Regression model on the first 300 samples of the Diabetes dataset and evaluates its performance on the remaining 200 samples. This serves as a benchmark score.

```default
est = Ridge()
est.fit(diabetes.data[:300, :], diabetes.target[:300])
print(est.score(diabetes.data[300:, :], diabetes.target[300:]))

0.43405742105789413
```

--------------------------------

### Visualize Program Parents with Fade Nodes

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/advanced.md

This snippet demonstrates how to visualize a parent program in gplearn, highlighting specific nodes that were altered during mutation. It utilizes the `export_graphviz` method and `pydotplus` for visualization.

```python
print(est_gp._program.parents)

{'parent_idx': 75, 'parent_nodes': [1, 10], 'method': 'Point Mutation'}

idx = est_gp._program.parents['parent_idx']
fade_nodes = est_gp._program.parents['parent_nodes']
print(est_gp._programs[-2][idx])
graph = est_gp._programs[-2][idx].export_graphviz(fade_nodes=fade_nodes)
graph = pydotplus.graphviz.graph_from_dot_data(graph)
Image(graph.create_png())
```

--------------------------------

### Create Random Training Data for Symbolic Regression (Python)

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

This snippet shows how to generate random training data points that lie on a predefined symbolic relationship. It uses `check_random_state` for reproducibility and `numpy.random.uniform` to create input features. The output `y_train` is calculated based on the known relationship.

```python
rng = check_random_state(0)

# Training samples
X_train = rng.uniform(-1, 1, 100).reshape(50, 2)
y_train = X_train[:, 0]**2 - X_train[:, 1]**2 + X_train[:, 1] - 1
```

--------------------------------

### Visualize Donor Parent Program in SymbolicRegressor

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/gp_examples.ipynb

Retrieves a specific parent program from the SymbolicRegressor's history based on the 'donor_idx' from the parent information. It then prints the parent program and its fitness, and visualizes it using Graphviz.

```python
idx = est_gp._program.parents['donor_idx']
fade_nodes = est_gp._program.parents['donor_nodes']
print(est_gp._programs[-2][idx])
print('Fitness:', est_gp._programs[-2][idx].fitness_)
dot_data = est_gp._programs[-2][idx].export_graphviz(fade_nodes=fade_nodes)
graph = graphviz.Source(dot_data)
graph
```

--------------------------------

### Visualize Program Tree with gplearn's export_graphviz

Source: https://context7.com/trevorstephens/gplearn/llms.txt

This snippet demonstrates how to visualize the tree structure of an evolved genetic program using the `export_graphviz` method. It generates a Graphviz DOT script that can be rendered into an image, allowing for interpretation of the discovered mathematical relationships. It also shows how to access parent information for tracing evolutionary steps and visualizing parent programs with faded nodes.

```python
import numpy as np
from gplearn.genetic import SymbolicRegressor
import graphviz

# Train a simple regressor
X = np.random.randn(100, 2)
y = X[:, 0] ** 2 - X[:, 1] + 0.5

est = SymbolicRegressor(
    population_size=500,
    generations=10,
    feature_names=['height', 'weight'],
    random_state=0
)
est.fit(X, y)

# Export to Graphviz format
dot_data = est._program.export_graphviz()
print(dot_data)

# Visualize with graphviz library
graph = graphviz.Source(dot_data)
graph.render('program_tree', format='png')  # Saves program_tree.png

# View parent information for evolution tracing
print(est._program.parents)

# Visualize parent with faded crossover nodes
if est._program.parents:
    parent_idx = est._program.parents.get('parent_idx')
    if parent_idx is not None:
        parent = est._programs[-2][parent_idx]
        fade_nodes = est._program.parents.get('parent_nodes', [])
        parent_dot = parent.export_graphviz(fade_nodes=fade_nodes)
```

--------------------------------

### Train Ridge Regressor on Transformed Data

Source: https://github.com/trevorstephens/gplearn/blob/main/doc/examples.md

Trains a Ridge Regression model on the first 300 samples of the dataset augmented with new symbolic features. It then evaluates the performance on the unseen final 200 samples.

```default
est = Ridge()
est.fit(new_diabetes[:300, :], diabetes.target[:300])
print(est.score(new_diabetes[300:, :], diabetes.target[300:]))

0.5336788517320445
```