### Install StellarGraph from Source with Demo Dependencies Source: https://stellargraph.readthedocs.io/en/stable/README.html Install StellarGraph from the local source code along with additional dependencies required for the demos. This is useful when working with the source code and wanting to run examples. ```bash pip install .[demos] ``` -------------------------------- ### Install StellarGraph from Source Source: https://stellargraph.readthedocs.io/en/stable/README.html After cloning the repository, navigate to the directory and install StellarGraph using pip. This command installs the library from the local source code. ```bash cd stellargraph pip install . ``` -------------------------------- ### Install StellarGraph Demos with Pip Source: https://stellargraph.readthedocs.io/en/stable/demos/index.html Install the necessary dependencies to run most StellarGraph demo notebooks locally using pip. ```bash pip install stellargraph[demos] ``` -------------------------------- ### Training Output Example Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/keras-node2vec-embeddings.ipynb Example output showing the training progress, including steps per epoch, loss, and binary accuracy for each epoch. ```text Train for 39760 steps Epoch 1/2 39760/39760 [==============================] - 155s 4ms/step - loss: 0.2924 - binary_accuracy: 0.8557 Epoch 2/2 39760/39760 [==============================] - 238s 6ms/step - loss: 0.1096 - binary_accuracy: 0.9641 ``` -------------------------------- ### Create Labeled Link Examples Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/ctdne-link-prediction.html Combines positive and negative link examples into a single dataset with corresponding binary labels. ```python def labelled_links(positive_examples, negative_examples): return ( positive_examples + negative_examples, np.repeat([1, 0], [len(positive_examples), len(negative_examples)]), ) link_examples, link_labels = labelled_links(pos, neg) link_examples_test, link_labels_test = labelled_links(pos_test, neg_test) ``` -------------------------------- ### Install StellarGraph using PyPI Source: https://stellargraph.readthedocs.io/en/stable/README.html Use this command to install the core StellarGraph library from the Python Package Index. ```bash pip install stellargraph ``` -------------------------------- ### Install StellarGraph Demos with Conda Source: https://stellargraph.readthedocs.io/en/stable/demos/index.html Install the necessary dependencies to run most StellarGraph demo notebooks locally using conda. ```bash conda install -c stellargraph stellargraph ``` -------------------------------- ### Generate and Verify Link Examples Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/ctdne-link-prediction.html This code generates training and testing link examples using the previously defined functions and then prints the graph information along with the counts of positive and negative links for both sets. It's useful for verifying the data generation process. ```python pos, neg = positive_and_negative_links(graph, edges_train) pos_test, neg_test = positive_and_negative_links(graph, edges_test) print( f"{graph.info()}\n" f"Training examples: {len(pos)} positive links, {len(neg)} negative links\n" f"Test examples: {len(pos_test)} positive links, {len(neg_test)} negative links" ) ``` -------------------------------- ### Attri2Vec Training Output Example Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/attri2vec-link-prediction.html Example output showing the loss and binary accuracy for each epoch during Attri2Vec model training. ```text Epoch 1/6 4711/4711 - 194s - loss: 0.7453 - binary_accuracy: 0.5219 Epoch 2/6 4711/4711 - 194s - loss: 0.6641 - binary_accuracy: 0.5677 Epoch 3/6 4711/4711 - 201s - loss: 0.5845 - binary_accuracy: 0.6375 Epoch 4/6 4711/4711 - 195s - loss: 0.5185 - binary_accuracy: 0.7198 Epoch 5/6 4711/4711 - 194s - loss: 0.4662 - binary_accuracy: 0.7767 Epoch 6/6 4711/4711 - 194s - loss: 0.4220 - binary_accuracy: 0.8110 ``` -------------------------------- ### Setup Callbacks for Early Stopping and Checkpointing Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gat-node-classification.ipynb Configure callbacks for early stopping based on validation accuracy and for saving the best model weights. Ensure the 'logs' directory exists. ```python from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint if not os.path.isdir("logs"): os.makedirs("logs") es_callback = EarlyStopping( monitor="val_acc", patience=20 ) # patience is the number of epochs to wait before early stopping in case of no further improvement mc_callback = ModelCheckpoint( "logs/best_model.h5", monitor="val_acc", save_best_only=True, save_weights_only=True ) ``` -------------------------------- ### Install StellarGraph with igraph for Community Detection Source: https://stellargraph.readthedocs.io/en/stable/README.html Install StellarGraph, demo dependencies, and the python-igraph library, which is necessary for community detection demos and may have platform-specific availability. ```bash pip install stellargraph[demos,igraph] ``` -------------------------------- ### APPNP Training Output Logs Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/ppnp-node-classification.ipynb Example output showing the training progress and validation metrics over multiple epochs. ```text Using GCN (local pooling) filters... ['...'] ['...'] Train for 1 steps, validate for 1 steps Epoch 1/120 1/1 - 1s - loss: 2.1611 - acc: 0.1571 - val_loss: 2.0960 - val_acc: 0.3500 Epoch 2/120 1/1 - 0s - loss: 2.0830 - acc: 0.3214 - val_loss: 2.0560 - val_acc: 0.3780 Epoch 3/120 1/1 - 0s - loss: 2.0771 - acc: 0.2214 - val_loss: 2.0171 - val_acc: 0.3200 Epoch 4/120 1/1 - 0s - loss: 2.0152 - acc: 0.3286 - val_loss: 1.9767 - val_acc: 0.3040 Epoch 5/120 1/1 - 0s - loss: 1.9555 - acc: 0.3143 - val_loss: 1.9280 - val_acc: 0.3040 Epoch 6/120 1/1 - 0s - loss: 1.9276 - acc: 0.3357 - val_loss: 1.8699 - val_acc: 0.3160 Epoch 7/120 1/1 - 0s - loss: 1.9307 - acc: 0.3500 - val_loss: 1.8084 - val_acc: 0.3820 Epoch 8/120 1/1 - 0s - loss: 1.8068 - acc: 0.4286 - val_loss: 1.7449 - val_acc: 0.5320 Epoch 9/120 1/1 - 0s - loss: 1.7419 - acc: 0.4357 - val_loss: 1.6791 - val_acc: 0.6180 Epoch 10/120 1/1 - 0s - loss: 1.7992 - acc: 0.4429 - val_loss: 1.6142 - val_acc: 0.6160 Epoch 11/120 1/1 - 0s - loss: 1.6373 - acc: 0.5429 - val_loss: 1.5286 - val_acc: 0.6260 Epoch 12/120 1/1 - 0s - loss: 1.6104 - acc: 0.5000 - val_loss: 1.4470 - val_acc: 0.6480 Epoch 13/120 1/1 - 0s - loss: 1.5940 - acc: 0.5000 - val_loss: 1.3990 - val_acc: 0.6360 Epoch 14/120 1/1 - 0s - loss: 1.6000 - acc: 0.5286 - val_loss: 1.3676 - val_acc: 0.6400 Epoch 15/120 1/1 - 0s - loss: 1.4582 - acc: 0.5786 - val_loss: 1.3376 - val_acc: 0.6620 Epoch 16/120 1/1 - 0s - loss: 1.4981 - acc: 0.5643 - val_loss: 1.3105 - val_acc: 0.7040 Epoch 17/120 1/1 - 0s - loss: 1.4196 - acc: 0.6500 - val_loss: 1.2935 - val_acc: 0.7060 Epoch 18/120 1/1 - 0s - loss: 1.4223 - acc: 0.6286 - val_loss: 1.2826 - val_acc: 0.7040 Epoch 19/120 1/1 - 0s - loss: 1.6010 - acc: 0.5786 - val_loss: 1.2728 - val_acc: 0.7060 Epoch 20/120 1/1 - 0s - loss: 1.4398 - acc: 0.7000 - val_loss: 1.2584 - val_acc: 0.7200 Epoch 21/120 1/1 - 0s - loss: 1.3107 - acc: 0.6786 - val_loss: 1.2481 - val_acc: 0.7240 Epoch 22/120 1/1 - 0s - loss: 1.3125 - acc: 0.6714 - val_loss: 1.2400 - val_acc: 0.7180 Epoch 23/120 1/1 - 0s - loss: 1.3205 - acc: 0.6929 - val_loss: 1.2298 - val_acc: 0.7120 Epoch 24/120 1/1 - 0s - loss: 1.1782 - acc: 0.7500 - val_loss: 1.2171 - val_acc: 0.7020 Epoch 25/120 1/1 - 0s - loss: 1.2335 - acc: 0.7286 - val_loss: 1.2071 - val_acc: 0.6980 Epoch 26/120 1/1 - 0s - loss: 1.2707 - acc: 0.6714 - val_loss: 1.1907 - val_acc: 0.6980 Epoch 27/120 1/1 - 0s - loss: 1.2500 - acc: 0.6643 - val_loss: 1.1814 - val_acc: 0.7020 Epoch 28/120 1/1 - 0s - loss: 1.1690 - acc: 0.7500 - val_loss: 1.1774 - val_acc: 0.7040 Epoch 29/120 1/1 - 0s - loss: 1.3786 - acc: 0.7214 - val_loss: 1.1625 - val_acc: 0.7240 Epoch 30/120 1/1 - 0s - loss: 1.2246 - acc: 0.7429 - val_loss: 1.1497 - val_acc: 0.7360 Epoch 31/120 1/1 - 0s - loss: 1.1109 - acc: 0.7929 - val_loss: 1.1388 - val_acc: 0.7440 Epoch 32/120 1/1 - 0s - loss: 1.0982 - acc: 0.7929 - val_loss: 1.1308 - val_acc: 0.7600 Epoch 33/120 ``` -------------------------------- ### Report Example Results Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/distmult-link-prediction.html Displays example evaluation results in a DataFrame format, similar to those reported in the DistMult paper for comparison. ```python results_as_dataframe(0.83, 0.942) ``` -------------------------------- ### Create a StellarGraph with tensor features Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-numpy.ipynb This example demonstrates constructing a StellarGraph where one node type has tensor features and another has vector features. The `info()` method confirms the feature shape. ```python square_foo_tensors_and_bar = StellarGraph( {"foo": square_foo_tensors, "bar": square_bar}, square_edges ) print(square_foo_tensors_and_bar.info()) ``` -------------------------------- ### Import StellarGraph and Dependencies Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/graphsage-node-classification.ipynb Imports necessary libraries for graph analysis, machine learning, and data manipulation. Ensure these are installed before running. ```python import networkx as nx import pandas as pd import os import stellargraph as sg from stellargraph.mapper import GraphSAGENodeGenerator from stellargraph.layer import GraphSAGE from tensorflow.keras import layers, optimizers, losses, metrics, Model from sklearn import preprocessing, feature_extraction, model_selection from stellargraph import datasets from IPython.display import display, HTML import matplotlib.pyplot as plt %matplotlib inline ``` -------------------------------- ### Clone StellarGraph Repository Source: https://stellargraph.readthedocs.io/en/stable/README.html Clone the official StellarGraph GitHub repository to your local machine. This is the first step for installing from source. ```bash git clone https://github.com/stellargraph/stellargraph.git ``` -------------------------------- ### Run Static Random Walks Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/ctdne-link-prediction.html Initialize and run static random walks using StellarGraph's BiasedRandomWalk. Specify the nodes to start walks from, the number of walks per node, and the length of each walk. ```python from stellargraph.data import BiasedRandomWalk static_rw = BiasedRandomWalk(graph) static_walks = static_rw.run( nodes=graph.nodes(), n=num_walks_per_node, length=walk_length ) print("Number of static random walks: {}".format(len(static_walks))) ``` -------------------------------- ### Create a StellarGraph with multiple node types Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-numpy.ipynb Construct a StellarGraph by combining IndexedArrays for different node types and defining edges. This example includes node types with and without features. ```python square_foo_and_bar = StellarGraph({"foo": square_foo, "bar": square_bar}, square_edges) print(square_foo_and_bar.info()) ``` -------------------------------- ### Define and compile a GCN model without pre-training Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb Creates and compiles a GCN model from scratch for supervised node classification. It uses the same configuration as the pre-trained model but starts with random weights. ```python direct_gcn_model = make_gcn_model() direct_x_in, direct_x_out = direct_gcn_model.in_out_tensors() direct_predictions = tf.keras.layers.Dense( units=train_targets.shape[1], activation="softmax" )(direct_x_out) direct_model = Model(inputs=direct_x_in, outputs=direct_predictions) direct_model.compile( optimizer=optimizers.Adam(lr=0.01), loss="categorical_crossentropy", metrics=["acc"], ) ``` -------------------------------- ### Sample Positive and Negative Links Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/ctdne-link-prediction.html Use this function to generate positive and negative link examples from a graph and a set of edges. It requires the graph object and the edges designated for the train/test split. Ensure `random` is imported. ```python def positive_and_negative_links(g, edges): pos = list(edges["source", "target"].itertuples(index=False)) neg = sample_negative_examples(g, pos) return pos, neg def sample_negative_examples(g, positive_examples): positive_set = set(positive_examples) def valid_neg_edge(src, tgt): return ( # no self-loops src != tgt and # neither direction of the edge should be a positive one (src, tgt) not in positive_set and (tgt, src) not in positive_set ) possible_neg_edges = [ (src, tgt) for src in g.nodes() for tgt in g.nodes() if valid_neg_edge(src, tgt) ] return random.sample(possible_neg_edges, k=len(positive_examples)) ``` -------------------------------- ### Set training parameters for HinSAGE model Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/hinsage-link-prediction.html Configures the minibatch size, number of training epochs, and the split ratio for training and testing data. This setup is crucial for controlling the training process and evaluating model performance. ```python batch_size = 200 epochs = 20 # Use 70% of edges for training, the rest for testing: train_size = 0.7 test_size = 0.3 ``` -------------------------------- ### Initialize DistMult Model and Compile Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/distmult-link-prediction.html Initializes the KGTripleGenerator and DistMult model, then compiles the model for training. Ensure embedding_dimension is set. ```python wn18_gen = KGTripleGenerator( wn18_graph, batch_size=len(wn18_train) // 10 # ~10 batches per epoch ) wn18_distmult = DistMult( wn18_gen, embedding_dimension=embedding_dimension, embeddings_regularizer=regularizers.l2(1e-7), ) wn18_inp, wn18_out = wn18_distmult.in_out_tensors() wn18_model = Model(inputs=wn18_inp, outputs=wn18_out) wn18_model.compile( optimizer=optimizers.Adam(lr=0.001), loss=losses.BinaryCrossentropy(from_logits=True), metrics=[metrics.BinaryAccuracy(threshold=0.0)], ) ``` -------------------------------- ### Importing Required Libraries Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/ppnp-node-classification.ipynb Initializes the environment by importing necessary libraries including TensorFlow, StellarGraph, and scikit-learn. ```python import networkx as nx import pandas as pd import numpy as np import os from tensorflow import keras from tensorflow.keras import backend as K from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint import stellargraph as sg from stellargraph.mapper import FullBatchNodeGenerator from stellargraph.layer.ppnp import PPNP from stellargraph.layer.appnp import APPNP from tensorflow.keras import layers, optimizers, losses, metrics, Model from sklearn import preprocessing, feature_extraction, model_selection from stellargraph import datasets from IPython.display import display, HTML import matplotlib.pyplot as plt %matplotlib inline ``` -------------------------------- ### Get Label Count Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/graphsage-inductive-node-classification.ipynb Returns the total number of labels in the dataset. ```python len(labels) ``` -------------------------------- ### Load the Blog Catalog dataset Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/metapath2vec-link-prediction.ipynb Loads the heterogeneous Blog Catalog 3 dataset and displays its description. ```python dataset = datasets.BlogCatalog3() display(HTML(dataset.description)) graph = dataset.load() ``` -------------------------------- ### Get Prediction Shape Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/cluster-gcn-node-classification.ipynb Check the shape of the generated predictions. The output should be (number_of_nodes, number_of_classes). ```python all_predictions.shape ``` -------------------------------- ### Get Sampled Label Count Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/graphsage-inductive-node-classification.ipynb Returns the count of labels in the sampled training set. ```python len(labels_sampled) ``` -------------------------------- ### Import required libraries Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gat-node-classification.ipynb Initializes the environment by importing necessary libraries for graph processing, machine learning, and visualization. ```python import networkx as nx import pandas as pd import os import stellargraph as sg from stellargraph.mapper import FullBatchNodeGenerator from stellargraph.layer import GAT from tensorflow.keras import layers, optimizers, losses, metrics, Model from sklearn import preprocessing, feature_extraction, model_selection from stellargraph import datasets from IPython.display import display, HTML import matplotlib.pyplot as plt %matplotlib inline ``` -------------------------------- ### Get Node IDs from Subgraph Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/attri2vec-link-prediction.html Retrieves and sorts the node identifiers from a NetworkX graph. ```python subgraph_node_ids = sorted(list(G_sub_nx.nodes)) ``` -------------------------------- ### Initialize Neo4j connection Source: https://stellargraph.readthedocs.io/en/stable/demos/connector/neo4j/load-cora-into-neo4j.ipynb Sets up the py2neo Graph object using environment variables. ```python import time import py2neo default_host = os.environ.get("STELLARGRAPH_NEO4J_HOST") # Create the Neo4j Graph database object; the arguments can be edited to specify location and authentication graph = py2neo.Graph(host=default_host, port=None, user=None, password=None) ``` -------------------------------- ### Split Graph for Testing Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/homogeneous-comparison-link-prediction.ipynb Performs an initial train/test split on a graph to isolate test examples. ```python graph_test, examples_test, labels_test = edge_splitter_test.train_test_split( p=0.1, method="global" ) print(graph_test.info()) ``` -------------------------------- ### Import StellarGraph Library Source: https://stellargraph.readthedocs.io/en/stable/demos/time-series/gcn-lstm-time-series.ipynb Initializes the StellarGraph library for data loading. ```python import stellargraph as sg ``` -------------------------------- ### Displaying Cora citation edges Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-pandas.ipynb Example format of the cora.cites TSV file showing citation relationships. ```text 35 1033 35 103482 35 103515 ... ``` -------------------------------- ### Initialize and Compile ComplEx Model Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/complex-link-prediction.html Sets up the KGTripleGenerator and ComplEx model, then compiles it with Keras optimizers and loss functions. ```python wn18_gen = KGTripleGenerator( wn18_graph, batch_size=len(wn18_train) // 100 # ~100 batches per epoch ) wn18_complex = ComplEx( wn18_gen, embedding_dimension=embedding_dimension, embeddings_regularizer=regularizers.l2(1e-7), ) wn18_inp, wn18_out = wn18_complex.in_out_tensors() wn18_model = Model(inputs=wn18_inp, outputs=wn18_out) wn18_model.compile( optimizer=optimizers.Adam(lr=0.001), loss=losses.BinaryCrossentropy(from_logits=True), metrics=[metrics.BinaryAccuracy(threshold=0.0)], ) ``` -------------------------------- ### Get Number of Nodes Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/cluster-gcn-node-classification.ipynb Determine the total number of nodes processed by the generator, which corresponds to the number of predictions made. ```python len(all_gen.node_order) ``` -------------------------------- ### Configure Training Callbacks Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/sgc-node-classification.ipynb Sets up early stopping and model checkpointing to save the best weights during training. ```python from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint if not os.path.isdir("logs"): os.makedirs("logs") es_callback = EarlyStopping( monitor="val_acc", patience=50 ) # patience is the number of epochs to wait before early stopping in case of no further improvement mc_callback = ModelCheckpoint( "logs/best_model.h5", monitor="val_acc", save_best_only=True, save_weights_only=True ) ``` -------------------------------- ### Initialize KGTripleGenerator and DistMult Model Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/distmult-link-prediction.html Sets up a KGTripleGenerator for batching graph data and initializes a DistMult model for knowledge graph embeddings. The model is configured with a specified embedding dimension and L2 regularization. ```python fb15k_gen = KGTripleGenerator( fb15k_graph, batch_size=len(fb15k_train) // 10 # ~100 batches per epoch ) fb15k_distmult = DistMult( fb15k_gen, embedding_dimension=embedding_dimension, embeddings_regularizer=regularizers.l2(1e-8), ) fb15k_inp, fb15k_out = fb15k_distmult.in_out_tensors() ``` -------------------------------- ### Create and Fit Model with Data Generator Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/watch-your-step-embeddings.ipynb Instantiate a data generator and fit the model. Ensure the `steps_per_epoch` is correctly calculated based on the graph size and batch size. ```python batch_size = 10 train_gen = generator.flow(batch_size=batch_size, num_parallel_calls=10) history = model.fit( train_gen, epochs=epochs, verbose=1, steps_per_epoch=int(len(G.nodes()) // batch_size) ) ``` -------------------------------- ### Get Model Input and Output Tensors Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/keras-node2vec-embeddings.ipynb Retrieve the input and output tensors for the Node2Vec model, which will be used to build the Keras model. ```python x_inp, x_out = node2vec.in_out_tensors() ``` -------------------------------- ### Displaying Cora node content Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-pandas.ipynb Example format of the cora.content TSV file containing node IDs, word vectors, and subject classes. ```text 31336 0 0 ... 0 1 0 0 0 0 0 0 Neural_Networks 1061127 0 0 ... 1 0 0 0 0 0 0 0 Rule_Learning 1106406 0 0 ... 0 0 0 0 0 0 0 0 Reinforcement_Learning ... ``` -------------------------------- ### Initialize StellarGraph from edges Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-pandas.ipynb Construct a basic StellarGraph instance using a DataFrame of edges. ```python square = StellarGraph(edges=square_edges) ``` -------------------------------- ### Get Node Features for Subgraph Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/attri2vec-link-prediction.html Selects and reindexes node features based on the sorted node IDs of the subgraph. Requires `node_data` and `feature_names` to be pre-defined. ```python subgraph_node_features = node_data[feature_names].reindex(subgraph_node_ids) ``` -------------------------------- ### Identify GAT Embedding Layer Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gat-node-classification.ipynb Finds the first Keras layer starting with 'graph_attention' to use as the embedding layer. Preserves previously trained weights. ```python emb_layer = next(l for l in model.layers if l.name.startswith("graph_attention")) print( "Embedding layer: {}, output shape {}".format(emb_layer.name, emb_layer.output_shape) ) ``` -------------------------------- ### Download and prepare BlogCatalog3 dataset Source: https://stellargraph.readthedocs.io/en/stable/demos/zzz-internal-developers/graph-resource-usage.ipynb Downloads the BlogCatalog3 dataset and defines the file paths for edges, group edges, groups, and nodes. This prepares the data for loading. ```python blogcatalog3 = sg.datasets.BlogCatalog3() blogcatalog3.download() blogcatalog3_edges = os.path.join(blogcatalog3.data_directory, "edges.csv") blogcatalog3_group_edges = os.path.join(blogcatalog3.data_directory, "group-edges.csv") blogcatalog3_groups = os.path.join(blogcatalog3.data_directory, "groups.csv") blogcatalog3_nodes = os.path.join(blogcatalog3.data_directory, "nodes.csv") ``` -------------------------------- ### Import required libraries Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-node-classification.ipynb Initializes the environment by importing StellarGraph, TensorFlow Keras, and standard data science libraries. ```python import pandas as pd import os import stellargraph as sg from stellargraph.mapper import FullBatchNodeGenerator from stellargraph.layer import GCN from tensorflow.keras import layers, optimizers, losses, metrics, Model from sklearn import preprocessing, model_selection from IPython.display import display, HTML import matplotlib.pyplot as plt %matplotlib inline ``` -------------------------------- ### Import Libraries for Node2Vec Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/keras-node2vec-embeddings.ipynb Imports necessary libraries for Node2Vec representation learning, including Keras, NetworkX, Pandas, and Stellargraph components. Ensure these are installed before running. ```python import matplotlib.pyplot as plt from sklearn.manifold import TSNE import os import networkx as nx import numpy as np import pandas as pd from tensorflow import keras from stellargraph import StellarGraph from stellargraph.data import BiasedRandomWalk from stellargraph.data import UnsupervisedSampler from stellargraph.data import BiasedRandomWalk from stellargraph.mapper import Node2VecLinkGenerator, Node2VecNodeGenerator from stellargraph.layer import Node2Vec, link_classification from stellargraph import datasets from IPython.display import display, HTML %matplotlib inline ``` -------------------------------- ### Import NetworkX Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-networkx.ipynb Import the NetworkX library for graph creation. ```python import networkx as nx ``` -------------------------------- ### Filter Nodes with Cypher WHERE Clause Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-saving-neo4j.ipynb Use a Cypher WHERE clause to select nodes based on their properties. This example filters nodes where 'left' is true or 'top' is false. ```python raw_subgraph_nodes = neo4j_graph.run( """ MATCH (n) WHERE n.left OR NOT n.top RETURN n.name AS name, n.left, n.top """ ).to_data_frame() subgraph_nodes = raw_subgraph_nodes.set_index("name") subgraph_nodes ``` -------------------------------- ### Initialize UnsupervisedSampler Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/graphsage-unsupervised-sampler-embeddings.ipynb Create an instance of UnsupervisedSampler using the graph and configured parameters. ```python unsupervised_samples = UnsupervisedSampler( G, nodes=nodes, length=length, number_of_walks=number_of_walks ) ``` -------------------------------- ### Extract k-hop nodes using Cypher Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-saving-neo4j.ipynb Uses a variable length relationship constraint to retrieve nodes within a specified hop distance from start nodes. ```python start_nodes = ["b"] raw_hop_nodes = neo4j_graph.run( """ MATCH (start) -[*0..1]- (n) WHERE start.name IN $start_nodes WITH DISTINCT n RETURN n.name AS name, n.top, n.left """, {"start_nodes": start_nodes}, ).to_data_frame() hop_nodes = raw_hop_nodes.set_index("name") hop_nodes ``` -------------------------------- ### Train Link Prediction Classifier Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/node2vec-link-prediction.ipynb Trains a logistic regression classifier for link prediction. Requires pre-defined link examples, labels, an embedding function, and a binary operator. ```python def train_link_prediction_model( link_examples, link_labels, get_embedding, binary_operator ): clf = link_prediction_classifier() link_features = link_examples_to_features( link_examples, get_embedding, binary_operator ) clf.fit(link_features, link_labels) return clf ``` ```python def link_prediction_classifier(max_iter=2000): lr_clf = LogisticRegressionCV(Cs=10, cv=10, scoring="roc_auc", max_iter=max_iter) return Pipeline(steps=[("sc", StandardScaler()), ("clf", lr_clf)]) ``` -------------------------------- ### Define Link Prediction Classifier Helpers Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/ctdne-link-prediction.html Provides functions to transform link examples into features, initialize a logistic regression pipeline, and evaluate performance using ROC AUC. ```python from sklearn.pipeline import Pipeline from sklearn.linear_model import LogisticRegressionCV from sklearn.metrics import roc_auc_score from sklearn.preprocessing import StandardScaler def link_examples_to_features(link_examples, transform_node): op_func = ( operator_func[binary_operator] if isinstance(binary_operator, str) else binary_operator ) return [ op_func(transform_node(src), transform_node(dst)) for src, dst in link_examples ] def link_prediction_classifier(max_iter=2000): lr_clf = LogisticRegressionCV(Cs=10, cv=10, scoring="roc_auc", max_iter=max_iter) return Pipeline(steps=[("sc", StandardScaler()), ("clf", lr_clf)]) def evaluate_roc_auc(clf, link_features, link_labels): predicted = clf.predict_proba(link_features) # check which class corresponds to positive links positive_column = list(clf.classes_).index(1) return roc_auc_score(link_labels, predicted[:, positive_column]) ``` -------------------------------- ### Initialize StellarGraph with Node and Edge Features Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-pandas.ipynb Constructs a StellarGraph with both node and edge features defined in their respective DataFrames. ```python square_named_features = StellarGraph( {"corner": square_node_data}, {"line": square_edge_data} ) print(square_named_features.info()) ``` -------------------------------- ### Compute Node Features from Node ID Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-networkx.ipynb Computes numerical features for nodes based on their ID. This is a basic example; real-world scenarios might involve more complex feature calculations. ```python g_feature_attr = g.copy() def compute_features(node_id): # in general this could compute something based on other features, but for this example, # we don't have any other features, so we'll just do something basic with the node_id return [ord(node_id), len(node_id)] for node_id, node_data in g_feature_attr.nodes(data=True): node_data["feature"] = compute_features(node_id) # let's see what some of them look like: g_feature_attr.nodes["a"], g_feature_attr.nodes["c"] ``` -------------------------------- ### Import required libraries Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/cluster-gcn-node-classification.ipynb Initializes the environment by importing necessary libraries for data manipulation, graph processing, and model building. ```python import networkx as nx import pandas as pd import itertools import json import os import numpy as np from networkx.readwrite import json_graph from sklearn.preprocessing import StandardScaler import stellargraph as sg from stellargraph.mapper import ClusterNodeGenerator from stellargraph.layer import GCN from stellargraph import globalvar from tensorflow.keras import backend as K from tensorflow.keras import layers, optimizers, losses, metrics, Model from sklearn import preprocessing, feature_extraction, model_selection from stellargraph import datasets from IPython.display import display, HTML from IPython.display import display, HTML import matplotlib.pyplot as plt %matplotlib inline ``` -------------------------------- ### Configure and Train APPNP Model Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/ppnp-node-classification.ipynb Initializes the FullBatchNodeGenerator with GCN method, defines the APPNP model architecture, and executes the training process with early stopping and model checkpointing. ```python generator = FullBatchNodeGenerator(G, method="gcn", sparse=True) train_gen = generator.flow(train_subjects.index, train_targets) val_gen = generator.flow(val_subjects.index, val_targets) test_gen = generator.flow(test_subjects.index, test_targets) appnp = APPNP( layer_sizes=[64, 64, train_targets.shape[-1]], activations=["relu", "relu", "relu"], bias=True, generator=generator, teleport_probability=0.1, dropout=0.5, kernel_regularizer=keras.regularizers.l2(0.001), ) x_inp, x_out = appnp.in_out_tensors() predictions = keras.layers.Softmax()(x_out) appnp_model = keras.models.Model(inputs=x_inp, outputs=predictions) appnp_model.compile( loss="categorical_crossentropy", metrics=["acc"], optimizer=keras.optimizers.Adam(lr=0.01), ) es_callback = EarlyStopping( monitor="val_acc", patience=50 ) # patience is the number of epochs to wait before early stopping in case of no further improvement mc_callback = ModelCheckpoint( "logs/best_appnp_model.h5", monitor="val_acc", save_best_only=True, save_weights_only=True, ) history = appnp_model.fit( train_gen, epochs=120, validation_data=val_gen, verbose=2, shuffle=False, # this should be False, since shuffling data means shuffling the whole graph callbacks=[es_callback, mc_callback], ) ``` -------------------------------- ### Initialize FullBatchNodeGenerator for PPNP Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/ppnp-node-classification.ipynb Use `FullBatchNodeGenerator` with `method='ppnp'` to preprocess the adjacency matrix and supply the personalized page rank matrix. Ensure `sparse=False` for dense matrices and set `teleport_probability` for the propagation step. ```python generator = FullBatchNodeGenerator( G, method="ppnp", sparse=False, teleport_probability=0.1 ) ``` -------------------------------- ### Create a StellarGraph object Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-pandas.ipynb Initialize a StellarGraph instance using node features and edge data. ```python cora_no_subject = StellarGraph({"paper": cora_content_no_subject}, {"cites": cora_cites}) print(cora_no_subject.info()) ``` -------------------------------- ### Load the BlogCatalog3 dataset Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/metapath2vec-embeddings.ipynb Load the graph dataset and display its basic properties. ```python dataset = datasets.BlogCatalog3() display(HTML(dataset.description)) g = dataset.load() print( "Number of nodes {} and number of edges {} in graph.".format( g.number_of_nodes(), g.number_of_edges() ) ) ``` -------------------------------- ### Create Unsupervised Sampler Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/keras-node2vec-embeddings.ipynb Initialize the UnsupervisedSampler with the graph, a list of nodes, and the configured biased random walker. ```python unsupervised_samples = UnsupervisedSampler(G, nodes=list(G.nodes()), walker=walker) ``` -------------------------------- ### Select Target Node and Get True/Predicted Labels Source: https://stellargraph.readthedocs.io/en/stable/demos/interpretability/gcn-sparse-node-link-importance.html Identify the target node for interpretation and retrieve its true class label and predicted scores. Ensure the generator is set up to flow graph nodes. ```python graph_nodes = list(G.nodes()) target_nid = 1109199 target_idx = graph_nodes.index(target_nid) y_true = all_targets[target_idx] # true class of the target node ``` ```python all_gen = generator.flow(graph_nodes) y_pred = model.predict(all_gen)[0, target_idx] class_of_interest = np.argmax(y_pred) print( "Selected node id: {}, \nTrue label: {}, \nPredicted scores: {}".format( target_nid, y_true, y_pred.round(2) ) ) ``` -------------------------------- ### Create Unsupervised Sampler Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/attri2vec-link-prediction.html Initializes an UnsupervisedSampler for Attri2Vec, which generates random walks on the graph for training. ```python unsupervised_samples = UnsupervisedSampler( G_sub, nodes=nodes, length=length, number_of_walks=number_of_walks ) ``` -------------------------------- ### Get Node Embeddings from Node IDs Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/keras-node2vec-embeddings.ipynb Generates node embeddings from node IDs using a Node2VecNodeGenerator and the previously defined embedding model. Ensure the Node2VecNodeGenerator is properly configured with the graph and batch size. ```python node_gen = Node2VecNodeGenerator(G, batch_size).flow(subjects.index) node_embeddings = embedding_model.predict(node_gen, workers=4, verbose=1) ``` -------------------------------- ### Compile and Train Keras Model Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/graphsage-inductive-node-classification.ipynb Configures the Keras model with an optimizer and loss function, then executes training. ```python model = Model(inputs=x_inp, outputs=prediction) model.compile( optimizer=optimizers.Adam(lr=0.005), loss=losses.categorical_crossentropy, metrics=["acc"], ) ``` ```python val_gen = generator.flow(val_labels.index, val_targets) ``` ```python history = model.fit( train_gen, epochs=15, validation_data=val_gen, verbose=0, shuffle=False ) ``` ```python sg.utils.plot_history(history) ``` -------------------------------- ### Initialize FullBatchNodeGenerator Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-node-classification.ipynb Creates a generator for the GCN algorithm using the normalized graph Laplacian matrix. ```python generator = FullBatchNodeGenerator(G, method="gcn") ``` -------------------------------- ### Helper Function for Link Feature Extraction Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/node2vec-link-prediction.ipynb Defines a helper function to convert link examples (pairs of nodes) into feature vectors. It applies a specified binary operator to the node embeddings of the source and destination nodes. ```python from sklearn.pipeline import Pipeline from sklearn.linear_model import LogisticRegressionCV from sklearn.metrics import roc_auc_score from sklearn.preprocessing import StandardScaler # 1. link embeddings def link_examples_to_features(link_examples, transform_node, binary_operator): return [ binary_operator(transform_node(src), transform_node(dst)) for src, dst in link_examples ] ``` -------------------------------- ### Create Link Generators Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/graphsage-link-prediction.html Initializes link generators for training and testing, and creates data flows. ```python train_gen = GraphSAGELinkGenerator(G_train, batch_size, num_samples) train_flow = train_gen.flow(edge_ids_train, edge_labels_train, shuffle=True) ``` ```python test_gen = GraphSAGELinkGenerator(G_test, batch_size, num_samples) test_flow = test_gen.flow(edge_ids_test, edge_labels_test) ``` -------------------------------- ### Create the training generator Source: https://stellargraph.readthedocs.io/en/stable/demos/interpretability/gcn-sparse-node-link-importance.html Prepares the training data flow for the GCN model. ```python train_gen = generator.flow(train_subjects.index, train_targets) ``` -------------------------------- ### Get Schema Type Adjacency List Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/hinsage-link-prediction.html Retrieves the type adjacency list from the generator's schema for the specified head node types and number of samples. This helps understand the graph structure relevant to the model. ```python generator.schema.type_adjacency_list(generator.head_node_types, len(num_samples)) ``` -------------------------------- ### Display model summary Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/hinsage-link-prediction.html Prints the architecture and layer details of the compiled model. ```python model.summary() ``` -------------------------------- ### Import necessary libraries for Node2Vec node classification Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/keras-node2vec-node-classification.ipynb Imports essential libraries for data manipulation, machine learning, and graph analysis, including Stellargraph components for Node2Vec and Keras for model implementation. Ensure these are installed before running. ```python import matplotlib.pyplot as plt from sklearn.manifold import TSNE from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegressionCV from sklearn.metrics import accuracy_score import os import networkx as nx import numpy as np import pandas as pd from tensorflow import keras from stellargraph import StellarGraph from stellargraph.data import BiasedRandomWalk from stellargraph.data import UnsupervisedSampler from stellargraph.data import BiasedRandomWalk from stellargraph.mapper import Node2VecLinkGenerator, Node2VecNodeGenerator from stellargraph.layer import Node2Vec, link_classification from stellargraph import datasets from IPython.display import display, HTML %matplotlib inline ``` -------------------------------- ### Initialize StellarGraph with Node Features Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-pandas.ipynb Constructs a StellarGraph object using a node feature DataFrame and an edge list. ```python square_node_features = StellarGraph(square_node_data, square_edges) print(square_node_features.info()) ``` -------------------------------- ### Download StellarGraph Demos Source: https://stellargraph.readthedocs.io/en/stable/demos/index.html Use this curl command to download a local copy of all StellarGraph demo notebooks. The archive is extracted and the master directory is stripped. ```bash curl -L https://github.com/stellargraph/stellargraph/archive/master.zip | tar -xz --strip=1 stellargraph-master/demos ``` -------------------------------- ### Initialize data generators Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gcn-deep-graph-infomax-fine-tuning-node-classification.ipynb Creates a FullBatchNodeGenerator and a CorruptedGenerator for unsupervised training. ```python fullbatch_generator = FullBatchNodeGenerator(G) corrupted_generator = CorruptedGenerator(fullbatch_generator) gen = corrupted_generator.flow(G.nodes()) ``` -------------------------------- ### Run Temporal Random Walks Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/ctdne-link-prediction.html Initialize and run temporal random walks using StellarGraph's TemporalRandomWalk. Specify the number of context windows, context window size, maximum walk length, and walk bias. ```python from stellargraph.data import TemporalRandomWalk temporal_rw = TemporalRandomWalk(graph) temporal_walks = temporal_rw.run( num_cw=num_cw, cw_size=context_window_size, max_walk_length=walk_length, walk_bias="exponential", ) print("Number of temporal random walks: {}".format(len(temporal_walks))) ``` -------------------------------- ### Train and Evaluate Model Source: https://stellargraph.readthedocs.io/en/stable/demos/time-series/gcn-lstm-time-series.ipynb Fit the model to training data and display the summary and performance metrics. ```python history = model.fit( trainX, trainY, epochs=100, batch_size=60, shuffle=True, verbose=0, validation_data=[testX, testY], ) ``` ```python model.summary() ``` ```python print( "Train loss: ", history.history["loss"][-1], "\nTest loss:", history.history["val_loss"][-1], ) ``` -------------------------------- ### Configure Early Stopping and Model Checkpointing Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/ppnp-node-classification.ipynb Set up callbacks for early stopping based on validation accuracy and for saving the best performing model weights during training. Patience determines the number of epochs to wait for improvement. ```python es_callback = EarlyStopping( monitor="val_acc", patience=50 ) # patience is the number of epochs to wait before early stopping in case of no further improvement mc_callback = ModelCheckpoint( "logs/best_fc_model.h5", monitor="val_acc", save_best_only=True, save_weights_only=True, ) ``` -------------------------------- ### Configure WatchYourStep model Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/watch-your-step-embeddings.ipynb Initializes the WatchYourStep layer with specified embedding dimensions and regularization. ```python wys = WatchYourStep( generator, num_walks=80, embedding_dimension=128, attention_regularizer=regularizers.l2(0.5), ) x_in, x_out = wys.in_out_tensors() ``` -------------------------------- ### Import required libraries Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/graphwave-embeddings.ipynb Load necessary libraries for graph manipulation, machine learning, and visualization. ```python import networkx as nx from stellargraph.mapper import GraphWaveGenerator from stellargraph import StellarGraph from sklearn.decomposition import PCA import numpy as np from matplotlib import pyplot as plt from scipy.sparse.linalg import eigs import tensorflow as tf from tensorflow.keras import backend as K ``` -------------------------------- ### Connect to Neo4j instance Source: https://stellargraph.readthedocs.io/en/stable/demos/basics/loading-saving-neo4j.ipynb Establish a connection to a running Neo4j database using the py2neo library. ```python import os import py2neo default_host = os.environ.get("STELLARGRAPH_NEO4J_HOST") # Create the Neo4j Graph database object; the parameters can be edited to specify location and authentication neo4j_graph = py2neo.Graph(host=default_host, port=None, user=None, password=None) ``` -------------------------------- ### Generate random walks Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/metapath2vec-embeddings.ipynb Initialize the random walker and generate the corpus of walks. ```python from stellargraph.data import UniformRandomMetaPathWalk # Create the random walker rw = UniformRandomMetaPathWalk(g) walks = rw.run( nodes=list(g.nodes()), # root nodes length=walk_length, # maximum length of a random walk n=1, # number of random walks per root node metapaths=metapaths, # the metapaths ) print("Number of random walks: {}".format(len(walks))) ``` -------------------------------- ### Initialize UnsupervisedSampler Source: https://stellargraph.readthedocs.io/en/stable/demos/embeddings/attri2vec-embeddings.ipynb Specify parameters for creating an UnsupervisedSampler instance, including root nodes, number of walks, and walk length. ```python nodes = list(G.nodes()) number_of_walks = 4 length = 5 ``` ```python unsupervised_samples = UnsupervisedSampler( G, nodes=nodes, length=length, number_of_walks=number_of_walks ) ``` -------------------------------- ### Create Data Generators for Training and Validation Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/distmult-link-prediction.html Creates data generators for training and validation using the KGTripleGenerator.flow method. The negative_samples parameter controls the ratio of negative to positive edges. ```python wn18_train_gen = wn18_gen.flow( wn18_train, negative_samples=negative_samples, shuffle=True ) wn18_valid_gen = wn18_gen.flow(wn18_valid, negative_samples=negative_samples) ``` -------------------------------- ### Initialize Knowledge Graph Data Generator and Model Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/complex-link-prediction.html Sets up the data generator for the knowledge graph and defines the ComplEx model architecture with L2 regularization. The model is then compiled with an Adam optimizer and binary crossentropy loss. ```python fb15k_gen = KGTripleGenerator( fb15k_graph, batch_size=len(fb15k_train) // 100 # ~100 batches per epoch ) fb15k_complex = ComplEx( fb15k_gen, embedding_dimension=embedding_dimension, embeddings_regularizer=regularizers.l2(1e-8), ) fb15k_inp, fb15k_out = fb15k_complex.in_out_tensors() fb15k_model = Model(inputs=fb15k_inp, outputs=fb15k_out) fb15k_model.compile( optimizer=optimizers.Adam(lr=0.001), loss=losses.BinaryCrossentropy(from_logits=True), metrics=[metrics.BinaryAccuracy(threshold=0.0)], ) ``` -------------------------------- ### Create Log Directory Source: https://stellargraph.readthedocs.io/en/stable/demos/node-classification/ppnp-node-classification.ipynb Ensures that a 'logs' directory exists for storing model checkpoints and other log files. This is a prerequisite for saving model weights. ```python if not os.path.isdir("logs"): os.makedirs("logs") ``` -------------------------------- ### Load MovieLens Dataset Source: https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/hinsage-link-prediction.html Loads the MovieLens dataset and displays its description. This is the initial step for working with graph data. ```python dataset = datasets.MovieLens() display(HTML(dataset.description)) G, edges_with_ratings = dataset.load() ```