### Start Federated Learning Server (Python) Source: https://context7.com/iwang05/fluid/llms.txt This Python code demonstrates how to start a federated learning server using the Flower framework. It configures a custom FedDropCIFARAndroid strategy, defines training parameters, initializes the server, and starts the server process for a specified number of rounds. Key components include the strategy definition, `fit_config` function, and `fl.server.start_server` call. ```python import flwr as fl from flwr.server.client_manager import SimpleClientManager import fedDropCIFAR_android import fl_server # Define the federated dropout strategy with client requirements strategy = fedDropCIFAR_android.FedDropCIFARAndroid( fraction_fit=1.0, fraction_eval=1.0, min_fit_clients=5, min_eval_clients=5, min_available_clients=5, eval_fn=None, on_fit_config_fn=fit_config, initial_parameters=None, ) # Configure training parameters for each round def fit_config(rnd: int, p=1.0): config = { "batch_size": 20, "local_epochs": 1, "p_val": p, } return config # Initialize custom server with client manager client_manager = SimpleClientManager() server = fl_server.Server(client_manager=client_manager, strategy=strategy) # Start server on specified IP and port for 100 rounds fl.server.start_server( "192.168.1.7:1999", config={"num_rounds": 100}, server=server, strategy=strategy ) ``` -------------------------------- ### Setup CIFAR10 Dataset Partitions (Bash) Source: https://context7.com/iwang05/fluid/llms.txt Downloads and extracts the partitioned CIFAR10 dataset. The dataset consists of multiple partitions, each containing training and testing image lists. These partitions are then copied into the Android application's assets directory for client-side loading. The strategy involves clients loading specific combinations of partitions. ```bash # Download partitioned CIFAR10 dataset wget https://www.dropbox.com/s/coeixr4kh8ljw6o/cifar10.zip unzip cifar10.zip # Dataset structure (10 partitions, each ~5000 images): # partition_0_train.txt # Lists image paths for partition 0 training # partition_0_test.txt # Lists image paths for partition 0 testing # ... up to partition_9 # Copy to Android app assets cp -r cifar10/* CIFAR10/client/app/src/main/assets/data/ # Client loading strategy: # Client 1 loads: partition_0 + partition_5 (double the data per client) ``` -------------------------------- ### Install FLuID and Dependencies (Shell) Source: https://github.com/iwang05/fluid/blob/main/README.md Installs the Flower framework, TensorFlow, and clones the FLuID repository. Requires Python version 3.7 to 3.9. ```shell pip install flwr==0.18.0 pip install tensorflow git clone https://github.com/iwang05/FLuID.git cd FLuID ``` -------------------------------- ### Setup Shakespeare Dataset with LEAF and Splitter Source: https://context7.com/iwang05/fluid/llms.txt This section details the process of setting up the Shakespeare dataset. It involves cloning the LEAF repository, generating the dataset, and then using a Python script to split the multi-user JSON data into individual files for each character, preparing them for the Android application. ```bash git clone https://github.com/TalwalkarLab/leaf.git cd leaf/data/shakespeare ./preprocess.sh -s niid --sf 1.0 -k 0 -t sample cd FLuID/Shakespeare python split_json_data.py \ --save_root ./split_data \ --leaf_train_json ../../leaf/data/shakespeare/data/train/train.json \ --leaf_test_json ../../leaf/data/shakespeare/data/test/test.json \ --val_frac 0 cp split_data/THE_KING/train.json Shakespeare/client/app/src/main/assets/data/1_train.json cp split_data/THE_KING/test.json Shakespeare/client/app/src/main/assets/data/1_test.json ``` -------------------------------- ### Start Flower Server (Python) Source: https://github.com/iwang05/fluid/blob/main/README.md Starts the Flower federated learning server. Configuration options include specifying the server IP, number of rounds, and strategy. It requires `min_fit_clients`, `min_eval_clients`, and `min_available_clients` to be configured in `server.py`. ```python import flwr as fl # Assuming 'server' and 'strategy' are defined elsewhere # server = ... # strategy = ... fl.server.start_server("192.168.1.7:1999", config={"num_rounds": 10}, server=server, strategy=strategy) ``` -------------------------------- ### Setup FEMNIST Dataset with LEAF Source: https://context7.com/iwang05/fluid/llms.txt This snippet demonstrates how to clone the LEAF repository, generate the FEMNIST dataset using the preprocess script, and copy the user data to the Android application's assets. The output includes JSON files containing user data with flattened images and labels. ```bash git clone https://github.com/TalwalkarLab/leaf.git cd leaf/data/femnist ./preprocess.sh -s niid --sf 1.0 -k 0 -t sample cp data/train/0_train.json FEMNIST/client/app/src/main/assets/data/ cp data/test/0_test.json FEMNIST/client/app/src/main/assets/data/ ``` -------------------------------- ### Build Android Client Application using Gradlew (Shell) Source: https://github.com/iwang05/fluid/blob/main/README.md Builds and installs the client application on an Android device using Gradle. This command is executed from the client directory. An alternative is to use Android Studio. ```shell cd /client/ gradlew installDebug # If on Max or Linux run ./gradlew installDebug ``` -------------------------------- ### Python: Aggregate Weights with Different Model Sizes Source: https://context7.com/iwang05/fluid/llms.txt Aggregates weights from clients, handling variations in model size and dropped neurons. It transforms sub-models to the full model shape by inserting zeros for dropped neurons and calculates a weighted average. Dependencies include `numpy` and `typing`. Inputs are a list of client results (weights, example count, client ID), dictionaries for dropped weights and their shapes, and the original full model weights shape. Outputs are the aggregated weights. ```python from typing import List, Tuple, Dict from functools import reduce import numpy as np # Assume Weights is defined as a list of numpy arrays Weights = List[np.ndarray] def aggregate_drop(self, results: List[Tuple[Weights, int, str]], dropWeights: Dict[str, List], origWeightsShape: List, dropWeightsShape: Dict[str, List]) -> Weights: """ Aggregate weights from clients with different model sizes. Sub-models are expanded to full size with zeros for dropped neurons, then aggregated with proper weighting based on training examples. """ num_examples_total = sum([num_examples for _, num_examples, _ in results]) # Track total examples for each weight parameter # Dropped weights have reduced example counts total_examples_wDrop = [] for i in range(len(origWeightsShape)): total_examples_wDrop.append(np.full(origWeightsShape[i], num_examples_total)) # Transform sub-models to full model shape transformedResults = [] for (clientWeights, num_examples, cid) in results: if cid not in dropWeights: # Non-straggler: just reshape weights transformedWeights = clientWeights for i in range(len(origWeightsShape)): if transformedWeights[i].shape != origWeightsShape[i]: transformedWeights[i] = np.reshape(transformedWeights[i], origWeightsShape[i]) transformedResults.append((transformedWeights, num_examples)) continue # Straggler: expand sub-model to full size transformedWeights = clientWeights for i in range(len(origWeightsShape)): if transformedWeights[i].shape != dropWeightsShape[cid][i]: transformedWeights[i] = np.reshape(transformedWeights[i], dropWeightsShape[cid][i]) # Insert zeros for dropped weights layer = 0 for [row, col] in dropWeights[cid]: # Insert zero rows for dropped output neurons if len(row) != 0: transformedWeights[layer] = np.insert( transformedWeights[layer], row - np.arange(len(row)), 0, axis=transformedWeights[layer].ndim - 1 ) # Reduce example count for these weights total_examples_wDrop[layer][..., row] -= num_examples # Insert zero columns for dropped input neurons if len(col) != 0: transformedWeights[layer] = np.insert( transformedWeights[layer], col - np.arange(len(col)), 0, axis=transformedWeights[layer].ndim - 2 ) total_examples_wDrop[layer][..., col, :] -= num_examples # Correct for double-subtraction at intersections # Assuming 'k' is defined correctly based on 'row' and 'col' for this calculation # For example: k = np.ix_(row, col) # transformedWeights[layer][np.ix_(*k)] += num_examples # This line was commented out in original, potential bug layer += 1 transformedResults.append((transformedWeights, num_examples)) # Weighted average: sum(weight * examples) / total_examples_per_weight weighted_weights = [[layer * num_examples for layer in weights] for weights, num_examples in transformedResults] weights_prime = [ np.divide(reduce(np.add, layer_updates), total_examples_wDrop[i]) for i, layer_updates in enumerate(zip(*weighted_weights)) ] return weights_prime ``` -------------------------------- ### Android Federated Learning Client Implementation (Java) Source: https://context7.com/iwang05/fluid/llms.txt Implements the `FlowerClient` for federated learning on Android. It handles model initialization, updating, training (`fit`), evaluation (`evaluate`), and data loading from assets. Dependencies include Android context and TensorFlow Lite model wrappers. It processes data partitions and manages local training epochs. ```java public class FlowerClient { private TransferLearningModelWrapper tlModel; private Context context; private int local_epochs = 1; // Initialize client with sub-model size public FlowerClient(Context context, double p_val) { this.tlModel = new TransferLearningModelWrapper(context, p_val); this.context = context; Log.e(TAG, "setting model to p = " + p_val); } // Update model size when straggler status changes public void updateModel(double p_val) { this.tlModel = new TransferLearningModelWrapper(this.context, p_val); Log.e(TAG, "changing model to p = " + p_val); } // Federated learning fit operation public Pair fit(ByteBuffer[] weights, int epochs) { this.local_epochs = epochs; // Update local model with global weights tlModel.updateParameters(weights); // Train on local data isTraining.close(); tlModel.train(this.local_epochs); tlModel.enableTraining((epoch, loss) -> setLastLoss(epoch, loss)); isTraining.block(); // Wait for training to complete // Return updated weights and training dataset size return Pair.create(getWeights(), tlModel.getSize_Training()); } // Federated learning evaluate operation public Pair, Integer> evaluate(ByteBuffer[] weights) { tlModel.updateParameters(weights); tlModel.disableTraining(); // Return (loss, accuracy) and test dataset size return Pair.create(tlModel.calculateTestStatistics(), tlModel.getSize_Testing()); } // Load dataset partitions for this client // For CIFAR10: each client loads 2 partitions for more data per device public void loadData(int device_id) { try { // Load first partition: partition_(device_id-1) BufferedReader reader = new BufferedReader( new InputStreamReader( this.context.getAssets().open( "data/partition_" + (device_id - 1) + "_train.txt"))); String line; while ((line = reader.readLine()) != null) { addSample("data/" + line, true); } reader.close(); // Load second partition: partition_(device_id+4) reader = new BufferedReader( new InputStreamReader( this.context.getAssets().open( "data/partition_" + (device_id + 5 - 1) + "_train.txt"))); while ((line = reader.readLine()) != null) { addSample("data/" + line, true); } reader.close(); // Load test data similarly // ... (test data loading code) } catch (IOException ex) { ex.printStackTrace(); } } } ``` -------------------------------- ### Invariant Dropout Strategy (FLuID) Implementation Placeholder (Python) Source: https://context7.com/iwang05/fluid/llms.txt This section indicates where the implementation for FLuID's novel invariant dropout strategy would reside within the `fedDropCIFAR_android.py` file. This strategy is designed to dynamically identify neurons with minimal weight updates across training rounds, creating optimized sub-models for stragglers. The actual code for `configure_fit` and the specific dropout logic is expected here. ```python # In fedDropCIFAR_android.py configure_fit method ``` -------------------------------- ### Random Dropout Strategy Implementation (Python) Source: https://context7.com/iwang05/fluid/llms.txt This Python code snippet, located within the `fedDropCIFAR_android.py` file, illustrates the implementation of the random dropout strategy for straggler clients. The `configure_fit` method selects clients and applies random dropout to a percentage of neurons, controlled by `self.p_val`. The `drop_rand` method calculates the number of neurons to drop and randomly selects their indices for removal from weights. ```python # In fedDropCIFAR_android.py configure_fit method # Select random neurons to dropout for stragglers def configure_fit(self, rnd: int, parameters: Parameters, client_manager: ClientManager): # Sample clients for this round clients = client_manager.sample(num_clients=sample_size, min_num_clients=min_num_clients) clientList = [] for client in clients: if (client.cid in self.straggler) and rnd > 2: # Apply random dropout: drops random (1-p)% of neurons # idxList=[0,2,4,6,8,12,14] specifies layer indices to apply dropout # 10 is the index before FC layer fit_ins_drop = FitIns( self.drop_rand(parameters, self.p_val, [0,2,4,6,8,12,14], 10, client.cid), config_drop ) clientList.append((client, fit_ins_drop)) else: clientList.append((client, fit_ins)) return clientList # The drop_rand method randomly selects neurons to remove # For a layer with shape (3,3,32,64) and p=0.75: # - Calculates numToDrop = 64 - int(0.75 * 64) = 16 neurons # - Randomly samples 16 neuron indices from [0-63] # - Removes corresponding weights from activation, bias, and next layer input ``` -------------------------------- ### Ordered Dropout Strategy (Fjord Baseline) Implementation (Python) Source: https://context7.com/iwang05/fluid/llms.txt This Python code snippet, part of `fedDropCIFAR_android.py`, implements the ordered dropout strategy, serving as a baseline from Fjord. The `configure_fit` method identifies straggler clients and applies dropout by selecting neurons from the end of each layer, determined by `self.p_val`. The `drop_order` method calculates the number of neurons to drop based on `p_val` and removes the rightmost ones, facilitating ordered sub-model extraction. ```python # In fedDropCIFAR_android.py configure_fit method # Select neurons from the end of each layer (ordered dropout) def configure_fit(self, rnd: int, parameters: Parameters, client_manager: ClientManager): clientList = [] for client in clients: if (client.cid in self.straggler) and rnd > 2: # Apply ordered dropout: drops rightmost (1-p)% of neurons fit_ins_drop = FitIns( self.drop_order(parameters, self.p_val, [0,2,4,6,8,12,14], 10, client.cid), config_drop ) clientList.append((client, fit_ins_drop)) else: clientList.append((client, fit_ins)) return clientList # The drop_order method selects neurons from the end of each layer # For a layer with 64 neurons and p=0.75: # - Calculates numToDrop = 64 - int(0.75 * 64) = 16 neurons # - Drops neurons [48, 49, 50, ..., 63] (rightmost 16) # - This allows ordered nested sub-model extraction ``` -------------------------------- ### Configure Fit with Invariant Dropout (Python) Source: https://github.com/iwang05/fluid/blob/main/README.md Demonstrates how to configure the fit method in `fedDrop_android.py` to use a specific dropout method, in this case, `drop_rand`. This function is part of the FLuID implementation for Android clients. ```python # Assuming 'FitIns', 'self.drop_rand', 'parameters', 'self.p_val', '[0,3]', '10', 'client.cid', and 'config_drop' are defined fit_ins_drop = FitIns(self.drop_rand(parameters, self.p_val, [0,3], 10, client.cid), config_drop) ``` -------------------------------- ### Configure Federated Learning Fit with Invariant Neuron Dropout (Python) Source: https://context7.com/iwang05/fluid/llms.txt Configures the client list for a federated learning round, applying dynamic invariant neuron dropout for straggler clients after a specified number of rounds. It prepares clients for training, potentially with modified parameters for dropout. ```python def configure_fit(self, rnd: int, parameters: Parameters, client_manager: ClientManager): clientList = [] for client in clients: if (client.cid in self.straggler) and rnd > 2: # Apply invariant dropout: drops neurons with minimal weight updates fit_ins_drop = FitIns( self.drop_dynamic(parameters, self.p_val, [0,2,4,6,8,12,14], 10, client.cid), config_drop ) clientList.append((client, fit_ins_drop)) else: clientList.append((client, fit_ins)) return clientList ``` -------------------------------- ### Split Shakespeare Dataset using Python Source: https://github.com/iwang05/fluid/blob/main/Shakespeare/README.md This script splits the combined Shakespeare dataset obtained from the LEAF benchmark into individual user datasets for training and testing. It takes paths to LEAF's train and test JSON files as input and saves the split data to a specified directory. ```shell python split_json_data.py --save_root --leaf_train_json --leaf_test_json --val_frac ``` -------------------------------- ### Generate TensorFlow Lite Sub-Models (Bash) Source: https://context7.com/iwang05/fluid/llms.txt Generates TensorFlow Lite models (.tflite) for different sub-model sizes by varying the 'p' parameter in a Python script. It involves navigating to the conversion directory, editing the script, running the conversion, and organizing the output files. These models are then copied to the Android application's assets. ```bash # Generate .tflite models for different sub-model sizes cd CIFAR10/tflite_convertor # Edit convert_to_tflite.py to define model architecture # Vary p parameter to create different sizes: 1.0, 0.95, 0.85, 0.75, 0.65, 0.5 python convert_to_tflite.py # Output files in tflite_convertor/tflite_model/: # bottleneck.tflite # initialize.tflite # optimizer_bottleneck.tflite # optimizer_head.tflite # train_head.tflite # Rename files with p-value prefix mv train_head.tflite 1.0_train_head.tflite # Repeat for p=0.95, 0.85, 0.75, 0.65, 0.5 # Copy to Android assets cp *.tflite ../client/app/src/main/assets/model/ # Required model files for each p value: # p_bottleneck.tflite # Feature extraction base model # p_train_head.tflite # Classification head for training # p_initialize.tflite # Model initialization # p_optimizer_bottleneck.tflite # Optimizer for base model # p_optimizer_head.tflite # Optimizer for classification head ``` -------------------------------- ### Calculate Minimum Weight Change Percentage (Python) Source: https://context7.com/iwang05/fluid/llms.txt This function calculates the minimum weight change percentage for each layer across all clients. It identifies the maximum change for each weight parameter and then determines the neuron with the minimum percentage change within each layer. This is used for initializing and validating thresholds, with specific logic for rounds 2 and 3, and a gradual increase after round 10. ```python def find_min(self, parameters: Parameters, results: List[Tuple[Weights, int]], idxList: List[int], rnd: int): """ Calculate minimum weight change percentage for each layer. Used to initialize and validate thresholds. """ weights = self.parameters_to_weights(parameters) difference = [np.full(shape, 0.0) for shape in self.weight_shapes] # Calculate maximum change of each weight parameter across all clients for cWeights, num_examples, cid in results: if cid in self.straggler: continue clientWeights = cWeights for i in range(len(weights)): difference[i] = np.maximum( difference[i], np.abs(clientWeights[i] - weights[i]) / np.abs(weights[i]) ) # For each layer, find neuron with minimum percentage change for idx in idxList: idx0Layer = np.amax(difference[idx], axis=tuple(dim)) idx1Layer = difference[idx + 1] idx2Layer = np.amax(difference[idx + 2], axis=tuple(dim)) sum = np.maximum(np.maximum(idx0Layer, idx1Layer), idx2Layer) noChangeIdx = np.argsort(sum) print("% difference: ", sum[noChangeIdx[0]]) # Initialize threshold at round 2 using minimum change if rnd == 2: self.changeThreshold[idx] = sum[noChangeIdx[0]] # Average with round 3 for stability if rnd == 3: self.changeThreshold[idx] = (self.changeThreshold[idx] + sum[noChangeIdx[0]]) / 2 # After round 10, gradually increase threshold every 2 rounds # This allows more neurons to be marked as invariant over time if rnd > 10: self.roundCounter += 1 if self.roundCounter >= 2: for idx in idxList: if not self.stopChange[idx]: self.changeThreshold[idx] += 0.2 # changeIncrement self.roundCounter = 0 ``` -------------------------------- ### Find Stable Neurons Across Training Rounds (Python) Source: https://context7.com/iwang05/fluid/llms.txt Analyzes weight updates from non-straggler clients to identify neurons with weight changes below a defined threshold across a majority of clients. It calculates the percentage of clients exhibiting minimal weight change for each neuron and identifies neurons where all related weights are stable. ```python # Server-side analysis after each training round # Identifies neurons with weight changes below threshold def find_stable(self, parameters: Parameters, results: List[Tuple[Weights, int]], idxList: List[int], idxConvFC: int): """ Analyzes weight updates from non-straggler clients to identify invariant neurons. A neuron is "invariant" if its weight parameters change less than the threshold for at least 75% of non-straggler clients. """ weights = self.parameters_to_weights(parameters) # For each layer, calculate which weights stayed within threshold for idx in idxList: difference = [np.full(shape, 0) for shape in self.weight_shapes] # Compare each client's weights against global model for cWeights, num_examples, cid in results: if cid in self.straggler: continue # Skip stragglers clientWeights = cWeights # Count weights within threshold: |client_weight - global_weight| <= threshold * |global_weight| for i in range(len(weights)): difference[i] += (np.abs(clientWeights[i] - weights[i]) <= np.abs(self.changeThreshold[idx] * weights[i])) * 1 # Mark as invariant if >= 75% of clients show minimal change for i in range(len(difference)): difference[i] = difference[i] >= (0.75 * (len(results) - len(self.straggler))) # Identify neuron indices where all related weights are invariant # Each neuron has weights in: activation, bias, and next layer input idx0Layer = np.all(difference[idx], axis=tuple(dim)) idx1Layer = difference[idx + 1] idx2Layer = np.all(difference[idx + 2], axis=tuple(dim)) noChangeIdx = idx0Layer & idx1Layer & idx2Layer # Build list of invariant neuron indices unchangedList = [i for i in range(len(noChangeIdx)) if noChangeIdx[i]] # Track neurons that were dropped last round and remain invariant self.defDropWeights[idx] = [i for i in self.prevDropWeights[idx] if i in unchangedList] return unchangedList ``` -------------------------------- ### Generate TFLite Models for Sub-models (Python) Source: https://github.com/iwang05/fluid/blob/main/README.md Generates TensorFlow Lite models for different sub-model sizes by varying the `p` variable in `convert_to_tflite.py`. The generated models are then renamed and placed in the client's asset directory. ```python python convert_to_tflite.py ``` -------------------------------- ### Python: Aggregate Fit and Identify Stragglers Source: https://context7.com/iwang05/fluid/llms.txt Aggregates federated learning results to identify stragglers based on training duration after round 2. It dynamically calculates and sets the sub-model size (p_val) based on the performance gap between the slowest and second slowest clients. The function also includes logic to re-identify stragglers in subsequent rounds, considering potential speedups due to dropout. ```python def aggregate_fit(self, rnd: int, results: List[Tuple[ClientProxy, FitRes]], failures: List[BaseException]): """ Aggregate results and identify stragglers based on training duration. Automatically sets sub-model size (p_val) based on performance gap. """ # At round 2, identify initial straggler if len(self.straggler) == 0 and rnd > 1: # Sort clients by training duration def time(elem): return elem[1].metrics.get('duration') results.sort(key=time) # Mark slowest client as straggler slowest_client = results[len(results) - 1] self.straggler[slowest_client[0].cid] = slowest_client[1].metrics.get('duration') # Calculate performance gap to set sub-model size stragglerDur = results[-1][1].metrics.get('duration') # e.g., 45.2 seconds nextSlowDur = results[-2][1].metrics.get('duration') # e.g., 38.5 seconds percentDiff = nextSlowDur / stragglerDur # e.g., 0.85 # Set p_val based on performance gap if percentDiff >= 0.90: # Gap < 10% self.p_val = 0.95 # 95% model size elif percentDiff >= 0.80: # Gap 10-20% self.p_val = 0.85 # 85% model size elif percentDiff >= 0.70: # Gap 20-30% self.p_val = 0.75 # 75% model size elif percentDiff >= 0.60: # Gap 30-40% self.p_val = 0.65 # 65% model size else: # Gap > 40% self.p_val = 0.5 # 50% model size print(f"Straggler detected: {slowest_client[0].cid}") print(f"Updated p_val to: {self.p_val}") # In subsequent rounds, check if straggler changed elif len(self.straggler) != 0 and rnd > 1 and not self.JustUpdatedStrag: results.sort(key=lambda x: x[1].metrics.get('duration')) slowest = results[-1] if slowest[0].cid not in self.straggler: # Estimate current straggler's original time (compensate for dropout speedup) for i in range(len(results)): if results[i][0].cid in self.straggler: # Estimate: measured_time / p_val * 1.15 (15% overhead buffer) self.straggler[results[i][0].cid] = ( results[i][1].metrics.get('duration') / self.p_val) * 1.15 # Check if new device is actually slower than estimated original time stragglerList = list(self.straggler.items()) if slowest[1].metrics.get('duration') > stragglerList[0][1]: # Update straggler designation and recalculate p_val self.straggler[slowest[0].cid] = slowest[1].metrics.get('duration') # Recalculate p_val with new straggler # ... (same logic as above) self.straggler.pop(stragglerList[0][0]) self.JustUpdatedStrag = True ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.