### Quick Start: Basic Classification with AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Demonstrates a 30-second setup for adaptive classification. It initializes a classifier with a HuggingFace model, adds training examples, and makes predictions. ```python from adaptive_classifier import AdaptiveClassifier # 🎯 Step 1: Initialize with any HuggingFace model classifier = AdaptiveClassifier("bert-base-uncased") # 📝 Step 2: Add training examples texts = ["The product works great!", "Terrible experience", "Neutral about this purchase"] labels = ["positive", "negative", "neutral"] classifier.add_examples(texts, labels) # 🔮 Step 3: Make predictions predictions = classifier.predict("This is amazing!") print(predictions) # Output: [('positive', 0.85), ('neutral', 0.12), ('negative', 0.03)] ``` -------------------------------- ### Development Setup for adaptive-classifier Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Sets up the development environment for the adaptive-classifier project. This involves cloning the repository, installing the package in editable mode, and optionally installing testing dependencies. ```bash # Clone the repository git clone https://github.com/codelion/adaptive-classifier.git cd adaptive-classifier # Install in development mode pip install -e . # Install test dependencies (optional) pip install pytest pytest-cov pytest-randomly ``` -------------------------------- ### Initialize and Use AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Demonstrates basic usage of the AdaptiveClassifier, including initialization, adding examples, making predictions, and saving/loading the model. Assumes the 'adaptive_classifier' library is installed. ```python from adaptive_classifier import AdaptiveClassifier # Initialize classifier = AdaptiveClassifier("bert-base-uncased") # Add examples texts = [ "Great product!", "Terrible experience", "Average performance" ] labels = ["positive", "negative", "neutral"] classifier.add_examples(texts, labels) # Make predictions prediction = classifier.predict("This is amazing!") print(prediction) # [('positive', 0.8), ('neutral', 0.15), ('negative', 0.05)] # Save and load classifier.save("./my_classifier") loaded = AdaptiveClassifier.load("./my_classifier") ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Installs the project's dependencies from 'requirements.txt' and then installs the project itself in editable mode. This ensures all necessary libraries and the project code are available for development. ```bash pip install -r requirements.txt pip install -e . ``` -------------------------------- ### Get Example Statistics Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Provides detailed statistics about the examples stored within the classifier, including total examples, examples per class, and memory usage. Returns a comprehensive dictionary of these statistics. ```python def get_example_statistics() -> Dict[str, Any] ``` -------------------------------- ### Install adaptive-classifier Package Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Installs the adaptive-classifier package using pip. This package includes ONNX Runtime for optimized CPU inference. ```bash pip install adaptive-classifier ``` -------------------------------- ### Python Function Documentation Example Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Demonstrates the Google-style docstring format used for documenting Python functions. It includes sections for a short description, a longer explanation, arguments, return values, and potential exceptions. ```python def function_name(param1: str, param2: int) -> bool: """Short description. Longer description if needed. Args: param1: Description of param1. param2: Description of param2. Returns: Description of return value. Raises: ValueError: Description of when this is raised. """ pass ``` -------------------------------- ### Initialize AdaptiveClassifier and Add Training Examples with Dynamic Class Addition Source: https://context7.com/codelion/adaptive-classifier/llms.txt Demonstrates basic initialization of the AdaptiveClassifier with a HuggingFace model, adding training examples with dynamic class addition support, and making predictions. The classifier returns sorted tuples of (label, confidence) and supports continuous learning without retraining from scratch. ```python from adaptive_classifier import AdaptiveClassifier # Initialize with any HuggingFace model classifier = AdaptiveClassifier("bert-base-uncased") # Add training examples - supports dynamic class addition texts = [ "The product works great!", "Terrible experience", "Neutral about this purchase", "Error code 404 appeared", "System crashed after update" ] labels = ["positive", "negative", "neutral", "technical", "technical"] classifier.add_examples(texts, labels) # Make predictions - returns sorted list of (label, confidence) tuples predictions = classifier.predict("This is amazing!") print(predictions) # Output: [('positive', 0.85), ('neutral', 0.12), ('negative', 0.03)] # Continuous learning - add new examples without retraining from scratch new_texts = ["Best purchase ever!", "Highly recommend this"] new_labels = ["positive", "positive"] classifier.add_examples(new_texts, new_labels) # Add completely new class at runtime error_texts = ["Database connection failed", "Null pointer exception"] error_labels = ["error", "error"] classifier.add_examples(error_texts, error_labels) ``` -------------------------------- ### Save and Load AdaptiveClassifier Models Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Provides examples for saving a trained AdaptiveClassifier model locally and loading it back. It also covers integration with the HuggingFace Hub for pushing and loading models. ```python # Save locally classifier.save("./my_classifier") loaded_classifier = AdaptiveClassifier.load("./my_classifier") # 🤗 HuggingFace Hub Integration classifier.push_to_hub("adaptive-classifier/my-model") hub_classifier = AdaptiveClassifier.from_pretrained("adaptive-classifier/my-model") ``` -------------------------------- ### Add Examples to Classifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Adds new text examples and their corresponding labels to the classifier. This method automatically handles the creation of new classes if they don't exist. It raises a ValueError if the input lists are empty or have mismatched lengths. ```python def add_examples(texts: List[str], labels: List[str]) ``` -------------------------------- ### Get Memory and Model Statistics with AdaptiveClassifier Source: https://context7.com/codelion/adaptive-classifier/llms.txt Retrieves and prints memory statistics (total examples, number of classes, examples per class) and detailed model statistics (memory usage, model parameters) from an AdaptiveClassifier instance. It also demonstrates how to clear memory for specific labels or all labels. ```python from adaptive_classifier import AdaptiveClassifier # Assuming 'classifier' is an initialized AdaptiveClassifier object # classifier = AdaptiveClassifier("your_model_name") # Get memory statistics stats = classifier.get_memory_stats() print(f"Total examples: {stats['total_examples']}") print(f"Number of classes: {stats['num_classes']}") print(f"Examples per class: {stats['examples_per_class']}") # Get detailed model statistics detailed_stats = classifier.get_example_statistics() print(f"Memory usage (prototypes): {detailed_stats['memory_usage']['prototypes']} bytes") print(f"Memory usage (examples): {detailed_stats['memory_usage']['examples']} bytes") print(f"Model parameters: {detailed_stats.get('model_params', 0)}") # Clear memory for specific labels classifier.clear_memory(labels=["technical", "error"]) # Clear all memory classifier.clear_memory() ``` -------------------------------- ### Continuous Learning with AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Illustrates how to add new examples to an existing classifier over time, enabling continuous learning and adaptation to new data. This method is crucial for models that need to evolve. ```python # Add new examples over time new_texts = ["New example 1", "New example 2"] new_labels = ["positive", "negative"] classifier.add_examples(new_texts, new_labels) ``` -------------------------------- ### Get Memory Statistics Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Retrieves statistics related to the classifier's memory usage, such as the number of stored examples and prototypes. Returns a dictionary containing these statistics. ```python def get_memory_stats() -> Dict[str, Any] ``` -------------------------------- ### Lint Code with Flake8 Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Runs flake8 to check the code for style guide violations and potential errors. This helps maintain code quality and adherence to standards. ```bash flake8 . ``` -------------------------------- ### Handling Order Dependency in Online Learning with Adaptive Classifier (Python) Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Demonstrates two scenarios illustrating the potential order dependency when adding examples incrementally to the Adaptive Classifier. This behavior is inherent to neural network training. It also shows how to configure the classifier to use only prototype-based predictions for order-independent results. ```python # Scenario 1 classifier.add_examples(["fish example"], ["aquatic"]) classifier.add_examples(["bird example"], ["aerial"]) # Scenario 2 classifier.add_examples(["bird example"], ["aerial"]) classifier.add_examples(["fish example"], ["aquatic"]) # Configure to use only prototypes (order-independent) config = { 'prototype_weight': 1.0, # Use only prototypes 'neural_weight': 0.0 # Disable neural network contribution } classifier = AdaptiveClassifier("bert-base-uncased", config=config) ``` -------------------------------- ### HuggingFace Hub Integration with ONNX Optimization and Pre-trained Models Source: https://context7.com/codelion/adaptive-classifier/llms.txt Demonstrates pushing trained classifiers to HuggingFace Hub with optional ONNX quantization, loading pre-trained models including enterprise hallucination detectors and LLM routers. Includes examples of using specialized models for detecting hallucinated content and routing queries to appropriate model tiers. ```python from adaptive_classifier import AdaptiveClassifier # Train classifier classifier = AdaptiveClassifier("bert-base-uncased") classifier.add_examples(texts, labels) # Push to HuggingFace Hub with ONNX optimization classifier.push_to_hub( "username/my-classifier", include_onnx=True, # Include ONNX version (default) quantize_onnx=True, # Quantize for smaller size and faster inference private=False ) # Load from Hub - automatically uses best inference backend hub_classifier = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-router") # Load enterprise pre-trained models hallucination_detector = AdaptiveClassifier.from_pretrained( "adaptive-classifier/llm-hallucination-detector" ) context = "France is in Western Europe. Capital: Paris." response = "Paris is the capital. Population is 70 million." prediction = hallucination_detector.predict(f"Context: {context}\nAnswer: {response}") # Returns: [('HALLUCINATED', 0.72), ('NOT_HALLUCINATED', 0.28)] # LLM routing for cost optimization router = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-router") query = "Write a function to calculate Fibonacci sequence" routing = router.predict(query) # Returns: [('HIGH', 0.92), ('LOW', 0.08)] - route to appropriate model tier ``` -------------------------------- ### Load and Use Enterprise Content Moderation Classifier in Python Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Provides an example of loading a pre-trained content moderation classifier from the adaptive-classifier library and using it to predict the nature of user-generated content. It highlights the availability of various pre-trained enterprise classifiers. ```python # Available pre-trained enterprise classifiers: classifiers = [ "adaptive-classifier/content-moderation", # Content safety "adaptive-classifier/business-sentiment", # Business communications "adaptive-classifier/pii-detection", # Privacy protection "adaptive-classifier/fraud-detection", # Financial security "adaptive-classifier/email-priority", # Email routing "adaptive-classifier/compliance-classification" # Regulatory compliance ] # Easy deployment moderator = AdaptiveClassifier.from_pretrained("adaptive-classifier/content-moderation") result = moderator.predict("User generated content here...") ``` -------------------------------- ### Save Classifier State Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Saves the current state of the adaptive classifier to a specified directory on disk. This allows for later reloading of the model and its learned examples. ```python def save(save_dir: str) ``` -------------------------------- ### Advanced Configuration and Memory Management in Python Source: https://context7.com/codelion/adaptive-classifier/llms.txt Details advanced configuration options for the AdaptiveClassifier, including model hyperparameters, memory management settings like maximum examples per class and prototype update frequency, and techniques to prevent catastrophic forgetting using Elastic Weight Consolidation (EWC). It also shows how to create classifiers for order-independent predictions by prioritizing prototype-based results. ```python from adaptive_classifier import AdaptiveClassifier, ModelConfig # Custom configuration for advanced scenarios config = { # Model settings 'max_length': 512, 'batch_size': 32, 'learning_rate': 0.001, # Memory settings 'max_examples_per_class': 1000, 'prototype_update_frequency': 100, 'similarity_threshold': 0.6, # EWC settings for preventing catastrophic forgetting 'ewc_lambda': 100.0, 'num_representative_examples': 5, # Training settings 'epochs': 10, 'early_stopping_patience': 3, 'min_examples_per_class': 3, # Prediction weights 'prototype_weight': 0.7, # Weight for prototype-based predictions 'neural_weight': 0.3, # Weight for neural network predictions 'min_confidence': 0.1 } classifier = AdaptiveClassifier("bert-base-uncased", config=config) # Order-independent predictions (use only prototypes) order_independent_config = { 'prototype_weight': 1.0, # Use only prototypes 'neural_weight': 0.0 # Disable neural network contribution } classifier = AdaptiveClassifier("bert-base-uncased", config=order_independent_config) ``` -------------------------------- ### Clear Classifier Memory Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Clears stored examples and prototypes from the classifier's memory. Optionally, specific labels can be provided to clear only those associated examples. If no labels are specified, all memory is cleared. ```python def clear_memory(labels: Optional[List[str]] = None) ``` -------------------------------- ### Implement Continuous Learning in Classifier Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Adds more training examples to existing classification classes over time. Supports incremental learning by appending new texts and labels to already-trained classes. Enables the classifier to adapt and improve predictions as new data becomes available. ```python # Add more examples to existing classes more_examples = [ "Best purchase ever!", "Highly recommend this" ] more_labels = ["positive"] * 2 classifier.add_examples(more_examples, more_labels) ``` -------------------------------- ### Multilingual and Cross-Domain Classification with AdaptiveClassifier Source: https://context7.com/codelion/adaptive-classifier/llms.txt Utilizes a multilingual AdaptiveClassifier model to perform classification across different languages. It demonstrates adding examples in multiple languages and predicting text in various languages, showcasing cross-language capabilities. It also shows how to move the model between CPU and GPU. ```python from adaptive_classifier import AdaptiveClassifier # Use multilingual model for cross-language classification classifier = AdaptiveClassifier("distilbert/distilbert-base-multilingual-cased") # Train with examples from multiple languages texts = [ # English examples "This is great", "This is terrible", "Average experience", # Spanish examples "Esto es excelente", "Esto es terrible", "Experiencia promedio", # French examples "C'est génial", "C'est terrible", "Expérience moyenne" ] labels = [ "positive", "negative", "neutral", "positive", "negative", "neutral", "positive", "negative", "neutral" ] classifier.add_examples(texts, labels) # Predict in any language test_cases = [ ("This is wonderful", "English"), ("Esto es maravilloso", "Spanish"), ("C'est merveilleux", "French") ] for text, language in test_cases: predictions = classifier.predict(text, k=2) print(f"{language}: {text}") print(f" Predictions: {predictions}\n") # Move model between devices classifier_gpu = classifier.to("cuda") classifier_cpu = classifier.to("cpu") ``` -------------------------------- ### Load and Use LLM Configuration Optimizer in Python Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Demonstrates loading a pre-trained LLM configuration optimizer and using it to predict optimal settings (like temperature and top_p) for different query types. This allows for generating more tailored and effective LLM responses. ```python config_optimizer = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-config-optimizer") query = "Explain quantum physics concepts" predictions = config_optimizer.predict(query) # Returns: [('BALANCED', 0.85), ('CREATIVE', 0.10), ...] # Automatically suggests temperature range: 0.6-1.0 for balanced responses ``` -------------------------------- ### Initialize and Train MultiLabelAdaptiveClassifier in Python Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Demonstrates how to initialize the MultiLabelAdaptiveClassifier with custom configurations, train it using multi-label data, and make predictions. It utilizes the 'adaptive_classifier' library and requires texts and corresponding labels for training. ```python from adaptive_classifier import MultiLabelAdaptiveClassifier # Initialize with configuration classifier = MultiLabelAdaptiveClassifier( "distilbert/distilbert-base-cased", default_threshold=0.5, min_predictions=1, max_predictions=5 ) # Multi-label training data texts = [ "Breaking: Scientists discover AI can help predict climate change patterns", "Tech giant announces breakthrough in quantum computing for healthcare", "Olympic committee adopts new sports technology for athlete performance" ] labels = [ ["science", "technology", "climate", "news"], ["technology", "healthcare", "quantum", "business"], ["sports", "technology", "performance", "news"] ] # Train the classifier classifier.add_examples(texts, labels) # Make predictions predictions = classifier.predict_multilabel( "Revolutionary medical AI system launched by tech startup" ) # Results: [('technology', 0.85), ('healthcare', 0.72), ('business', 0.45)] ``` -------------------------------- ### Clone Repository and Navigate Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Clones the Adaptive Classifier repository from GitHub and changes the directory to the project's root. This is the initial step for setting up the local development environment. ```bash git clone https://github.com/codelion/adaptive-classifier.git cd adaptive-classifier ``` -------------------------------- ### Create and Activate Virtual Environment Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Creates a Python virtual environment named 'venv' and activates it. This isolates project dependencies. The activation command differs between Unix-like systems and Windows. ```bash python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate ``` -------------------------------- ### Load and Use Intelligent LLM Router in Python Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Illustrates loading a pre-trained LLM router and using it to predict the appropriate model tier for a given query. This helps optimize costs by routing complex queries to more powerful models and simpler ones to less expensive tiers. ```python router = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-router") query = "Write a function to calculate Fibonacci sequence" predictions = router.predict(query) # Returns: [('HIGH', 0.92), ('LOW', 0.08)] # Route to GPT-4 for complex tasks, GPT-3.5 for simple ones ``` -------------------------------- ### Initialize AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Creates an instance of the AdaptiveClassifier. Requires a HuggingFace model name and optionally accepts a device, configuration dictionary, and random seed. The device can be 'cuda' or 'cpu', or left to auto-detect GPU. Configuration details are specified in ModelConfig. ```python AdaptiveClassifier( model_name: str, device: Optional[str] = None, config: Optional[Dict[str, Any]] = None, seed: int = 42 ) ``` -------------------------------- ### Initialize ModelConfig Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Creates a configuration object for the adaptive classifier. It accepts an optional dictionary to pre-populate configuration settings. ```python ModelConfig(config: Optional[Dict[str, Any]] = None) ``` -------------------------------- ### Format Code with Black and Sort Imports with isort Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Applies code formatting using Black and sorts imports using isort across the entire project. These tools ensure consistent code style. ```bash black . isort . ``` -------------------------------- ### Run Project Tests Source: https://github.com/codelion/adaptive-classifier/blob/main/CONTRIBUTING.md Executes the project's test suite using pytest. It's recommended to run tests to ensure code integrity before submitting changes. ```bash pytest tests/ ``` -------------------------------- ### Save and Deploy Models with ONNX Export Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Demonstrates saving and loading adaptive classifier models with ONNX optimization. Supports both quantized (4x smaller, faster) and unquantized versions. By default, both versions are saved and quantized is loaded automatically on CPU for optimal performance. Includes options to prefer unquantized models, force PyTorch, or skip ONNX export entirely. ```python # Save with ONNX export (both quantized & unquantized versions) classifier.save("./model") # Push to Hub with ONNX (both versions included by default) classifier.push_to_hub("username/model") # Load automatically uses quantized ONNX on CPU (fastest, 4x smaller) fast_classifier = AdaptiveClassifier.load("./model") # Choose unquantized ONNX for maximum accuracy accurate_classifier = AdaptiveClassifier.load("./model", prefer_quantized=False) # Force PyTorch (no ONNX) pytorch_classifier = AdaptiveClassifier.load("./model", use_onnx=False) # Opt-out of ONNX export when saving classifier.save("./model", include_onnx=False) ``` -------------------------------- ### Load and Use LLM Hallucination Detector in Python Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Shows how to load a pre-trained hallucination detection model using AdaptiveClassifier.from_pretrained and use it to predict whether a given response contains hallucinations based on provided context. The output indicates the probability of hallucination. ```python detector = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-hallucination-detector") context = "France is in Western Europe. Capital: Paris. Population: ~67 million." response = "Paris is the capital. Population is 70 million." # Contains hallucination prediction = detector.predict(f"Context: {context}\nAnswer: {response}") # Returns: [('HALLUCINATED', 0.72), ('NOT_HALLUCINATED', 0.28)] ``` -------------------------------- ### Multi-Label Classification with MultiLabelAdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Shows how to perform multi-label classification, where a single text can belong to multiple categories. It includes configuration for minimum and maximum predictions and demonstrates adding multi-label training data. ```python from adaptive_classifier import MultiLabelAdaptiveClassifier # Initialize multi-label classifier classifier = MultiLabelAdaptiveClassifier( "bert-base-uncased", min_predictions=1, # Ensure at least 1 prediction max_predictions=5 # Limit to top 5 predictions ) # Multi-label training data (each text can have multiple labels) texts = [ "AI researchers study climate change using machine learning", "Tech startup develops healthcare solutions" ] labels = [ ["technology", "science", "climate", "ai"], ["technology", "business", "healthcare"] ] classifier.add_examples(texts, labels) # Make multi-label predictions predictions = classifier.predict_multilabel("Medical AI breakthrough announced") # Output: [('healthcare', 0.72), ('technology', 0.68), ('ai', 0.45)] ``` -------------------------------- ### Configure Multi-Label Classification with Advanced Settings Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Sets up MultiLabelAdaptiveClassifier with configurable thresholds, prediction limits, and adaptive threshold adaptation. Handles scenarios where texts belong to multiple categories simultaneously. Supports custom thresholds, label statistics, and per-label threshold configuration for nuanced predictions. ```python from adaptive_classifier import MultiLabelAdaptiveClassifier # Configure advanced multi-label settings classifier = MultiLabelAdaptiveClassifier( "bert-base-uncased", default_threshold=0.5, # Base threshold for predictions min_predictions=1, # Minimum labels to return max_predictions=10 # Maximum labels to return ) # Training with diverse multi-label examples texts = [ "Scientists develop AI for medical diagnosis and climate research", "Tech company launches sustainable energy and healthcare products", "Olympic athletes use sports science and nutrition technology" ] labels = [ ["science", "ai", "healthcare", "research"], ["technology", "business", "environment", "healthcare"], ["sports", "science", "health", "technology"] ] classifier.add_examples(texts, labels) # Advanced prediction options predictions = classifier.predict_multilabel( "New research on AI applications in environmental science", threshold=0.3, # Custom threshold max_labels=5 # Limit results ) # Get detailed statistics stats = classifier.get_label_statistics() print(f"Adaptive threshold: {stats['adaptive_threshold']}") print(f"Label-specific thresholds: {stats['label_thresholds']}") ``` -------------------------------- ### Benchmark ONNX vs PyTorch Performance Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Compares inference performance between PyTorch and ONNX implementations using the benchmark script. Measures query latency across multiple runs to evaluate optimization effectiveness. ONNX typically provides 4x speedup on CPU with quantization enabled. ```bash # Compare PyTorch vs ONNX performance python scripts/benchmark_onnx.py --model bert-base-uncased --runs 100 ``` -------------------------------- ### Add New Classes Dynamically to Classifier Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Enables adding completely new classification classes after model initialization. Uses the add_examples method to integrate new texts and corresponding labels into the classifier's training set. Useful for extending classifier functionality without retraining from scratch. ```python # Add a completely new class new_texts = [ "Error code 404 appeared", "System crashed after update" ] new_labels = ["technical"] * 2 classifier.add_examples(new_texts, new_labels) ``` -------------------------------- ### Batch Processing and Model Persistence with ONNX Export Source: https://context7.com/codelion/adaptive-classifier/llms.txt Shows batch prediction for multiple texts with configurable batch sizes and top-k results. Demonstrates model persistence operations including saving with optional ONNX export for CPU inference acceleration, loading with quantization options, and choosing between ONNX and PyTorch inference backends. ```python from adaptive_classifier import AdaptiveClassifier # Process large datasets efficiently classifier = AdaptiveClassifier("distilbert/distilbert-base-cased") # Batch prediction for multiple texts test_texts = [ "Great product, highly recommend", "Poor quality, don't buy", "Average experience", "API giving timeout error" ] batch_predictions = classifier.predict_batch(test_texts, k=3, batch_size=32) for text, preds in zip(test_texts, batch_predictions): print(f"\n{text}") for label, score in preds: print(f" {label}: {score:.3f}") # Save model locally with ONNX export (default for 2-4x faster CPU inference) classifier.save("./my_classifier") # Load from disk - automatically uses quantized ONNX on CPU loaded = AdaptiveClassifier.load("./my_classifier") # Load with unquantized ONNX for maximum accuracy accurate_classifier = AdaptiveClassifier.load("./my_classifier", prefer_quantized=False) # Force PyTorch mode (no ONNX) pytorch_classifier = AdaptiveClassifier.load("./my_classifier", use_onnx=False) # Save without ONNX export classifier.save("./pytorch_only", include_onnx=False) ``` -------------------------------- ### Multi-Label Classification with Adaptive Thresholds in Python Source: https://context7.com/codelion/adaptive-classifier/llms.txt Demonstrates multi-label classification using the MultiLabelAdaptiveClassifier. It covers initialization with custom thresholds, adding training data with multiple labels per text, and performing predictions with adaptive thresholding. The statistics of label distribution and adaptive thresholds are also shown. ```python from adaptive_classifier import MultiLabelAdaptiveClassifier # Initialize with multi-label configuration classifier = MultiLabelAdaptiveClassifier( "bert-base-uncased", default_threshold=0.5, # Base threshold for predictions min_predictions=1, # Ensure at least 1 label per input max_predictions=5 # Limit to top 5 labels ) # Multi-label training data - each text can have multiple labels texts = [ "AI researchers study climate change using machine learning", "Tech startup develops healthcare solutions", "Olympic athletes use sports science and nutrition technology", "Breaking news: Scientists discover new quantum computing application" ] labels = [ ["technology", "science", "climate", "ai"], ["technology", "business", "healthcare"], ["sports", "science", "health", "technology"], ["science", "technology", "quantum", "news"] ] classifier.add_examples(texts, labels) # Predict multiple labels with adaptive thresholds predictions = classifier.predict_multilabel( "Medical AI breakthrough announced by tech company", threshold=0.3, # Custom threshold (uses adaptive default if None) max_labels=5 # Limit results ) print(predictions) # Returns: [('healthcare', 0.72), ('technology', 0.68), ('ai', 0.45), ('business', 0.38)] # Get label statistics and adaptive thresholds stats = classifier.get_label_statistics() print(f"Adaptive threshold: {stats['adaptive_threshold']}") print(f"Total labels: {stats['num_labels']}") print(f"Label distribution: {stats['examples_per_label']}") # Threshold automatically adjusts based on number of labels: # 2-4 labels: 0.5 (default) # 5-9 labels: 0.4 (20% lower) # 10-19 labels: 0.3 (40% lower) # 20-29 labels: 0.2 (60% lower) # 30+ labels: 0.1 (80% lower) - prevents empty predictions ``` -------------------------------- ### Optimized CPU Inference with ONNX Runtime Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Highlights the automatic use of ONNX Runtime for optimized CPU inference in Adaptive Classifier, offering 2-4x speed improvements without code changes. It contrasts ONNX CPU performance with PyTorch on both CPU and GPU. ```python # Automatically uses ONNX on CPU, PyTorch on GPU classifier = AdaptiveClassifier("bert-base-uncased") # That's it! Predictions are 2-4x faster on CPU predictions = classifier.predict("Fast inference!") ``` -------------------------------- ### Load Saved Classifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Loads a previously saved AdaptiveClassifier instance from a given directory. Optionally, a device ('cuda' or 'cpu') can be specified for loading the model. Returns the loaded classifier object. ```python @classmethod def load(cls, save_dir: str, device: Optional[str] = None) -> 'AdaptiveClassifier' ``` -------------------------------- ### Dynamic Class Addition in AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Demonstrates how to introduce entirely new classes to the classifier dynamically. This feature allows the model to handle previously unseen categories without requiring a full re-initialization. ```python # Add completely new class technical_texts = ["Error 404", "System crash"] technical_labels = ["technical", "technical"] classifier.add_examples(technical_texts, technical_labels) ``` -------------------------------- ### Batch Prediction with AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Shows how to use the `predict_batch` method for efficient processing of multiple text inputs. This is useful for high-throughput scenarios. The `k` parameter specifies the number of top predictions to return. ```python # Process multiple texts efficiently texts = ["Text 1", "Text 2", "Text 3"] predictions = classifier.predict_batch(texts, k=3) ``` -------------------------------- ### Implement Strategic Classification for Adversarial Robustness Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Enables strategic mode to defend against adversarial inputs and text manipulation. Configures cost functions for sentiment words, text length changes, and word substitutions. Provides dual predictions (regular + strategic blend), pure strategic predictions, and robust predictions assuming already-manipulated input. ```python # Enable strategic mode to defend against adversarial inputs config = { 'enable_strategic_mode': True, 'cost_function_type': 'linear', 'cost_coefficients': { 'sentiment_words': 0.5, # Cost to change sentiment-bearing words 'length_change': 0.1, # Cost to modify text length 'word_substitution': 0.3 # Cost to substitute words }, 'strategic_blend_regular_weight': 0.6, # Weight for regular predictions 'strategic_blend_strategic_weight': 0.4 # Weight for strategic predictions } classifier = AdaptiveClassifier("bert-base-uncased", config=config) classifier.add_examples(texts, labels) # Robust predictions that consider potential manipulation text = "This product has amazing quality features!" # Dual prediction (automatic blend of regular + strategic) predictions = classifier.predict(text) # Pure strategic prediction (simulates adversarial manipulation) strategic_preds = classifier.predict_strategic(text) # Robust prediction (assumes input may already be manipulated) robust_preds = classifier.predict_robust(text) print(f"Dual: {predictions}") print(f"Strategic: {strategic_preds}") print(f"Robust: {robust_preds}") ``` -------------------------------- ### Optimize AdaptiveClassifier with ONNX Runtime Source: https://context7.com/codelion/adaptive-classifier/llms.txt Configures and uses the AdaptiveClassifier with ONNX Runtime for optimized inference on CPU. It covers automatic ONNX usage, explicit control, exporting models to ONNX format with quantization, and saving/loading models with ONNX preferences. ```python from adaptive_classifier import AdaptiveClassifier # Initialize with automatic ONNX on CPU (default behavior) classifier = AdaptiveClassifier("bert-base-uncased") # Automatically uses ONNX Runtime on CPU for 2-4x speedup # Explicit ONNX control classifier_onnx = AdaptiveClassifier("bert-base-uncased", use_onnx=True) classifier_pytorch = AdaptiveClassifier("bert-base-uncased", use_onnx=False) classifier_auto = AdaptiveClassifier("bert-base-uncased", use_onnx="auto") # Export existing model to ONNX onnx_path = classifier.export_onnx( "./onnx_model", quantize=True, # Apply INT8 quantization quantization_config="arm64" # Options: "arm64", "avx512", "avx2" ) # Both quantized (model_quantized.onnx) and unquantized (model.onnx) versions saved # Quantized: INT8, 4x smaller, ~1.14x faster on ARM, 2-4x faster on x86 # Unquantized: Full precision, maximum accuracy, larger file size # Save with ONNX export (both versions included by default) classifier.save("./production_model", include_onnx=True, quantize_onnx=True) # Load with preference for quantized (default - best performance) fast_classifier = AdaptiveClassifier.load("./production_model", prefer_quantized=True) # Load unquantized for maximum accuracy accurate_classifier = AdaptiveClassifier.load("./production_model", prefer_quantized=False) # Device-aware behavior # On GPU: Automatically uses PyTorch # On CPU: Automatically uses quantized ONNX (if available) ``` -------------------------------- ### Debug Specific Classifier with Pytest (Bash) Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/integration_tests.md This command allows for focused debugging of a specific classifier by running only the relevant integration tests. The `-k` flag filters tests by name, `-v` provides verbose output, and `-s` allows print statements to be shown. ```bash pytest tests/test_enterprise_classifiers_integration.py -k "fraud-detection" -v -s ``` -------------------------------- ### AdaptiveClassifier Update Method Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md The `update` method allows dynamic modification of the AdaptiveClassifier's configuration parameters after initialization. It accepts keyword arguments to change specific settings. ```python def update(**kwargs) ``` -------------------------------- ### Memory Management in AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Shows how to retrieve memory statistics and clear specific classes from the classifier's memory. This is useful for managing resource usage and fine-tuning the model's data. ```python # Get memory statistics stats = classifier.get_memory_stats() print(stats) # Clear specific classes classifier.clear_memory(labels=["technical"]) ``` -------------------------------- ### Strategic Classification for Adversarial Robustness in Python Source: https://context7.com/codelion/adaptive-classifier/llms.txt Introduces strategic classification mode for enhanced adversarial robustness. This section details how to configure cost functions and blending weights for predictions that are resilient to manipulation. It also shows how to perform dual, pure strategic, and robust predictions, and evaluate the model's robustness against varying levels of input manipulation. ```python from adaptive_classifier import AdaptiveClassifier # Enable strategic mode with cost function configuration config = { 'enable_strategic_mode': True, 'cost_function_type': 'linear', 'cost_coefficients': { 'sentiment_words': 0.5, # Cost to modify sentiment words 'length_change': 0.1, # Cost to change text length 'word_substitution': 0.3 # Cost to substitute words }, 'strategic_blend_regular_weight': 0.6, # Weight for regular predictions 'strategic_blend_strategic_weight': 0.4 # Weight for strategic predictions } classifier = AdaptiveClassifier("bert-base-uncased", config=config) classifier.add_examples(texts, labels) text = "This product has amazing quality features!" # Dual prediction - automatic blend of regular + strategic # Default predict() method in strategic mode predictions = classifier.predict(text) print(f"Dual prediction: {predictions}") # Uses game-theoretic approach to combine regular and strategic-aware predictions # Pure strategic prediction - simulates adversarial manipulation strategic_preds = classifier.predict_strategic(text) print(f"Strategic: {strategic_preds}") # Predicts what classification would be if user strategically modified input # Robust prediction - assumes input may already be manipulated robust_preds = classifier.predict_robust(text) print(f"Robust: {robust_preds}") # Applies techniques resistant to strategic manipulation # Evaluate strategic robustness test_texts = ["Sample text 1", "Sample text 2", "Sample text 3"] test_labels = ["positive", "negative", "neutral"] robustness_metrics = classifier.evaluate_strategic_robustness( test_texts, test_labels, gaming_levels=[0.0, 0.5, 1.0] # Different manipulation intensities ) print(f"Robustness metrics: {robustness_metrics}") ``` -------------------------------- ### Move Model to Device Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Transfers the model's parameters to a specified device (e.g., 'cuda' or 'cpu'). This method supports method chaining and returns the classifier instance after moving. ```python def to(device: str) -> 'AdaptiveClassifier' ``` -------------------------------- ### Predict Multiple Texts in Batch Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Performs batched predictions for a list of input texts, improving efficiency. Returns a list of prediction lists, each containing (label, confidence) tuples. Configurable parameters include the number of top predictions (k) and the batch size. ```python def predict_batch( texts: List[str], k: int = 5, batch_size: int = 32 ) -> List[List[Tuple[str, float]]] ``` -------------------------------- ### AdaptiveClassifier to_dict Method Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md The `to_dict` method converts the current configuration of the AdaptiveClassifier into a Python dictionary. This is useful for saving, logging, or inspecting the model's settings. ```python def to_dict() -> Dict[str, Any] ``` -------------------------------- ### Run Pytest Without Integration Tests (Bash) Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/integration_tests.md This command skips integration tests during local development using pytest. It's useful for faster feedback loops when focusing on unit tests or specific modules. ```bash pytest tests/ -m "not integration" ``` -------------------------------- ### Merge AdaptiveClassifiers Source: https://context7.com/codelion/adaptive-classifier/llms.txt Demonstrates how to merge two AdaptiveClassifier instances. This is useful when you want to combine the knowledge from two independently trained models into a single model. ```python from adaptive_classifier import AdaptiveClassifier classifier1 = AdaptiveClassifier("bert-base-uncased") classifier2 = AdaptiveClassifier("bert-base-uncased") # ... train both classifiers ... merged = classifier1.merge_classifiers(classifier2) ``` -------------------------------- ### Classifier Merging in AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Illustrates the process of merging two trained AdaptiveClassifier instances into one. This feature is useful for combining knowledge from different training runs or datasets. ```python # Merge two classifiers classifier1 = AdaptiveClassifier("bert-base-uncased") classifier2 = AdaptiveClassifier("bert-base-uncased") # ... train both classifiers ... classifier1.merge_classifiers(classifier2) ``` -------------------------------- ### Predict Single Text Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Predicts labels for a single input text. Returns a list of (label, confidence) tuples sorted by confidence, up to the specified number (k). An empty input text will raise a ValueError. ```python def predict(text: str, k: int = 5) -> List[Tuple[str, float]] ``` -------------------------------- ### Strategic Defense (Anti-Gaming) with AdaptiveClassifier Source: https://github.com/codelion/adaptive-classifier/blob/main/README.md Enables strategic classification mode within AdaptiveClassifier to enhance robustness against adversarial manipulation or gaming attempts. Predictions made in this mode consider potential attempts to exploit the model. ```python # Enable strategic classification for adversarial robustness config = {'enable_strategic_mode': True} strategic_classifier = AdaptiveClassifier("bert-base-uncased", config=config) # Robust predictions against manipulation predictions = strategic_classifier.predict("This product has amazing quality features!") # Returns predictions that consider potential gaming attempts ``` -------------------------------- ### Merge Another Classifier Source: https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md Merges the data and state of another AdaptiveClassifier instance into the current one. This operation requires that both classifiers have compatible embedding dimensions. Returns the current classifier instance after merging. ```python def merge_classifiers(other: 'AdaptiveClassifier') -> 'AdaptiveClassifier' ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.