### Install NTextCat via NuGet Source: https://github.com/ivanakcheurov/ntextcat/blob/master/README.md Command to add the NTextCat package to your .NET project. ```bash dotnet add package NTextCat ``` -------------------------------- ### Identify Language with NTextCat Source: https://github.com/ivanakcheurov/ntextcat/blob/master/README.md Example showing how to load a language profile and identify the language of a text string. Ensure the profile file is deployed with the application. ```csharp using NTextCat; ... // Don't forget to deploy a language profile (e.g. Core14.profile.xml) with your application. // (take a look at "content" folder inside of NTextCat nupkg and here: https://github.com/ivanakcheurov/ntextcat/tree/master/src/LanguageModels). var factory = new RankedLanguageIdentifierFactory(); var identifier = factory.Load("Core14.profile.xml"); // can be an absolute or relative path. Beware of 260 chars limitation of the path length in Windows. Linux allows 4096 chars. var languages = identifier.Identify("your text to get its language identified"); var mostCertainLanguage = languages.FirstOrDefault(); if (mostCertainLanguage != null) Console.WriteLine("The language of the text is '{0}' (ISO639-3 code)", mostCertainLanguage.Item1.Iso639_3); else Console.WriteLine("The language couldn’t be identified with an acceptable degree of certainty"); // outputs: The language of the text is 'eng' (ISO639-3 code) ``` -------------------------------- ### Identify Language with RankedLanguageIdentifier Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Shows how to use the Identify method to get multiple language matches ranked by similarity. This is useful for comparing potential languages and filtering results based on a score threshold. ```csharp using NTextCat; using System; using System.Linq; var factory = new RankedLanguageIdentifierFactory(); var identifier = factory.Load("Core14.profile.xml"); // Identify language with multiple results for comparison var results = identifier.Identify("Die Entwicklung der künstlichen Intelligenz schreitet voran"); // Display top 3 language matches Console.WriteLine("Top language matches:"); foreach (var result in results.Take(3)) { var langInfo = result.Item1; var distance = result.Item2; Console.WriteLine($" {langInfo.Iso639_3}: score {distance:F0}"); } // Filter results within acceptable threshold (e.g., 1.05 times the best score) double bestScore = results.First().Item2; double threshold = 1.05; var acceptableResults = results.Where(r => r.Item2 <= bestScore * threshold).ToList(); Console.WriteLine($"\nAcceptable matches (within {threshold}x of best): {acceptableResults.Count}"); // Output: // Top language matches: // deu: score 3842 // nld: score 8156 // dan: score 9234 // // Acceptable matches (within 1.05x of best): 1 ``` -------------------------------- ### CLI: Train Custom Models from Directory Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Train new language models from a directory containing text files. Each file should be named with its ISO language code (e.g., `eng.txt`). The output is a custom profile XML. ```bash # Train from directory containing language files # Each file should be named with ISO language code (e.g., eng.txt, deu.txt) NTextCat.Cli -train=./training_corpus > custom.profile.xml ``` -------------------------------- ### Create and Use LanguageInfo in C# Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Demonstrates creating a LanguageInfo object for custom classification and retrieving language metadata. Also shows how to identify a language from text using a loaded profile. ```csharp using NTextCat; using System; // Create language info for custom classification var spanishInfo = new LanguageInfo( iso6392T: "spa", // ISO 639-2T code iso6393: "spa", // ISO 639-3 code englishName: "Spanish", // English name localName: "Español" // Native name ); Console.WriteLine($"ISO 639-2T: {spanishInfo.Iso639_2T}"); Console.WriteLine($"ISO 639-3: {spanishInfo.Iso639_3}"); Console.WriteLine($"English Name: {spanishInfo.EnglishName}"); Console.WriteLine($"Local Name: {spanishInfo.LocalName}"); // Language info from identification result var factory = new RankedLanguageIdentifierFactory(); var identifier = factory.Load("Core14.profile.xml"); var result = identifier.Identify("Hola mundo").First(); var detectedLang = result.Item1; Console.WriteLine($"\nDetected: {detectedLang.Iso639_3} ({detectedLang.EnglishName ?? "Unknown"})"); ``` -------------------------------- ### CLI: Use Custom Threshold Settings Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Configure custom threshold settings for language identification. `-a` sets the maximum number of alternative languages to show, and `-u` adjusts the threshold multiplier for alternatives. ```bash # Use custom threshold settings # -a: max alternative languages to show (default: 10) # -u: threshold multiplier for alternatives (default: 1.05) NTextCat.Cli -a 3 -u 1.1 -l "texto en español" # Output: spa ``` -------------------------------- ### CLI: Train Custom Models from Wildcard Pattern Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Train language models using a wildcard pattern to select training files. This is useful for specifying files in a specific format or location. ```bash # Train from wildcard pattern NTextCat.Cli -train="./corpus/*.txt" > languages.profile.xml ``` -------------------------------- ### Train and Save Language Models to XML Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Trains a model from file-based corpus and persists it to an XML profile for future reuse. ```csharp using NTextCat; using System; using System.IO; using System.Linq; // Prepare training corpus from files string[] trainingFiles = Directory.GetFiles("./training_data", "*.txt"); var trainingData = trainingFiles.Select(filePath => { string langCode = Path.GetFileNameWithoutExtension(filePath); // e.g., "eng.txt" -> "eng" var langInfo = new LanguageInfo(langCode, langCode, null, null); return Tuple.Create(langInfo, (TextReader)new StreamReader(filePath)); }); var factory = new RankedLanguageIdentifierFactory( maxNGramLength: 5, maximumSizeOfDistribution: 4000, occuranceNumberThreshold: 0, onlyReadFirstNLines: int.MaxValue, allowMultithreading: true ); // Train and save to file in one operation var identifier = factory.TrainAndSave(trainingData, "custom.profile.xml"); // The profile can now be loaded in future sessions var reloadedFactory = new RankedLanguageIdentifierFactory(); var reloadedIdentifier = reloadedFactory.Load("custom.profile.xml"); // Verify it works var result = reloadedIdentifier.Identify("This is a test sentence"); Console.WriteLine($"Language: {result.First().Item1.Iso639_3}"); ``` -------------------------------- ### Load Language Models with Filter Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Use the `Load` method with a filter predicate to load only specific languages from a profile, reducing memory usage. The filter predicate receives each language model and should return true if the model should be loaded. ```csharp using NTextCat; using System; using System.Linq; var factory = new RankedLanguageIdentifierFactory(); // Load only Western European languages from a large profile var europeanLanguages = new[] { "eng", "deu", "fra", "spa", "ita", "por", "nld" }; var identifier = factory.Load( "Wikipedia82.profile.xml", filterPredicate: model => europeanLanguages.Contains(model.Language.Iso639_3) ); // Check loaded languages Console.WriteLine("Loaded languages:"); foreach (var model in identifier.LanguageModels) { Console.WriteLine($" {model.Language.Iso639_3}"); } // Identify text using filtered model var result = identifier.Identify("Questa è una frase in italiano"); Console.WriteLine($"\nDetected: {result.First().Item1.Iso639_3}"); // Output: // Loaded languages: // eng // deu // fra // spa // ita // por // nld // // Detected: ita ``` -------------------------------- ### Load Language Identifier with RankedLanguageIdentifierFactory Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Demonstrates how to create a RankedLanguageIdentifierFactory, load a language profile (Core14), and identify the language of a text snippet. Use this to initialize language detection in your .NET application. ```csharp using NTextCat; using System; using System.Linq; // Create the factory with default settings var factory = new RankedLanguageIdentifierFactory(); // Load a language identification profile (Core14 contains 14 popular languages) // The path can be absolute or relative var identifier = factory.Load("Core14.profile.xml"); // Identify the language of a text snippet string textToIdentify = "Bonjour, comment allez-vous aujourd'hui?"; var languages = identifier.Identify(textToIdentify); // Get the most likely language var mostCertainLanguage = languages.FirstOrDefault(); if (mostCertainLanguage != null) { Console.WriteLine($"Language: {mostCertainLanguage.Item1.EnglishName}"); Console.WriteLine($"ISO 639-3 Code: {mostCertainLanguage.Item1.Iso639_3}"); Console.WriteLine($"ISO 639-2T Code: {mostCertainLanguage.Item1.Iso639_2T}"); Console.WriteLine($"Distance Score: {mostCertainLanguage.Item2}"); } else { Console.WriteLine("Could not identify the language with acceptable certainty"); } // Output: // Language: (null - depends on profile) // ISO 639-3 Code: fra // ISO 639-2T Code: fra // Distance Score: 4523.0 ``` -------------------------------- ### NaiveBayesLanguageIdentifierFactory.Load Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Loads a Naive Bayes language identifier from a profile file and demonstrates its usage for identifying the language of given text. ```APIDOC ## NaiveBayesLanguageIdentifierFactory.Load ### Description Loads a pre-trained Naive Bayes language identifier from a profile file. This classifier can be used to identify the language of a given text. ### Method Factory Method (Implicit) ### Endpoint N/A (Local Class Method) ### Parameters None directly for `Load` method, but the factory requires a profile file path. ### Request Body N/A ### Request Example ```csharp using NTextCat; using System; using System.Linq; var factory = new NaiveBayesLanguageIdentifierFactory(); var identifier = factory.Load("Core14.profile.xml"); string russianText = "Главная задача программы - определить язык текста"; var languages = identifier.Identify(russianText); var topResult = languages.First(); Console.WriteLine($"Detected language: {topResult.Item1.Iso639_2T}"); Console.WriteLine($"Confidence score: {topResult.Item2}"); ``` ### Response #### Success Response (200) An instance of `ILanguageIdentifier` capable of identifying languages. #### Response Example ```csharp // No direct response body, but the identifier object is returned. // Example output from Identify: // Detected language: rus // Confidence score: -245.67 ``` ``` -------------------------------- ### CLI: Train with Custom Parameters Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Train language models with custom parameters to optimize the model. `-t` sets the maximum n-grams, `-f` the minimum occurrence threshold, and `-i` limits the lines read per file. ```bash # Train with custom parameters # -t: maximum n-grams in distribution (default: 4000) # -f: minimum occurrence threshold (default: 0) # -i: only read first N lines from each file NTextCat.Cli -train=./corpus -t 5000 -f 2 -i 10000 > optimized.profile.xml ``` -------------------------------- ### CLI: Verify Trained Model Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Verify a trained language model by using it to identify the language of a test sentence. Use the `-p` flag to specify the custom profile. ```bash # Verify the trained model NTextCat.Cli -p custom.profile.xml -l "Test sentence" ``` -------------------------------- ### Identify Language with NaiveBayesLanguageIdentifierFactory Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Uses the Naive Bayes approach to classify text. Requires a pre-existing profile file. ```csharp using NTextCat; using System; using System.Linq; // Create Naive Bayes factory var factory = new NaiveBayesLanguageIdentifierFactory(); // Load the same profile format as RankedLanguageIdentifier var identifier = factory.Load("Core14.profile.xml"); // Test with Russian text string russianText = "Главная задача программы - определить язык текста"; var languages = identifier.Identify(russianText); var topResult = languages.First(); Console.WriteLine($"Detected language: {topResult.Item1.Iso639_2T}"); Console.WriteLine($"Confidence score: {topResult.Item2}"); // Test with Japanese text string japaneseText = "人工知能の開発は進んでいます"; var japaneseResult = identifier.Identify(japaneseText).First(); Console.WriteLine($"Japanese text detected as: {japaneseResult.Item1.Iso639_3}"); // Output: // Detected language: rus // Confidence score: -245.67 // Japanese text detected as: jpn ``` -------------------------------- ### Identify language via command line Source: https://github.com/ivanakcheurov/ntextcat/blob/master/src/README.txt Executes language identification on a text file using default settings. ```bash NTextCat.exe -e=UTF-8 -noprompt < Evaluation\russian-utf8.txt ``` -------------------------------- ### CLI: Identify Language with Specific Profile Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Identify the language of a given text using a specific profile file. Use the `-p` flag to specify the profile. ```bash # Identify language with specific profile NTextCat.Cli -p Core14.profile.xml -l "Ceci est un texte français" # Output: fra ``` -------------------------------- ### CLI: Train with Multithreading Disabled Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Train language models with multithreading disabled using the `-m` flag. This is useful for debugging training processes. ```bash # Train with multithreading disabled (useful for debugging) NTextCat.Cli -train=./corpus -m > single_thread.profile.xml ``` -------------------------------- ### Train Custom Language Models with RankedLanguageIdentifierFactory Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Trains a language identifier using custom training data provided via TextReader objects. ```csharp using NTextCat; using System; using System.IO; using System.Linq; // Prepare training data for custom languages/dialects var trainingData = new[] { Tuple.Create( new LanguageInfo("eng", "eng", "English", "English"), (TextReader)new StringReader(@"The quick brown fox jumps over the lazy dog. Shakespeare was born in Stratford-upon-Avon. Many consider him the greatest writer.")), Tuple.Create( new LanguageInfo("deu", "deu", "German", "Deutsch"), (TextReader)new StringReader(@"Der schnelle braune Fuchs springt über den faulen Hund. Die deutsche Sprache ist bekannt für ihre langen zusammengesetzten Wörter.")), Tuple.Create( new LanguageInfo("fra", "fra", "French", "Français"), (TextReader)new StringReader(@"Le renard brun rapide saute par-dessus le chien paresseux. La langue française est connue pour sa beauté et son élégance.")) }; // Create factory with custom parameters var factory = new RankedLanguageIdentifierFactory( maxNGramLength: 5, // Use 1-5 character n-grams maximumSizeOfDistribution: 4000, // Keep top 4000 n-grams per language occuranceNumberThreshold: 0, // Include all n-grams (0 = no minimum) onlyReadFirstNLines: int.MaxValue, allowMultithreading: true ); // Train the identifier var identifier = factory.Train(trainingData); // Test the custom model var testResult = identifier.Identify("Guten Morgen, wie geht es Ihnen?"); Console.WriteLine($"Detected: {testResult.First().Item1.Iso639_3}"); // Output: // Detected: deu ``` -------------------------------- ### RankedLanguageIdentifierFactory.Load Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Loads a language identification profile from an XML file and returns an identifier instance. ```APIDOC ## METHOD RankedLanguageIdentifierFactory.Load ### Description Loads pre-trained language models from XML profile files to initialize the identifier. ### Parameters #### Path Parameters - **path** (string) - Required - The file path to the language profile XML (e.g., "Core14.profile.xml"). ### Response - **RankedLanguageIdentifier** (object) - An instance configured with the loaded language models. ``` -------------------------------- ### RankedLanguageIdentifierFactory.Train Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Trains a custom language model using the Ranked classifier approach with provided training data and demonstrates its usage. ```APIDOC ## RankedLanguageIdentifierFactory.Train ### Description Trains a custom language model using the Ranked classifier. It accepts an array of tuples, where each tuple contains `LanguageInfo` and a `TextReader` for the training data of a specific language. ### Method Factory Method (Implicit) ### Endpoint N/A (Local Class Method) ### Parameters - **trainingData** (Tuple[]) - Required - An array of tuples containing language information and text readers for training data. - **maxNGramLength** (int) - Optional - The maximum length of n-grams to consider (default is typically 5). - **maximumSizeOfDistribution** (int) - Optional - The maximum number of n-grams to keep in the distribution for each language (default is typically 4000). - **occuranceNumberThreshold** (int) - Optional - The minimum number of occurrences for an n-gram to be included (0 means all n-grams are included). - **onlyReadFirstNLines** (int) - Optional - Limits reading to the first N lines of each text reader. - **allowMultithreading** (bool) - Optional - Enables or disables multithreading for training. ### Request Body N/A ### Request Example ```csharp using NTextCat; using System; using System.IO; using System.Linq; var trainingData = new[] { Tuple.Create( new LanguageInfo("eng", "eng", "English", "English"), (TextReader)new StringReader(@"The quick brown fox jumps over the lazy dog. Shakespeare was born in Stratford-upon-Avon. Many consider him the greatest writer.")), Tuple.Create( new LanguageInfo("deu", "deu", "German", "Deutsch"), (TextReader)new StringReader(@"Der schnelle braune Fuchs springt über den faulen Hund. Die deutsche Sprache ist bekannt für ihre langen zusammengesetzten Wörter.")), Tuple.Create( new LanguageInfo("fra", "fra", "French", "Français"), (TextReader)new StringReader(@"Le renard brun rapide saute par-dessus le chien paresseux. La langue française est connue pour sa beauté et son élégance.")) }; var factory = new RankedLanguageIdentifierFactory( maxNGramLength: 5, maximumSizeOfDistribution: 4000, occuranceNumberThreshold: 0, onlyReadFirstNLines: int.MaxValue, allowMultithreading: true ); var identifier = factory.Train(trainingData); var testResult = identifier.Identify("Guten Morgen, wie geht es Ihnen?"); Console.WriteLine($"Detected: {testResult.First().Item1.Iso639_3}"); ``` ### Response #### Success Response (200) An instance of `ILanguageIdentifier` trained with the provided data. #### Response Example ```csharp // No direct response body, but the identifier object is returned. // Example output from Identify: // Detected: deu ``` ``` -------------------------------- ### CLI: Identify from Stdin Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Identify language from standard input by piping text to the CLI tool. The `-e` flag specifies the encoding, and `-noprompt` prevents interactive prompts. ```bash # Identify from stdin (pipe text input) echo "Der schnelle braune Fuchs" | NTextCat.Cli -e UTF-8 -noprompt # Output: deu ``` -------------------------------- ### CLI: Process Each Line Separately Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Process each line of piped input separately using the `-s` flag. This is useful for multi-line input where each line should be identified independently. ```bash # Process each line separately with -s flag echo -e "Hello world\nBonjour monde\nHallo Welt" | NTextCat.Cli -s -noprompt # Output: # eng # fra # deu ``` -------------------------------- ### RankedLanguageIdentifierFactory.TrainAndSave Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Trains a custom language model and saves it to an XML profile file for later use. This combines training with persistence. ```APIDOC ## RankedLanguageIdentifierFactory.TrainAndSave ### Description Trains a custom language model using the Ranked classifier and immediately saves the resulting profile to a specified XML file. This is useful for creating reusable language models. ### Method Factory Method (Implicit) ### Endpoint N/A (Local Class Method) ### Parameters - **trainingData** (IEnumerable>) - Required - An enumerable of tuples containing language information and text readers for training data. - **profilePath** (string) - Required - The file path where the trained profile will be saved. - **maxNGramLength** (int) - Optional - The maximum length of n-grams to consider (default is typically 5). - **maximumSizeOfDistribution** (int) - Optional - The maximum number of n-grams to keep in the distribution for each language (default is typically 4000). - **occuranceNumberThreshold** (int) - Optional - The minimum number of occurrences for an n-gram to be included (0 means all n-grams are included). - **onlyReadFirstNLines** (int) - Optional - Limits reading to the first N lines of each text reader. - **allowMultithreading** (bool) - Optional - Enables or disables multithreading for training. ### Request Body N/A ### Request Example ```csharp using NTextCat; using System; using System.IO; using System.Linq; string[] trainingFiles = Directory.GetFiles("./training_data", "*.txt"); var trainingData = trainingFiles.Select(filePath => { string langCode = Path.GetFileNameWithoutExtension(filePath); // e.g., "eng.txt" -> "eng" var langInfo = new LanguageInfo(langCode, langCode, null, null); return Tuple.Create(langInfo, (TextReader)new StreamReader(filePath)); }); var factory = new RankedLanguageIdentifierFactory( maxNGramLength: 5, maximumSizeOfDistribution: 4000, occuranceNumberThreshold: 0, onlyReadFirstNLines: int.MaxValue, allowMultithreading: true ); var identifier = factory.TrainAndSave(trainingData, "custom.profile.xml"); // The profile can now be loaded in future sessions var reloadedFactory = new RankedLanguageIdentifierFactory(); var reloadedIdentifier = reloadedFactory.Load("custom.profile.xml"); var result = reloadedIdentifier.Identify("This is a test sentence"); Console.WriteLine($"Language: {result.First().Item1.Iso639_3}"); ``` ### Response #### Success Response (200) An instance of `ILanguageIdentifier` trained with the provided data and the profile saved to the specified file. #### Response Example ```csharp // No direct response body, but the identifier object is returned. // Example output from Identify: // Language: eng ``` ``` -------------------------------- ### CLI: Basic Language Identification Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Perform basic language identification from a command-line argument. The `-l` flag specifies the text to identify. ```bash # Basic language identification from command line argument NTextCat.Cli -l "This is English text" # Output: eng ``` -------------------------------- ### CLI: Specify Input Encoding for File Processing Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Specify the input encoding when processing files from standard input. Use the `<` operator to redirect file content to stdin. ```bash # Specify input encoding for file processing NTextCat.Cli -e UTF-8 -noprompt < input.txt ``` -------------------------------- ### RankedLanguageIdentifier.Identify Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Analyzes a text string and returns an ordered collection of language matches. ```APIDOC ## METHOD RankedLanguageIdentifier.Identify ### Description Accepts a text string and returns an ordered collection of language matches ranked by similarity using distance calculation. ### Parameters #### Request Body - **text** (string) - Required - The text snippet to identify. ### Response - **IEnumerable>** - An ordered collection where Item1 is LanguageInfo (ISO codes/names) and Item2 is the distance score (lower is better). ``` -------------------------------- ### Azure Function for Language Identification in C# Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt An Azure Function that accepts JSON input with a 'text' property and returns the ISO 639-3 language code and confidence score. Ensure the 'content' directory with profile XML is accessible. ```csharp // Azure Function (run.csx) #r "Newtonsoft.Json" using System.Net; using Microsoft.AspNetCore.Mvc; using Newtonsoft.Json; using NTextCat; using System.IO; public static async Task Run(HttpRequest req, ILogger log) { log.LogInformation("Language identification request received"); string requestBody = await new StreamReader(req.Body).ReadToEndAsync(); dynamic data = JsonConvert.DeserializeObject(requestBody); string text = data?.text; if (string.IsNullOrWhiteSpace(text)) return new BadRequestObjectResult("Request body must contain a 'text' property"); var factory = new RankedLanguageIdentifierFactory(); var assemblyLocation = typeof(RankedLanguageIdentifierFactory).Assembly.Location; var profilePath = Path.Combine( Path.GetDirectoryName(assemblyLocation), "content", "Core14.profile.xml"); var identifier = factory.Load(profilePath); var languages = identifier.Identify(text); var mostCertainLanguage = languages.FirstOrDefault(); return new OkObjectResult(new { language = mostCertainLanguage?.Item1.Iso639_3, score = mostCertainLanguage?.Item2 }); } // Example HTTP request: // POST /api/identify // Content-Type: application/json // {"text": "Bonjour tout le monde"} // // Response: {"language": "fra", "score": 4523.0} ``` -------------------------------- ### CharacterNGramExtractor: Extract N-grams from Text Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Extract character n-grams from a given text using `CharacterNGramExtractor`. The `maxNGramLength` parameter defines the maximum length of n-grams to extract. Word boundaries are marked with underscores. ```csharp using NTextCat; using System; using System.Linq; // Create extractor with max 5-gram length var extractor = new CharacterNGramExtractor(maxNGramLength: 5); // Extract n-grams from text string text = "Hello"; var ngrams = extractor.GetFeatures(text).ToList(); // The word "Hello" becomes "_Hello_" and produces: // Unigrams: H, e, l, l, o // Bigrams: _H, He, el, ll, lo, o_ // Trigrams: _He, Hel, ell, llo, lo_, o_ // etc. Console.WriteLine($"Total n-grams extracted: {ngrams.Count}"); Console.WriteLine("Sample n-grams:"); foreach (var ngram in ngrams.Distinct().Take(15)) { Console.WriteLine($" '{ngram}'"); } ``` -------------------------------- ### CharacterNGramExtractor: Extract N-grams from TextReader Source: https://context7.com/ivanakcheurov/ntextcat/llms.txt Extract character n-grams from a `TextReader`, which is suitable for processing large files efficiently. The `GetFeatures` method can accept a `TextReader` directly. ```csharp // Extract from TextReader for large files using (var reader = new StreamReader("large_document.txt")) { var fileNgrams = extractor.GetFeatures(reader); int count = fileNgrams.Count(); Console.WriteLine($"N-grams from file: {count}"); } ``` -------------------------------- ### Identify Language with NTextCat API Source: https://github.com/ivanakcheurov/ntextcat/blob/master/docs/index.html This JavaScript code snippet uses jQuery and XMLHttpRequest to send text to the NTextCat API for language detection. It handles user input, displays a loading message, and updates the result with the detected language or an error message. The API call is debounced to avoid excessive requests. ```javascript $(function () { var timer; var origvalue; function identify() { if ($('#txtLanguage').val().length == 0) { $('#result').html('Please enter some text in the field above.'); return; } $('#result').html('Identifying the language...'); var xhttp = new XMLHttpRequest(); xhttp.onreadystatechange = function () { if (this.readyState == 4) { if (this.status == 200) { $("#dialog:ui-dialog").dialog("destroy"); if (this.responseText.length > 0) { $('#result').html('The language of the text entered is ' + this.responseText.toUpperCase() + '.'); } else { $('#result').html('Cannot determine the language.'); } } else { $('#result').html('Error:' + this.responseText); } } }; xhttp.open("POST", "https://ntextcat-func-win.azurewebsites.net/api/HttpTrigger1", true); xhttp.send(JSON.stringify({ text: $('#txtLanguage').val() })); } $("#txtLanguage").bind('input propertychange', function () { clearTimeout(timer); origvalue = $('#txtLanguage').val(); timer = setTimeout(function () { if (origvalue == $('#txtLanguage').val()) { identify(); } }, 1000); }); $('#lnkButton').button().click(identify); }); ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.