### Convert File Encoding to UTF-8 - C# Source: https://context7.com/charsetdetector/utf-unknown/llms.txt This example demonstrates reading a file with its detected encoding and writing it to a new file encoded as UTF-8. It includes error handling for detection failures and unsupported .NET encodings. ```csharp using UtfUnknown; using System; using System.IO; using System.Text; public class EncodingConverter { public static async Task ConvertToUtf8Async(string inputPath, string outputPath) { // Detect source encoding var result = await CharsetDetector.DetectFromFileAsync(inputPath); if (result.Detected == null) { throw new InvalidOperationException($"Could not detect encoding for: {inputPath}"); } var sourceEncoding = result.Detected.Encoding; var encodingName = result.Detected.EncodingName; var confidence = result.Detected.Confidence; Console.WriteLine($"Detected: {encodingName} (confidence: {confidence:P0})"); if (sourceEncoding == null) { throw new NotSupportedException($"Encoding '{encodingName}' is not supported in .NET"); } // Read with detected encoding, write as UTF-8 string content = await File.ReadAllTextAsync(inputPath, sourceEncoding); await File.WriteAllTextAsync(outputPath, content, new UTF8Encoding(encoderShouldEmitUTF8Identifier: true)); Console.WriteLine($"Converted {inputPath} from {encodingName} to UTF-8"); } public static void ProcessDirectory(string directoryPath) { foreach (var file in Directory.GetFiles(directoryPath, "*.txt")) { var result = CharsetDetector.DetectFromFile(file); if (result.Detected != null) { var detail = result.Detected; Console.WriteLine($"{Path.GetFileName(file)}: {detail.EncodingName} " + $ ``` ```csharp } else { Console.WriteLine($"{Path.GetFileName(file)}: Unknown encoding"); } } } } // Usage await EncodingConverter.ConvertToUtf8Async("legacy-document.txt", "output-utf8.txt"); EncodingConverter.ProcessDirectory("./documents"); ``` -------------------------------- ### Get Detailed Encoding Information - C# Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Access detailed encoding information like name, confidence, and BOM presence using the DetectionDetail class. The System.Text.Encoding instance may be null if not available in .NET. ```csharp using UtfUnknown; using System.Text; DetectionResult result = CharsetDetector.DetectFromFile("document.txt"); DetectionDetail detail = result.Detected; if (detail != null) { // Encoding name (short identifier) string name = detail.EncodingName; // e.g., "utf-8", "windows-1251", "shift-jis" // System.Text.Encoding (null if not available in .NET) Encoding encoding = detail.Encoding; // Confidence level (0.0 to 1.0) float confidence = detail.Confidence; // BOM detection bool hasBom = detail.HasBOM; // String representation Console.WriteLine(detail.ToString()); // Output: "Detected utf-8 with confidence of 1. (BOM: False)" // Safe encoding usage if (encoding != null) { string content = File.ReadAllText("document.txt", encoding); Console.WriteLine($"Successfully read {content.Length} characters"); } else { // Some encodings like CP949 are mapped to compatible alternatives // CP949 -> ks_c_5601-1987 // ISO-2022-CN -> x-cp50227 Console.WriteLine($"Encoding '{name}' not directly available, check for alternatives"); } } ``` -------------------------------- ### Work with Detection Results Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Access detailed detection information, including the best match, confidence levels, and alternative candidates. Filter results by confidence and check for .NET compatibility. ```csharp using UtfUnknown; using System.Linq; DetectionResult result = CharsetDetector.DetectFromFile("multilingual.txt"); // Access best detection DetectionDetail best = result.Detected; if (best != null) { Console.WriteLine($"Best match: {best.EncodingName}"); Console.WriteLine($"Confidence: {best.Confidence:P0}"); Console.WriteLine($"Has BOM: {best.HasBOM}"); Console.WriteLine($"Encoding object: {best.Encoding?.WebName ?? "Not available"}"); } // Access all detection candidates (sorted by confidence) IList allResults = result.Details; if (allResults != null && allResults.Count > 1) { Console.WriteLine("\nAll candidates:"); foreach (var detail in allResults) { Console.WriteLine($" {detail.EncodingName}: {detail.Confidence:P0}"); } } // Filter results by minimum confidence var highConfidenceResults = result.Details? .Where(d => d.Confidence >= 0.8f) .ToList(); Console.WriteLine($"\nHigh confidence matches: {highConfidenceResults?.Count ?? 0}"); // Check if encoding is usable in .NET if (result.Detected?.Encoding != null) { // Safe to use for reading/writing var encoding = result.Detected.Encoding; Console.WriteLine($"Can use System.Text.Encoding: {encoding.EncodingName}"); } else { // Encoding detected but not available in .NET Console.WriteLine($"Detected {result.Detected?.EncodingName} but no .NET Encoding available"); } ``` -------------------------------- ### Detect Character Set from File (Asynchronous) Source: https://github.com/charsetdetector/utf-unknown/blob/master/README.md Use DetectFromFileAsync for asynchronous character set detection from a file. A CancellationToken can be provided to manage the asynchronous operation. ```csharp // Detect from File asynchronously DetectionResult result = await CharsetDetector.DetectFromFileAsync("path/to/file.txt", cancellationToken); // or pass FileInfo ``` -------------------------------- ### Accessing Detection Details Source: https://github.com/charsetdetector/utf-unknown/blob/master/README.md After detection, you can access the best detected encoding, its alias, the System.Text.Encoding object, and the confidence level. All detected details are available in the Details list. ```csharp // Get the best Detection DetectionDetail resultDetected = results.Detected; // Get the alias of the found encoding string encodingName = resultDetected.EncodingName; // Get the System.Text.Encoding of the found encoding (can be null if not available) Encoding encoding = resultDetected.Encoding; // Get the confidence of the found encoding (between 0 and 1) float confidence = resultDetected.Confidence; // Get all the details of the result IList allDetails = result.Details; ``` -------------------------------- ### Detect Character Set from File (Synchronous) Source: https://github.com/charsetdetector/utf-unknown/blob/master/README.md Use DetectFromFile to determine the character encoding of a file. You can pass a file path or a FileInfo object. This method is synchronous. ```csharp // Detect from File DetectionResult result = CharsetDetector.DetectFromFile("path/to/file.txt"); // or pass FileInfo ``` -------------------------------- ### DetectFromFileAsync Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Asynchronously detects the character encoding of a file, suitable for UI applications or non-blocking operations. ```APIDOC ## DetectFromFileAsync ### Description Asynchronously detects the character encoding of a file with optional cancellation support. ### Method Static Method (Async) ### Parameters #### Path Parameters - **path** (string) - Required - The file path to detect. - **fileInfo** (FileInfo) - Required - The FileInfo object to detect. - **cancellationToken** (CancellationToken) - Optional - Token to cancel the operation. ### Response - **DetectionResult** (object) - Contains Detected (EncodingInfo) with encoding details. ``` -------------------------------- ### Detect Character Set from Stream (Asynchronous) Source: https://github.com/charsetdetector/utf-unknown/blob/master/README.md Use DetectFromStreamAsync for asynchronous character set detection from a stream. This method accepts a CancellationToken for operation management. ```csharp // Detect from Stream asynchronously result = await CharsetDetector.DetectFromStreamAsync(stream, cancellationToken); ``` -------------------------------- ### DetectionResult Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Describes the structure of the detection result, including the best match and alternative candidates. ```APIDOC ## DetectionResult ### Description The DetectionResult class contains the primary detection result via the Detected property, as well as alternative results in the Details list sorted by confidence level. ### Response Fields - **Detected** (DetectionDetail) - The best match found. - **Details** (IList) - All detection candidates sorted by confidence. ### DetectionDetail Fields - **EncodingName** (string) - The name of the detected encoding. - **Confidence** (float) - The confidence level of the detection. - **HasBOM** (bool) - Indicates if a Byte Order Mark was present. - **Encoding** (Encoding) - The .NET Encoding object if available. ``` -------------------------------- ### Detect Character Set from Stream (Synchronous) Source: https://github.com/charsetdetector/utf-unknown/blob/master/README.md Use DetectFromStream to determine the character encoding of a stream. This method is synchronous and requires an open stream. ```csharp // Detect from Stream result = CharsetDetector.DetectFromStream(stream); ``` -------------------------------- ### DetectFromFile Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Detects the character encoding of a file by reading its contents from a file path or FileInfo object. ```APIDOC ## DetectFromFile ### Description Detects the character encoding of a file by reading its contents. The method automatically handles file stream management and returns complete detection results including the encoding name, confidence level, and whether a BOM was detected. ### Method Static Method ### Parameters #### Path Parameters - **path** (string) - Required - The file path to detect. - **fileInfo** (FileInfo) - Required - The FileInfo object to detect. ### Response - **DetectionResult** (object) - Contains Detected (EncodingInfo), which includes EncodingName, Encoding (System.Text.Encoding), Confidence (float), and HasBOM (bool). ``` -------------------------------- ### Detect Character Set from Bytes (Synchronous) Source: https://github.com/charsetdetector/utf-unknown/blob/master/README.md Use DetectFromBytes to determine the character encoding of a byte array. This method is synchronous and returns a list of potential detections. ```csharp // Detect from bytes results = CharsetDetector.DetectFromBytes(byteArray); ``` -------------------------------- ### Asynchronous File Encoding Detection - C# Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Asynchronously detects a file's character encoding, supporting cancellation. This is suitable for UI applications to prevent blocking. Accepts a FileInfo object and a CancellationToken. ```csharp using UtfUnknown; using System.Threading; // Basic async detection DetectionResult result = await CharsetDetector.DetectFromFileAsync("path/to/large-file.txt"); Console.WriteLine($"Detected: {result.Detected?.EncodingName ?? "Unknown"}"); // With cancellation token using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30)); try { result = await CharsetDetector.DetectFromFileAsync("path/to/file.txt", cts.Token); if (result.Detected != null) { Console.WriteLine($"Encoding: {result.Detected.EncodingName}"); Console.WriteLine($"Confidence: {result.Detected.Confidence:P0}"); } } catch (OperationCanceledException) { Console.WriteLine("Detection was cancelled"); } // Using FileInfo with cancellation var fileInfo = new FileInfo("path/to/document.txt"); result = await CharsetDetector.DetectFromFileAsync(fileInfo, cts.Token); ``` -------------------------------- ### Detect Encoding from File Path - C# Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Detects the character encoding of a file using its path. Handles file stream management and returns encoding details. Can also accept a FileInfo object. ```csharp using UtfUnknown; // Basic file detection DetectionResult result = CharsetDetector.DetectFromFile("path/to/document.txt"); if (result.Detected != null) { // Get encoding information string encodingName = result.Detected.EncodingName; // e.g., "utf-8", "shift-jis" System.Text.Encoding encoding = result.Detected.Encoding; // System.Text.Encoding object float confidence = result.Detected.Confidence; // 0.0 to 1.0 bool hasBom = result.Detected.HasBOM; // true if BOM detected Console.WriteLine($"Encoding: {encodingName}"); Console.WriteLine($"Confidence: {confidence:P0}"); Console.WriteLine($"Has BOM: {hasBom}"); // Read file with detected encoding if (encoding != null) { string content = File.ReadAllText("path/to/document.txt", encoding); } } else { Console.WriteLine("Could not detect encoding"); } // Using FileInfo var fileInfo = new FileInfo("path/to/document.txt"); result = CharsetDetector.DetectFromFile(fileInfo); ``` -------------------------------- ### DetectFromStreamAsync Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Asynchronously detects character encoding from a stream with optional byte limit and cancellation support. ```APIDOC ## DetectFromStreamAsync ### Description Asynchronously detects character encoding from a stream with optional byte limit and cancellation support. Best for non-blocking I/O operations with large or slow streams. ### Parameters #### Method Parameters - **stream** (Stream) - Required - The input stream to analyze. - **maxBytesToRead** (int) - Optional - Maximum number of bytes to read for detection. - **cancellationToken** (CancellationToken) - Optional - Token to cancel the operation. ### Response - **DetectionResult** (Object) - Contains the detected encoding details. ``` -------------------------------- ### DetectFromStream Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Detects the character encoding from any readable stream, such as network or memory streams. ```APIDOC ## DetectFromStream ### Description Detects the character encoding from any readable stream. Supports an optional parameter to limit the number of bytes read for performance optimization. ### Method Static Method ### Parameters #### Path Parameters - **stream** (Stream) - Required - The readable stream to analyze. - **maxBytesToRead** (int) - Optional - Limit the number of bytes read for performance optimization. ### Response - **DetectionResult** (object) - Contains Detected (EncodingInfo) with encoding details. ``` -------------------------------- ### Detect Character Encoding from Byte Array Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Use DetectFromBytes for direct encoding detection from in-memory byte arrays. Supports specifying an offset and length to analyze a specific portion of the data. ```csharp using UtfUnknown; using System.Text; // Basic byte array detection string text = "こんにちは世界"; // Japanese text byte[] bytes = Encoding.UTF8.GetBytes(text); DetectionResult result = CharsetDetector.DetectFromBytes(bytes); Console.WriteLine($"Encoding: {result.Detected?.EncodingName}"); // Output: utf-8 Console.WriteLine($"Confidence: {result.Detected?.Confidence:P0}"); // Output: 100% ``` ```csharp // Detect with offset and length (partial array) byte[] mixedData = new byte[1000]; // ... populate with data result = CharsetDetector.DetectFromBytes(mixedData, offset: 100, len: 500); Console.WriteLine($"Partial detection: {result.Detected?.EncodingName}"); ``` ```csharp // Detecting specific BOM patterns byte[] utf8Bom = { 0xEF, 0xBB, 0xBF, 0x68, 0x65, 0x6C, 0x6C, 0x6F }; // UTF-8 with BOM result = CharsetDetector.DetectFromBytes(utf8Bom); Console.WriteLine($"Has BOM: {result.Detected?.HasBOM}"); // Output: True Console.WriteLine($"Encoding: {result.Detected?.EncodingName}"); // Output: utf-8 byte[] utf16LeBom = { 0xFF, 0xFE, 0x68, 0x00, 0x65, 0x00 }; // UTF-16 LE with BOM result = CharsetDetector.DetectFromBytes(utf16LeBom); Console.WriteLine($"Encoding: {result.Detected?.EncodingName}"); // Output: utf-16le ``` ```csharp // Korean text detection string koreanText = "UTF-Unknown은 파일, 스트림의 캐릭터 셋을 탐지합니다."; bytes = Encoding.UTF8.GetBytes(koreanText); result = CharsetDetector.DetectFromBytes(bytes); Console.WriteLine($"Korean text encoding: {result.Detected?.EncodingName}"); ``` -------------------------------- ### DetectFromBytes Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Detects character encoding directly from a byte array, supporting optional offset and length parameters. ```APIDOC ## DetectFromBytes ### Description Detects character encoding directly from a byte array. Useful when data is already loaded in memory. Supports optional offset and length parameters for detecting a specific portion of the data. ### Parameters #### Method Parameters - **bytes** (byte[]) - Required - The byte array to analyze. - **offset** (int) - Optional - The starting position in the array. - **len** (int) - Optional - The number of bytes to process. ### Response - **DetectionResult** (Object) - Contains the detected encoding details. ``` -------------------------------- ### Detect Encoding from Stream - C# Source: https://context7.com/charsetdetector/utf-unknown/llms.txt Detects character encoding from any readable stream, such as MemoryStream or network streams. Supports limiting the number of bytes read for performance. Note that the stream position is not reset after detection. ```csharp using UtfUnknown; using System.IO; // From MemoryStream byte[] data = File.ReadAllBytes("document.txt"); using var memoryStream = new MemoryStream(data); DetectionResult result = CharsetDetector.DetectFromStream(memoryStream); Console.WriteLine($"Detected: {result.Detected?.EncodingName}"); // Limit bytes read for large streams (improves performance) using var largeStream = new FileStream("large-file.txt", FileMode.Open); result = CharsetDetector.DetectFromStream(largeStream, maxBytesToRead: 4096); if (result.Detected != null) { Console.WriteLine($"Encoding: {result.Detected.EncodingName}"); Console.WriteLine($"Confidence: {result.Detected.Confidence:P0}"); } // Note: Stream position is not reset before or after detection // Reset manually if needed: largeStream.Position = 0; // From network stream (example with HttpClient) using var httpClient = new HttpClient(); using var response = await httpClient.GetStreamAsync("https://example.com/data.txt"); using var bufferedStream = new MemoryStream(); await response.CopyToAsync(bufferedStream); bufferedStream.Position = 0; result = CharsetDetector.DetectFromStream(bufferedStream); Console.WriteLine($"Remote file encoding: {result.Detected?.EncodingName}"); ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.