### Run JamSpell HTTP Server Source: https://github.com/bakwc/jamspell/blob/master/README.md Start the web server using a pre-trained model file. ```bash ./web_server/web_server en.bin localhost 8080 ``` -------------------------------- ### Install jamspell Python package Source: https://github.com/bakwc/jamspell/blob/master/README.md Install the jamspell Python package using pip. Ensure swig3 is installed beforehand. ```bash pip install jamspell ``` -------------------------------- ### HTTP API JSON Response Format Source: https://github.com/bakwc/jamspell/blob/master/README.md Example structure of the JSON response returned by the candidates endpoint. ```json { "results": [ { "candidates": [ "best", "beat", "belt", "bet", "bent", "beet", "beit" ], "len": 4, "pos_from": 9 }, { "candidates": [ "checker", "chicken", "checked", "wherein", "coherent", "cheered", "cherokee" ], "len": 7, "pos_from": 20 } ] } ``` -------------------------------- ### JamSpell C++ Usage Source: https://github.com/bakwc/jamspell/blob/master/README.md Example of how to use the JamSpell library in C++ for spell correction and candidate generation. ```APIDOC ## C++ Usage 1. Add `jamspell` and `contrib` directories to your project. 2. Include the necessary header and use the `TSpellCorrector` class. ### Request Example ```cpp #include int main(int argc, const char** argv) { NJamSpell::TSpellCorrector corrector; corrector.LoadLangModel("model.bin"); corrector.FixFragment(L"I am the begt spell cherken!"); // Expected output: "I am the best spell checker!" corrector.GetCandidates({L"i", L"am", L"the", L"begt", L"spell", L"cherken"}, 3); // Expected output: "best", "beat", "belt", "bet", "bent", ... ) corrector.GetCandidates({L"i", L"am", L"the", L"begt", L"spell", L"cherken"}, 3); // Expected output: "checker", "chicken", "checked", "wherein", "coherent", ... ) return 0; } ``` ``` -------------------------------- ### Run interactive correction mode Source: https://context7.com/bakwc/jamspell/llms.txt Start the JamSpell corrector in interactive mode to test corrections directly in the terminal. ```bash # Start interactive mode ./main/jamspell correct en.bin ``` -------------------------------- ### Correct Spelling via HTTP GET Source: https://context7.com/bakwc/jamspell/llms.txt Use the /fix endpoint to correct text passed as a query parameter. ```bash # Start the web server ./web_server/web_server en.bin localhost 8080 # GET request with text parameter curl "http://localhost:8080/fix?text=I%20am%20the%20begt%20spell%20cherken" # Output: I am the best spell checker # URL encode special characters curl "http://localhost:8080/fix?text=Ths%20sentance%20has%20erors" # Output: This sentence has errors ``` -------------------------------- ### HTTP API - GET /candidates Source: https://context7.com/bakwc/jamspell/llms.txt Returns JSON with spelling error positions and ranked correction candidates for misspelled words. ```APIDOC ## GET /candidates - Get Correction Candidates ### Description Returns JSON with spelling error positions and ranked correction candidates for each misspelled word. ### Method GET ### Endpoint `/candidates` ### Parameters #### Path Parameters None #### Query Parameters - **text** (string) - Required - The text to analyze for candidates. #### Request Body None ### Request Example ```bash # GET request for candidates curl "http://localhost:8080/candidates?text=I%20am%20the%20begt%20spell%20cherken" ``` ### Response #### Success Response (200) JSON object containing an array of results, where each result details the position, length, and candidates for a misspelled word. #### Response Example ```json { "results": [ { "candidates": [ "best", "beat", "belt", "bet", "bent", "beet", "beit" ], "len": 4, "pos_from": 9 }, { "candidates": [ "checker", "chicken", "checked", "wherein", "coherent", "cheered", "cherokee" ], "len": 7, "pos_from": 20 } ] } ``` ``` -------------------------------- ### Build JamSpell from Source Source: https://github.com/bakwc/jamspell/blob/master/README.md Standard build procedure using CMake for the library and HTTP server. ```bash git clone https://github.com/bakwc/JamSpell.git cd JamSpell mkdir build cd build cmake .. make ``` -------------------------------- ### Generate training and test datasets Source: https://context7.com/bakwc/jamspell/llms.txt Split a source text file into training and testing sets for model development. ```bash # Generate train/test split from text file python evaluate/generate_dataset.py source_text.txt output_prefix ``` -------------------------------- ### Train a Custom Model Source: https://github.com/bakwc/jamspell/blob/master/README.md Execute the training command using an alphabet file and a corpus of text. ```bash ./main/jamspell train ../test_data/alphabet_en.txt ../test_data/sherlockholmes.txt model_sherlock.bin ``` -------------------------------- ### Configure Jamspell Tests with CMake Source: https://github.com/bakwc/jamspell/blob/master/tests/CMakeLists.txt Sets up the build environment for Jamspell tests, linking against the Jamspell library and Google Test. ```cmake enable_testing() include_directories(${GTEST_INCLUDE_DIRS}) add_executable(jamspell_tests test_perfect_hash.cpp) target_link_libraries(jamspell_tests jamspell_lib ${GTEST_BOTH_LIBRARIES} pthread) add_test(jamspell_tests jamspell_tests) ``` -------------------------------- ### Build and train a JamSpell model Source: https://context7.com/bakwc/jamspell/llms.txt Compile the JamSpell source and train a new model using an alphabet file and a training dataset. ```bash # Build JamSpell git clone https://github.com/bakwc/JamSpell.git cd JamSpell mkdir build && cd build cmake .. make # Train a model # Usage: jamspell train alphabet.txt dataset.txt resultModel.bin ./main/jamspell train ../test_data/alphabet_en.txt ../test_data/sherlockholmes.txt sherlock_model.bin ``` -------------------------------- ### C++ API - Training a Model Source: https://context7.com/bakwc/jamspell/llms.txt Instructions for training a custom language model from scratch using the C++ API. ```APIDOC ## C++ API - Training a Model ### Description Train a custom language model from scratch using the C++ API. ### Method `TrainLangModel(const std::string& textFile, const std::string& alphabetFile, const std::string& outputModelFile)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```cpp #include #include int main() { NJamSpell::TSpellCorrector corrector; // Train a new model // Parameters: textFile, alphabetFile, outputModelFile bool success = corrector.TrainLangModel( "training_data.txt", // UTF-8 text corpus "alphabet_en.txt", // Alphabet file "my_model.bin" // Output model ); if (!success) { std::cerr << "Training failed" << std::endl; return 1; } // Model is now loaded and ready to use std::wstring result = corrector.FixFragment(L"tset input"); std::wcout << result << std::endl; // Output: L"test input" return 0; } ``` ### Response Returns a boolean indicating success or failure of the training process. #### Success Response (200) `true` if training was successful. #### Response Example `true` ``` -------------------------------- ### Interact with JamSpell HTTP API Source: https://github.com/bakwc/jamspell/blob/master/README.md Perform spell correction or candidate retrieval using GET or POST requests. ```bash curl "http://localhost:8080/fix?text=I am the begt spell cherken" ``` ```bash curl -d "I am the begt spell cherken" http://localhost:8080/fix ``` ```bash curl "http://localhost:8080/candidates?text=I am the begt spell cherken" # or curl -d "I am the begt spell cherken" http://localhost:8080/candidates ``` -------------------------------- ### Jamspell CMake Configuration Source: https://github.com/bakwc/jamspell/blob/master/CMakeLists.txt Sets up the CMake build system for the Jamspell project. Includes C++ standard, PIC flag, debug symbols, and finds Google Test. ```cmake cmake_minimum_required(VERSION 3.18) project(jamspell) option(USE_BOOST_CONVERT "use Boost.Locale instead of std::codecvt for string conversion" OFF) set(CMAKE_CXX_FLAGS "-std=c++11 -fPIC -g") find_package(GTest) link_directories(${PROJECT_BINARY_DIR}/jamspell) include_directories(${CMAKE_SOURCE_DIR}) # find Boost if necessary if(USE_BOOST_CONVERT) set(Boost_USE_STATIC_LIBS ON) set(Boost_USE_MULTITHREADED ON) find_package(Boost REQUIRED) message(STATUS "Using Boost string convert: Enabled") add_definitions(-DUSE_BOOST_CONVERT) endif() find_package (Threads) add_subdirectory(jamspell) add_subdirectory(main) add_subdirectory(contrib) add_subdirectory(web_server) if(GTest_FOUND) add_subdirectory(tests) endif() ``` -------------------------------- ### HTTP API - GET /fix Source: https://context7.com/bakwc/jamspell/llms.txt Corrects spelling errors in text provided as a query parameter. Returns plain text with corrections. ```APIDOC ## GET /fix - Fix Text Spelling ### Description Corrects spelling errors in text passed as a query parameter. Returns plain text with corrections applied. ### Method GET ### Endpoint `/fix` ### Parameters #### Path Parameters None #### Query Parameters - **text** (string) - Required - The text to correct. #### Request Body None ### Request Example ```bash # Start the web server ./web_server/web_server en.bin localhost 8080 # GET request with text parameter curl "http://localhost:8080/fix?text=I%20am%20the%20begt%20spell%20cherken" # Output: I am the best spell checker # URL encode special characters curl "http://localhost:8080/fix?text=Ths%20sentance%20has%20erors" # Output: This sentence has errors ``` ### Response #### Success Response (200) Plain text with spelling corrections applied. #### Response Example ``` I am the best spell checker ``` ``` -------------------------------- ### Evaluate a Model Source: https://github.com/bakwc/jamspell/blob/master/README.md Run the evaluation script to test model performance against a dataset. ```bash python evaluate/evaluate.py -a alphabet_file.txt -jsp your_model.bin -mx 50000 your_test_data.txt ``` -------------------------------- ### JamSpell Pre-trained Models Source: https://github.com/bakwc/jamspell/blob/master/README.md Information on downloading pre-trained language models for JamSpell. ```APIDOC ## Download Models Pre-trained models are available for download. However, training your own model on a larger dataset is recommended for better quality. - [en.tar.gz](https://github.com/bakwc/JamSpell-models/raw/master/en.tar.gz) (35Mb) - [fr.tar.gz](https://github.com/bakwc/JamSpell-models/raw/master/fr.tar.gz) (31Mb) - [ru.tar.gz](https://github.com/bakwc/JamSpell-models/raw/master/ru.tar.gz) (38Mb) ``` -------------------------------- ### Configure Candidate Evaluation in Python Source: https://context7.com/bakwc/jamspell/llms.txt Adjust the number of candidates checked to balance accuracy and performance. ```python import jamspell corrector = jamspell.TSpellCorrector() corrector.LoadLangModel('en.bin') # Increase candidates for potentially better accuracy (default: 14) corrector.SetMaxCandidatesToCheck(20) # Or decrease for faster processing corrector.SetMaxCandidatesToCheck(7) text = "speling erors here" corrected = corrector.FixFragment(text) print(corrected) ``` -------------------------------- ### JamSpell Model Training Source: https://github.com/bakwc/jamspell/blob/master/README.md Instructions on how to train custom language models for JamSpell. ```APIDOC ## Train Custom Model To train a custom model, follow these steps: 1. Install ```cmake```. 2. Clone and build JamSpell: ```bash git clone https://github.com/bakwc/JamSpell.git cd JamSpell mkdir build cd build cmake .. make ``` 3. Prepare training data: - A UTF-8 text file with sentences (e.g., `sherlockholmes.txt`). - A file with the language alphabet (e.g., `alphabet_en.txt`). 4. Train the model: ```bash ./main/jamspell train ../test_data/alphabet_en.txt ../test_data/sherlockholmes.txt model_sherlock.bin ``` 5. Evaluate the spellchecker using the provided script: ```bash python evaluate/evaluate.py -a alphabet_file.txt -jsp your_model.bin -mx 50000 your_test_data.txt ``` 6. Use `evaluate/generate_dataset.py` to create training/testing data from various formats like text files, Leipzig Corpora Collection, and fb2 books. ``` -------------------------------- ### Train a Language Model in C++ Source: https://context7.com/bakwc/jamspell/llms.txt Create a custom language model from a text corpus and alphabet file. ```cpp #include #include int main() { NJamSpell::TSpellCorrector corrector; // Train a new model // Parameters: textFile, alphabetFile, outputModelFile bool success = corrector.TrainLangModel( "training_data.txt", // UTF-8 text corpus "alphabet_en.txt", // Alphabet file "my_model.bin" // Output model ); if (!success) { std::cerr << "Training failed" << std::endl; return 1; } // Model is now loaded and ready to use std::wstring result = corrector.FixFragment(L"tset input"); std::wcout << result << std::endl; // Output: L"test input" return 0; } ``` -------------------------------- ### Train a Custom Language Model Source: https://context7.com/bakwc/jamspell/llms.txt Trains a new model using a UTF-8 text corpus and an alphabet file. ```python import jamspell corrector = jamspell.TSpellCorrector() # Train a new model from text data # - text_file: UTF-8 text file with training sentences # - alphabet_file: File containing valid alphabet characters # - model_file: Output path for the trained model success = corrector.TrainLangModel( 'training_corpus.txt', # Training text 'alphabet_en.txt', # Alphabet file (one char per line) 'custom_model.bin' # Output model file ) if success: print("Model trained and saved successfully") # The corrector is now loaded with the trained model result = corrector.FixFragment("tset text") print(result) ``` -------------------------------- ### Load a Language Model in Python Source: https://context7.com/bakwc/jamspell/llms.txt Initializes the spell corrector and loads a binary model file. Returns a boolean indicating success. ```python import jamspell # Initialize the spell corrector corrector = jamspell.TSpellCorrector() # Load a pre-trained model (returns True on success) success = corrector.LoadLangModel('en.bin') if not success: raise Exception('Failed to load language model') # The corrector is now ready for spell checking print("Model loaded successfully") ``` -------------------------------- ### Configure Jamspell Web Server Executable in CMake Source: https://github.com/bakwc/jamspell/blob/master/web_server/CMakeLists.txt Defines the web_server executable and links platform-specific network libraries and the Jamspell library. ```cmake add_executable(web_server main.cpp) if(WIN32) target_link_libraries(web_server wsock32 ws2_32 jamspell_lib ${CMAKE_THREAD_LIBS_INIT}) else() target_link_libraries(web_server jamspell_lib ${CMAKE_THREAD_LIBS_INIT}) endif() ``` -------------------------------- ### Score sentences with the language model Source: https://context7.com/bakwc/jamspell/llms.txt Output the language model probability for sentences to debug model performance. ```bash # Score mode - outputs language model probability ./main/jamspell score en.bin ``` -------------------------------- ### Integrate JamSpell in C++ Source: https://github.com/bakwc/jamspell/blob/master/README.md Initialize the spell corrector with a model file and use it to fix text or retrieve spelling candidates. ```cpp #include int main(int argc, const char** argv) { NJamSpell::TSpellCorrector corrector; corrector.LoadLangModel("model.bin"); corrector.FixFragment(L"I am the begt spell cherken!"); // "I am the best spell checker!" corrector.GetCandidates({L"i", L"am", L"the", L"begt", L"spell", L"cherken"}, 3); // "best", "beat", "belt", "bet", "bent", ... ) corrector.GetCandidates({L"i", L"am", L"the", L"begt", L"spell", L"cherken"}, 3); // "checker", "chicken", "checked", "wherein", "coherent", ... ) return 0; } ``` -------------------------------- ### Evaluate model performance programmatically Source: https://context7.com/bakwc/jamspell/llms.txt Use the Python evaluation script to benchmark the accuracy of a trained model. ```python from evaluate.evaluate import evaluateJamspell # Returns: (errorsRate, fixRate, broken, topNerr, topNfix) results = evaluateJamspell( modelFile='en.bin', testText='test_data.txt', alphabetFile='alphabet_en.txt', maxWords=50000 ) errors_rate, fix_rate, broken_rate, top7_errors, top7_fix = results print(f"Error rate: {errors_rate:.2%}") print(f"Fix rate: {fix_rate:.2%}") print(f"Broken rate: {broken_rate:.2%}") print(f"Top-7 error rate: {top7_errors:.2%}") print(f"Top-7 fix rate: {top7_fix:.2%}") ``` -------------------------------- ### Query candidates via HTTP API Source: https://context7.com/bakwc/jamspell/llms.txt Send a POST request to the JamSpell HTTP server to retrieve spelling candidates for a given string. ```bash curl -d "I am the begt spell cherken" http://localhost:8080/candidates ``` -------------------------------- ### Process files in batch mode Source: https://context7.com/bakwc/jamspell/llms.txt Use the CLI to fix spelling errors in an entire text file. ```bash # Fix all errors in a file # Usage: jamspell fix model.bin input.txt output.txt ./main/jamspell fix en.bin input_with_errors.txt corrected_output.txt ``` -------------------------------- ### Define Jamspell Library Source: https://github.com/bakwc/jamspell/blob/master/jamspell/CMakeLists.txt Defines a CMake library named 'jamspell_lib' and lists its source files. This is a foundational step for building the project. ```cmake add_library(jamspell_lib spell_corrector.cpp lang_model.cpp utils.cpp perfect_hash.cpp bloom_filter.cpp) ``` -------------------------------- ### Python API - TSpellCorrector.SetMaxCandidatesToCheck Source: https://context7.com/bakwc/jamspell/llms.txt Demonstrates how to adjust the maximum number of candidates to check for each word in the Python API. Higher values can improve accuracy at the cost of performance. ```APIDOC ## Python API - TSpellCorrector.SetMaxCandidatesToCheck ### Description Sets the maximum number of candidates to evaluate for each word. Higher values may improve accuracy but reduce performance. ### Method `SetMaxCandidatesToCheck(self, num_candidates: int)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import jamspell corrector = jamspell.TSpellCorrector() corrector.LoadLangModel('en.bin') # Increase candidates for potentially better accuracy (default: 14) corrector.SetMaxCandidatesToCheck(20) # Or decrease for faster processing corrector.SetMaxCandidatesToCheck(7) text = "speling erors here" corrected = corrector.FixFragment(text) print(corrected) ``` ### Response None (This is a method call that modifies corrector state) #### Success Response (200) None #### Response Example None ``` -------------------------------- ### Retrieve Correction Candidates via HTTP Source: https://context7.com/bakwc/jamspell/llms.txt Fetch JSON containing spelling error positions and ranked candidates. ```bash # GET request for candidates curl "http://localhost:8080/candidates?text=I%20am%20the%20begt%20spell%20cherken" ``` ```json { "results": [ { "candidates": [ "best", "beat", "belt", "bet", "bent", "beet", "beit" ], "len": 4, "pos_from": 9 }, { "candidates": [ "checker", "chicken", "checked", "wherein", "coherent", "cheered", "cherokee" ], "len": 7, "pos_from": 20 } ] } ``` -------------------------------- ### Link Boost Libraries Conditionally Source: https://github.com/bakwc/jamspell/blob/master/jamspell/CMakeLists.txt Conditionally includes Boost directories and links Boost libraries to 'jamspell_lib' if Boost is found. This allows Jamspell to leverage Boost functionalities when available. ```cmake if(Boost_FOUND) include_directories(${Boost_INCLUDE_DIRS}) target_link_libraries(jamspell_lib ${Boost_LIBRARIES}) endif() ```