### Start Voikko Development Environment Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/devenv/README.md Run this command to start a new bash session within the pre-configured Voikko development environment. ```bash docker-compose run devenv bash ``` -------------------------------- ### Voikkotest Quick Mode Output Example Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting This is an example of a successful test run output when using voikkotest in quick mode. It shows the compilation status and the results of various test suites. ```text Starting test suite under /home/harri/git/corevoikko/tests/voikkotest/fi-x-malmor "voikko-fi_FI.pro" is up to date. "voikko-fi_FI.pro" is up to date. install -m 755 -d /home/harri/tmp/voikkotest/inst/2/mor-malmor install -m 644 /home/harri/tmp/voikkotest/build/fi-x-malmor/voikko-fi_FI.pro /home/harri/tmp/voikkotest/build/fi-x-malmor/voikko-fi_FI.lex_? /home/harri/tmp/voikkotest/build/fi-x-malmor/voikko-fi_FI.mor_? /home/harri/tmp/voikkotest/build/fi-x-malmor/voikko-fi_FI.sym_? /home/harri/tmp/voikkotest/inst/2/mor-malmor Build complete, starting tests All 33 morphological tests were successful. Starting test suite under /home/harri/git/corevoikko/tests/voikkotest/fi-x-vfst make: Kohteelle "vvfst" ei tarvitse tehdä mitään. install -m 755 -d /home/harri/tmp/voikkotest/inst/2/mor-vfstd install -m 644 /home/harri/tmp/voikkotest/build/fi-x-vfstd/voikko-fi_FI.pro /home/harri/tmp/voikkotest/build/fi-x-vfstd/mor.vfst /home/harri/tmp/voikkotest/inst/2/mor-vfstd Build complete, starting tests All 24 spelling tests were successful. All 5 morphological tests were successful. Starting test suite under /home/harri/git/corevoikko/tests/voikkotest/fi-x-malstd "voikko-fi_FI.pro" is up to date. "voikko-fi_FI.pro" is up to date. install -m 755 -d /home/harri/tmp/voikkotest/inst/2/mor-malstd install -m 644 /home/harri/tmp/voikkotest/build/fi-x-malstd/voikko-fi_FI.pro /home/harri/tmp/voikkotest/build/fi-x-malstd/voikko-fi_FI.lex_? /home/harri/tmp/voikkotest/build/fi-x-malstd/voikko-fi_FI.mor_? /home/harri/tmp/voikkotest/build/fi-x-malstd/voikko-fi_FI.sym_? /home/harri/tmp/voikkotest/inst/2/mor-malstd Build complete, starting tests All 863 spelling tests were successful. All 50 suggestion tests were successful. All 127 morphological tests were successful. All 72 hyphenation tests were successful. All 164 grammar tests were successful. All 31 tokenizer tests were successful. All 18 sentence splitting tests were successful. All 4 compatibility tests were successful. ``` -------------------------------- ### Voikko Dictionary Compatibility Test Files Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting Example file structure for Voikko dictionary compatibility tests. Dictionaries should be placed in subdirectories under $HOME/tmp/voikkotest/compatibility/ starting with 'mor-'. ```shell $HOME/tmp/voikkotest/compatibility/mor-sm1.4/2/mor-standard/voikko-fi_FI.pro $HOME/tmp/voikkotest/compatibility/mor-sm1.4/2/mor-standard/voikko-fi_FI.sym_l $HOME/tmp/voikkotest/compatibility/mor-sm1.4/2/mor-standard/voikko-fi_FI.mor_l $HOME/tmp/voikkotest/compatibility/mor-sm1.4/2/mor-standard/voikko-fi_FI.lex_l $HOME/tmp/voikkotest/compatibility/mor-sm1.5/2/mor-standard/voikko-fi_FI.pro $HOME/tmp/voikkotest/compatibility/mor-sm1.5/2/mor-standard/voikko-fi_FI.sym_l $HOME/tmp/voikkotest/compatibility/mor-sm1.5/2/mor-standard/voikko-fi_FI.mor_l $HOME/tmp/voikkotest/compatibility/mor-sm1.5/2/mor-standard/voikko-fi_FI.lex_l ``` -------------------------------- ### Testing Java Interface Installation on Windows Source: https://github.com/voikko/corevoikko/wiki/java-install Run unit tests for the Java interface on Windows to verify installation and functionality. Ensure JUnit and JNA JAR files are in the classpath. ```bash java -cp libvoikko-3.2-SNAPSHOT.jar;libvoikko-3.2-SNAPSHOT-tests.jar;junit4.jar;jna.jar org.junit.runner.JUnitCore org.puimula.libvoikko.VoikkoTest org.puimula.libvoikko.VoikkoJnaLatinTest ``` -------------------------------- ### Testing Java Interface Installation on Linux Source: https://github.com/voikko/corevoikko/wiki/java-install Run unit tests for the Java interface on Linux to verify installation and functionality. Ensure JUnit and JNA are installed and accessible. ```bash java -cp libvoikko-3.2-SNAPSHOT.jar:libvoikko-3.2-SNAPSHOT-tests.jar:/usr/share/java/junit4.jar:/usr/share/java/jna.jar org.junit.runner.JUnitCore org.puimula.libvoikko.VoikkoTest org.puimula.libvoikko.VoikkoJnaLatinTest ``` -------------------------------- ### Build Voikko-fi Source: https://github.com/voikko/corevoikko/wiki/build-windows To build voikko-fi, ensure Python 3, Foma, GNU Make, sed, sort, and cat are installed and in your PATH. Then run 'make vvfst'. Building on Linux is recommended due to easier dependency management. ```bash make vvfst ``` -------------------------------- ### TENSE Attribute Examples Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Demonstrates the 'TENSE' attribute for verbs, indicating tense and aspect. Examples include 'present_simple' and 'past_imperfective'. ```text Word: juoksen -> TENSE: present_simple Word: juoksin -> TENSE: past_imperfective ``` -------------------------------- ### Cross-compile Libvoikko from Linux with GCC Source: https://github.com/voikko/corevoikko/wiki/build-windows This is often the easiest method for building Libvoikko on Windows. It involves installing mingw-w64 and configuring the build for either 32-bit or 64-bit Windows. You can specify a dictionary path using --with-dictionary-path. ```bash apt-get install mingw-w64 ./autogen.sh CXXFLAGS=-Wno-long-long ac_cv_func_malloc_0_nonnull=yes ac_cv_func_realloc_0_nonnull=yes ./configure --host=i686-w64-mingw32 make make install ``` ```bash i686-w64-mingw32-strip --strip-unneeded *.dll *.exe ``` ```bash x86_64-w64-mingw32-strip --strip-unneeded *.exe *.dll ``` -------------------------------- ### MOOD Attribute Examples Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Shows examples of the 'MOOD' attribute for verbs, specifying the mood such as 'indicative' or 'conditional'. ```text Word: juoksen -> MOOD: indicative Word: juoksisin -> MOOD: conditional ``` -------------------------------- ### Launch Voikko Docker Container Source: https://github.com/voikko/corevoikko/blob/master/docker/README.md Builds and launches the Voikko Docker container using Docker Compose. Ensure Docker and Docker Compose are installed. ```bash docker compose up --build ``` -------------------------------- ### FSTOUTPUT Attribute Example Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Provides an example of the 'FSTOUTPUT' attribute, which is used by analyzers implemented with finite state transducers to expose raw transducer output. This attribute is only produced when 'fullMorphology' is set to true. ```text Word: kissalla -> FSTOUTPUT: [Ln][Xp]kissa[X][Xs]505527[X]kissa[Sade][Ny]lla ``` -------------------------------- ### Run NUnit Tests for Voikko .NET Wrapper Source: https://github.com/voikko/corevoikko/wiki/cs-install Execute NUnit tests to verify the functionality of the .NET wrapper. This requires NUnit console runner and a Finnish standard dictionary to be installed. ```bash nunit-console libvoikko.tests/bin/Release/libvoikko.tests.dll ``` -------------------------------- ### NUMBER Attribute Examples Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Demonstrates the 'NUMBER' attribute, indicating the grammatical number of a word. Suggested values include 'singular' and 'plural'. ```text Word: kissa -> NUMBER: singular Word: kissat -> NUMBER: plural ``` -------------------------------- ### Test Accept First Uppercase Setting Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Checks the `setAcceptFirstUppercase` setting, which allows words starting with an uppercase letter (like proper nouns) to be accepted. ```javascript QUnit.test("setAcceptFirstUppercase", function(assert) { voikko.setAcceptFirstUppercase(false); assert.strictEqual(voikko.spell("Kissa"), false); voikko.setAcceptFirstUppercase(true); assert.strictEqual(voikko.spell("Kissa"), true); }); ``` -------------------------------- ### Node.js libvoikko analyze output Source: https://github.com/voikko/corevoikko/wiki/JavaScript Example output from the analyze function in Node.js, showing detailed linguistic information for the word 'alusta'. ```json [ { BASEFORM: 'alustaa', CLASS: 'teonsana', FSTOUTPUT: '[Lt][Xp]alustaa[X]alusta[Tk][Ap][P2][Ny][Eb]', MOOD: 'imperative', NEGATIVE: 'both', NUMBER: 'singular', PERSON: '2', STRUCTURE: '=pppppp', TENSE: 'present_simple', WORDBASES: '+alustaa(alustaa)' }, { BASEFORM: 'alku', CLASS: 'nimisana', FSTOUTPUT: '[Ln][Xp]alku[X]alu[Sela][Ny]sta', NUMBER: 'singular', SIJAMUOTO: 'sisaeronto', STRUCTURE: '=pppppp', WORDBASES: '+alku(alku)' }, { BASEFORM: 'alus', CLASS: 'nimisana', FSTOUTPUT: '[Ln][Xp]alus[X]alu[Sp][Ny]sta', NUMBER: 'singular', SIJAMUOTO: 'osanto', STRUCTURE: '=pppppp', WORDBASES: '+alus(alus)' }, { BASEFORM: 'alunen', CLASS: 'nimisana', FSTOUTPUT: '[Ln][Xp]alunen[X]alu[Sp][Ny]sta', NUMBER: 'singular', SIJAMUOTO: 'osanto', STRUCTURE: '=pppppp', WORDBASES: '+alunen(alunen)' }, { BASEFORM: 'alusta', CLASS: 'nimisana', FSTOUTPUT: '[Ln][Xp]alusta[X]alust[Sn][Ny]a', NUMBER: 'singular', SIJAMUOTO: 'nimento', STRUCTURE: '=pppppp', WORDBASES: '+alusta(alusta)'} ] ``` -------------------------------- ### Re-linking for Self-Contained Libvoikko DLL Source: https://github.com/voikko/corevoikko/wiki/build-windows After the initial build with static flags, re-link the libvoikko-1.dll using a modified command. This command is found in your shell's command history and should start with 'i686-w64-mingw32-g++ -shared -nostdlib'. Replace '-shared' with '-shared -static -static-libgcc -static-libstdc++' and execute in the 'src' subdirectory. ```bash i686-w64-mingw32-g++ -shared -nostdlib -static -static-libgcc -static-libstdc++ ``` -------------------------------- ### Get Attribute Values for Unknown Attribute Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `attributeValues` for an unknown attribute. ```javascript let values = voikko.attributeValues("XYZ"); assert.equal(values, null); ``` -------------------------------- ### BASEFORM Attribute Example Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Shows the 'BASEFORM' attribute, representing the base or dictionary form of a given word. This attribute is provided only when 'fullMorphology' is set to true. ```text Word: kissalla -> BASEFORM: kissa ``` -------------------------------- ### Voikko Dictionary Compatibility Assertions Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting Example assertions file for Voikko compatibility tests. This file contains Python expressions that evaluate to True when using the dictionaries, with 'v' representing the Voikko object. ```python v.spell(u"kissa") not v.spell(u"kisssa") ``` -------------------------------- ### Voikko Dictionary Variant Name Modification Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting Example of modifying a dictionary variant name within a .pro file to include the 'mor-' prefix from its directory name for compatibility testing. ```text info: Language-Variant: standardsm1.5 ``` -------------------------------- ### Test Sentence Segmentation Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Checks if sentences are correctly identified and segmented, including their text and the type of the next segment's start. ```javascript QUnit.test("sentences", function(assert) { var sentences = voikko.sentences("Kissa ei ole koira. Koira ei ole kissa."); assert.deepEqual(sentences, [ {nextStartType: "PROBABLE", text: "Kissa ei ole koira. "}, {nextStartType: "NONE", text: "Koira ei ole kissa."} ]); }); ``` -------------------------------- ### STRUCTURE Attribute Examples Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Illustrates the 'STRUCTURE' attribute which describes morpheme boundaries, character case, and hyphenation restrictions for a word. This attribute is essential for understanding word segmentation and linguistic properties. ```text Word: Matti-niminen -> STRUCTURE: =ipppp-=ppppppp Word: DNA-näyte -> STRUCTURE: =jjj-=ppppp Word: autokauppa -> STRUCTURE: =pppp=pppppp ``` -------------------------------- ### PERSON Attribute Examples Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/doc/morphological-analysis.txt Illustrates the 'PERSON' attribute for verbs, indicating the grammatical person (first, second, or third). For passive voice, it can represent a fourth voice. ```text Word: juoksen -> PERSON: 1 Word: juokset -> PERSON: 2 ``` -------------------------------- ### Build .NET Assembly with mdtool Source: https://github.com/voikko/corevoikko/wiki/cs-install Use this command to build the .NET wrapper assembly for Voikko. Ensure you are in the 'cs' directory of the libvoikko source package. ```bash mdtool build --configuration:Release --target:Build ``` -------------------------------- ### Build libvoikko with Preload Mode Source: https://github.com/voikko/corevoikko/wiki/JavaScript Build the libvoikko library using 'preload' mode, which is efficient for web environments but not compatible with Node.js. ```bash js/build.sh preload ``` -------------------------------- ### Configure Default Morphology Path (Linux/OS X) Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting Sets up the default morphology path for libvoikko in a Linux or OS X environment. This is required for slow mode testing. ```bash mkdir -p ~/.voikko/2 ln -s $CORE/voikko-fi/vvfst ~/.voikko/2/mor-standard ``` -------------------------------- ### Build Libvoikko with GCC on Windows Source: https://github.com/voikko/corevoikko/wiki/build-windows Use this method when building Libvoikko directly on Windows with MinGW GCC. Ensure Python is in your PATH and run autoreconf if working from a Git clone. ```bash autoreconf --force --install ./configure make make install ``` -------------------------------- ### Get Attribute Values for Enumerated Attribute Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `attributeValues` for an enumerated attribute like 'NUMBER'. ```javascript let values = voikko.attributeValues("NUMBER"); assert.equal(values.length, 2); assert.ok(values.indexOf("singular") >= 0); assert.ok(values.indexOf("plural") >= 0); ``` -------------------------------- ### Get Attribute Values for Non-Enumerated Attribute Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `attributeValues` for a non-enumerated attribute like 'BASEFORM'. ```javascript let values = voikko.attributeValues("BASEFORM"); assert.equal(values, null); ``` -------------------------------- ### Initialize and Terminate Libvoikko Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Sets up and tears down the libvoikko instance for each test. Ensures proper cleanup to prevent test interference. ```javascript QUnit.config.autostart = false; Libvoikko({ onLoad : function(libvoikko) { var voikko; QUnit.module("libvoikko", { beforeEach: function() { voikko = libvoikko.init("fi"); }, afterEach: function() { voikko.terminate(); voikko = null; } }); ``` -------------------------------- ### Initialize and Analyze with Libvoikko in TypeScript Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/typescript/README.md Demonstrates how to import, initialize, and use the Libvoikko library for word analysis in a TypeScript project. Ensure libvoikko is compiled for JavaScript as per the wiki. ```typescript import Libvoikko from './libvoikko'; const voikko = new Libvoikko(); const v = voikko.init('fi'); const a = v.analyze('alustaa'); console.log(a); ``` -------------------------------- ### Build libvoikko with Plain Mode Source: https://github.com/voikko/corevoikko/wiki/JavaScript Build the libvoikko library using 'plain' mode, which does not load dictionaries by default. This allows for separate dictionary building or direct use of the native file system in Node.js. ```bash js/build.sh plain ``` -------------------------------- ### Clone and Configure libvoikko Source: https://github.com/voikko/corevoikko/wiki/JavaScript Clone the corevoikko repository and configure the libvoikko library for JavaScript compilation. Ensure your Emscripten environment is sourced. ```bash git clone https://github.com/voikko/corevoikko cd corevoikko/libvoikko source ~/emscripten/emsdk/emsdk_env.sh ./autogen.sh js/configure.sh ``` -------------------------------- ### Test libvoikko with Node.js Source: https://github.com/voikko/corevoikko/wiki/JavaScript Initialize and test the libvoikko library in a Node.js environment by analyzing a Finnish word. ```javascript const Libvoikko = await require('./js/libvoikko.js')() let v = Libvoikko.init("fi") v.analyze("alusta") ``` -------------------------------- ### Transpile TypeScript and Run with Node.js Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/typescript/README.md Shows the commands to transpile the TypeScript code to JavaScript using `tsc` and then execute the resulting JavaScript file with Node.js. ```bash npx -p typescript tsc --esModuleInterop --target es5 main.ts node main.js ``` -------------------------------- ### Specify Build Directory for Voikko Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use VVFST_BUILDDIR to specify the output directory for build files. Defaults to 'vvfst' in the source directory if not set. ```bash VVFST_BUILDDIR=path/to/directory ``` -------------------------------- ### Download Finnish Dictionary Source: https://github.com/voikko/corevoikko/wiki/JavaScript Download and extract the latest snapshot of the standard Finnish dictionary for use with libvoikko. ```bash wget https://www.puimula.org/htp/testing/voikko-snapshot-v5/dict.zip unzip dict.zip rm dict.zip ``` -------------------------------- ### Build libvoikko with Embed Mode Source: https://github.com/voikko/corevoikko/wiki/JavaScript Build the libvoikko library using 'embed' mode. This mode works with Node.js but can be slow to load in web environments. ```bash js/build.sh embed ``` -------------------------------- ### Create Voikko Test Directory Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting Creates the necessary directory for storing temporary files required by the test suite. This is a prerequisite for running the integration tests. ```bash mkdir -p $HOME/tmp/voikkotest ``` -------------------------------- ### Setting Voikko Property in .NET Source: https://github.com/voikko/corevoikko/wiki/cs-install Demonstrates how to set a Voikko object's setting using a .NET property. This is an alternative to Java's setter methods. ```csharp voikko.AcceptTitlesInGc = false; ``` -------------------------------- ### Set Description for Voikko Vocabulary Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use VOIKKO_DESCRIPTION to provide a long description for the language variant of the vocabulary. ```bash VOIKKO_DESCRIPTION="Description of the vocabulary" ``` -------------------------------- ### Configure for Statically Linked Libvoikko DLL with GCC Source: https://github.com/voikko/corevoikko/wiki/build-windows Use these CXXFLAGS and LDFLAGS when configuring the build to produce a self-contained DLL that does not depend on external compiler or C++ support libraries. This is typically used with the cross-compiler. ```bash LDFLAGS='-static-libstdc++ -static-libgcc' CXXFLAGS='-static-libstdc++ -static-libgcc -Wno-long-long' ``` -------------------------------- ### Set Lexicon Generator Options for Voikko Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use GENLEX_OPTS to pass options to the lexicon generator script. Available options include --min-frequency, --extra-usage, --style, and --sourceid. ```bash GENLEX_OPTS="--option1=xxx --option2=yyy ..." ``` -------------------------------- ### Voikkotest Slow Mode Commands Source: https://github.com/voikko/corevoikko/wiki/libvoikko-IntegrationTesting Commands for operating voikkotest in slow mode. Use '--base' to generate baseline word lists, '--current' to test changes against the baseline, and '--compare' to view differences. ```bash voikkotest --base ``` ```bash voikkotest --current ``` ```bash voikkotest --compare ``` -------------------------------- ### Configure Base Forms Inclusion in Voikko Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Set VVFST_BASEFORMS to 'yes' or 'no'. 'yes' includes BASEFORM attribute information, resulting in a larger dictionary. 'no' produces a smaller dictionary but may yield incorrect BASEFORM values, suitable only for specific use cases like spell checking. ```bash VVFST_BASEFORMS=yes|no ``` -------------------------------- ### Specify Patch Information for Voikko Distribution Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use SM_PATCHINFO to describe any modifications made to the released version, especially for prerelease versions or modified source code. ```bash SM_PATCHINFO="Information about applied patches" ``` -------------------------------- ### Configure Minimum Frequency for Lexicon Generation Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use the --min-frequency option within GENLEX_OPTS to include words with a specified or higher frequency class in the .lex files. ```bash --min-frequency=n ``` -------------------------------- ### Hyphenation with Ugly Hyphens Enabled Source: https://github.com/voikko/corevoikko/blob/master/tests/voikkotest/fi-x-vfst/hyphen.txt Demonstrates hyphenation when `setNoUglyHyphenation` is true. This setting allows for 'ugly' hyphenations, which might be less conventional. ```python import voikko voikko.setNoUglyHyphenation(True) # Example words and their hyphenations # "hienosto" has a compound and non-compound analysis. The non-compound analysis is assumed to be correct. hienosto hie-nos-to # If there is a compound analysis with two components and another with three components, do not guess which one is correct. pöytähienosto pöy-tä-hienos-to ``` -------------------------------- ### Rebuild Voikko Development Environment Container Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/devenv/README.md Execute this command to rebuild the Docker container for the development environment, typically after changes to the Dockerfile. ```bash docker-compose build devenv ``` -------------------------------- ### Configure Suggestion Strategy Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setSuggestionStrategy` to change how suggestions are generated. ```javascript voikko.setSuggestionStrategy("OCR"); assert.ok(voikko.suggest("koari").indexOf("koira") == -1); assert.ok(voikko.suggest("koir_").indexOf("koira") >= 0); voikko.setSuggestionStrategy("TYPO"); assert.ok(voikko.suggest("koari").indexOf("koira") >= 0); ``` -------------------------------- ### Configure Style Flags for Lexicon Generation Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use the --style option within GENLEX_OPTS to include words that match all specified style flags. Defaults to 'old,international,inappropriate'. ```bash --style=style1,style2,... ``` -------------------------------- ### Test Tokenization Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Ensures that text is correctly tokenized into words and whitespace elements. ```javascript QUnit.test("tokens", function(assert) { var tokens = voikko.tokens("kissa ja koira sekä härkä"); assert.deepEqual(tokens, [ {type: "WORD", text: "kissa"}, {type: "WHITESPACE", text: " "}, {type: "WORD", text: "ja"}, {type: "WHITESPACE", text: " "}, {type: "WORD", text: "koira"}, {type: "WHITESPACE", text: " "}, {type: "WORD", text: "sekä"}, {type: "WHITESPACE", text: " "}, {type: "WORD", text: "härkä"} ]); }); ``` -------------------------------- ### Analyze a Word with Voikko Docker Container Source: https://github.com/voikko/corevoikko/blob/master/docker/README.md Executes a Python command within the running Voikko Docker container to analyze a specific word using the Finnish Voikko dictionary. ```bash docker exec -ti docker-voikkocontainer-1 python3 -c 'from libvoikko import Voikko; print(Voikko("fi").analyze("alusta"))' ``` -------------------------------- ### Configure Extra Usage Flags for Lexicon Generation Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use the --extra-usage option within GENLEX_OPTS to include words with specific usage flags. Listing 'sukija' ignores application-specific exclusions. ```bash --extra-usage=usage1,usage2,... ``` -------------------------------- ### Set Language Variant for Voikko Vocabulary Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Use VOIKKO_VARIANT to set a short name for the language variant, matching the regex [a-z][a-z0-9]*. Defaults to 'standard'. ```bash VOIKKO_VARIANT=variant ``` -------------------------------- ### Accept Old Finnish Word Forms in Voikko Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Set VANHAT_MUODOT to 'yes' to accept word forms valid in old Finnish but not standard Finnish. Defaults to 'no'. ```bash VANHAT_MUODOT=yes|no ``` -------------------------------- ### Configure Uppercase Word Acceptance Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests the `setAcceptAllUppercase` function to control whether all-uppercase words are accepted. ```javascript voikko.setIgnoreUppercase(false); voikko.setAcceptAllUppercase(false); assert.strictEqual(voikko.spell("KISSA"), false); voikko.setAcceptAllUppercase(true); assert.strictEqual(voikko.spell("KISSA"), true); assert.strictEqual(voikko.spell("KAAAA"), false); ``` -------------------------------- ### Default Hyphenation with No Ugly Hyphens Source: https://github.com/voikko/corevoikko/blob/master/tests/voikkotest/fi-x-vfst/hyphen.txt Demonstrates default hyphenation behavior when `setNoUglyHyphenation` is true and `setIgnoreDot` is false. This setting aims to avoid awkward hyphenations. ```python import voikko voikko.setNoUglyHyphenation(True) voikko.setIgnoreDot(False) # Example words and their hyphenations # kissa kis-sa # kerrostalo ker-ros-ta-lo # elävä elä-vä # määyntä määyn-tä # maito mai-to # peili pei-li # noita noi-ta # kuiva kui-va # lyijy lyi-jy # Joensuu Joen-suu # kuoria kuo-ria # Äetsä Äet-sä # kieuttiin kieut-tiin # aiottu aiot-tu # sprintteri sprint-te-ri # traktori trak-to-ri # vaieta vaie-ta # proteiini pro-teii-ni # intiaani in-tiaa-ni # jesuiitta je-suiit-ta # AA AA # hienosto hie-nos-to # pöytähienosto pöy-tä-hienos-to # 90-vuotias 90-vuo-tias # hiili-12 hiili-12 # hiili12 hiili12 # http://www.lappeenranta.fi http://www.lappeenranta.fi # www.lappeenranta.fi www.lappeenranta.fi # asukkaat.hoas.fi asuk-kaat.hoas.fi # tuhataskelinen tu-hat-as-ke-li-nen # jok'ikinen jok-iki-nen # nougat'kaan nou-gat'kaan # te'vaste te'vas-te # Ei tavutusta heittomerkin jälkeen, eikä kohdalla, kun perässä on konsonantti # juurihoitoklubi juu-ri-hoi-to-klu-bi # demokraattiosavaltio de-mo-kraat-ti-osa-val-tio # kissa/koira kis-sa/koi-ra # hatapitk@iki.fi hatapitk@iki.fi # 23.5.2014 23.5.2014 # Bach-trumpetti Bach-trum-pet-ti ``` -------------------------------- ### Configure Bulleted Lists in Grammar Check Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setAcceptBulletedListsInGc` to control acceptance of bulleted lists in grammar checks. ```javascript voikko.setAcceptBulletedListsInGc(false); assert.ok(voikko.grammarErrors("kissa", "fi").length > 0); voikko.setAcceptBulletedListsInGc(true); assert.ok(voikko.grammarErrors("kissa", "fi").length == 0); ``` -------------------------------- ### Configure Hyphenation of Unknown Words Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setHyphenateUnknownWords` to control hyphenation for words not in the dictionary. ```javascript voikko.setHyphenateUnknownWords(false); assert.equal(voikko.hyphenate("kirjutepo"), "kirjutepo"); voikko.setHyphenateUnknownWords(true); assert.equal(voikko.hyphenate("kirjutepo"), "kir-ju-te-po"); ``` -------------------------------- ### Enable Source ID Insertion in Lexicon Source: https://github.com/voikko/corevoikko/blob/master/voikko-fi/README.md Set the --sourceid option within GENLEX_OPTS to insert word identifiers from Joukahainen into the lexicon. This option requires VOIKKO_DEBUG=yes to be set and is off by default. ```bash --sourceid ``` -------------------------------- ### Test Suggestion Functionality Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Checks the word suggestion feature, including cases with no suggestions and when the input word is already correct. ```javascript QUnit.test("suggest", function(assert) { assert.ok(voikko.suggest("koirra").indexOf("koira") >= 0); assert.ok(voikko.suggest("määärä").indexOf("määrä") >= 0); assert.equal(voikko.suggest("lasjkblvankirknaslvethikertvhgn").length, 0); }); QUnit.test("suggestReturnsArgumentIfWordIsCorrect", function(assert) { assert.deepEqual(voikko.suggest("koira"), ["koira"]) }); ``` -------------------------------- ### Configure Title Acceptance in Grammar Check Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setAcceptTitlesInGc` to control acceptance of titles in grammar checks. ```javascript voikko.setAcceptTitlesInGc(false); assert.equal(voikko.grammarErrors("Kissa on eläin", "fi").length, 1); voikko.setAcceptTitlesInGc(true); assert.equal(voikko.grammarErrors("Kissa on eläin", "fi").length, 0); ``` -------------------------------- ### Configure Non-Word Acceptance Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests the `setIgnoreNonwords` function to control whether non-word characters in strings are ignored. ```javascript voikko.setIgnoreNonwords(false); assert.strictEqual(voikko.spell("hatapitk@iki.fi"), false); voikko.setIgnoreNonwords(true); assert.strictEqual(voikko.spell("hatapitk@iki.fi"), true); assert.strictEqual(voikko.spell("ashdaksd"), false); ``` -------------------------------- ### Little Endian Magic Number Source: https://github.com/voikko/corevoikko/wiki/vfst-fileformat The magic number for a little endian Varissuo finite state transducer file. ```text 6E 3A 01 00 FA 51 03 00 ``` -------------------------------- ### Configure Ugly Hyphenation Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setNoUglyHyphenation` to control the hyphenation of words. ```javascript voikko.setNoUglyHyphenation(false); assert.equal(voikko.hyphenate("iva"), "i-va"); voikko.setNoUglyHyphenation(true); assert.equal(voikko.hyphenate("iva"), "iva"); ``` -------------------------------- ### Configure Unfinished Paragraphs in Grammar Check Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setAcceptUnfinishedParagraphsInGc` to control acceptance of unfinished paragraphs in grammar checks. ```javascript voikko.setAcceptUnfinishedParagraphsInGc(false); assert.equal(voikko.grammarErrors("Kissa on ", "fi").length, 1); voikko.setAcceptUnfinishedParagraphsInGc(true); assert.equal(voikko.grammarErrors("Kissa on ", "fi").length, 0); ``` -------------------------------- ### Tokenize String with Null Characters Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests the `tokens` function to handle null characters as unknown tokens. ```javascript assert.deepEqual(voikko.tokens("kissa\0koira"), [ {type: "WORD", text: "kissa"}, {type: "UNKNOWN", text: "\0"}, {type: "WORD", text: "koira"} ]); assert.deepEqual(voikko.tokens("kissa\0\0koira"), [ {type: "WORD", text: "kissa"}, {type: "UNKNOWN", text: "\0"}, {type: "UNKNOWN", text: "\0"}, {type: "WORD", text: "koira"} ]); assert.deepEqual(voikko.tokens("kissa\0"), [ {type: "WORD", text: "kissa"}, {type: "UNKNOWN", text: "\0"} ]); assert.deepEqual(voikko.tokens("\0kissa"), [ {type: "UNKNOWN", text: "\0"}, {type: "WORD", text: "kissa"} ]); assert.deepEqual(voikko.tokens("\0"), [ {type: "UNKNOWN", text: "\0"} ]); assert.equal(voikko.tokens("").length, 0); ``` -------------------------------- ### Configure Minimum Hyphenated Word Length Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests `setMinHyphenatedWordLength` to set the minimum length for hyphenation. ```javascript voikko.setMinHyphenatedWordLength(6); assert.equal(voikko.hyphenate("koira"), "koira"); voikko.setMinHyphenatedWordLength(2); assert.equal(voikko.hyphenate("koira"), "koi-ra"); ``` -------------------------------- ### Configure Missing Hyphen Acceptance Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests the `setAcceptMissingHyphens` function to control acceptance of words missing hyphens. ```javascript voikko.setAcceptMissingHyphens(false); assert.strictEqual(voikko.spell("sosiaali"), false); voikko.setAcceptMissingHyphens(true); assert.strictEqual(voikko.spell("sosiaali"), true); ``` -------------------------------- ### Big Endian Magic Number Source: https://github.com/voikko/corevoikko/wiki/vfst-fileformat The magic number for a big endian Varissuo finite state transducer file. ```text 00 01 3A 6E 00 03 51 FA ``` -------------------------------- ### Test Hyphenation with Custom Separator Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests hyphenation with a custom soft hyphen separator and verifies the behavior when the separator is not used. ```javascript QUnit.test("hyphenateWithCustomSeparator", function(assert) { assert.equal(voikko.hyphenate("kissa", "­", true), "kis­sa"); assert.equal(voikko.hyphenate("kuorma-auto", "­", true), "kuor­ma-au­to"); assert.equal(voikko.hyphenate("vaa'an", "­", true), "vaa­an"); assert.equal(voikko.hyphenate("vaa'an", "­", false), "vaa'an"); }); ``` -------------------------------- ### Test Word Analysis Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Verifies the analysis of a word to determine its structure, represented by a pattern of prefixes, roots, and suffixes. ```javascript QUnit.test("analyze", function(assert) { var analysisList = voikko.analyze("kansaneläkehakemus"); assert.equal(analysisList.length, 1); assert.equal(analysisList[0]["STRUCTURE"], "=pppppp=ppppp=ppppppp"); }); ``` -------------------------------- ### Configure Extra Hyphen Acceptance Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Tests the `setAcceptExtraHyphens` function to control acceptance of words with extra hyphens. ```javascript voikko.setAcceptExtraHyphens(false); assert.strictEqual(voikko.spell("kerros-talo"), false); voikko.setAcceptExtraHyphens(true); assert.strictEqual(voikko.spell("kerros-talo"), true); ``` -------------------------------- ### Test Ignore Dot Setting Source: https://github.com/voikko/corevoikko/blob/master/libvoikko/js/qunit.html Verifies the `setIgnoreDot` setting, which controls whether trailing dots are ignored during spell checking. ```javascript QUnit.test("setIgnoreDot", function(assert) { voikko.setIgnoreDot(false); assert.strictEqual(voikko.spell("kissa."), false); voikko.setIgnoreDot(true); assert.strictEqual(voikko.spell("kissa."), true); }); ```