### Installation and Setup Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt Instructions on how to install the Elasticsearch Sudachi plugin and configure the Sudachi dictionary. ```APIDOC ## Installation and Setup Download and install the plugin using the Elasticsearch plugin manager, then configure the Sudachi dictionary files. ```bash # Install the plugin from GitHub releases bin/elasticsearch-plugin install https://github.com/WorksApplications/elasticsearch-sudachi/releases/download/v3.1.1/analysis-sudachi-8.13.4-3.1.1.zip # Download and extract the Sudachi dictionary wget https://d2ej7fkh96fzlu.cloudfront.net/sudachidict/sudachi-dictionary-latest-core.zip unzip sudachi-dictionary-latest-core.zip # Create sudachi config directory and copy dictionary mkdir config/sudachi cp sudachi-dictionary-*/system_core.dic config/sudachi/system_core.dic # Start Elasticsearch bin/elasticsearch ``` ``` -------------------------------- ### Install Elasticsearch Sudachi Plugin and Dictionary Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt This section provides bash commands to install the Elasticsearch Sudachi plugin from GitHub releases and download/configure the Sudachi dictionary. It covers plugin installation, dictionary download, and Elasticsearch startup. ```bash bin/elasticsearch-plugin install https://github.com/WorksApplications/elasticsearch-sudachi/releases/download/v3.1.1/analysis-sudachi-8.13.4-3.1.1.zip wget https://d2ej7fkh96fzlu.cloudfront.net/sudachidict/sudachi-dictionary-latest-core.zip unzip sudachi-dictionary-latest-core.zip mkdir config/sudachi cp sudachi-dictionary-*/system_core.dic config/sudachi/system_core.dic bin/elasticsearch ``` -------------------------------- ### Complete Analysis Configuration Example Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt This example demonstrates a comprehensive index configuration using multiple Sudachi features for production-level Japanese text search. It includes custom filters, tokenizers, and analyzers tailored for Japanese language processing. ```APIDOC ## PUT /production_japanese ### Description Configures an Elasticsearch index named `production_japanese` with advanced analysis settings using the Sudachi plugin. This setup is designed for production Japanese text search, incorporating features like synonym expansion, reading form conversion, and part-of-speech filtering. ### Method PUT ### Endpoint `/production_japanese` ### Request Body ```json { "settings": { "analysis": { "filter": { "search_split": { "type": "sudachi_split", "mode": "search" }, "synonym_filter": { "type": "synonym", "synonyms": ["関西国際空港,関空", "関西 => 近畿"] }, "romaji_reading": { "type": "sudachi_readingform", "use_romaji": true }, "katakana_reading": { "type": "sudachi_readingform", "use_romaji": false }, "pos_filter": { "type": "sudachi_part_of_speech", "stoptags": ["助詞", "助動詞", "補助記号,句点", "補助記号,読点"] } }, "tokenizer": { "sudachi_c_tokenizer": { "type": "sudachi_tokenizer", "split_mode": "C" }, "sudachi_a_tokenizer": { "type": "sudachi_tokenizer", "split_mode": "A" } }, "analyzer": { "sudachi_baseform_analyzer": { "type": "custom", "tokenizer": "sudachi_c_tokenizer", "filter": ["sudachi_baseform"] }, "sudachi_normalized_analyzer": { "type": "custom", "tokenizer": "sudachi_c_tokenizer", "filter": ["sudachi_normalizedform"] }, "sudachi_reading_analyzer": { "type": "custom", "tokenizer": "sudachi_c_tokenizer", "filter": ["katakana_reading"] }, "sudachi_romaji_analyzer": { "type": "custom", "tokenizer": "sudachi_c_tokenizer", "filter": ["romaji_reading"] }, "sudachi_search_analyzer": { "type": "custom", "tokenizer": "sudachi_c_tokenizer", "filter": ["search_split"] }, "sudachi_synonym_analyzer": { "type": "custom", "tokenizer": "sudachi_c_tokenizer", "filter": ["synonym_filter", "search_split"] }, "sudachi_clean_analyzer": { "type": "custom", "tokenizer": "sudachi_a_tokenizer", "filter": ["sudachi_baseform", "pos_filter", "sudachi_ja_stop"] } } } }, "mappings": { "properties": { "title": { "type": "text", "analyzer": "sudachi_search_analyzer" }, "content": { "type": "text", "analyzer": "sudachi_clean_analyzer" }, "reading": { "type": "text", "analyzer": "sudachi_reading_analyzer" } } } } ``` ### Response #### Success Response (200 OK) Returns an acknowledgment of the index creation or update. #### Response Example ```json { "acknowledged": true, "shards_acknowledged": true, "index": "production_japanese" } ``` ``` -------------------------------- ### Install analysis-sudachi Plugin for Elasticsearch Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Instructions for installing the analysis-sudachi plugin in Elasticsearch. Supports installation from a release package URL or a self-built package. ```bash # Install from release package $ bin/elasticsearch-plugin install https://github.com/WorksApplications/elasticsearch-sudachi/releases/download/v3.1.1/analysis-sudachi-8.13.4-3.1.1.zip ``` ```bash # Install from self-build package (use absolute path) $ bin/elasticsearch-plugin install file:///path/to/analysis-sudachi-8.13.4-3.1.1.zip ``` -------------------------------- ### Sudachi Tokenizer Configuration Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Example configurations for setting up the Sudachi tokenizer with different split modes and resource paths. ```APIDOC ## Tokenizer Configuration Examples ### Basic Sudachi Tokenizer Configuration This configuration sets up the `sudachi_tokenizer` with a specific split mode and discards punctuation. ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer", "split_mode": "C", "discard_punctuation": true, "resources_path": "/etc/elasticsearch/config/sudachi" } }, "analyzer": { "sudachi_analyzer": { "type": "custom", "tokenizer": "sudachi_tokenizer" } } } } } } ``` ### Sudachi Tokenizer with Additional Dictionary Settings This configuration shows how to specify custom system dictionaries and user dictionaries for the Sudachi tokenizer. ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer", "additional_settings": "{\"systemDict\":\"system_full.dic\",\"userDict\":[\"user.dic\"]}" } }, "analyzer": { "sudachi_analyzer": { "type": "custom", "tokenizer": "sudachi_tokenizer" } } } } } } ``` ``` -------------------------------- ### Elasticsearch Synonym Filter Setup Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/docs/synonym.md This section provides an example of how to configure Elasticsearch settings to use the converted Solr synonym file with the `synonym_graph` token filter. ```APIDOC ## Elasticsearch Synonym Filter Configuration ### Description This endpoint details the Elasticsearch settings required to integrate the converted Solr synonym file using the `synonym_graph` token filter. It assumes the converted file is placed in the Elasticsearch configuration directory. ### Method PUT ### Endpoint `/` (Index Settings API) ### Parameters #### Request Body - **`settings`** (object) - Required - The main settings object for the index. - **`analysis`** (object) - Required - Analysis settings. - **`filter`** (object) - Required - Token filter definitions. - **`sudachi_synonym`** (object) - Required - Configuration for the Sudachi synonym filter. - **`type`** (string) - Required - Must be `synonym_graph`. - **`synonyms_path`** (string) - Required - Path to the synonym file (e.g., `sudachi/synonym.txt`). Assumed to be relative to `$ES_PATH_CONF`. - **`tokenizer`** (object) - Required - Tokenizer definitions. - **`sudachi_tokenizer`** (object) - Required - Configuration for the Sudachi tokenizer. - **`type`** (string) - Required - Must be `sudachi_tokenizer`. - **`analyzer`** (object) - Required - Analyzer definitions. - **`sudachi_synonym_analyzer`** (object) - Required - Custom analyzer using Sudachi tokenizer and the synonym filter. - **`type`** (string) - Required - Must be `custom`. - **`tokenizer`** (string) - Required - Must be `sudachi_tokenizer`. - **`filter`** (array) - Required - List of token filters, including `sudachi_synonym`. ### Request Example ```json { "settings": { "analysis": { "filter": { "sudachi_synonym": { "type": "synonym_graph", "synonyms_path": "sudachi/synonym.txt" } }, "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer" } }, "analyzer": { "sudachi_synonym_analyzer": { "type": "custom", "tokenizer": "sudachi_tokenizer", "filter": [ "sudachi_synonym" ] } } } } } ``` ### Response #### Success Response (200 OK) - Index settings updated successfully. #### Response Example (Standard Elasticsearch acknowledgment response) ### Notes - The `sudachi_split` filter cannot be used with the synonym filter because it produces a token graph. ``` -------------------------------- ### Test Sudachi Normalized Form (Bash) Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt These bash scripts use `curl` to test the `sudachi_normalized_analyzer`. The first example shows normalization of a spelling variant ('呑み' to '飲む'), and the second demonstrates converting hiragana ('おおきく') to kanji ('大きい'). ```bash # Test normalized form - handles spelling variants curl -X GET "localhost:9200/sudachi_normalized/_analyze?pretty" -H 'Content-Type: application/json' -d' { "analyzer": "sudachi_normalized_analyzer", "text": "呑み" }' # Response (variant "呑み" normalized to standard "飲む"): # { # "tokens": [ # { "token": "飲む", "start_offset": 0, "end_offset": 2, "type": "word", "position": 0 } # ] # } # Another example with hiragana to kanji normalization curl -X GET "localhost:9200/sudachi_normalized/_analyze?pretty" -H 'Content-Type: application/json' -d' { "analyzer": "sudachi_normalized_analyzer", "text": "おおきく" }' # Response: # { # "tokens": [ # { "token": "大きい", "start_offset": 0, "end_offset": 4, "type": "word", "position": 0 } # ] # } ``` -------------------------------- ### Sudachi Split Token Filter Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Documentation and examples for the `sudachi_split` token filter, which provides additional segmentation for search. ```APIDOC ## Sudachi Split Token Filter The `sudachi_split` token filter enhances tokenization for search purposes, similar to Kuromoji's `mode`. ### Modes - **search**: Provides additional segmentation suitable for search queries. It uses modes 'C' and 'A'. - Example: `関西国際空港` -> `関西国際空港`, `関西`, `国際`, `空港` - **extended**: Similar to 'search' mode but also segments unknown words. - Example: `関西国際空港` -> `関西国際空港`, `関西`, `国際`, `空港` / `アバラカダブラ` -> `アバラカダブラ`, `ア`, `バ`, `ラ`, `カ`, `ダ`, `ブ`, `ラ` **Note**: When used in a search query, subwords split by this filter are treated as a phrase. For searching with both 'A' and 'C' units, consider using multiple tokenizers. ### PUT sudachi_sample This example demonstrates configuring an index with the `sudachi_split` filter in 'search' mode. ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer" } }, "analyzer": { "sudachi_analyzer": { "filter": ["my_searchfilter"], "tokenizer": "sudachi_tokenizer", "type": "custom" } }, "filter":{ "my_searchfilter": { "type": "sudachi_split", "mode": "search" } } } } } } ``` ### POST sudachi_sample/_analyze Analyzes the text "関西国際空港" using the `sudachi_analyzer` configured with the `sudachi_split` filter. **Request Body** ```json { "analyzer": "sudachi_analyzer", "text": "関西国際空港" } ``` **Success Response (200)** ```json { "tokens" : [ { "token" : "関西国際空港", "start_offset" : 0, "end_offset" : 6, "type" : "word", "position" : 0, "positionLength" : 3 }, { "token" : "関西", "start_offset" : 0, "end_offset" : 2, "type" : "word", "position" : 0 }, { "token" : "国際", "start_offset" : 2, "end_offset" : 4, "type" : "word", "position" : 1 }, { "token" : "空港", "start_offset" : 4, "end_offset" : 6, "type" : "word", "position" : 2 } ] } ``` ``` -------------------------------- ### Elasticsearch Sudachi Index Creation and Analysis Testing Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt These bash commands demonstrate how to create an Elasticsearch index using a Sudachi analysis configuration and how to test specific analyzers with sample Japanese text. ```bash # Create index curl -X PUT 'localhost:9200/test_sudachi' -H 'Content-Type: application/json' -d @analysis_sudachi.json # Test different analyzers curl -X GET "localhost:9200/test_sudachi/_analyze?pretty" -H 'Content-Type: application/json' -d'{"analyzer":"sudachi_search_analyzer", "text": "関西国際空港と関西"}' ``` -------------------------------- ### Test Sudachi Reading Form Conversion (Bash) Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt These bash scripts use `curl` to test the Sudachi reading form analyzers. The first tests the `katakana_analyzer` on '寿司' to get 'スシ', and the second tests the `romaji_analyzer` to get 'susi'. ```bash # Test katakana reading form curl -X GET "localhost:9200/sudachi_reading/_analyze?pretty" -H 'Content-Type: application/json' -d' { "analyzer": "katakana_analyzer", "text": "寿司" }' # Response: # { # "tokens": [ # { "token": "スシ", "start_offset": 0, "end_offset": 2, "type": "word", "position": 0 } # ] # } # Test romaji reading form curl -X GET "localhost:9200/sudachi_reading/_analyze?pretty" -H 'Content-Type: application/json' -d' { "analyzer": "romaji_analyzer", "text": "寿司" }' # Response: # { # "tokens": [ # { "token": "susi", "start_offset": 0, "end_offset": 2, "type": "word", "position": 0 } # ] # } ``` -------------------------------- ### Create Index with Sudachi Split Mode Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt Demonstrates creating an Elasticsearch index using the Sudachi tokenizer with a specified split mode ('A'). ```APIDOC ## POST /test_sudachi_user_a ### Description Creates an Elasticsearch index named `test_sudachi_user_a` configured with the Sudachi tokenizer and a specific split mode ('A'). ### Method PUT ### Endpoint `/test_sudachi_user_a` ### Parameters #### Request Body - **settings** (object) - Required - The index settings. - **analysis** (object) - Required - **tokenizer** (object) - Required - **sudachi_tokenizer** (object) - Required - Sudachi tokenizer configuration. - **type** (string) - Required - Must be `sudachi_tokenizer`. - **additional_settings** (string) - Required - JSON string specifying the user dictionary. Example: `{"userDict": ["user_test.dic"]}` - **split_mode** (string) - Required - The Sudachi split mode. Example: `A` ### Request Example ```bash curl -X PUT "localhost:9200/test_sudachi_user_a" -H 'Content-Type: application/json' -d' { "settings": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer", "additional_settings": "{\"userDict\": [\"user_test.dic\"]}", "split_mode": "A" } } } } }' ``` ### Response #### Success Response (200) An acknowledgment that the index has been created or updated. #### Response Example ```json { "acknowledged": true } ``` ``` -------------------------------- ### User Dictionary Configuration Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt Provides instructions on how to compile a custom user dictionary using Sudachi's `UserDictionaryBuilder` for domain-specific vocabulary. ```APIDOC ## User Dictionary Configuration Configure custom user dictionaries to add domain-specific vocabulary or override default tokenization behavior. ### Method N/A (Command Line Tool) ### Endpoint N/A ### Description This section details the command to compile a user dictionary from a CSV file using the Sudachi Java library. ### Command Example ```bash java -Dfile.encoding=UTF-8 -cp sudachi-*.jar \ com.worksap.nlp.sudachi.dictionary.UserDictionaryBuilder \ -o user_custom.dic -s system_core.dic \ -d 'Custom domain dictionary' \ user_vocabulary.csv ``` ### Parameters - **`-o `**: Specifies the output file name for the compiled user dictionary. - **`-s `**: Path to the system core dictionary. - **`-d `**: An optional description for the user dictionary. - **``**: Path to the CSV file containing the user vocabulary. ``` -------------------------------- ### Compile Sudachi User Dictionary (Bash) Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt This bash command demonstrates how to compile a custom Sudachi user dictionary. It uses the `UserDictionaryBuilder` Java class, specifying input CSV and system dictionary files, and an output dictionary file. ```bash # Compile user dictionary from CSV java -Dfile.encoding=UTF-8 -cp sudachi-*.jar \ com.worksap.nlp.sudachi.dictionary.UserDictionaryBuilder \ -o user_custom.dic -s system_core.dic \ -d 'Custom domain dictionary' \ user_vocabulary.csv ``` -------------------------------- ### Sudachi Part-of-Speech Token Filter Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Documentation and examples for the `sudachi_part_of_speech` token filter, used to remove tokens based on their part-of-speech tags. ```APIDOC ## Sudachi Part-of-Speech Token Filter The `sudachi_part_of_speech` token filter allows you to remove tokens based on specified part-of-speech and inflectional tags. ### Settings - **stoptags** (Array): An array of part-of-speech and/or inflectional tags to be removed. If not provided, it defaults to the `stoptags.txt` file embedded in the `lucene-analysis-sudachi.jar`. Sudachi POS information is a CSV list with 6 items: 1. Part-of-speech hierarchy (e.g., `名詞`) 2. Part-of-speech hierarchy (e.g., `固有名詞`) 3. Part-of-speech hierarchy (e.g., `地名`) 4. Part-of-speech hierarchy (e.g., `一般`) 5. Inflectional type (e.g., `五段-カ行`) 6. Inflectional form (e.g., `終止形-一般`) `stoptags` support forward matching on these fields. ### PUT sudachi_sample This example configures an index with the `sudachi_part_of_speech` filter to remove specific Japanese grammatical elements. ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer" } }, "analyzer": { "sudachi_analyzer": { "filter": ["my_posfilter"], "tokenizer": "sudachi_tokenizer", "type": "custom" } }, "filter":{ "my_posfilter":{ "type":"sudachi_part_of_speech", "stoptags":[ "助詞", "助動詞", "補助記号,句点", "補助記号,読点" ] } } } } } } ``` ### POST sudachi_sample/_analyze Analyzes the text "寿司がおいしいね" using the `sudachi_analyzer` configured with the `sudachi_part_of_speech` filter. **Request Body** ```json { "analyzer": "sudachi_analyzer", "text": "寿司がおいしいね" } ``` **Success Response (200)** ```json { "tokens": [ { "token": "寿司", "start_offset": 0, "end_offset": 2, "type": "word", "position": 0 }, { "token": "おいしい", "start_offset": 3, "end_offset": 7, "type": "word", "position": 2 } ] } ``` ``` -------------------------------- ### Configure Sudachi Tokenizer and Stopwords Filter (JSON) Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Defines a custom Elasticsearch index setting that includes the 'sudachi_tokenizer' and a 'sudachi_ja_stop' filter named 'my_stopfilter'. This setup allows for Japanese text analysis with custom stopwords. ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer" } }, "analyzer": { "sudachi_analyzer": { "filter": ["my_stopfilter"], "tokenizer": "sudachi_tokenizer", "type": "custom" } }, "filter":{ "my_stopfilter":{ "type":"sudachi_ja_stop", "stopwords":[ "_japanese_", "は", "です" ] } } } } } } ``` -------------------------------- ### Create Elasticsearch Index with Sudachi Tokenizer and Split Mode Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt Creates an Elasticsearch index using the Sudachi tokenizer with a specified user dictionary and 'A' split mode. This configures how Sudachi tokenizes text. ```bash curl -X PUT "localhost:9200/test_sudachi_user_a" -H 'Content-Type: application/json' -d' { "settings": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer", "additional_settings": "{\"userDict\": [\"user_test.dic\"]}", "split_mode": "A" } } } } }' ``` -------------------------------- ### Create Index and Test Analyzers Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt Provides `curl` commands to create an Elasticsearch index with the specified Sudachi analysis configuration and to test the functionality of different analyzers. ```APIDOC ## Create Index ### Description Uses `curl` to send a PUT request to create or update an Elasticsearch index named `test_sudachi` using the analysis configuration defined in `analysis_sudachi.json`. ### Method PUT ### Endpoint `localhost:9200/test_sudachi` ### Request Body (Assumes `analysis_sudachi.json` contains the JSON configuration from the previous example) ### Request Example ```bash curl -X PUT 'localhost:9200/test_sudachi' -H 'Content-Type: application/json' -d @analysis_sudachi.json ``` ## Test Analyzers ### Description Uses `curl` to send a GET request to the `_analyze` endpoint of the `test_sudachi` index. This command tests the `sudachi_search_analyzer` with the input text "関西国際空港と関西". ### Method GET ### Endpoint `localhost:9200/test_sudachi/_analyze` ### Query Parameters - **pretty** (boolean) - Optional - Formats the JSON output for readability. ### Request Body ```json { "analyzer": "sudachi_search_analyzer", "text": "関西国際空港と関西" } ``` ### Request Example ```bash curl -X GET "localhost:9200/test_sudachi/_analyze?pretty" -H 'Content-Type: application/json' -d'{"analyzer":"sudachi_search_analyzer", "text": "関西国際空港と関西"}' ``` ### Response (Success Response 200 OK) Returns the analysis results, showing how the input text is tokenized and processed by the specified analyzer. ### Response Example ```json { "tokens": [ { "token": "関西国際空港", "start_offset": 0, "end_offset": 7, "type": "word", "position": 0 }, { "token": "と", "start_offset": 7, "end_offset": 8, "type": "word", "position": 1 }, { "token": "関西", "start_offset": 8, "end_offset": 10, "type": "word", "position": 2 } ] } ``` ``` -------------------------------- ### Configure Sudachi Tokenizer with User Dictionary (Split Mode A) Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/docs/user-dict-tutorial.md This snippet demonstrates configuring the Sudachi tokenizer with a user dictionary in split mode A (short tokens). The `split_mode` parameter is added to the `sudachi_tokenizer` settings to enable this behavior. ```bash curl -X PUT "localhost:9200/test_sudachi_user_a" -H 'Content-Type: application/json' -d @- </config/sudachi` path for proper loading. ```bash cp user_test.dic `/config/sudachi/user_test.dic` ``` -------------------------------- ### Configure User Dictionary for Sudachi Tokenizer (A Split Mode) Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/docs/user-dict-tutorial.md This section details how to configure a user dictionary for the Sudachi tokenizer when using the 'A' split mode (short tokens) by specifying additional settings and the split mode in the index configuration. ```APIDOC ## PUT /test_sudachi_user_a ### Description Configures an Elasticsearch index with a Sudachi tokenizer that uses a user-defined dictionary for the 'A' split mode (short tokens). ### Method PUT ### Endpoint `/test_sudachi_user_a` ### Parameters #### Request Body - **settings** (object) - Required - The settings for the index. - **analysis** (object) - Required - Analysis settings. - **tokenizer** (object) - Required - Tokenizer settings. - **sudachi_tokenizer** (object) - Required - Sudachi tokenizer configuration. - **type** (string) - Required - Must be `sudachi_tokenizer`. - **additional_settings** (string) - Required - JSON string containing tokenizer settings. Example: `{"userDict": ["user_test.dic"]}`. - **split_mode** (string) - Optional - The split mode for the tokenizer. Use `A` for short tokens. ### Request Example ```json { "settings": { "analysis": { "tokenizer" : { "sudachi_tokenizer": { "type": "sudachi_tokenizer", "additional_settings": "{\"userDict\": [\"user_test.dic\"]}", "split_mode": "A" } } } } } ``` ### Response #### Success Response (200) - **acknowledged** (boolean) - Indicates if the index creation was acknowledged. - **shards_acknowledged** (boolean) - Indicates if the shard allocation was acknowledged. - **index** (string) - The name of the created index. #### Response Example ```json { "acknowledged": true, "shards_acknowledged": true, "index": "test_sudachi_user_a" } ``` ``` -------------------------------- ### Build analysis-sudachi Plugin for Elasticsearch Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Command to build the analysis-sudachi plugin for a specific Elasticsearch version. The engineVersion parameter specifies the target Elasticsearch version. ```bash # Build for Elasticsearch ./gradlew -PengineVersion=es:8.17.10 build ``` ```bash # Build for OpenSearch ./gradlew -PengineVersion=os:2.19.3 build ``` -------------------------------- ### Analysis API with Sudachi Synonyms Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/docs/synonym.md Demonstrates how to use the configured `sudachi_synonym_analyzer` with the Elasticsearch Analysis API to see how synonyms are processed. ```APIDOC ## Analyze Text with Sudachi Synonyms ### Description This endpoint shows how to use the `sudachi_synonym_analyzer` with the Elasticsearch Analyze API to test synonym expansion. Two cases are presented: one with a term that has synonyms and one without. ### Method POST ### Endpoint `//_analyze` ### Parameters #### Path Parameters - **``** (string) - Required - The name of the Elasticsearch index. #### Request Body - **`analyzer`** (string) - Required - The name of the analyzer to use (e.g., `sudachi_synonym_analyzer`). - **`text`** (string) - Required - The text to analyze. ### Request Example (Case 1) ```json { "analyzer": "sudachi_synonym_analyzer", "text": "アイスクリーム" } ``` ### Response (Case 1) #### Success Response (200 OK) - **`tokens`** (array) - An array of token objects representing the analyzed text. - **`token`** (string) - The token text. - **`start_offset`** (integer) - The starting character offset of the token. - **`end_offset`** (integer) - The ending character offset of the token. - **`type`** (string) - The token type (e.g., `SYNONYM`, `word`). - **`position`** (integer) - The position of the token in the sequence. #### Response Example (Case 1) ```json { "tokens": [ { "token": "アイスクリーム", "start_offset": 0, "end_offset": 7, "type": "SYNONYM", "position": 0 }, { "token": "ice cream", "start_offset": 0, "end_offset": 7, "type": "SYNONYM", "position": 0 }, { "token": "ice", "start_offset": 0, "end_offset": 7, "type": "SYNONYM", "position": 0 }, { "token": "アイス", "start_offset": 0, "end_offset": 7, "type": "SYNONYM", "position": 0 } ] } ``` ### Request Example (Case 2) ```json { "analyzer": "sudachi_synonym_analyzer", "text": "アイス" } ``` ### Response (Case 2) #### Success Response (200 OK) - **`tokens`** (array) - An array of token objects representing the analyzed text. #### Response Example (Case 2) ```json { "tokens": [ { "token": "アイス", "start_offset": 0, "end_offset": 3, "type": "word", "position": 0 } ] } ``` ``` -------------------------------- ### Configure Sudachi Tokenizer with Split Mode C Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md This configuration sets up the Sudachi tokenizer with 'split_mode' set to 'C' and punctuation discarding enabled. It specifies a path for Sudachi resources. This is useful for precise tokenization based on Sudachi's modes. ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer", "split_mode": "C", "discard_punctuation": true, "resources_path": "/etc/elasticsearch/config/sudachi" } }, "analyzer": { "sudachi_analyzer": { "type": "custom", "tokenizer": "sudachi_tokenizer" } } } } } } ``` -------------------------------- ### PUT /sudachi_sample (with sudachi_baseform) Source: https://github.com/worksapplications/elasticsearch-sudachi/blob/develop/README.md Configures an Elasticsearch index with the Sudachi tokenizer and the `sudachi_baseform` token filter for lemmatization. ```APIDOC ## PUT /sudachi_sample ### Description Configures an Elasticsearch index with the Sudachi tokenizer and the `sudachi_baseform` token filter, which replaces terms with their dictionary base form (lemmatization). ### Method PUT ### Endpoint /sudachi_sample ### Request Body ```json { "settings": { "index": { "analysis": { "tokenizer": { "sudachi_tokenizer": { "type": "sudachi_tokenizer" } }, "analyzer": { "sudachi_analyzer": { "filter": ["sudachi_baseform"], "tokenizer": "sudachi_tokenizer", "type": "custom" } } } } } } ``` ### Response (Index creation response, typically 200 OK) ``` -------------------------------- ### Default Sudachi Analyzer Configuration Source: https://context7.com/worksapplications/elasticsearch-sudachi/llms.txt Configuration for the default `sudachi` analyzer, which includes baseform normalization, part-of-speech filtering, and Japanese stopword removal. ```APIDOC ## Default Sudachi Analyzer Configuration The built-in `sudachi` analyzer provides a pre-configured analysis chain with baseform normalization, part-of-speech filtering, and Japanese stopword removal. ### Request Body ```json { "settings": { "index": { "analysis": { "analyzer": { "default_sudachi_analyzer": { "type": "custom", "tokenizer": "sudachi_tokenizer", "filter": [ "sudachi_baseform", "sudachi_part_of_speech", "sudachi_ja_stop" ] } } } } } } ``` ```