ExAwabi

https://hexdocs.pm/exawabi/
Admin
ExAwabi is an Elixir wrapper for Awabi, providing morphological analysis using MeCab dictionaries...
Tokens:3,339
Snippets:26
Trust Score:9
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install MeCab on Debian/Ubuntu

Source: https://hexdocs.pm/exawabi/readme.html

Install MeCab and the UTF-8 IPADIC dictionary on Debian/Ubuntu systems using apt.

```bash
$ sudo apt install mecab mecab-ipadic-utf8
```

--------------------------------

### Install MeCab on macOS with Homebrew

Source: https://hexdocs.pm/exawabi/readme.html

Install MeCab and the IPADIC dictionary on macOS using Homebrew.

```bash
$ brew install mecab
$ brew install mecab-ipadic
```

--------------------------------

### tokenize/1 example

Source: https://hexdocs.pm/exawabi/ExAwabi.html

Example usage of ExAwabi.tokenize/1 with a Japanese string. Returns a list of tuples with the token and its morphological information.

```elixir
iex> ExAwabi.tokenize("すもももももももものうち")
[
  {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
  {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
  {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
  {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
  {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
  {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
  {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
]
```

--------------------------------

### tokenize_n_best/2 example

Source: https://hexdocs.pm/exawabi/ExAwabi.html

Example usage of ExAwabi.tokenize_n_best/2 with a Japanese string and N=3. Returns a list of tokenization results, each a list of tuples.

```elixir
iex> ExAwabi.tokenize_n_best("すもももももももものうち", 3)
[
  [
    {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
    {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
  ],
  [
    {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
    {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
  ],
  [
    {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
    {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
  ]
]
```

--------------------------------

### ExAwabi.tokenize_n_best/2

Source: https://hexdocs.pm/exawabi/index.html

Tokenizes the input string and returns the N best matches found by the analyzer.

```APIDOC
## ExAwabi.tokenize_n_best(s, n)

### Description
Tokenize the input string with N best matches.

### Parameters
- **s** (binary) - Required - The string to be tokenized.
- **n** (integer) - Required - The number of best matches to return.

### Response
- **Returns** ([[binary]]) - A list of lists, where each inner list contains the tokens for a specific match result.
```

--------------------------------

### Add ExAwabi dependency to mix.exs

Source: https://hexdocs.pm/exawabi/readme.html

Add the exawabi package to your Elixir project's dependencies in mix.exs.

```elixir
def deps do
  [
    {:exawabi, "~> 0.2.1"}
  ]
end
```

--------------------------------

### tokenize/1 type specification

Source: https://hexdocs.pm/exawabi/ExAwabi.html

Type specification for the tokenize/1 function. It takes a binary and returns a list of binaries.

```elixir
@spec tokenize(binary()) :: [binary()]
```

--------------------------------

### tokenize_n_best(s, n)

Source: https://hexdocs.pm/exawabi/ExAwabi.html

Tokenizes the string with N best matches. Returns a list of lists, each containing a tokenization result.

```APIDOC
## tokenize_n_best(s, n)

### Description
Tokenize the string with `N` best matches.

### Function Signature
```elixir
@spec tokenize_n_best(binary(), integer()) :: [[binary()]]
```

### Parameters
- **s** (binary()) - Required - The string to tokenize.
- **n** (integer()) - Required - The number of best matches to return.

### Returns
- List of lists, each containing a tokenization result (list of tuples).

### Example
```elixir
iex> ExAwabi.tokenize_n_best("すもももももももものうち", 3)
[
  [
    {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
    {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
  ],
  [
    {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
    {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
  ],
  [
    {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
    {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
    {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
    {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
  ]
]
```
```

--------------------------------

### ExAwabi.tokenize/1

Source: https://hexdocs.pm/exawabi/index.html

Tokenizes the provided input string into a list of tuples containing the token and its morphological information.

```APIDOC
## ExAwabi.tokenize(s)

### Description
Tokenize the input string using the MeCab analyzer.

### Parameters
- **s** (binary) - Required - The string to be tokenized.

### Response
- **Returns** ([binary]) - A list of tuples containing the token and its associated morphological features.
```

--------------------------------

### tokenize(s)

Source: https://hexdocs.pm/exawabi/ExAwabi.html

Tokenizes the given string using the MeCab dictionary. Returns a list of tuples containing the token and its morphological analysis.

```APIDOC
## tokenize(s)

### Description
Tokenize the string.

### Function Signature
```elixir
@spec tokenize(binary()) :: [binary()]
```

### Parameters
- **s** (binary()) - Required - The string to tokenize.

### Returns
- List of tuples, each containing the token and its morphological analysis.

### Example
```elixir
iex> ExAwabi.tokenize("すもももももももものうち")
[
  {"すもも", "名詞,一般,*,*,*,*,すもも,スモモ,スモモ"},
  {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
  {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
  {"も", "助詞,係助詞,*,*,*,*,も,モ,モ"},
  {"もも", "名詞,一般,*,*,*,*,もも,モモ,モモ"},
  {"の", "助詞,連体化,*,*,*,*,の,ノ,ノ"},
  {"うち", "名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ"}
]
```
```

--------------------------------

### tokenize_n_best/2 type specification

Source: https://hexdocs.pm/exawabi/ExAwabi.html

Type specification for the tokenize_n_best/2 function. It takes a binary and an integer, and returns a list of lists of binaries.

```elixir
@spec tokenize_n_best(binary(), integer()) :: [[binary()]]
```
Tokens:
Raw