MeCab

https://github.com/taku910/mecab
Admin
MeCab is an open-source text segmentation and morphological analysis engine designed for Japanese...
Tokens:41,967
Snippets:332
Trust Score:8.4
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install MeCab Dictionaries

Source: https://github.com/taku910/mecab/blob/master/index.html

Build and installation sequence for MeCab dictionary packages.

```shell
% tar zxfv mecab-ipadic-2.7.0-XXXX.tar.gz
% mecab-ipadic-2.7.0-XXXX
% ./configure
% make
% su
```

--------------------------------

### Example Output Format Configurations

Source: https://github.com/taku910/mecab/blob/master/format.html

Examples of custom format definitions extracted from dicrc, demonstrating various output styles like yomi, simple, csv, and ChaSen compatibility.

```text
; yomi
node-format-yomi = %pS%f[7]
unk-format-yomi = %M
eos-format-yomi  = \n

; simple
node-format-simple = %m\t%F-[0,1,2,3]\n
eos-format-simple  = EOS\n

; csv
node-format-csv = %m,%f[7],%f[8],%f[6],%F-[0,1,2,3],%f[4],%f[5]\n
unk-format-csv  = %m,%m,%m,%f[6],%F-[0,1,2,3],,\n
eos-format-csv  = EOS,,,,,,\n

; ChaSen
node-format-chasen = %m\t%f[7]\t%f[6]\t%F-[0,1,2,3]\t%f[4]\t%f[5]\n
unk-format-chasen  = %m\t%m\t%m\t%F-[0,1,2,3]\t\t\n
eos-format-chasen  = EOS\n
```

--------------------------------

### Partial Parsing Input Format Examples

Source: https://github.com/taku910/mecab/blob/master/partial.html

Examples of input formats for the -p option, demonstrating wildcard usage and explicit part-of-speech specification.

```text
にわ    *
に      *
はにわ  *
にわとり        *
が      *
いる    *
EOS
```

```text
にわ  *
に      助詞
はにわ  *
にわとり        *
が      接続詞
いる    *,非自立
EOS
```

```text
にわ
には
にわ
にわとり
がいる
EOS
```

--------------------------------

### Install MeCab on UNIX

Source: https://github.com/taku910/mecab/blob/master/index.html

Standard build and installation sequence for MeCab source packages on UNIX-like systems.

```shell
% tar zxfv mecab-X.X.tar.gz
% cd mecab-X.X
% ./configure 
% make
% make check
% su
# make install
```

--------------------------------

### Auto Link implementation examples

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/dic-detail.html

Configuration files and commands for implementing an Auto Link feature.

```c
cost = (int)max(-36000, -400 * (length^1.5)) 
```

```csv
Google,0,0,-5878,http://www.google.com/
Yahoo,0,0,-4472,http://www.yahoo.com/
ChaSen,0,0,-5878,http://chasen.org/
京都,0,0,-3200,http://www.city.kyoto.jp/
...
```

```text
1 1
0 0 0
```

```text
cost-factor = 800
bos-feature = BOS/EOS
output-format-type=autolink

node-format-autolink = <a href="%H">%M</a>
unk-format-autolink = %M
eos-format-autolink = \n
```

```bash
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -c euc-jp
reading ./unk.def ..  2
emitting double-array: 100% |###########################################| 
reading ./dic.csv ..  4
emitting double-array: 100% |###########################################| 
emitting matrix      : 100% |###########################################
done!

% mecab -d .
京都に行った. 
<a href="http://www.city.kyoto.jp/">京都</a>に行った。
YahooとGoogle
<a href="http://www.yahoo.com/">Yahoo</a>と<a href="http://www.google.com/">Google</a>
```

--------------------------------

### List Seed Dictionary Files

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/learn.html

Example of listing the contents of the seed directory.

```bash
% cd $WORK/seed
% ls 
Adj.csv          Interjection.csv   Noun.name.csv    Noun.verbal.csv  Symbol.csv        rewrite.def
Adnominal.csv    Noun.adjv.csv      Noun.number.csv  Others.csv       Verb.csv          unk.def
Adverb.csv       Noun.adverbal.csv  Noun.org.csv     Postp-col.csv    char.def
Auxil.csv        Noun.csv           Noun.others.csv  Postp.csv        corpus
Conjunction.csv  Noun.demonst.csv   Noun.place.csv   Prefix.csv       dicrc
Filler.csv       Noun.nai.csv       Noun.proper.csv  Suffix.csv       feature.def
```

--------------------------------

### Define char.def Categories

Source: https://github.com/taku910/mecab/blob/master/learn.html

Examples of category definitions and codepoint mappings for unknown word processing in char.def.

```text
KANJI          0 0 2
SYMBOL         1 1 0
NUMERIC        1 1 0
ALPHA          1 1 0
HIRAGANA       0 1 2 
```

```text
0x0009 SPACE
0x30A1..0x30FF  KATAKANA
0x30FC          KATAKANA HIRAGANA  # ー
```

```text
DEFAULT        0 1 0  # DEFAULT is a mandatory category!
SPACE          0 1 0  
KANJI          0 0 2
SYMBOL         1 1 0
NUMERIC        1 1 0
ALPHA          1 1 0
HIRAGANA       0 1 2 
KATAKANA       1 1 0
KANJINUMERIC   1 1 0
GREEK          1 1 0
CYRILLIC       1 1 0
```

--------------------------------

### Define Rewrite Rules

Source: https://github.com/taku910/mecab/blob/master/learn.html

Examples of rewrite rules for left and right contexts in MeCab configuration files.

```text
# 読み,発音をとりのぞいて, 品詞1,2,3,4,活用形,活用型,原形,よみ を使う
* ,* ,* ,* ,* ,* ,* ,*  $1,$2,$3,$4,$5,$6,$7,$8
# 読みがない場合は無視
* ,* ,* ,* ,* ,* ,*    $1,$2,$3,$4,$5,$6,$7,*

[left rewrite]
(助詞|助動詞),* ,* ,* ,* ,* ,(ない|無い)    $1,$2,$3,$4,$5,$6,無い
(助詞|助動詞),終助詞,* ,* ,* ,* ,(よ|ヨ)   $1,$2,$3,$4,$5,$6,よ
...

[right rewrite]
(助詞|助動詞),* ,* ,* ,* ,* ,(ない|無い)    $1,$2,$3,$4,$5,$6,無い
(助詞|助動詞),終助詞,* ,* ,* ,* ,(よ|ヨ)   $1,$2,$3,$4,$5,$6,よ
..
```

--------------------------------

### Partial Parsing Command Line Usage

Source: https://github.com/taku910/mecab/blob/master/partial.html

Example of running MeCab with the -p option to force a specific input string into a single morpheme.

```bash
% mecab -p
こんな長い入力を一形態素にしてみる      動詞
EOS
こんな長い入力を一形態素にしてみる      動詞
EOS
```

--------------------------------

### Example Inflected Dictionary Entries

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/learn.html

Example of how to define inflected forms in the dictionary, as MeCab does not perform inflection processing automatically.

```csv
連れ出す,0,0,0,動詞,自立,*,*,五段・サ行,基本形,連れ出す,ツレダス,ツレダス
連れ出さ,0,0,0,動詞,自立,*,*,五段・サ行,未然形,連れ出す,ツレダサ,ツレダサ
連れ出そ,0,0,0,動詞,自立,*,*,五段・サ行,未然ウ接続,連れ出す,ツレダソ,ツレダソ
連れ出し,0,0,0,動詞,自立,*,*,五段・サ行,連用形,連れ出す,ツレダシ,ツレダシ
連れ出せ,0,0,0,動詞,自立,*,*,五段・サ行,仮定形,連れ出す,ツレダセ,ツレダセ
連れ出せ,0,0,0,動詞,自立,*,*,五段・サ行,命令ｅ,連れ出す,ツレダセ,ツレダセ
連れ出しゃ,0,0,0,動詞,自立,*,*,五段・サ行,仮定縮約１,連れ出す,ツレダシャ,ツレダシャ
```

--------------------------------

### Unigram Rewrite Section

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/learn.html

Example header for the unigram rewrite section in rewrite.def.

```text
[unigram rewrite]
```

--------------------------------

### Monitor Training Output

Source: https://github.com/taku910/mecab/blob/master/learn.html

Example of the console output generated during the CRF training process.

```text
reading corpus ... adding virtual node: 名詞,固有名詞,地域,一般,*,*,東日,トウニチ,トウニチ
adding virtual node: 副詞,助詞類接続,*,*,*,*,かなり,カナリ,カナリ

Number of sentences: 32
Number of features:  47547
eta:                 0.00010
freq:                1
C(sigma^2):          1.00000

iter=0 err=1.00000 F=0.41186 target=1691.68869 diff=1.00000
iter=1 err=1.00000 F=0.68727 target=1077.14848 diff=0.36327
iter=2 err=0.87500 F=0.81904 target=621.20311 diff=0.42329
iter=3 err=0.81250 F=0.86354 target=384.72432 diff=0.38068
iter=4 err=0.68750 F=0.93685 target=233.72722 diff=0.39248
..
```

--------------------------------

### Auto Link url.csv entries

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Example CSV entries for mapping keywords to URLs.

```csv
Google,0,0,-5878,http://www.google.com/
Yahoo,0,0,-4472,http://www.yahoo.com/
ChaSen,0,0,-5878,http://chasen.org/
京都,0,0,-3200,http://www.city.kyoto.jp/
...
```

--------------------------------

### C++ Node Traversal and Lattice Operations

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/libmecab.html

Examples of iterating through lattice nodes, retrieving N-best results, calculating marginal probabilities, and accessing dictionary metadata.

```cpp
  std::cout << ' ' << node->feature
	      << ' ' << (int)(node->surface - input)
	      << ' ' << (int)(node->surface - input + node->length)
	      << ' ' << node->rcAttr
	      << ' ' << node->lcAttr
	      << ' ' << node->posid
	      << ' ' << (int)node->char_type
	      << ' ' << (int)node->stat
	      << ' ' << (int)node->isbest
	      << ' ' << node->alpha
	      << ' ' << node->beta
	      << ' ' << node->prob
	      << ' ' << node->cost << std::endl;
  }


  // begin_nodes/end_nodes
  const size_t len = lattice->size();
  for (int i = 0; i <= len; ++i) {
    MeCab::Node *b = lattice->begin_nodes(i);
    MeCab::Node *e = lattice->end_nodes(i);
    for (; b; b = b->bnext) {
      printf("B[%d] %s\t%s\n", i, b->surface, b->feature);
    }
    for (; e; e = e->enext) {
      printf("E[%d] %s\t%s\n", i, e->surface, e->feature);
    }
  }

  // N best results
  lattice->set_request_type(MECAB_NBEST);
  lattice->set_sentence(input);
  CHECK(tagger->parse(lattice));
  for (int i = 0; i < 10; ++i) {
    std::cout << "NBEST: " << i << std::endl;
    std::cout << lattice->toString();
    if (!lattice->next()) {
      // No more results
      break;
    }
  }

  // Marginal probabilities
  lattice->remove_request_type(MECAB_NBEST);
  lattice->set_request_type(MECAB_MARGINAL_PROB);
  lattice->set_sentence(input);
  CHECK(tagger->parse(lattice));
  std::cout << lattice->theta() << std::endl;
  for (const MeCab::Node *node = lattice->bos_node();
       node; node = node->next) {
    std::cout.write(node->surface, node->length);
    std::cout << "\t" << node->feature;
    std::cout << "\t" << node->prob << std::endl;
  }

  // Dictionary info
  const MeCab::DictionaryInfo *d = model->dictionary_info();
  for (; d; d = d->next) {
    std::cout << "filename: " <<  d->filename << std::endl;
    std::cout << "charset: " <<  d->charset << std::endl;
    std::cout << "size: " <<  d->size << std::endl;
    std::cout << "type: " <<  d->type << std::endl;
    std::cout << "lsize: " <<  d->lsize << std::endl;
    std::cout << "rsize: " <<  d->rsize << std::endl;
    std::cout << "version: " <<  d->version << std::endl;
  }

  // Swap model atomically.
  MeCab::Model *another_model = MeCab::createModel("");
  model->swap(another_model);

  delete lattice;
  delete tagger;
  delete model;

  return 0;
}
```

--------------------------------

### Auto Link cost calculation formula

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Example function to calculate word cost for Auto Link based on length.

```c
cost = (int)max(-36000, -400 * (length^1.5)) 
```

--------------------------------

### Example Dictionary Entries

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/learn.html

Format for dictionary entries in CSV, where the first four columns are surface form, left connection ID, right connection ID, and cost.

```csv
進学校,0,0,0,名詞,一般,*,*,*,*,進学校,シンガクコウ,シンガクコー
梅暦,0,0,0,名詞,一般,*,*,*,*,梅暦,ウメゴヨミ,ウメゴヨミ
気圧,0,0,0,名詞,一般,*,*,*,*,気圧,キアツ,キアツ
水中翼船,0,0,0,名詞,一般,*,*,*,*,水中翼船,スイチュウヨクセン,スイチューヨクセン
```

--------------------------------

### Initialize Workspace and Directories

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/learn.html

Commands to set up the working directory structure for training.

```bash
cd $WORK
mkdir seed final
```

--------------------------------

### Initialize Training Directory

Source: https://github.com/taku910/mecab/blob/master/learn.html

Commands to set up the working directory and list required files for dictionary indexing.

```bash
cd $WORK
mkdir seed final
```

```bash
% cd $WORK/seed
% ls 
Adj.csv          Interjection.csv   Noun.name.csv    Noun.verbal.csv  Symbol.csv        rewrite.def
Adnominal.csv    Noun.adjv.csv      Noun.number.csv  Others.csv       Verb.csv          unk.def
Adverb.csv       Noun.adverbal.csv  Noun.org.csv     Postp-col.csv    char.def
Auxil.csv        Noun.csv           Noun.others.csv  Postp.csv        corpus
Conjunction.csv  Noun.demonst.csv   Noun.place.csv   Prefix.csv       dicrc
Filler.csv       Noun.nai.csv       Noun.proper.csv  Suffix.csv       feature.def
```

--------------------------------

### システム辞書の再コンパイルとインストール

Source: https://github.com/taku910/mecab/blob/master/dic.html

システム辞書を変更した後に実行するコマンド例です。

```bash
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -t euc-jp
% su
# make install
```

```bash
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -t utf8
```

--------------------------------

### ユーザ辞書のコンパイル

Source: https://github.com/taku910/mecab/blob/master/dic.html

ユーザ辞書ファイルを作成し、システム辞書を参照してコンパイルするコマンドです。

```bash
% /usr/local/libexec/mecab/mecab-dict-index -d/usr/local/lib/mecab/dic/ipadic \
-u foo.dic -f euc-jp -t euc-jp foo.csv
```

--------------------------------

### Build Binary Dictionary

Source: https://github.com/taku910/mecab/blob/master/learn.html

Compile the dictionary into a binary format for analysis using mecab-dict-index.

```bash
% cd $WORK/final
% /usr/local/libexec/mecab/mecab-dict-index
```

```bash
% /usr/local/libexec/mecab/mecab-dict-index -d $WORK/final -o $WORK/final
```

--------------------------------

### Define Feature Templates in feature.def

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/learn.html

Examples of UNIGRAM and BIGRAM feature templates used to extract features for CRF training.

```text
UNIGRAM W0:%F[6]
UNIGRAM W1:%F[0]/%F[6]
UNIGRAM W2:%F[0],%F?[1]/%F[6]
UNIGRAM W3:%F[0],%F[1],%F?[2]/%F[6]
UNIGRAM W4:%F[0],%F[1],%F[2],%F?[3]/%F[6]

UNIGRAM T0:%t
UNIGRAM T1:%F[0]/%t
UNIGRAM T2:%F[0],%F?[1]/%t
UNIGRAM T3:%F[0],%F[1],%F?[2]/%t
UNIGRAM T4:%F[0],%F[1],%F[2],%F?[3]/%t

BIGRAM B00:%L[0]/%R[0]
BIGRAM B01:%L[0],%L?[1]/%R[0]
BIGRAM B02:%L[0]/%R[0],%R?[1]
BIGRAM B03:%L[0]/%R[0],%R[1],%R?[2]
BIGRAM B04:%L[0],%L?[1]/%R[0],%R[1],%R?[2]
BIGRAM B05:%L[0]/%R[0],%R[1],%R[2],%R?[3]
BIGRAM B06:%L[0],%L?[1]/%R[0],%R[1],%R[2],%R?[3]
```

--------------------------------

### Simple pos-id.def Configuration

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/posid.html

A basic mapping of POS categories to numeric IDs.

```text
名詞 1
動詞 2
形容詞 3
副詞 4
助詞 5
接続詞 6
助動詞 7
連体詞 8
感動詞 9
* 10
```

--------------------------------

### Configure ASCII Unknown Word Handling

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/unk.html

Settings for char.def and unk.def to treat ASCII strings as unknown words, similar to kakasi behavior.

```text
ASCII       1 1 0

0x0021..0x007E ASCII
```

```text
ASCII,1192,1192,0,名詞,サ変接続,*,*,*,*,*
```

--------------------------------

### Compile and Test the Dictionary

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Commands to index the dictionary and test the conversion using the mecab command-line tool.

```bash
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -c euc-jp
reading ./unk.def ..  2
emitting double-array: 100% |###########################################| 
reading ./dic.csv ..  4
emitting double-array: 100% |###########################################| 
emitting matrix      : 100% |###########################################
done!
% mecab -d .
これはてすとです
コレハテストデス
```

--------------------------------

### Format Training Corpus

Source: https://github.com/taku910/mecab/blob/master/learn.html

Example of the training data format where surface forms are tab-separated from CSV-formatted feature arrays, with EOS marking sentence boundaries.

```text
太郎    名詞,固有名詞,人名,名,*,*,太郎,タロウ,タロー
は      助詞,係助詞,*,*,*,*,は,ハ,ワ
花子    名詞,固有名詞,人名,名,*,*,花子,ハナコ,ハナコ
が      助詞,格助詞,一般,*,*,*, が,ガ,ガ
好き    名詞,形容動詞語幹,*,*,*,*, 好き,スキ,スキ
だ      助動詞,*,*,*, 特殊・ダ,基本形,だ,ダ,ダ
.       記号,句点,*,*,*,*, . , . , . 
EOS
焼酎    名詞,一般,*,*,*,*,焼酎,ショウチュウ,ショーチュー
好き    名詞,形容動詞語幹,*,*,*,*,好き,スキ,スキ
の      助詞,連体化,*,*,*,*, の,ノ,ノ
親父    名詞,一般,*,*,*,*,親父,オヤジ,オヤジ
.       記号,句点,*,*,*,*, . , . , . 
EOS
```

--------------------------------

### Single-Threaded C API Usage

Source: https://github.com/taku910/mecab/blob/master/libmecab.html

Demonstrates basic parsing, N-best retrieval, and node inspection in a single-threaded environment.

```c
#include <mecab.h>
#include <stdio.h>

#define CHECK(eval) if (! eval) { \
    fprintf (stderr, "Exception:%s\n", mecab_strerror (mecab)); \
    mecab_destroy(mecab); \
    return -1; }

int main (int argc, char **argv)  {
  char input[] = "太郎は次郎が持っている本を花子に渡した。";
  mecab_t *mecab;
  const mecab_node_t *node;
  const char *result;
  int i;
  size_t len;

  // Create tagger object
  mecab = mecab_new(argc, argv);
  CHECK(mecab);

  // Gets tagged result in string.
  result = mecab_sparse_tostr(mecab, input);
  CHECK(result)
  printf ("INPUT: %s\n", input);
  printf ("RESULT:\n%s", result);

  // Gets N best results
  result = mecab_nbest_sparse_tostr (mecab, 3, input);
  CHECK(result);
  fprintf (stdout, "NBEST:\n%s", result);

  CHECK(mecab_nbest_init(mecab, input));
  for (i = 0; i < 3; ++i) {
    printf ("%d:\n%s", i, mecab_nbest_next_tostr (mecab));
  }

  // Gets node object
  node = mecab_sparse_tonode(mecab, input);
  CHECK(node);
  for (; node; node = node->next) {
    if (node->stat == MECAB_NOR_NODE || node->stat == MECAB_UNK_NODE) {
      fwrite (node->surface, sizeof(char), node->length, stdout);
      printf("\t%s\n", node->feature);
    }
  }

  // Dictionary info
  const mecab_dictionary_info_t *d = mecab_dictionary_info(mecab);
  for (; d; d = d->next) {
    printf("filename: %s\n", d->filename);
    printf("charset: %s\n", d->charset);
    printf("size: %d\n", d->size);
    printf("type: %d\n", d->type);
    printf("lsize: %d\n", d->lsize);
    printf("rsize: %d\n", d->rsize);
    printf("version: %d\n", d->version);
  }

  mecab_destroy(mecab);

  return 0;
}
```

--------------------------------

### Define char.def

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Minimal configuration for character definitions including DEFAULT and SPACE settings.

```text
DEFAULT 1 0 0
SPACE   0 1 0
0x0020 SPACE
```

--------------------------------

### MeCab::Tagger::parseNBestInit

Source: https://github.com/taku910/mecab/blob/master/bindings.html

Initializes the N-best parsing process.

```APIDOC
## bool parseNBestInit(string str, int len)

### Description
Initializes the N-best parsing process to retrieve results in order of probability.

### Parameters
- **str** (string) - Required - The input string to parse.
- **len** (int) - Optional - The length of the input string.
```

--------------------------------

### 辞書リコンパイルコマンド

Source: https://github.com/taku910/mecab/blob/master/posid.html

pos-id.defを変更した後に実行する辞書リコンパイル用コマンド。

```bash
/usr/local/libexec/mecab/mecab-dict-index
```

--------------------------------

### Analyze file and output to file

Source: https://github.com/taku910/mecab/blob/master/index.html

Specify an input file for analysis and redirect the output to a specified file using the -o option.

```bash
% mecab INPUT -o OUTPUT
```

--------------------------------

### MeCab動作確認例

Source: https://github.com/taku910/mecab/blob/master/dic.html

追加した単語を含むテキストを解析する例です。

```bash
% mecab 
ユーザ設定が必要です。
ユーザ設定      名詞,一般,*,*,*,*,ユーザ設定,ユーザセッテイ,ユーザセッテイ,追加エントリ
が      助詞,格助詞,一般,*,*,*,が,ガ,ガ
必要    名詞,形容動詞語幹,*,*,*,*,必要,ヒツヨウ,ヒツヨー
です    助動詞,*,*,*,特殊・デス,基本形,です,デス,デス
。      記号,句点,*,*,*,*,。,。,。
EOS
```

--------------------------------

### Output all morphemes with -a option

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/soft.html

Use the -a option to output all morphemes in the lattice instead of just the optimal solution.

```bash
% mecab -a
東京都庁
東京    名詞,固有名詞,地域,一般,*,*,東京,トウキョウ,トーキョー
東      名詞,固有名詞,一般,*,*,*,東,ヒガシ,ヒガシ
東      名詞,固有名詞,地域,一般,*,*,東,ヒガシ,ヒガシ
東      名詞,固有名詞,人名,名,*,*,東,ヒガシ,ヒガシ
東      名詞,固有名詞,人名,姓,*,*,東,アズマ,アズマ
東      名詞,一般,*,*,*,*,東,ヒガシ,ヒガシ
京都    名詞,固有名詞,一般,*,*,*,京都,キョウト,キョート
京都    名詞,固有名詞,地域,一般,*,*,京都,キョウト,キョート
京      名詞,固有名詞,地域,一般,*,*,京,キョウ,キョー
京      名詞,固有名詞,人名,名,*,*,京,ミヤコ,ミヤコ
京      名詞,固有名詞,人名,姓,*,*,京,キョウ,キョー
都庁    名詞,一般,*,*,*,*,都庁,トチョウ,トチョー
都      名詞,接尾,地域,*,*,*,都,ト,ト
都      名詞,固有名詞,地域,一般,*,*,都,ミヤコ,ミヤコ
都      名詞,固有名詞,人名,姓,*,*,都,ミヤコ,ミヤコ
都      名詞,固有名詞,人名,名,*,*,都,ミヤコ,ミヤコ
都      名詞,一般,*,*,*,*,都,ト,ト
庁      名詞,接尾,一般,*,*,*,庁,チョウ,チョー
庁      名詞,固有名詞,地域,一般,*,*,庁,チョウ,チョー
庁      名詞,一般,*,*,*,*,庁,チョウ,チョー
EOS
```

--------------------------------

### コストの自動推定コマンド

Source: https://github.com/taku910/mecab/blob/master/dic.html

モデルファイルを使用してコストを自動推定するコマンド例です。

```bash
% /usr/local/libexec/mecab/mecab-dict-index -m model_file -d/usr/local/lib/mecab/dic/ipadic \
-u foo.dic -f euc-jp -t euc-jp foo.csv
```

```bash
% /usr/local/libexec/mecab/mecab-dict-index -m model_file -d/usr/local/lib/mecab/dic/ipadic \
-u foo2.csv -f euc-jp -t euc-jp -a foo.csv
```

--------------------------------

### Perform basic morphological analysis

Source: https://github.com/taku910/mecab/blob/master/bindings.html

Initialize a MeCab::Tagger instance with parameters and parse text to retrieve results as a string.

```perl
use MeCab;
$m = new MeCab::Tagger ("-Ochasen");
print $m->parse ("今日もしないとね");
```

```ruby
require 'MeCab'
m = MeCab::Tagger.new ("-Ochasen")
print m.parse ("今日もしないとね")
```

```python
import sys
import MeCab
m = MeCab.Tagger ("-Ochasen")
print m.parse ("今日もしないとね")
```

```java
import org.chasen.mecab.Tagger;
import org.chasen.mecab.Node
 public static void main(String[] argv) {
 Tagger tagger = new Tagger ("-Ochasen");
 System.out.println (tagger.parse ("太郎は二郎にこの本を渡した.")); 
}
```

--------------------------------

### Compile and test Auto Link

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Commands to compile the dictionary and test the Auto Link functionality.

```bash
% /usr/local/libexec/mecab/mecab-dict-index -f euc-jp -c euc-jp
reading ./unk.def ..  2
emitting double-array: 100% |###########################################| 
reading ./dic.csv ..  4
emitting double-array: 100% |###########################################| 
emitting matrix      : 100% |###########################################
done!

% mecab -d .
京都に行った. 
<a href="http://www.city.kyoto.jp/">京都</a>に行った。
YahooとGoogle
<a href="http://www.yahoo.com/">Yahoo</a>と<a href="http://www.google.com/">Google</a>
```

--------------------------------

### Auto Link matrix.def

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Connection table for a single-state Auto Link configuration.

```text
1 1
0 0 0
```

--------------------------------

### createModel

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/doxygen/namespaceMeCab.html

Creates a new Model object using command-line arguments or a configuration string.

```APIDOC
## Model* createModel(int argc, char** argv)
## Model* createModel(const char* arg)

### Description
Creates a new Model object. Can be initialized using standard command-line arguments (argc/argv) or a single configuration string.

### Parameters
- **argc** (int) - Number of arguments.
- **argv** (char**) - Array of argument strings.
- **arg** (const char*) - Configuration string.

### Returns
- **Model*** - A pointer to the created Model object.
```

--------------------------------

### Generate Distribution Dictionary

Source: https://github.com/taku910/mecab/blob/master/learn.html

Create a distribution dictionary from a seed directory using the mecab-dict-gen utility.

```bash
% cd $WORK/seed
% /usr/local/libexec/mecab/mecab-dict-gen -o ../final -m model
```

```bash
% /usr/local/libexec/mecab/mecab-dict-gen -o $WORK/final -d $WORK/seed -m $WORK/seed/model
```

--------------------------------

### Configure ASCII string handling in char.def

Source: https://github.com/taku910/mecab/blob/master/unk.html

Map ASCII characters to a single category for custom unknown word processing.

```text
ASCII       1 1 0

0x0021..0x007E ASCII
```

--------------------------------

### Analyze text with MeCab

Source: https://github.com/taku910/mecab/blob/master/index.html

Run MeCab from the command line to analyze text provided via standard input.

```bash
% mecab
すもももももももものうち
すもも  名詞,一般,*,*,*,*,すもも,スモモ,スモモ
も      助詞,係助詞,*,*,*,*,も,モ,モ
もも    名詞,一般,*,*,*,*,もも,モモ,モモ
も      助詞,係助詞,*,*,*,*,も,モ,モ
もも    名詞,一般,*,*,*,*,もも,モモ,モモ
の      助詞,連体化,*,*,*,*,の,ノ,ノ
うち    名詞,非自立,副詞可能,*,*,*,うち,ウチ,ウチ
EOS
```

--------------------------------

### Generate Binary Dictionary

Source: https://github.com/taku910/mecab/blob/master/learn.html

Commands to execute the dictionary indexer to create binary dictionaries from seed files.

```bash
% cd $WORK/seed
% /usr/local/libexec/mecab/mecab-dict-index
```

```bash
% /usr/local/libexec/mecab/mecab-dict-index -d $WORK/seed -o $WORK/seed
```

--------------------------------

### Define unk.def

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Minimal configuration for unknown word part-of-speech definitions.

```text
DEFAULT,0,0,0,*
SPACE,0,0,0,*
```

--------------------------------

### Detailed pos-id.def Configuration

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/posid.html

A comprehensive mapping covering specific POS sub-categories.

```text
その他,間投,*,* 0
フィラー,*,*,* 1
感動詞,*,*,* 2
記号,アルファベット,*,* 3
記号,一般,*,* 4
記号,括弧開,*,* 5
記号,括弧閉,*,* 6
記号,句点,*,* 7
記号,空白,*,* 8
記号,読点,*,* 9
形容詞,自立,*,* 10
形容詞,接尾,*,* 11
形容詞,非自立,*,* 12
助詞,格助詞,一般,* 13
助詞,格助詞,引用,* 14
助詞,格助詞,連語,* 15
助詞,係助詞,*,* 16
助詞,終助詞,*,* 17
助詞,接続助詞,*,* 18
助詞,特殊,*,* 19
助詞,副詞化,*,* 20
助詞,副助詞,*,* 21
助詞,副助詞／並立助詞／終助詞,*,* 22
助詞,並立助詞,*,* 23
助詞,連体化,*,* 24
助動詞,*,*,* 25
接続詞,*,*,* 26
接頭詞,形容詞接続,*,* 27
接頭詞,数接続,*,* 28
接頭詞,動詞接続,*,* 29
接頭詞,名詞接続,*,* 30
動詞,自立,*,* 31
動詞,接尾,*,* 32
動詞,非自立,*,* 33
副詞,一般,*,* 34
副詞,助詞類接続,*,* 35
名詞,サ変接続,*,* 36
名詞,ナイ形容詞語幹,*,* 37
名詞,一般,*,* 38
名詞,引用文字列,*,* 39
名詞,形容動詞語幹,*,* 40
名詞,固有名詞,一般,* 41
名詞,固有名詞,人名,一般 42
名詞,固有名詞,人名,姓 43
名詞,固有名詞,人名,名 44
名詞,固有名詞,組織,* 45
名詞,固有名詞,地域,一般 46
名詞,固有名詞,地域,国 47
名詞,数,*,* 48
名詞,接続詞的,*,* 49
名詞,接尾,サ変接続,* 50
名詞,接尾,一般,* 51
名詞,接尾,形容動詞語幹,* 52
名詞,接尾,助数詞,* 53
名詞,接尾,助動詞語幹,* 54
名詞,接尾,人名,* 55
名詞,接尾,地域,* 56
名詞,接尾,特殊,* 57
名詞,接尾,副詞可能,* 58
名詞,代名詞,一般,* 59
名詞,代名詞,縮約,* 60
名詞,動詞非自立的,*,* 61
名詞,特殊,助動詞語幹,* 62
名詞,非自立,一般,* 63
名詞,非自立,形容動詞語幹,* 64
名詞,非自立,助動詞語幹,* 65
名詞,非自立,副詞可能,* 66
名詞,副詞可能,*,* 67
連体詞,*,*,* 68
```

--------------------------------

### Specify Output Formats via Command Line

Source: https://github.com/taku910/mecab/blob/master/format.html

Use command-line arguments to override default output formatting strings.

```bash
% mecab --node-format=STR --bos-format=STR --eos-format=STR --unk-format=STR --eon-format=STR
```

--------------------------------

### Define matrix.def

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Specifies the size of the connection table followed by the transition costs between context IDs.

```text
100 120
0 0 1
0 1 10
0 2 5
```

--------------------------------

### Train CRF Parameters

Source: https://github.com/taku910/mecab/blob/master/learn.html

Commands to train the CRF model using the prepared corpus and binary dictionary.

```bash
% cd $WORK/seed
% /usr/local/libexec/mecab/mecab-cost-train -c 1.0 corpus model
```

```bash
% /usr/local/libexec/mecab/mecab-cost-train -d $WORK/seed -c 1.0 $WORK/seed/corpus $WORK/seed/model
```

--------------------------------

### createLattice

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/doxygen/namespaceMeCab.html

Creates a new Lattice object.

```APIDOC
## Lattice* createLattice()

### Description
Creates and returns a new instance of a Lattice object.

### Returns
- **Lattice*** - A pointer to the newly created Lattice object.
```

--------------------------------

### Change output format

Source: https://github.com/taku910/mecab/blob/master/index.html

Use the -O option to switch between different output formats like reading, ChaSen compatibility, or full dump.

```bash
% mecab -Oyomi (ヨミ付与)
% mecab -Ochasen (ChaSen互換)
% mecab -Odump (全情報を出力)
```

--------------------------------

### Configure Numeric Unknown Word Handling

Source: https://github.com/taku910/mecab/blob/master/mecab/doc/unk.html

Settings for char.def and unk.def to treat consecutive numbers as a single unknown word.

```text
NUMERIC        1 1 0
```

```text
NUMERIC,1204,1204,0,名詞,数,*,*,*,*,*
```

--------------------------------

### Compile MeCab dictionary

Source: https://github.com/taku910/mecab/blob/master/dic-detail.html

Command to generate the binary dictionary for analysis.

```bash
% /usr/local/libexec/mecab/mecab-dict-index 
```

--------------------------------

### Multi-Threaded C API Usage

Source: https://github.com/taku910/mecab/blob/master/libmecab.html

Demonstrates using models and lattices for thread-safe parsing.

```c
#include <mecab.h>
#include <stdio.h>

#define CHECK(eval) if (! eval) { \
    fprintf (stderr, "Exception:%s\n", mecab_strerror (mecab)); \
    mecab_destroy(mecab); \
    return -1; }

int main (int argc, char **argv)  {
  char input[] = "太郎は次郎が持っている本を花子に渡した。";
  mecab_model_t *model, *another_model;
  mecab_t *mecab;
  mecab_lattice_t *lattice;
  const mecab_node_t *node;
  const char *result;
  int i;
  size_t len;

  model = mecab_model_new(argc, argv);
  CHECK(model);

  mecab = mecab_model_new_tagger(model);
  CHECK(mecab);

  lattice = mecab_model_new_lattice(model);
  CHECK(lattice);

  mecab_lattice_set_sentence(lattice, input);
  mecab_parse_lattice(mecab, lattice);

  printf("RESULT: %s\n", mecab_lattice_tostr(lattice));

  node = mecab_lattice_get_bos_node(lattice);
  for (;  node; node = node->next) {
    printf("%d ", node->id);
```
Tokens:
Raw