### Install natural via NPM Source: https://naturalnode.github.io/natural/installation.html Use this command to install the natural library into your project. ```bash npm install natural ``` -------------------------------- ### Initialize PresentVerbInflector Source: https://naturalnode.github.io/natural/inflectors.html Setup the PresentVerbInflector instance. Note that this feature is experimental. ```javascript var verbInflector = new natural.PresentVerbInflector(); ``` -------------------------------- ### Initialize Phonetic Algorithms Source: https://naturalnode.github.io/natural/phonetics.html Setup the Metaphone and SoundEx instances for phonetic comparison. ```javascript const natural = require('natural'); const metaphone = new natural.Metaphone(); const soundEx = new natural.SoundEx(); const wordA = 'phonetics'; const wordB = 'fonetix'; ``` -------------------------------- ### Initialize ShortestPathTree Source: https://naturalnode.github.io/natural/shortest_path_tree.html Instantiate the ShortestPathTree with an EdgeWeightedDigraph and a starting vertex. ```javascript var ShortestPathTree = natural.ShortestPathTree; var spt = new ShortestPathTree(digraph, 5); ``` -------------------------------- ### Initialize and use various tokenizers Source: https://naturalnode.github.io/natural/Tokenizers.html Examples of Treebank, Regexp, WordPunct, and Orthography tokenizers. ```javascript tokenizer = new natural.TreebankWordTokenizer(); console.log(tokenizer.tokenize("my dog hasn't any fleas.")); // [ 'my', 'dog', 'has', 'n\'t', 'any', 'fleas', '.' ] tokenizer = new natural.RegexpTokenizer({pattern: /\-/}); console.log(tokenizer.tokenize("flea-dog")); // [ 'flea', 'dog' ] tokenizer = new natural.WordPunctTokenizer(); console.log(tokenizer.tokenize("my dog hasn't any fleas.")); // [ 'my', 'dog', 'hasn', '\'', 't', 'any', 'fleas', '.' ] tokenizer = new natural.OrthographyTokenizer({language: "fi"}); console.log(tokenizer.tokenize("Mikä sinun nimesi on?")); // [ 'Mikä', 'sinun', 'nimesi', 'on' ] ``` -------------------------------- ### Install WordNet database Source: https://naturalnode.github.io/natural/wordnet.html Required command to install the necessary WordNet database files via npm. ```bash npm install wordnet-db ``` -------------------------------- ### Importing a tokenizer module Source: https://naturalnode.github.io/natural/development.html Example of importing a specific module to maintain library modularity. ```javascript const tokenenizers = require('natural/lib/natural/tokenizers') const tokenizer = new tokenizers.SentenceTokenizer() ``` -------------------------------- ### Initialize LongestPathTree Source: https://naturalnode.github.io/natural/longest_path_tree.html Instantiate the LongestPathTree with an EdgeWeightedDigraph and a starting vertex. ```javascript var LongestPathTree = natural.LongestPathTree; var lpt = new LongestPathTree(digraph, 5); ``` -------------------------------- ### keysWithPrefix Source: https://naturalnode.github.io/natural/tries.html Retrieves all words in the Trie that start with the specified prefix. ```APIDOC ## keysWithPrefix(prefix) ### Description Returns an array of all words in the Trie that begin with the given prefix, or an empty array if no matches are found. ``` -------------------------------- ### Tagger Output Example Source: https://naturalnode.github.io/natural/brill_pos_tagger.html The structure of the object returned by the tagger. ```text Sentence { taggedWords: [ { token: 'I', tag: 'NN' }, { token: 'see', tag: 'VB' }, { token: 'the', tag: 'DT' }, { token: 'man', tag: 'NN' }, { token: 'with', tag: 'IN' }, { token: 'the', tag: 'DT' }, { token: 'telescope', tag: 'NN' } ] } ``` -------------------------------- ### Define rule predicate examples Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Examples of specific transformation rules using predicates to check word categories or properties. ```text NN CD CURRENT-WORD-IS-NUMBER YES ``` ```text VBD NN PREV-TAG DT ``` -------------------------------- ### new ShortestPathTree(digraph, startVertex) Source: https://naturalnode.github.io/natural/shortest_path_tree.html Constructor to initialize a new ShortestPathTree instance for a given directed acyclic graph and starting vertex. ```APIDOC ## Constructor ### Description Initializes a new ShortestPathTree instance to solve the single-source shortest paths problem. ### Parameters - **digraph** (EdgeWeightedDigraph) - Required - The edge-weighted directed acyclic graph. - **startVertex** (number) - Required - The starting vertex for the shortest path calculation. ``` -------------------------------- ### Retrieve all keys with a prefix Source: https://naturalnode.github.io/natural/tries.html Returns an array of all words in the Trie that start with the specified prefix. ```javascript console.log(trie.keysWithPrefix("string")); // ["string1", "string2", "string3"] ``` -------------------------------- ### Initialize NounInflector Source: https://naturalnode.github.io/natural/inflectors.html Setup the NounInflector instance to perform pluralization and singularization tasks. ```javascript var natural = require('natural'); var nounInflector = new natural.NounInflector(); ``` -------------------------------- ### getDistTo(vertex) Source: https://naturalnode.github.io/natural/longest_path_tree.html Returns the distance from the start vertex to the specified target vertex. ```APIDOC ## Method: getDistTo(vertex) ### Description Returns the longest path distance to the specified vertex. ### Parameters - **vertex** (number) - Required - The target vertex. ### Returns - **number** - The distance to the vertex. ``` -------------------------------- ### new LongestPathTree(digraph, startVertex) Source: https://naturalnode.github.io/natural/longest_path_tree.html Initializes a new instance of the LongestPathTree to compute longest paths from a specified start vertex in a DAG. ```APIDOC ## Constructor: new LongestPathTree(digraph, startVertex) ### Description Initializes the LongestPathTree with a directed acyclic graph and a starting vertex. ### Parameters - **digraph** (EdgeWeightedDigraph) - Required - An instance of EdgeWeightedDigraph. - **startVertex** (number) - Required - The starting vertex for the longest path calculation. ``` -------------------------------- ### Get definition by synset offset Source: https://naturalnode.github.io/natural/wordnet.html Retrieves a specific definition directly using a synset offset and part of speech identifier. ```javascript var wordnet = new natural.WordNet(); wordnet.get(4424418, 'n', function(result) { console.log('------------------------------------'); console.log(result.lemma); console.log(result.pos); console.log(result.gloss); console.log(result.synonyms); }); ``` -------------------------------- ### Apply Padding to N-Grams Source: https://naturalnode.github.io/natural/n_grams.html Generate n-grams with specified start and end padding symbols. ```javascript console.log(NGrams.ngrams('some other words here for you', 4, '[start]', '[end]')); ``` ```javascript console.log(NGrams.ngrams('some other words here for you', 4, null, '[end]')); ``` -------------------------------- ### Get classification probabilities Source: https://naturalnode.github.io/natural/bayesian_classifier.html Retrieve the set of matched classes and their associated values. ```javascript [ { label: 'buy', value: 0.39999999999999997 }, { label: 'sell', value: 0.19999999999999998 } ] ``` ```javascript console.log(classifier.getClassifications('i am long copper')); ``` -------------------------------- ### Define maximum entropy model test data Source: https://naturalnode.github.io/natural/maxent_classifier.html Example JSON structure for defining context and class mappings in a test specification. ```json { "a": "x", "b": { "data": "0" } } ``` -------------------------------- ### Get Phonetics via String Method Source: https://naturalnode.github.io/natural/phonetics.html Retrieve phonetic representation directly from a string using the phonetics() method. ```javascript console.log('phonetics'.phonetics()); ``` -------------------------------- ### Get Distance to Vertex Source: https://naturalnode.github.io/natural/longest_path_tree.html Retrieve the longest path distance to a specific vertex. ```javascript console.log(lpt.getDistTo(4)); ``` -------------------------------- ### Retrieve vertex count Source: https://naturalnode.github.io/natural/edge_weighted_digraph.html Use the v() method to get the total number of vertices in the digraph. ```javascript console.log(digraph.v()); ``` -------------------------------- ### Retrieve edge count Source: https://naturalnode.github.io/natural/edge_weighted_digraph.html Use the e() method to get the total number of edges in the digraph. ```javascript console.log(digraph.e()); ``` -------------------------------- ### Get Longest Path to Vertex Source: https://naturalnode.github.io/natural/longest_path_tree.html Retrieve the sequence of vertices representing the longest path to a specific vertex. ```javascript console.log(lpt.pathTo(4)); ``` ```text [5, 1, 3, 6, 4] ``` -------------------------------- ### Retrieve Spelling Corrections Source: https://naturalnode.github.io/natural/spellcheck.html Get a list of suggested corrections sorted by probability, limited by a maximum edit distance. ```javascript spellcheck.getCorrections('soemthing', 1); // ['something'] spellcheck.getCorrections('soemthing', 2); // ['something', 'soothing'] ``` -------------------------------- ### Initialize and use file-based storage Source: https://naturalnode.github.io/natural/storage.html Demonstrates how to instantiate the StorageBackend with the FILE type and perform basic store and retrieve operations. ```javascript let storage = require('natural').StorageBackend; const object = { attr1: 'val1', attr2: 'val2' } storageBackend = new storage.StorageBackend(storage.STORAGE_TYPES.FILE) const key = await storageBackend.store(object) console.log(key) // UUID const result = await storageBackend.retrieve(key) console.log(result) // { attr1: 'val1', attr2: 'val2' } ``` -------------------------------- ### Initialize Elements and Samples Source: https://naturalnode.github.io/natural/maxent_classifier.html Create custom element instances and organize them into a sample object for training. ```javascript var MyElement = require('MyElementClass'); var Context = require('Context'); var Sample = require('Sample'); var x = new MyElementClass("x", new Context("0")); // A sample is created from an array of elements var sample = new Sample(); sample.addElement(x); ``` -------------------------------- ### Initialize with custom database path Source: https://naturalnode.github.io/natural/wordnet.html Configures the WordNet instance to use a manually downloaded database directory. ```javascript var wordnet = new natural.WordNet('/my/wordnet/dict'); ``` -------------------------------- ### Initialize and populate a Trie Source: https://naturalnode.github.io/natural/tries.html Demonstrates how to import the Trie class, instantiate it, and add words individually or in batches. ```javascript var natural = require('natural'); var Trie = natural.Trie; var trie = new Trie(); // Add one string at a time trie.addString("test"); // Or add many strings trie.addStrings(["string1", "string2", "string3"]); ``` -------------------------------- ### Importing from the main library Source: https://naturalnode.github.io/natural/typescript.html Shows how to import components directly from the main library entry point. ```typescript import { WordTokenizer } from '../lib/natural' ``` -------------------------------- ### Importing and using the WordTokenizer module Source: https://naturalnode.github.io/natural/typescript.html Demonstrates importing a specific tokenizer class and performing basic tokenization. ```typescript import { WordTokenizer } from '../lib/natural/tokenizers' let tokenizer = new WordTokenizer() console.log(tokenizer.tokenize('your dog has fleas.')) // [ 'your', 'dog', 'has', 'fleas' ] ``` -------------------------------- ### Initialize Brill POS Tagger Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Create a lexicon, rule set, and tagger instance using the natural library. ```javascript var natural = require("natural"); const language = "EN" const defaultCategory = 'N'; const defaultCategoryCapitalized = 'NNP'; var lexicon = new natural.Lexicon(language, defaultCategory, defaultCategoryCapitalized); var ruleSet = new natural.RuleSet('EN'); var tagger = new natural.BrillPOSTagger(lexicon, ruleSet); ``` -------------------------------- ### Create a Feature Set Source: https://naturalnode.github.io/natural/maxent_classifier.html Initialize a FeatureSet and add features to it. ```javascript var FeatureSet = require('FeatureSet'); var set = new FeatureSet(); set.addFeature(f, "f", ["0"]); ``` -------------------------------- ### Initialize and use SentimentAnalyzer Source: https://naturalnode.github.io/natural/sentiment_analysis.html Demonstrates how to instantiate the analyzer with a language, stemmer, and vocabulary type, then calculate sentiment for an array of strings. ```javascript var Analyzer = require('natural').SentimentAnalyzer; var stemmer = require('natural').PorterStemmer; var analyzer = new Analyzer("English", stemmer, "afinn"); // getSentiment expects an array of strings console.log(analyzer.getSentiment(["I", "like", "cherries"])); // 0.6666666666666666 ``` -------------------------------- ### Initialize natural library Source: https://naturalnode.github.io/natural/stemmers.html Import the natural library to access stemming functionality. ```javascript var natural = require('natural'); ``` -------------------------------- ### Load Sample from File Source: https://naturalnode.github.io/natural/maxent_classifier.html Load a sample from a file, requiring the original element class to reconstruct objects correctly. ```javascript sample.load('sample.json', MyElementClass, function(err, sample) { }); ``` -------------------------------- ### Create RuleSet Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Initialize a rule set for a specific language. ```javascript var ruleSet = new natural.RuleSet('EN'); ``` -------------------------------- ### Initialize and populate an EdgeWeightedDigraph Source: https://naturalnode.github.io/natural/edge_weighted_digraph.html Create a new instance of EdgeWeightedDigraph and add directed edges with specific weights. ```javascript var EdgeWeightedDigraph = natural.EdgeWeightedDigraph; var digraph = new EdgeWeightedDigraph(); digraph.add(5,4,0.35); digraph.add(5,1,0.32); digraph.add(1,3,0.29); digraph.add(6,2,0.40); digraph.add(3,6,0.52); digraph.add(6,4,0.93); ``` -------------------------------- ### Load Corpus and Build Lexicon Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Initializes the corpus from a Brown corpus source and generates a lexicon required for sentence tagging. ```javascript var natural = require(natural); const JSON_FLAG = 2; var brownCorpus = require('../lib/natural/brill_pos_tagger/lib/Corpus'); var corpus = new Corpus(brownCorpus, JSON_FLAG, natural.Sentence); var lexicon = corpus.buildLexicon(); ``` -------------------------------- ### Initialize a Classifier Source: https://naturalnode.github.io/natural/maxent_classifier.html Create a new classifier instance using classes, a feature set, and training samples. ```javascript var Classifier = require('Classifier'); var classifier = new Classifier(classes, featureSet, sample); ``` -------------------------------- ### Lookup word definitions Source: https://naturalnode.github.io/natural/wordnet.html Demonstrates how to initialize the WordNet object and retrieve definitions for a specific word. ```javascript var wordnet = new natural.WordNet(); wordnet.lookup('node', function(results) { results.forEach(function(result) { console.log('------------------------------------'); console.log(result.synsetOffset); console.log(result.pos); console.log(result.lemma); console.log(result.synonyms); console.log(result.pos); console.log(result.gloss); }); }); ``` -------------------------------- ### Initialize and use SentenceTokenizer Source: https://naturalnode.github.io/natural/Tokenizers.html Splits text into sentences based on punctuation, with support for custom abbreviations. ```javascript const abbreviations = ['i.e.', 'e.g.', 'Dr.'] tokenizer = new natural.SentenceTokenizer(abbreviations); console.log(tokenizer.tokenize("This is a sentence. This is another sentence")); // ["This is a sentence.", "This is another sentence."] ``` -------------------------------- ### Initialize a BayesClassifier Source: https://naturalnode.github.io/natural/bayesian_classifier.html Instantiate a new BayesClassifier using the natural library. ```javascript var natural = require('natural'); var classifier = new natural.BayesClassifier(); ``` -------------------------------- ### Save Sample to File Source: https://naturalnode.github.io/natural/maxent_classifier.html Persist a trained sample to a JSON file. ```javascript sample.save('sample.json', function(error, sample) { ... }); ``` -------------------------------- ### Import specific modules Source: https://naturalnode.github.io/natural/installation.html Import only the necessary components to keep your application lightweight. ```javascript const tokenizers = require('natural/lib/natural/tokenizers') const tokenizer = new tokenizers.SentenceTokenizer() ``` -------------------------------- ### Set NODE_PATH for unit tests Source: https://naturalnode.github.io/natural/development.html Required environment variable configuration for running the unit test suite. ```bash export NODE_PATH=. ``` -------------------------------- ### Monitor training progress Source: https://naturalnode.github.io/natural/bayesian_classifier.html Subscribe to the trainedWithDocument event to track training progress. ```javascript classifier.events.on('trainedWithDocument', function (obj) { console.log(obj); /* { * total: 23 // There are 23 total documents being trained against * index: 12 // The index/number of the document that's just been trained against * doc: {...} // The document that has just been indexed * } */ }); ``` -------------------------------- ### Initialize NGrams Source: https://naturalnode.github.io/natural/n_grams.html Access the NGrams module from the natural library. ```javascript var NGrams = natural.NGrams; ``` -------------------------------- ### SentimentAnalyzer Constructor Source: https://naturalnode.github.io/natural/sentiment_analysis.html Initializes a new instance of the SentimentAnalyzer. ```APIDOC ## new SentimentAnalyzer(language, stemmer, vocabulary) ### Description Initializes the sentiment analyzer with a specific language, stemmer, and vocabulary type. ### Parameters - **language** (string) - Required - The language code for the analyzer. - **stemmer** (object) - Required - A stemmer instance to increase coverage, or null. - **vocabulary** (string) - Required - The type of vocabulary to use: "afinn", "senticon", or "pattern". ``` -------------------------------- ### Find all prefixes on a path Source: https://naturalnode.github.io/natural/tries.html Returns all words in the Trie that match prefixes along the provided search string path. ```javascript trie.addString("tes"); trie.addString("test"); console.log(trie.findMatchesOnPath("tester")); // ['tes', 'test']; ``` -------------------------------- ### Initialize and use WordTokenizer Source: https://naturalnode.github.io/natural/Tokenizers.html Basic usage of the WordTokenizer to split a string into an array of words. ```javascript var natural = require('natural'); var tokenizer = new natural.WordTokenizer(); console.log(tokenizer.tokenize("your dog has fleas.")); // [ 'your', 'dog', 'has', 'fleas' ] ``` -------------------------------- ### Output Rule Set Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Prints the generated transformation rules in a readable format for verification or later use. ```javascript console.log(ruleSet.prettyPrint()); ``` -------------------------------- ### Configure case-sensitivity Source: https://naturalnode.github.io/natural/tries.html Demonstrates how to toggle case-insensitivity by passing false to the Trie constructor. ```javascript trie.contains("TEST"); // false var ciTrie = new Trie(false); ciTrie.addString("test"); ciTrie.contains("TEsT"); // true ``` -------------------------------- ### Train the Brill POS Tagger Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Executes the training process using the prepared corpus, templates, and lexicon. ```javascript var trainer = new natural.BrillPOSTrainer(/* optional threshold */); var ruleSet = trainer.train(corpus, templates, lexicon); ``` -------------------------------- ### Trie Constructor Source: https://naturalnode.github.io/natural/tries.html Initializes a new Trie instance. The constructor accepts a boolean parameter to toggle case-sensitivity. ```APIDOC ## new Trie(caseSensitive) ### Description Creates a new Trie instance. By default, the trie is case-sensitive. ### Parameters - **caseSensitive** (boolean) - Optional - Set to false to enable case-insensitive mode. ``` -------------------------------- ### Load documents from files Source: https://naturalnode.github.io/natural/tfidf.html Adds documents to the TfIdf instance directly from files on disk. ```javascript var tfidf = new TfIdf(); tfidf.addFileSync('data_files/one.txt'); tfidf.addFileSync('data_files/two.txt'); ``` -------------------------------- ### Train the classifier Source: https://naturalnode.github.io/natural/bayesian_classifier.html Add documents to the classifier and trigger the training process. ```javascript classifier.addDocument('i am long qqqq', 'buy'); classifier.addDocument('buy the q\'s', 'buy'); classifier.addDocument('short gold', 'sell'); classifier.addDocument('sell gold', 'sell'); classifier.train(); ``` -------------------------------- ### Test Brill POS Tagger Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Initializes the tester and tagger to evaluate performance against a test corpus. ```javascript var tester = new natural.BrillPOSTester(); var tagger = new natural.BrillPOSTagger(lexicon, ruleSet); var scores = tester.test(corpora[1], tagger); ``` -------------------------------- ### Calculate term weights in a corpus Source: https://naturalnode.github.io/natural/tfidf.html Demonstrates adding documents to a corpus and calculating the tf-idf weight for specific terms across all documents. ```javascript var natural = require('natural'); var TfIdf = natural.TfIdf; var tfidf = new TfIdf(); tfidf.addDocument('this document is about node.'); tfidf.addDocument('this document is about ruby.'); tfidf.addDocument('this document is about ruby and node.'); tfidf.addDocument('this document is about node. it has node examples'); console.log('node --------------------------------'); tfidf.tfidfs('node', function(i, measure) { console.log('document #' + i + ' is ' + measure); }); console.log('ruby --------------------------------'); tfidf.tfidfs('ruby', function(i, measure) { console.log('document #' + i + ' is ' + measure); }); ``` -------------------------------- ### EdgeWeightedDigraph Methods Source: https://naturalnode.github.io/natural/edge_weighted_digraph.html Methods available for interacting with an EdgeWeightedDigraph instance, including adding edges and retrieving graph statistics. ```APIDOC ## EdgeWeightedDigraph.add(from, to, weight) ### Description Adds a directed edge with a specified weight to the graph. ### Parameters - **from** (number) - The starting vertex index. - **to** (number) - The ending vertex index. - **weight** (number) - The weight associated with the edge. ## EdgeWeightedDigraph.v() ### Description Returns the total number of vertices in the graph. ## EdgeWeightedDigraph.e() ### Description Returns the total number of edges in the graph. ``` -------------------------------- ### Retrieve distance to vertex Source: https://naturalnode.github.io/natural/shortest_path_tree.html Use getDistTo to return the shortest distance from the source to the specified vertex. ```javascript console.log(spt.getDistTo(4)); ``` -------------------------------- ### Apply transformation rules algorithm Source: https://naturalnode.github.io/natural/brill_pos_tagger.html The core algorithm iterates through the sentence and applies all defined rules to each word position. ```javascript Brill_POS_Tagger.prototype.applyRules = function(sentence) { for (var i = 0, size = sentence.taggedWords.length; i < size; i++) { this.ruleSet.getRules().forEach(function(rule) { rule.apply(sentence, i); }); } return sentence; }; ``` -------------------------------- ### Classifier.load(filename, callback) Source: https://naturalnode.github.io/natural/maxent_classifier.html Loads a classifier state from a JSON file. ```APIDOC ## Classifier.load(filename, callback) ### Description Loads a previously saved classifier model from a file. ### Parameters - **filename** (string) - The path to the file. - **callback** (function) - Callback function receiving error and classifier object. ``` -------------------------------- ### Use DoubleMetaphone Source: https://naturalnode.github.io/natural/phonetics.html Access the two encodings returned by the experimental DoubleMetaphone algorithm. ```javascript const natural = require('natural'); const dm = natural.DoubleMetaphone(); const encodings = dm.process('Matrix'); console.log(encodings[0]); console.log(encodings[1]); ``` -------------------------------- ### Classifier.train(maxIterations, minImprovement) Source: https://naturalnode.github.io/natural/maxent_classifier.html Trains the classifier using Generalised Iterative Scaling. ```APIDOC ## Classifier.train(maxIterations, minImprovement) ### Description Trains the classifier until the maximum iterations are reached or the improvement in likelihood falls below the specified threshold. ### Parameters - **maxIterations** (number) - The maximum number of iterations for training. - **minImprovement** (number) - The minimum improvement threshold for likelihood. ``` -------------------------------- ### Train the Classifier Source: https://naturalnode.github.io/natural/maxent_classifier.html Train the classifier using specified iteration limits and improvement thresholds. ```javascript var maxIterations = 100; var minImprovement = .01; var p = classifier.train(maxIterations, minImprovement); ``` -------------------------------- ### Define a Feature Source: https://naturalnode.github.io/natural/maxent_classifier.html Create a new feature by defining a function that maps elements to 0 or 1, along with a name and parameters. ```javascript var Feature = require('Feature'); function f(x) { if (x.b === "0") { return 1; } return 0; } var feature = new Feature(f, name, parameters); ``` -------------------------------- ### Define Rule Templates Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Creates a collection of rule templates used by the learning algorithm to generate transformation rules. ```javascript var templateNames = [ "NEXT-TAG", "NEXT-WORD-IS-CAP", "PREV-1-OR-2-OR-3-TAG", "...", ]; var templates = templateNames.map(function(name) { return new natural.RuleTemplate(name); }); ``` -------------------------------- ### Add document as tokens Source: https://naturalnode.github.io/natural/bayesian_classifier.html Train the classifier using an array of tokens instead of a string. ```javascript classifier.addDocument(['sell', 'gold'], 'sell'); ``` -------------------------------- ### Initialize Spellcheck with a Corpus Source: https://naturalnode.github.io/natural/spellcheck.html Create a new Spellcheck instance using an array of tokens from a corpus or a dictionary list. ```javascript var corpus = ['something', 'soothing']; var spellcheck = new natural.Spellcheck(corpus); ``` -------------------------------- ### addString / addStrings Source: https://naturalnode.github.io/natural/tries.html Methods to populate the Trie with one or multiple strings. ```APIDOC ## addString(string) ### Description Adds a single string to the Trie. ## addStrings(strings) ### Description Adds an array of strings to the Trie. ``` -------------------------------- ### getDistTo(vertex) Source: https://naturalnode.github.io/natural/shortest_path_tree.html Retrieves the shortest distance from the source vertex to the specified target vertex. ```APIDOC ## getDistTo(vertex) ### Description Returns the shortest distance from the source vertex to the provided vertex. ### Parameters - **vertex** (number) - Required - The target vertex. ### Returns - **number** - The distance to the vertex. ``` -------------------------------- ### pathTo(vertex) Source: https://naturalnode.github.io/natural/shortest_path_tree.html Retrieves the sequence of vertices representing the shortest path to the specified target vertex. ```APIDOC ## pathTo(vertex) ### Description Returns the shortest path from the source vertex to the provided vertex as an array of vertices. ### Parameters - **vertex** (number) - Required - The target vertex. ### Returns - **Array** - An array representing the path. ``` -------------------------------- ### tokenizer.tokenize(text) Source: https://naturalnode.github.io/natural/Tokenizers.html The tokenize method is common to all tokenizer classes in the Natural library. It accepts a string of text and returns an array of tokens based on the specific rules of the instantiated tokenizer. ```APIDOC ## Method: tokenize(text) ### Description Splits the provided input string into an array of tokens based on the specific logic of the tokenizer instance. ### Parameters - **text** (string) - Required - The input string to be tokenized. ### Returns - **Array** - An array containing the resulting tokens. ### Example ```javascript var natural = require('natural'); var tokenizer = new natural.WordTokenizer(); var tokens = tokenizer.tokenize("your dog has fleas."); // Returns: ['your', 'dog', 'has', 'fleas'] ``` ``` -------------------------------- ### Compare Phonetic Similarity Source: https://naturalnode.github.io/natural/phonetics.html Check if two words sound alike using the Metaphone or SoundEx algorithms. ```javascript if(metaphone.compare(wordA, wordB)) console.log('they sound alike!'); ``` ```javascript if(soundEx.compare(wordA, wordB)) console.log('they sound alike!'); ``` -------------------------------- ### Implement a parameter values function Source: https://naturalnode.github.io/natural/brill_pos_tagger.html A values function returns an array of possible parameter values for a given location in a tagged sentence. ```javascript function nextTagParameterValues(sentence, i) { if (i < sentence.length - 1) { return [sentence[i + 1].tag]; } else { return []; } } ``` -------------------------------- ### Use custom stemmer Source: https://naturalnode.github.io/natural/bayesian_classifier.html Initialize a classifier with a specific stemmer for non-English languages. ```javascript const PorterStemmerRu = require('./node_modules/natural/lib/natural/stemmers/porter_stemmer_ru'); var classifier = new natural.BayesClassifier(PorterStemmerRu); ``` -------------------------------- ### Use pre-tokenized input Source: https://naturalnode.github.io/natural/tfidf.html Passes arrays of tokens directly to bypass default string preprocessing. ```javascript var natural = require('natural'); var TfIdf = natural.TfIdf; var tfidf = new TfIdf(); tfidf.addDocument(['document', 'about', 'node']); tfidf.addDocument(['document', 'about', 'ruby']); tfidf.addDocument(['document', 'about', 'ruby', 'node']); tfidf.addDocument(['document', 'about', 'node', 'node', 'examples']); tfidf.tfidfs(['node', 'ruby'], function(i, measure) { console.log('document #' + i + ' is ' + measure); }); ``` -------------------------------- ### Retrieve shortest path to vertex Source: https://naturalnode.github.io/natural/shortest_path_tree.html Use pathTo to return an array representing the sequence of vertices in the shortest path. ```javascript console.log(spt.pathTo(4)); ``` ```javascript [5, 4] ``` -------------------------------- ### Generate Features with Closures Source: https://naturalnode.github.io/natural/maxent_classifier.html Use closures to generate feature functions dynamically within a loop. ```javascript var FeatureSet = require('FeatureSet'); var Feature = require('Feature'); var listOfTags = ['NN', 'DET', 'PREP', 'ADJ']; var featureSet = new FeatureSet(); listofTags.forEach(function(tag) { function isTag(x) { if (x.b.data.tag === tag) { return 1 } return 0; } featureSet.addFeature(new Feature(isTag, "isTag", [tag])); }); ``` -------------------------------- ### natural.WordNet.get Source: https://naturalnode.github.io/natural/wordnet.html Retrieves a definition directly using a synset offset and part of speech. ```APIDOC ## natural.WordNet.get(synsetOffset, pos, callback) ### Description Retrieves a specific definition using a synset offset and part of speech identifier. ### Parameters - **synsetOffset** (number) - Required - The unique offset for the synset. - **pos** (string) - Required - The part of speech (e.g., 'n' for noun). - **callback** (function) - Required - A function that receives the result object containing lemma, pos, gloss, and synonyms. ``` -------------------------------- ### Initialize CountInflector Source: https://naturalnode.github.io/natural/inflectors.html Access the CountInflector utility to generate ordinal numbers. ```javascript var countInflector = natural.CountInflector; ``` -------------------------------- ### Lexicon JSON Structure Source: https://naturalnode.github.io/natural/brill_pos_tagger.html The expected format for the lexicon JSON file. ```json { "word1": ["cat1"], "word2": ["cat2", "cat3"], ... } ``` -------------------------------- ### NGrams.ngrams(sequence, n, startSymbol, endSymbol) Source: https://naturalnode.github.io/natural/n_grams.html Generates n-grams of a specified length, with optional padding. ```APIDOC ## NGrams.ngrams(sequence, n, startSymbol, endSymbol) ### Description Generates n-grams of length n. Supports optional start and end padding symbols. ### Parameters - **sequence** (string|Array) - Required - The input text or array of tokens. - **n** (number) - Required - The length of the n-grams. - **startSymbol** (string|null) - Optional - Symbol to use for left padding. - **endSymbol** (string|null) - Optional - Symbol to use for right padding. ``` -------------------------------- ### natural.BrillPOSTagger.tag(sentence) Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Tags an array of tokens with their respective part-of-speech categories using the initialized lexicon and ruleset. ```APIDOC ## natural.BrillPOSTagger.tag(sentence) ### Description Tags an array of tokens (a sentence) based on the transformation rules and lexicon provided during initialization. ### Parameters - **sentence** (Array) - Required - An array of strings representing the tokens to be tagged. ### Returns - **Sentence** (Object) - An object containing a `taggedWords` property, which is an array of objects with `token` and `tag` fields. ``` -------------------------------- ### Pluralize and Singularize Nouns Source: https://naturalnode.github.io/natural/inflectors.html Use the NounInflector instance to convert words between singular and plural forms. ```javascript console.log(nounInflector.pluralize('radius')); ``` ```javascript console.log(nounInflector.singularize('beers')); ``` -------------------------------- ### Persist and recall classifier Source: https://naturalnode.github.io/natural/bayesian_classifier.html Save a trained classifier to a file and load it back for reuse. ```javascript classifier.save('classifier.json', function(err, classifier) { // the classifier is saved to the classifier.json file! }); ``` ```javascript natural.BayesClassifier.load('classifier.json', null, function(err, classifier) { console.log(classifier.classify('long SUNW')); console.log(classifier.classify('short SUNW')); }); ``` -------------------------------- ### Tag a Sentence Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Use the tagger to process an array of tokens and output the tagged results. ```javascript var tagger = new natural.BrillPOSTagger(lexicon, ruleSet); var sentence = ["I", "see", "the", "man", "with", "the", "telescope"]; console.log(tagger.tag(sentence)); ``` -------------------------------- ### Define a rule template entry Source: https://naturalnode.github.io/natural/brill_pos_tagger.html A template entry maps a predicate name to its function, window, parameter count, and value-generating functions. ```javascript "NEXT-TAG": { // maps to the predicate function "function": next_tag_is, // Minimum required window before or after current position to be a relevant predicate "window": [0, 1], // The number of parameters the predicate takes "nrParameters": 1, // Function that returns relevant values for parameter 1 "parameter1Values": nextTagParameterValues } ``` -------------------------------- ### List terms by importance Source: https://naturalnode.github.io/natural/tfidf.html Retrieves all terms in a document sorted by their calculated tf-idf weight. ```javascript tfidf.listTerms(0 /*document index*/).forEach(function(item) { console.log(item.term + ': ' + item.tfidf); }); ``` -------------------------------- ### Tokenize and Phoneticize Strings Source: https://naturalnode.github.io/natural/phonetics.html Convert a full text string into an array of phonetic representations. ```javascript console.log('phonetics rock'.tokenizeAndPhoneticize()); ``` -------------------------------- ### Feature(f, name, parameters) Source: https://naturalnode.github.io/natural/maxent_classifier.html Creates a new feature function that maps elements to zero or one. ```APIDOC ## Feature(f, name, parameters) ### Description Defines a feature function that maps elements to 0 or 1. The name and parameters uniquely identify the feature. ### Parameters - **f** (function) - The feature function logic. - **name** (string) - The name of the feature function. - **parameters** (array) - An array of strings for the parameters of the feature function. ``` -------------------------------- ### Classifier.save(filename, callback) Source: https://naturalnode.github.io/natural/maxent_classifier.html Saves the classifier state to a JSON file. ```APIDOC ## Classifier.save(filename, callback) ### Description Persists the classifier's probability distribution to a file. ### Parameters - **filename** (string) - The path to the file. - **callback** (function) - Callback function receiving error and classifier object. ``` -------------------------------- ### Output Test Scores Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Displays the accuracy percentages for the lexicon-only tagging and the rule-based tagging. ```javascript console.log("Test score lexicon " + scores[0] + "%"); console.log("Test score after applying rules " + scores[1] + "%"); ``` -------------------------------- ### findMatchesOnPath Source: https://naturalnode.github.io/natural/tries.html Returns all words in the Trie that match prefixes along the path of the provided search string. ```APIDOC ## findMatchesOnPath(string) ### Description Returns an array of all prefix matches found along the path of the search string. ``` -------------------------------- ### Process Phonetic Encodings Source: https://naturalnode.github.io/natural/phonetics.html Retrieve the raw phonetic representation of a word. ```javascript console.log(metaphone.process('phonetics')); ``` -------------------------------- ### natural.WordNet.lookup Source: https://naturalnode.github.io/natural/wordnet.html Looks up definitions for a given word. ```APIDOC ## natural.WordNet.lookup(word, callback) ### Description Looks up definitions for a specified word and returns an array of results via a callback. ### Parameters - **word** (string) - Required - The word to look up. - **callback** (function) - Required - A function that receives an array of results containing synsetOffset, pos, lemma, synonyms, and gloss. ``` -------------------------------- ### Retrieve classification probabilities Source: https://naturalnode.github.io/natural/maxent_classifier.html Iterates through all classes to log their respective labels and probability scores. ```javascript var classifications = classifier.getClassifications(context); classifications.forEach(function(classPlusProbability) { console.log('Class ' + classPlusProbability.label + ' has score ' + classPlusProbability.value); }); ``` -------------------------------- ### PresentVerbInflector Source: https://naturalnode.github.io/natural/inflectors.html The PresentVerbInflector class provides methods to pluralize and singularize present tense verbs. ```APIDOC ## PresentVerbInflector ### Description Provides methods to convert present tense verbs between singular and plural forms. Note: This feature is experimental. ### Methods - **pluralize(word)**: Returns the plural form of the provided verb. - **singularize(word)**: Returns the singular form of the provided verb. ``` -------------------------------- ### SentenceTokenizer(abbreviations) Source: https://naturalnode.github.io/natural/Tokenizers.html The SentenceTokenizer splits text into sentences based on punctuation marks (period, question mark, exclamation mark, and ellipsis) while handling abbreviations and decimal points. ```APIDOC ## Constructor: new natural.SentenceTokenizer(abbreviations) ### Description Initializes a new sentence tokenizer that respects specific abbreviations to avoid incorrect sentence splitting. ### Parameters - **abbreviations** (Array) - Optional - An array of strings representing abbreviations that should not trigger a sentence break (e.g., ['i.e.', 'Dr.']). ### Example ```javascript const abbreviations = ['i.e.', 'e.g.', 'Dr.'] const tokenizer = new natural.SentenceTokenizer(abbreviations); const sentences = tokenizer.tokenize("This is a sentence. This is another sentence"); // Returns: ["This is a sentence.", "This is another sentence."] ``` ``` -------------------------------- ### pathTo(vertex) Source: https://naturalnode.github.io/natural/longest_path_tree.html Returns the sequence of vertices representing the longest path to the specified target vertex. ```APIDOC ## Method: pathTo(vertex) ### Description Returns the longest path to the specified vertex as an array of vertices. ### Parameters - **vertex** (number) - Required - The target vertex. ### Returns - **Array** - An array representing the path from the start vertex to the target vertex. ``` -------------------------------- ### Pluralize and Singularize Present Tense Verbs Source: https://naturalnode.github.io/natural/inflectors.html Convert present tense verbs between singular and plural forms using the verb inflector. ```javascript console.log(verbInflector.singularize('become')); ``` ```javascript console.log(verbInflector.pluralize('becomes')); ``` -------------------------------- ### findPrefix Source: https://naturalnode.github.io/natural/tries.html Finds the longest prefix of the input string that exists as a word in the Trie. ```APIDOC ## findPrefix(string) ### Description Returns an array containing the longest matching prefix found in the Trie and the remaining unmatched portion of the string. ``` -------------------------------- ### Implement a predicate function Source: https://naturalnode.github.io/natural/brill_pos_tagger.html A predicate function evaluates a condition based on the sentence, current position, and a parameter. ```javascript function next_tag_is(sentence, i, parameter) { if (i < sentence.taggedWords.length - 1) { return(sentence.taggedWords[i + 1][1] === parameter); } else { return(false); } } ``` -------------------------------- ### natural.LevenshteinDistanceSearch(source, target) Source: https://naturalnode.github.io/natural/string_distance.html Performs an approximate string matching search using the Levenshtein algorithm. ```APIDOC ## natural.LevenshteinDistanceSearch(source, target) ### Description Searches for the best matching substring within a target string. ### Parameters - **source** (string) - Required - The string to search for. - **target** (string) - Required - The string to search within. ``` -------------------------------- ### Generate Bigrams Source: https://naturalnode.github.io/natural/n_grams.html Create bigrams from a string or an array of tokens. ```javascript console.log(NGrams.bigrams('some words here')); console.log(NGrams.bigrams(['some', 'words', 'here'])); ``` -------------------------------- ### Classify context Source: https://naturalnode.github.io/natural/maxent_classifier.html Returns the single highest scoring class for a given context. ```javascript var class = classifier.classify(context); console.log(class); ``` -------------------------------- ### Find longest prefix Source: https://naturalnode.github.io/natural/tries.html Uses findPrefix to identify the longest matching word prefix and the remaining unmatched string segment. ```javascript console.log(trie.findPrefix("tester")); // ['test', 'er'] console.log(trie.findPrefix("string4")); // [null, '4'] console.log(trie.findPrefix("string3")); // ['string3', ''] ``` -------------------------------- ### Perform Approximate String Matching Source: https://naturalnode.github.io/natural/string_distance.html Uses the Levenshtein algorithm to find the best matching substring within a target string. ```javascript var natural = require('natural'); var source = 'The RainCoat BookStore'; var target = 'All the best books are here at the Rain Coats Book Store'; console.log(natural.LevenshteinDistanceSearch(source, target)); ``` -------------------------------- ### Generate Arbitrary N-Grams Source: https://naturalnode.github.io/natural/n_grams.html Create n-grams of a specified length from a string or an array of tokens. ```javascript console.log(NGrams.ngrams('some other words here for you', 4)); console.log(NGrams.ngrams(['some', 'other', 'words', 'here', 'for', 'you'], 4)); ``` -------------------------------- ### Serialize and deserialize classifier Source: https://naturalnode.github.io/natural/bayesian_classifier.html Convert a classifier to a JSON string and restore it later. ```javascript var classifier = new natural.BayesClassifier(); classifier.addDocument(['sell', 'gold'], 'sell'); classifier.addDocument(['buy', 'silver'], 'buy'); // serialize var raw = JSON.stringify(classifier); // deserialize var restoredClassifier = natural.BayesClassifier.restore(JSON.parse(raw)); console.log(restoredClassifier.classify('i should sell that')); ``` -------------------------------- ### Generate Trigrams Source: https://naturalnode.github.io/natural/n_grams.html Create trigrams from a string or an array of tokens. ```javascript console.log(NGrams.trigrams('some other words here')); console.log(NGrams.trigrams(['some', 'other', 'words', 'here'])); ``` -------------------------------- ### NGrams.bigrams(sequence) Source: https://naturalnode.github.io/natural/n_grams.html Generates bigrams from a string or an array of tokens. ```APIDOC ## NGrams.bigrams(sequence) ### Description Generates bigrams from the provided input. If a string is provided, it is tokenized automatically. ### Parameters - **sequence** (string|Array) - Required - The input text or array of tokens to process. ``` -------------------------------- ### Measure multiple terms Source: https://naturalnode.github.io/natural/tfidf.html Calculates the combined weight of multiple terms across documents. ```javascript var natural = require('natural'); var TfIdf = natural.TfIdf; var tfidf = new TfIdf(); tfidf.addDocument('this document is about node.'); tfidf.addDocument('this document is about ruby.'); tfidf.addDocument('this document is about ruby and node.'); tfidf.tfidfs('node ruby', function(i, measure) { console.log('document #' + i + ' is ' + measure); }); ``` -------------------------------- ### Serialize and deserialize TfIdf instance Source: https://naturalnode.github.io/natural/tfidf.html Converts a TfIdf instance to JSON for storage and restores it later. ```javascript var tfidf = new TfIdf(); tfidf.addDocument('document one', 'un'); tfidf.addDocument('document Two', 'deux'); var s = JSON.stringify(tfidf); // save "s" to disk, database or otherwise // assuming you pulled "s" back out of storage. var tfidf = new TfIdf(JSON.parse(s)); ``` -------------------------------- ### FeatureSet.addFeature(f, name, parameters) Source: https://naturalnode.github.io/natural/maxent_classifier.html Adds a feature to a feature set. ```APIDOC ## FeatureSet.addFeature(f, name, parameters) ### Description Adds a new feature to the feature set, ensuring uniqueness based on name and parameters. ### Parameters - **f** (function) - The feature function. - **name** (string) - The name of the feature. - **parameters** (array) - The parameters for the feature. ``` -------------------------------- ### contains Source: https://naturalnode.github.io/natural/tries.html Checks if a specific string exists as a word in the Trie. ```APIDOC ## contains(string) ### Description Returns true if the provided string is marked as a word in the Trie, otherwise returns false. ``` -------------------------------- ### Measure term weight in a specific document Source: https://naturalnode.github.io/natural/tfidf.html Retrieves the tf-idf score for a specific term in a single document by index. ```javascript console.log(tfidf.tfidf('node', 0)); console.log(tfidf.tfidf('node', 1)); ``` -------------------------------- ### Classify text input Source: https://naturalnode.github.io/natural/bayesian_classifier.html Predict the label for a given input string. ```javascript console.log(classifier.classify('i am short silver')); ``` ```javascript console.log(classifier.classify('i am long copper')); ``` -------------------------------- ### natural.LevenshteinDistance(s1, s2, options) Source: https://naturalnode.github.io/natural/string_distance.html Calculates the Levenshtein distance between two strings with configurable edit costs. ```APIDOC ## natural.LevenshteinDistance(s1, s2, options) ### Description Calculates the Levenshtein distance between two strings. ### Parameters - **s1** (string) - Required - The first string. - **s2** (string) - Required - The second string. - **options** (object) - Optional - Configuration for edit costs (insertion_cost, deletion_cost, substitution_cost). ``` -------------------------------- ### getSentiment(tokens) Source: https://naturalnode.github.io/natural/sentiment_analysis.html Calculates the sentiment score for an array of string tokens. ```APIDOC ## getSentiment(tokens) ### Description Calculates the sentiment of a piece of text by summing the polarity of each word and normalizing with the length of the sentence. ### Parameters - **tokens** (Array) - Required - An array of strings representing the text to analyze. ### Response - **score** (number) - The calculated sentiment score. ``` -------------------------------- ### Process Phonetics with Length Limit Source: https://naturalnode.github.io/natural/phonetics.html Retrieve phonetic representation with a specified maximum code length. ```javascript console.log(metaphone.process('phonetics', 3)); ``` -------------------------------- ### stem(word) Source: https://naturalnode.github.io/natural/stemmers.html Stems a single word using the selected stemmer class. ```APIDOC ## stem(word) ### Description Stems a single word using the specified stemmer instance. ### Parameters - **word** (string) - Required - The word to be stemmed. ### Usage Example ```javascript var natural = require('natural'); console.log(natural.PorterStemmer.stem("words")); // Returns "word" console.log(natural.PorterStemmerRu.stem("падший")); console.log(natural.PorterStemmerEs.stem("jugaría")); ``` ``` -------------------------------- ### Calculate Dice Coefficient Source: https://naturalnode.github.io/natural/string_distance.html Computes the Dice coefficient for string similarity. ```javascript var natural = require('natural'); console.log(natural.DiceCoefficient('thing', 'thing')); console.log(natural.DiceCoefficient('not', 'same')); ``` -------------------------------- ### Specify transformation rules in JSON Source: https://naturalnode.github.io/natural/brill_pos_tagger.html Define transformation rules using a JSON structure where each rule string follows the format OLD_CAT NEW_CAT PREDICATE PARAMETER. ```json { "rules": [ "OLD_CAT NEW_CAT PREDICATE PARAMETER", ... ] } ``` -------------------------------- ### NGrams.trigrams(sequence) Source: https://naturalnode.github.io/natural/n_grams.html Generates trigrams from a string or an array of tokens. ```APIDOC ## NGrams.trigrams(sequence) ### Description Generates trigrams from the provided input. If a string is provided, it is tokenized automatically. ### Parameters - **sequence** (string|Array) - Required - The input text or array of tokens to process. ``` -------------------------------- ### Generate Chinese N-Grams Source: https://naturalnode.github.io/natural/n_grams.html Use NGramsZH for Chinese language processing with the same API as standard NGrams. ```javascript var NGramsZH = natural.NGramsZH; console.log(NGramsZH.bigrams('中文测试')); console.log(NGramsZH.bigrams(['中', '文', '测', '试'])); console.log(NGramsZH.trigrams('中文测试')); console.log(NGramsZH.trigrams(['中', '文', '测', '试'])); console.log(NGramsZH.ngrams('一个中文测试', 4)); console.log(NGramsZH.ngrams(['一', '个', '中', '文', '测', '试'], 4)); ``` -------------------------------- ### Generate Ordinal Numbers Source: https://naturalnode.github.io/natural/inflectors.html Convert integers into their ordinal string representations. ```javascript console.log(countInflector.nth(1)); ``` ```javascript console.log(countInflector.nth(111)); ``` -------------------------------- ### natural.JaroWinklerDistance(s1, s2, distance, ignoreCase) Source: https://naturalnode.github.io/natural/string_distance.html Calculates the Jaro-Winkler distance, returning a value between 0 and 1 representing string similarity. ```APIDOC ## natural.JaroWinklerDistance(s1, s2, distance, ignoreCase) ### Description Returns a number between 0 and 1 indicating how closely two strings match. ### Parameters - **s1** (string) - Required - The first string. - **s2** (string) - Required - The second string. - **distance** (number) - Optional - Known distance between strings. - **ignoreCase** (boolean) - Optional - Whether to ignore case. ``` -------------------------------- ### Stem English words with PorterStemmer Source: https://naturalnode.github.io/natural/stemmers.html Use the PorterStemmer to reduce an English word to its stem. ```javascript console.log(natural.PorterStemmer.stem("words")); // stem a single word ```