### Install wink-bm25-text-search Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Use npm to install the package as a dependency. ```sh npm install wink-bm25-text-search --save ``` -------------------------------- ### Define document structure for addDoc Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Example JSON structure for documents added to the BM25 model. ```json { title: 'Barack Obama', body: 'Barack Hussein Obama II born August 4, 1961 is an American politician...' tags: 'democratic nobel peace prize columbia michelle...' } ``` -------------------------------- ### Initialize and Use BM25 Search Engine Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Demonstrates configuring the engine, defining preprocessing tasks, adding documents, and performing a search query. ```javascript // Load wink-bm25-text-search var bm25 = require( 'wink-bm25-text-search' ); // Create search engine's instance var engine = bm25(); // Load sample data (load any other JSON data instead of sample) var docs = require( 'wink-bm25-text-search/sample-data/demo-data-for-wink-bm25.json' ); // Load wink nlp and its model const winkNLP = require( 'wink-nlp' ); // Use web model const model = require( 'wink-eng-lite-web-model' ); const nlp = winkNLP( model ); const its = nlp.its; const prepTask = function ( text ) { const tokens = []; nlp.readDoc(text) .tokens() // Use only words ignoring punctuations etc and from them remove stop words .filter( (t) => ( t.out(its.type) === 'word' && !t.out(its.stopWordFlag) ) ) // Handle negation and extract stem of the word .each( (t) => tokens.push( (t.out(its.negationFlag)) ? '!' + t.out(its.stem) : t.out(its.stem) ) ); return tokens; }; // Contains search query. var query; // Step I: Define config // Only field weights are required in this example. engine.defineConfig( { fldWeights: { title: 1, body: 2 } } ); // Step II: Define PrepTasks pipe. // Set up 'default' preparatory tasks i.e. for everything else engine.definePrepTasks( [ prepTask ] ); // Step III: Add Docs // Add documents now... docs.forEach( function ( doc, i ) { // Note, 'i' becomes the unique id for 'doc' engine.addDoc( doc, i ); } ); // Step IV: Consolidate // Consolidate before searching engine.consolidate(); // All set, start searching! query = 'not studied law'; // `results` is an array of [ doc-id, score ], sorted by score var results = engine.search( query ); // Print number of results. console.log( '%d entries found.', results.length ); // -> 1 entries found. // results[ 0 ][ 0 ] i.e. the top result is: console.log( docs[ results[ 0 ][ 0 ] ].body ); // -> George Walker Bush (born July 6, 1946) is an... // -> ... He never studied Law... // Whereas if you search for `law` then multiple entries will be // found except the above entry! ``` -------------------------------- ### Accessor Methods Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md A collection of read-only methods to retrieve internal state and configuration from the BM25 search instance. ```APIDOC ## Accessor Methods ### getDocs() Returns the Term Frequencies and length of each document. ### getTokens() Returns the `token: index` mapping. ### getIDF() Returns IDF for each token. Tokens are referenced via their numerical index, which is accessed via `getTokens()`. ### getConfig() Returns the BM25F Configuration as set up by `defineConfig()`. ### getTotalCorpusLength() Returns the total number of tokens across all documents added. ### getTotalDocs() Returns total documents added. ``` -------------------------------- ### definePrepTasks( tasks [, field ] ) Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Defines the text preparation tasks to transform raw text into tokens. ```APIDOC ## definePrepTasks( tasks [, field ] ) ### Description Defines the text preparation tasks to transform raw incoming text into an array of tokens required during `addDoc()` and `search()` operations. ### Parameters - **tasks** (array) - Required - An array of functions where the first accepts a string and the last returns an array of tokens. - **field** (string) - Optional - The document field for which the tasks are defined. ``` -------------------------------- ### defineConfig( config ) Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Defines the configuration for the BM25 model, including field weights, BM25 parameters, and fields to retain. ```APIDOC ## defineConfig( config ) ### Description Defines the configuration from the `config` object. This object defines field weights, BM25 parameters (k1, b, k), and fields whose original values must be retained. ### Parameters - **config** (object) - Required - Object containing `fldWeights` (mandatory), `bm25Params` (optional), and `ovFldNames` (optional). ``` -------------------------------- ### exportJSON() / importJSON( json ) Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Methods for exporting and importing the BM25 model as JSON. ```APIDOC ## exportJSON() Exports the BM25 model as JSON text. ## importJSON( json ) Imports an existing JSON BM25 model for searching. - **json** (string) - Required - The JSON string representing the model. ``` -------------------------------- ### consolidate( fp ) Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Consolidates the BM25 model for all added documents. ```APIDOC ## consolidate( fp ) ### Description Consolidates the BM25 model. This is a prerequisite for `search()` and prevents further document additions. ### Parameters - **fp** (number) - Optional - The precision at which term frequency values are stored (default: 4). ``` -------------------------------- ### reset() Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Resets the BM25 model. ```APIDOC ## reset() Completely resets the BM25 model by re-initializing all variables, except the preparatory tasks. ``` -------------------------------- ### search( text [, limit, filter, params ] ) Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Searches for text within the BM25 model. ```APIDOC ## search( text [, limit, filter, params ] ) ### Description Searches for the `text` and returns results sorted by relevance score. ### Parameters - **text** (string) - Required - The search query. - **limit** (number) - Optional - Maximum number of results (default: 10). - **filter** (function) - Optional - A function to filter results. - **params** (any) - Optional - Parameters passed to the filter function. ``` -------------------------------- ### addDoc( doc, uniqueId ) Source: https://github.com/winkjs/wink-bm25-text-search/blob/master/README.md Adds a document to the BM25 model. ```APIDOC ## addDoc( doc, uniqueId ) ### Description Adds the `doc` with the `uniqueId` to the BM25 model. Requires `defineConfig()` and `definePrepTasks()` to be called first. ### Parameters - **doc** (object) - Required - The structured JSON document to add. - **uniqueId** (string/number) - Required - A unique identifier for the document. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.