### Main Method Execution Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.html Orchestrates the execution of both the search and drill-down examples, printing the results to the console. Includes setup and output formatting. ```java public static void main(String[] args) throws Exception { System.out.println("Facet counting example:"); System.out.println("-----------------------"); SimpleSortedSetFacetsExample example = new SimpleSortedSetFacetsExample(); List results = example.runSearch(); System.out.println("Author: " + results.get(0)); System.out.println("Publish Year: " + results.get(1)); System.out.println("\n"); System.out.println("Facet drill-down example (Publish Year/2010):"); System.out.println("---------------------------------------------"); System.out.println("Author: " + example.runDrillDown()); } ``` -------------------------------- ### Main Method for Example Execution Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/DistanceFacetsExample.html Entry point for the example. It initializes the index, runs the search and drill-down examples, and prints the results. ```Java public static void main(String[] args) throws Exception { DistanceFacetsExample example = new DistanceFacetsExample(); example.index(); System.out.println("Distance facet counting example:"); System.out.println("-----------------------"); System.out.println(example.search()); System.out.println("Distance facet drill-down example (field/< 2 km):"); System.out.println("---------------------------------------------"); TopDocs hits = example.drillDown(example.TWO_KM); System.out.println(hits.totalHits + " totalHits"); example.close(); } ``` -------------------------------- ### Main Method for Example Execution Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/ExpressionAggregationFacetsExample.html The entry point for the example. It initializes the example, runs the search, and prints the resulting facet counts to the console. ```Java public static void main(String[] args) throws Exception { System.out.println("Facet counting example:"); System.out.println("-----------------------"); FacetResult result = new ExpressionAggregationFacetsExample().runSearch(); System.out.println(result); } ``` -------------------------------- ### Main Method for Example Execution Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/RangeFacetsExample.html Orchestrates the execution of the range facets example, including indexing, searching, and drill-down operations, then prints the results. ```java public static void main(String[] args) throws Exception { RangeFacetsExample example = new RangeFacetsExample(); example.index(); System.out.println("Facet counting example:"); System.out.println("-----------------------"); System.out.println(example.search()); System.out.println("\n"); System.out.println("Facet counting example:"); System.out.println("-----------------------"); System.out.println(example.searchTopChildren()); System.out.println("\n"); System.out.println("Facet drill-down example (timestamp/Past six hours):"); System.out.println("---------------------------------------------"); TopDocs hits = example.drillDown(example.PAST_SIX_HOURS); System.out.println(hits.totalHits + " totalHits"); System.out.println("\n"); System.out.println("Facet drill-sideways example (timestamp/Past six hours):"); System.out.println("---------------------------------------------"); DrillSideways.DrillSidewaysResult sideways = example.drillSideways(example.PAST_SIX_HOURS); System.out.println(sideways.hits.totalHits + " totalHits"); System.out.println(sideways.facets.getTopChildren(10, "timestamp")); } ``` -------------------------------- ### Main Method for Examples Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SimpleFacetsExample.html Executes all the faceting examples and prints their results to the console. This serves as a demonstration of how to use the different faceting methods. ```Java public static void main(String[] args) throws Exception { System.out.println("Facet counting example:"); System.out.println("-----------------------"); SimpleFacetsExample example = new SimpleFacetsExample(); List results1 = example.runFacetOnly(); System.out.println("Author: " + results1.get(0)); System.out.println("Publish Date: " + results1.get(1)); System.out.println("Facet counting example (combined facets and search):"); System.out.println("-----------------------"); List results = example.runSearch(); System.out.println("Author: " + results.get(0)); System.out.println("Publish Date: " + results.get(1)); } ``` -------------------------------- ### Run Drill Down Example Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SandboxFacetsExample.html Executes the drill-down example. This method first indexes the data and then calls the internal drillDown method. ```Java /** Runs the drill-down example. */ public FacetResult runDrillDown() throws IOException { index(); return drillDown(); } ``` -------------------------------- ### FST Visualization Example Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/fst/Util.html Example demonstrating how to dump an FST to a GraphViz 'dot' file for visualization and how to render it using the 'dot' command-line tool. ```Java PrintWriter pw = new PrintWriter("out.dot"); Util.toDot(fst, pw, true, true); pw.close(); ``` ```Shell dot -Tpng -o out.png out.dot ``` -------------------------------- ### Run Facets with Search Example Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SandboxFacetsExample.html Executes the combined facets and search example. This method first indexes the data and then calls the internal facetsWithSearch method. ```Java /** Runs the search example. */ public List runSearch() throws IOException { index(); return facetsWithSearch(); } ``` -------------------------------- ### FileBasedQueryMaker Configuration Example Source: https://lucene.apache.org/core/10_4_0/benchmark/org/apache/lucene/benchmark/byTask/feeds/FileBasedQueryMaker.html Example configuration properties for FileBasedQueryMaker. Set the 'file.query.maker.file' to the path of your query file and 'file.query.maker.default.field' to specify the default search field. ```properties file.query.maker.file=c:/myqueries.txt file.query.maker.default.field=body ``` -------------------------------- ### Run Facet Only Example Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SandboxFacetsExample.html Executes the facet counting example. This method first indexes the data and then calls the internal facetsOnly method. ```Java /** Runs the search example. */ public List runFacetOnly() throws IOException { index(); return facetsOnly(); } ``` -------------------------------- ### startOffset Source: https://lucene.apache.org/core/10_4_0/highlighter/org/apache/lucene/search/uhighlight/OffsetsEnum.MultiOffsetsEnum.html Gets the starting offset of the current term at the current position. ```APIDOC ## startOffset ### Description Retrieves the starting offset of the term at the current position. ### Signature `public int startOffset() throws IOException` ### Returns The starting offset of the current term. ### Throws * `IOException` - If an I/O error occurs. ``` -------------------------------- ### Get Character Array Length from Start Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/compound/hyphenation/TernaryTree.html Calculates the length of a null-terminated character array starting from a specified index. This is useful for measuring substrings within a character array. ```java static int strlen(char[] a, int start) ``` -------------------------------- ### Prefix Wildcard Example Source: https://lucene.apache.org/core/10_4_0/queryparser/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.html Selects documents containing words that start with 'tes', such as 'test' or 'testing'. ```lucene tes* ``` -------------------------------- ### Main Method for Demonstrating Facet Examples Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/AssociationsFacetsExample.html This is the main entry point for the example. It demonstrates both the sum associations and drill-down functionalities. It prints the results of summing 'tags' and 'genre' associations, and then iterates through the facet results to display counts per label. ```java public static void main(String[] args) throws Exception { System.out.println("Sum associations example:"); System.out.println("-------------------------"); List results = new AssociationsFacetsExample().runSumAssociations(); System.out.println("tags: " + results.get(0)); System.out.println("genre: " + results.get(1)); System.out.println("-------------------------"); System.out.println("Counts per label are also available:"); for (FacetResult facetResult : results) { for (LabelAndValue lv : facetResult.labelValues) { System.out.println("\t" + lv.label + ": " + lv.count); } } } ``` -------------------------------- ### start Source: https://lucene.apache.org/core/10_4_0/spatial-extras/org/apache/lucene/spatial/prefix/AbstractVisitingPrefixTreeQuery.VisitorTemplate.html Abstract method called first to set up traversal. ```APIDOC ## start ### Description Abstract method called first to set up traversal and any necessary initializations. ### Throws * `IOException` ``` -------------------------------- ### Example Usage of RegexCompletionQuery Source: https://lucene.apache.org/core/10_4_0/suggest/org/apache/lucene/search/suggest/document/RegexCompletionQuery.html Demonstrates how to create a RegexCompletionQuery to search for terms starting with 'sug' or 'sub' in the 'suggest_field'. ```java CompletionQuery query = new RegexCompletionQuery(new Term("suggest_field", "su[g|b]")); ``` -------------------------------- ### Main Method for SandboxFacetsExample Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SandboxFacetsExample.html The main entry point for the SandboxFacetsExample. It instantiates the example class and runs various faceting demonstrations, printing the results to the console. ```Java /** Runs the search and drill-down examples and prints the results. */ public static void main(String[] args) throws Exception { SandboxFacetsExample example = new SandboxFacetsExample(); System.out.println("Simple facet counting example:"); System.out.println("---------------------------------------------"); for (FacetResult result : example.runSimpleFacetsWithSearch()) { System.out.println(result); } System.out.println("Simple facet counting for drill sideways example:"); System.out.println("---------------------------------------------"); for (FacetResult result : example.runSimpleFacetsWithDrillSideways()) { System.out.println(result); } System.out.println("Facet counting example:"); System.out.println("-----------------------"); List results1 = example.runFacetOnly(); System.out.println("Author: " + results1.get(0)); System.out.println("Publish Date: " + results1.get(1)); System.out.println("Facet counting example (combined facets and search):"); System.out.println("-----------------------"); List results = example.runSearch(); System.out.println("Author: " + results.get(0)); System.out.println("Publish Date: " + results.get(1)); System.out.println("Facet drill-down example (Publish Date/2010):"); System.out.println("---------------------------------------------"); System.out.println("Author: " + example.runDrillDown()); System.out.println("Facet drill-sideways example (Publish Date/2010):"); System.out.println("---------------------------------------------"); for (FacetResult result : example.runDrillSideways()) { System.out.println(result); } System.out.println("Facet counting example with exclusive ranges:"); System.out.println("---------------------------------------------"); for (FacetResult result : example.runNonOverlappingRangesCountFacetsOnly()) { System.out.println(result); } System.out.println("Facet counting example with overlapping ranges:"); System.out.println("---------------------------------------------"); } ``` -------------------------------- ### OrdRange Methods Source: https://lucene.apache.org/core/10_4_0/facet/org/apache/lucene/facet/sortedset/SortedSetDocValuesReaderState.OrdRange.html Provides methods to access the start and end ordinals, iterate through the range, and get string/hash representations. ```APIDOC ## OrdRange Methods ### `start()` Returns the value of the `start` record component. * **Returns**: (int) The starting ordinal of the range. ### `end()` Returns the value of the `end` record component. * **Returns**: (int) The ending ordinal of the range. ### `iterator()` Iterates from start to end ord (inclusive). * **Returns**: (PrimitiveIterator.OfInt) An iterator for the ordinal range. ### `toString()` Returns a string representation of this record class, including the name of the class and the values of its components. * **Returns**: (String) A string representation of the OrdRange object. ### `equals(Object o)` Indicates whether some other object is "equal" to this one. The objects are equal if the other object is of the same class and all record components are equal. * **Parameters**: * **o** (Object) - The object with which to compare. * **Returns**: (boolean) `true` if this object is the same as the `o` argument; `false` otherwise. ### `hashCode()` Returns a hash code value for this object, derived from the hash code of each record component. * **Returns**: (int) A hash code value for this object. ``` -------------------------------- ### ICUTokenizerFactory Configuration Example Source: https://lucene.apache.org/core/10_4_0/analysis/icu/org/apache/lucene/analysis/icu/segmentation/ICUTokenizerFactory.html Demonstrates how to configure ICUTokenizerFactory with custom rule files. ```APIDOC ## ICUTokenizerFactory Configuration ### Description This example shows how to use `ICUTokenizerFactory` with custom per-script rule files. ### Configuration Example ```xml ``` ### Parameters * **cjkAsWords** (boolean) - Optional. If true, CJK characters are treated as words. * **rulefiles** (String) - Optional. A comma-separated list of `code:rulefile` pairs, where `code` is a four-letter ISO 15924 script code and `rulefile` is a resource path to the rule file. ``` -------------------------------- ### Get Character Array Substring Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/util/RollingCharBuffer.html Retrieves a portion of the buffered characters as a char array, starting from a specified position and for a given length. ```Java public char[] get(int posStart, int length) ``` -------------------------------- ### FastVectorHighlighter Initialization with SingleFragListBuilder Source: https://lucene.apache.org/core/10_4_0/highlighter/org/apache/lucene/search/vectorhighlight/SingleFragListBuilder.html Example of initializing FastVectorHighlighter using SingleFragListBuilder and SimpleFragmentsBuilder. This setup is typical for retrieving entire field contents. ```java FastVectorHighlighter h = new FastVectorHighlighter( true, true, new SingleFragListBuilder(), new SimpleFragmentsBuilder() ); ``` -------------------------------- ### HitQueue Constructor and Pre-population Example Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/search/HitQueue.html Demonstrates how to create a HitQueue with pre-population enabled and how to correctly extract elements from it. This is crucial when `prePopulate` is true, as you need to distinguish between sentinel values and actual added elements. ```java PriorityQueue pq = new HitQueue(10, true); // pre-populate. ScoreDoc top = pq.top(); // Add/Update one element. top.score = 1.0f; top.doc = 0; top = (ScoreDoc) pq.updateTop(); int totalHits = 1; // Now pop only the elements that were *truly* inserted. // First, pop all the sentinel elements (there are pq.size() - totalHits). for (int i = pq.size() - totalHits; i > 0; i--) pq.pop(); // Now pop the truly added elements. ScoreDoc[] results = new ScoreDoc[totalHits]; for (int i = totalHits - 1; i >= 0; i--) { results[i] = (ScoreDoc) pq.pop(); } ``` -------------------------------- ### Solr Field Type with TrimFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/miscellaneous/TrimFilterFactory.html Example of configuring a Solr field type to use TrimFilterFactory. This setup applies trimming after tokenization. ```xml ``` -------------------------------- ### Implement Custom Attribute Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/analysis/package-summary.html Example of how to start implementing a custom attribute in Lucene. This class extends AttributeImpl and implements the custom attribute interface. ```java public class FirstTokenOfSentenceAttributeImpl extends AttributeImpl implements FirstTokenOfSentenceAttribute { private boolean firstToken; public void setFirstToken(boolean firstToken) { this.firstToken = firstToken; } public boolean getFirstToken() { return firstToken; } @Override public void clear() { firstToken = false; } ... ``` -------------------------------- ### Example of ContextQuery with Boosted Contexts Source: https://lucene.apache.org/core/10_4_0/suggest/org/apache/lucene/search/suggest/document/ContextQuery.html Demonstrates how to create a ContextQuery and add contexts with specific boost values. This is useful for prioritizing suggestions from certain contexts. ```java CompletionQuery completionQuery = ...; ContextQuery query = new ContextQuery(completionQuery); query.addContext("context1", 2); query.addContext("context2", 1); ``` -------------------------------- ### Get Suffix Length Source: https://lucene.apache.org/core/10_4_0/codecs/org/apache/lucene/codecs/uniformsplit/TermBytes.html Returns the length of this term's incremental encoding suffix. This suffix, starting from the MDP, is what's actually encoded. ```java public int getSuffixLength() ``` -------------------------------- ### Solr Field Type with ASCIIFoldingFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/miscellaneous/ASCIIFoldingFilterFactory.html Example of how to configure a Solr field type to use ASCIIFoldingFilterFactory. This setup includes a WhitespaceTokenizerFactory followed by ASCIIFoldingFilterFactory. ```xml ``` -------------------------------- ### Constructor Details Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/PriorityQueue.html Details on how to instantiate a PriorityQueue. ```APIDOC ## Constructor Details ### PriorityQueue(int maxSize) Create an empty priority queue of the configured size. ### PriorityQueue(int maxSize, Supplier sentinelObjectSupplier) Create a priority queue that is pre-filled with sentinel objects, so that the code which uses that queue can always assume it's full and only change the top without attempting to insert any new object. Those sentinel values should always compare worse than any non-sentinel value (i.e., `lessThan(T, T)` should always favor the non-sentinel values). By default, the supplier returns null, which means the queue will not be filled with sentinel values. Otherwise, the value returned will be used to pre-populate the queue. If this method is extended to return a non-null value, then the following usage pattern is recommended: ``` PriorityQueue pq = new MyQueue(numHits); // save the 'top' element, which is guaranteed to not be null. MyObject pqTop = pq.top(); <...> // now in order to add a new element, which is 'better' than top (after // you've verified it is better), it is as simple as: pqTop.change(). pqTop = pq.updateTop(); ``` **NOTE:** the given supplier will be called `maxSize` times, relying on a new object to be returned and will not check if it's null again. Therefore you should ensure any call to this method creates a new instance and behaves consistently, e.g., it cannot return null if it previously returned non-null and all returned instances must `compare equal`. ``` -------------------------------- ### Initialize Index and Taxonomy Writers Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SandboxFacetsExample.html Sets up the index and taxonomy directories and initializes the IndexWriter and DirectoryTaxonomyWriter. The IndexWriter is configured to create a new index, and the taxonomy writer is set to use a separate directory. ```java private final Directory indexDir = new ByteBuffersDirectory(); private final Directory taxoDir = new ByteBuffersDirectory(); private final FacetsConfig config = new FacetsConfig(); private SandboxFacetsExample() { config.setHierarchical("Publish Date", true); config.setHierarchical("Author", false); } /** Build the example index. */ void index() throws IOException { IndexWriter indexWriter = new IndexWriter( indexDir, new IndexWriterConfig(new WhitespaceAnalyzer()).setOpenMode(OpenMode.CREATE)); // Writes facet ords to a separate directory from the main index DirectoryTaxonomyWriter taxoWriter = new DirectoryTaxonomyWriter(taxoDir); ``` -------------------------------- ### Grouping with Boolean Operators Example Source: https://lucene.apache.org/core/10_4_0/queryparser/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.html Selects documents containing 'test' in the 'title' field and a word starting with 'pass' or 'fail' in the default search fields. ```lucene title:test AND (pass* OR fail*) ``` -------------------------------- ### Running Drill-Down Example Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/AssociationsFacetsExample.html This method provides a convenient way to run the drill-down example. It first calls `index()` to prepare the data and then executes the `drillDown()` method to perform and return the drill-down facet results. ```java public FacetResult runDrillDown() throws IOException { index(); return drillDown(); } ``` -------------------------------- ### Facet Counting Example Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.html Performs a faceted search and retrieves top children for 'Author' and 'Publish Year' facets. Requires setup of index and configuration. ```java private List search() throws IOException { DirectoryReader indexReader = DirectoryReader.open(indexDir); IndexSearcher searcher = new IndexSearcher(indexReader); SortedSetDocValuesReaderState state = new DefaultSortedSetDocValuesReaderState(indexReader, config); // Facets will be computed for "Author" and "Publish Year" dimensions FacetsCollectorManager fcm = new FacetsCollectorManager(); FacetsCollector fc = FacetsCollectorManager.search(searcher, new MatchAllDocsQuery(), 10, fcm).facetsCollector(); // Retrieve results Facets facets = new SortedSetDocValuesFacetCounts(state, fc); List results = new ArrayList<>(); results.add(facets.getTopChildren(10, "Author")); results.add(facets.getTopChildren(10, "Publish Year")); indexReader.close(); return results; } ``` -------------------------------- ### Solr Field Type Configuration Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/miscellaneous/RemoveDuplicatesTokenFilterFactory.html Example of how to configure RemoveDuplicatesTokenFilterFactory within a Solr field type definition. This setup uses WhitespaceTokenizerFactory followed by RemoveDuplicatesTokenFilterFactory. ```xml ``` -------------------------------- ### StandardQueryParser Usage and Configuration Source: https://lucene.apache.org/core/10_4_0/queryparser/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.html Demonstrates how to instantiate and configure the StandardQueryParser, set analysis, and parse a query string with a default field. ```java StandardQueryParser qpHelper = new StandardQueryParser(); StandardQueryConfigHandler config = qpHelper.getQueryConfigHandler(); config.setAllowLeadingWildcard(true); config.setAnalyzer(new WhitespaceAnalyzer()); Query query = qpHelper.parse("apache AND lucene", "defaultField"); ``` -------------------------------- ### start Source: https://lucene.apache.org/core/10_4_0/replicator/org/apache/lucene/replicator/nrt/CopyJob.html Begins the file copying process for this job. This is an abstract method that must be implemented by subclasses. ```APIDOC ## public abstract void start() throws IOException ### Description Begin copying files. ### Throws: - `IOException` - If an I/O error occurs during the start of the copy process. ``` -------------------------------- ### Get the next iteration seed Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/internal/hppc/IntHashSet.html Provides a seed for building iteration starting slots and offset increments. Used internally to ensure varied iteration orders. ```java set.nextIterationSeed(); ``` -------------------------------- ### Run IndexUpgrader from Command Line Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/index/IndexUpgrader.html This command-line example shows how to use IndexUpgrader to upgrade an index directory. It includes options for deleting prior commits and verbose output. ```bash java -cp lucene-core.jar org.apache.lucene.index.IndexUpgrader [-delete-prior-commits] [-verbose] indexDir ``` -------------------------------- ### Pattern File Format Example Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/pattern/PatternTypingFilterFactory.html Illustrates the format for a pattern file used by PatternTypingFilterFactory. Each line specifies flags, a pattern, and a replacement string, separated by ':::'. Comments start with '#'. ```plaintext (flags) (pattern) ::: (replacement) ``` ```plaintext 3 (\d+)\(?([a-z])\)? ::: legal2_$1_$2 ``` -------------------------------- ### SearchFiles Main Method and Argument Parsing Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/SearchFiles.html The main method initializes the search environment, parses command-line arguments for index directory, search field, query source, and search parameters like hits per page and KNN vector count. It sets up the IndexSearcher, Analyzer, and optionally a KnnVectorDict. ```Java public static void main(String[] args) throws Exception { String usage = "Usage:\tjava org.apache.lucene.demo.SearchFiles [-index dir] [-field f] [-repeat n] [-queries file] [-query string] [-raw] [-paging hitsPerPage] [-knn_vector knnHits]\n\nSee http://lucene.apache.org/core/9_0_0/demo/ for details."; if (args.length > 0 && ("-h".equals(args[0]) || "-help".equals(args[0]))) { System.out.println(usage); System.exit(0); } String index = "index"; String field = "contents"; String queries = null; int repeat = 0; boolean raw = false; int knnVectors = 0; String queryString = null; int hitsPerPage = 10; for (int i = 0; i < args.length; i++) { switch (args[i]) { case "-index": index = args[++i]; break; case "-field": field = args[++i]; break; case "-queries": queries = args[++i]; break; case "-query": queryString = args[++i]; break; case "-repeat": repeat = Integer.parseInt(args[++i]); break; case "-raw": raw = true; break; case "-paging": hitsPerPage = Integer.parseInt(args[++i]); if (hitsPerPage <= 0) { System.err.println("There must be at least 1 hit per page."); System.exit(1); } break; case "-knn_vector": knnVectors = Integer.parseInt(args[++i]); break; default: System.err.println("Unknown argument: " + args[i]); System.exit(1); } } DirectoryReader reader = DirectoryReader.open(FSDirectory.open(Paths.get(index))); IndexSearcher searcher = new IndexSearcher(reader); Analyzer analyzer = new StandardAnalyzer(); KnnVectorDict vectorDict = null; if (knnVectors > 0) { vectorDict = new KnnVectorDict(reader.directory(), IndexFiles.KNN_DICT); } BufferedReader in; if (queries != null) { in = Files.newBufferedReader(Paths.get(queries), StandardCharsets.UTF_8); } else { in = new BufferedReader(new InputStreamReader(System.in, StandardCharsets.UTF_8)); } QueryParser parser = new QueryParser(field, analyzer); while (true) { if (queries == null && queryString == null) { // prompt the user System.out.println("Enter query: "); } ``` -------------------------------- ### Configure CommonGramsQueryFilterFactory in Solr Schema Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/commongrams/CommonGramsQueryFilterFactory.html Example of how to integrate CommonGramsQueryFilterFactory into a Solr field type definition. This setup uses a custom word list for generating common n-grams. ```xml ``` -------------------------------- ### Sample Benchmark Algorithm Configuration Source: https://lucene.apache.org/core/10_4_0/benchmark/org/apache/lucene/benchmark/byTask/package-summary.html This configuration defines a benchmark test to compare the indexing time of adding documents with different sizes. It sets up parameters for merging, buffering, and document making, and defines two main tasks: PopulateShort and PopulateLong, each with specific document counts and optimization steps. The benchmark is run twice with different merge factors and buffer sizes. ```plaintext # -------------------------------------------------------- # # Sample: what is the effect of doc size on indexing time? # # There are two parts in this test: # - PopulateShort adds 2N documents of length L # - PopulateLong adds N documents of length 2L # Which one would be faster? # The comparison is done twice. # # -------------------------------------------------------- # ------------------------------------------------------------------------------------- # multi val params are iterated by NewRound's, added to reports, start with column name. merge.factor=mrg:10:20 max.buffered=buf:100:1000 compound=true analyzer=org.apache.lucene.analysis.standard.StandardAnalyzer directory=FSDirectory doc.stored=true doc.tokenized=true doc.term.vector=false doc.add.log.step=500 docs.dir=reuters-out doc.maker=org.apache.lucene.benchmark.byTask.feeds.SimpleDocMaker query.maker=org.apache.lucene.benchmark.byTask.feeds.SimpleQueryMaker # task at this depth or less would print when they start task.max.depth.log=2 log.queries=false # ------------------------------------------------------------------------------------- { { "PopulateShort" CreateIndex { AddDoc(4000) > : 20000 Optimize CloseIndex > ResetSystemErase { "PopulateLong" CreateIndex { AddDoc(8000) > : 10000 Optimize CloseIndex > ResetSystemErase NewRound } : 2 RepSumByName RepSelectByPref Populate ``` -------------------------------- ### Solr Field Type Configuration Source: https://lucene.apache.org/core/10_4_0/analysis/icu/org/apache/lucene/analysis/icu/ICUFoldingFilterFactory.html Example of how to configure a Solr field type to use ICUFoldingFilterFactory for text analysis. This setup includes a whitespace tokenizer followed by the ICU folding filter. ```xml ``` -------------------------------- ### Index Files Command-Line Example Source: https://lucene.apache.org/core/10_4_0/core/index.html This command demonstrates how to use the IndexFiles utility from the Lucene demo JAR to create an index for files within a specified directory. It requires lucene-core, lucene-demo, and lucene-analysis-common JARs. ```bash java -cp lucene-core.jar:lucene-demo.jar:lucene-analysis-common.jar org.apache.lucene.demo.IndexFiles -index index -docs rec.food.recipes/soups ``` -------------------------------- ### Solr Field Type Configuration with TruncateTokenFilter Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/miscellaneous/TruncateTokenFilterFactory.html Example of configuring a Solr field type to use TruncateTokenFilterFactory for diacritics-insensitive search in Turkish. This setup truncates tokens to a prefix length of 5. ```xml ``` -------------------------------- ### Search Files Command-Line Example Source: https://lucene.apache.org/core/10_4_0/core/index.html This command demonstrates how to use the SearchFiles utility from the Lucene demo JAR to search an existing index. It requires lucene-core, lucene-demo, lucene-queryparser, and lucene-analysis-common JARs. ```bash java -cp lucene-core.jar:lucene-demo.jar:lucene-queryparser.jar:lucene-analysis-common.jar org.apache.lucene.demo.SearchFiles ``` -------------------------------- ### Get Or Default Example Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/internal/hppc/LongFloatHashMap.html Demonstrates the getOrDefault method, which retrieves the value associated with a key or returns a specified default value if the key is not found. This helps avoid null checks and simplifies code. ```java public float getOrDefault(long key, float defaultValue) ``` -------------------------------- ### Get DirectReader Instance with Offset Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/packed/DirectReader.html Retrieves a LongValues instance from a specific offset within a slice, decoding a specified number of bits per value. Useful when the data does not start at the beginning of the slice. ```java public static LongValues getInstance(RandomAccessInput slice, int bitsPerValue, long offset) ``` -------------------------------- ### Sample Main Method Source: https://lucene.apache.org/core/10_4_0/benchmark/org/apache/lucene/benchmark/byTask/programmatic/Sample.html The entry point for running Lucene benchmarks programmatically. It allows for setting up and executing performance tests without relying on external algorithm files. ```Java public static void main(String[] args) throws Exception ``` -------------------------------- ### Solr Field Type with ShingleFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/shingle/ShingleFilterFactory.html Example of configuring ShingleFilterFactory within a Solr field type. This setup creates bigrams (shingles of size 2) and also outputs unigrams if no bigrams are formed. ```xml ``` -------------------------------- ### N-Shortest Paths by Weight Example Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/fst/package-summary.html Shows how to find the N-shortest paths (by weight) in an FST. Requires a comparator for the output type. ```java Comparator comparator = new Comparator() { public int compare(Long left, Long right) { return left.compareTo(right); } }; Arc firstArc = fst.getFirstArc(new Arc()); Util.TopResults paths = Util.shortestPaths(fst, firstArc, fst.outputs.getNoOutput(), comparator, 3, true); System.out.println(Util.toBytesRef(paths.topN.get(0).input, scratchBytes).utf8ToString()); // cat System.out.println(paths.topN.get(0).output); // 5 System.out.println(Util.toBytesRef(paths.topN.get(1).input, scratchBytes).utf8ToString()); // dog System.out.println(paths.topN.get(1).output); // 7 ``` -------------------------------- ### start Method Source: https://lucene.apache.org/core/10_4_0/highlighter/org/apache/lucene/search/highlight/NullFragmenter.html Initializes the Fragmenter. This method is called once at the beginning of the highlighting process. It allows the Fragmenter to access attributes from the TokenStream. ```APIDOC ## start Method ### Description Initializes the Fragmenter. You can grab references to the Attributes you are interested in from tokenStream and then access the values in `Fragmenter.isNewFragment()`. ### Signature `public void start(String s, TokenStream tokenStream)` ### Parameters - **s** (String) - The original source text. - **tokenStream** (TokenStream) - The TokenStream to be fragmented. ``` -------------------------------- ### Example Usage of DirectWriter Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/packed/DirectWriter.html Demonstrates how to use DirectWriter to write packed integers to a directory. This involves calculating bits per value, getting an instance of the writer, adding values, and finishing the write operation. ```java int bitsPerValue = DirectWriter.bitsRequired(100); // values up to and including 100 IndexOutput output = dir.createOutput("packed", IOContext.DEFAULT); DirectWriter writer = DirectWriter.getInstance(output, numberOfValues, bitsPerValue); for (int i = 0; i < numberOfValues; i++) { writer.add(value); } writer.finish(); output.close(); ``` -------------------------------- ### ClassicTokenizerFactory.create Method Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/classic/ClassicTokenizerFactory.html Documentation for the create method, which instantiates a ClassicTokenizer. ```APIDOC ## create(AttributeFactory factory) ### Description Creates a `ClassicTokenizer` instance with the specified `AttributeFactory`. ### Method `create` ### Parameters #### Path Parameters - **factory** (AttributeFactory) - Description not available in source. ``` -------------------------------- ### Example Usage of LegacyDirectWriter Source: https://lucene.apache.org/core/10_4_0/backward-codecs/org/apache/lucene/backward_codecs/packed/LegacyDirectWriter.html Demonstrates how to use LegacyDirectWriter to write packed integers to an IndexOutput. This involves calculating the bits per value, getting an instance of the writer, adding values, and finishing the write operation. ```java int bitsPerValue = LegacyDirectWriter.bitsRequired(100); // values up to and including 100 IndexOutput output = dir.createOutput("packed", IOContext.DEFAULT); DirectWriter writer = LegacyDirectWriter.getInstance(output, numberOfValues, bitsPerValue); for (int i = 0; i < numberOfValues; i++) { writer.add(value); } writer.finish(); output.close(); ``` -------------------------------- ### init Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/codecs/lucene104/Lucene104PostingsReader.html Performs any initialization, such as reading and verifying the header from the provided terms dictionary IndexInput. ```APIDOC ## init(IndexInput termsIn, SegmentReadState state) ### Description Performs any initialization, such as reading and verifying the header from the provided terms dictionary `IndexInput`. ### Method `void` ### Parameters #### Path Parameters - **termsIn** (IndexInput) - Required - The input stream for terms. - **state** (SegmentReadState) - Required - The state for reading the segment. ### Throws - `IOException` ``` -------------------------------- ### Get Merge-Optimized DirectReader Instance with Base Offset Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/packed/DirectReader.html Retrieves a LongValues instance optimized for merges, starting from a specified base offset. This is also faster for sequential access and useful for large-scale data processing. ```java public static LongValues getMergeInstance(RandomAccessInput slice, int bitsPerValue, long baseOffset, long numValues) ``` -------------------------------- ### Benchmark Report Output Example Source: https://lucene.apache.org/core/10_4_0/benchmark/org/apache/lucene/benchmark/byTask/package-summary.html This sample output shows the results of the benchmark test, detailing performance metrics for 'PopulateShort' and 'PopulateLong' tasks across different configurations (merge factor and buffer size). It includes columns for operation, round, record count per run, records per second, elapsed time, and memory usage. ```text Operation round mrg buf runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem PopulateShort 0 10 100 1 20003 119.6 167.26 12,959,120 14,241,792 PopulateLong - - 0 10 100 - - 1 - - 10003 - - - 74.3 - - 134.57 - 17,085,208 - 20,635,648 PopulateShort 1 20 1000 1 20003 143.5 139.39 63,982,040 94,756,864 PopulateLong - - 1 20 1000 - - 1 - - 10003 - - - 77.0 - - 129.92 - 87,309,608 - 100,831,232 ``` -------------------------------- ### Building a PhraseQuery with Relative Term Positions Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/search/PhraseQuery.html Shows how to construct a PhraseQuery using a builder where terms are added with relative positions starting from 0. This is equivalent to the previous example but uses a different positional offset. ```java PhraseQuery.Builder builder = new PhraseQuery.Builder(); builder.add(new Term("body", "one"), 0); builder.add(new Term("body", "two"), 1); PhraseQuery pq = builder.build(); ``` -------------------------------- ### startDocument Source: https://lucene.apache.org/core/10_4_0/codecs/org/apache/lucene/codecs/simpletext/SimpleTextStoredFieldsWriter.html Starts writing a new document. ```APIDOC ## startDocument() ### Description Starts writing a new document. ### Method void ### Throws - `IOException` ``` -------------------------------- ### Solr Field Type Configuration with BeiderMorseFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/phonetic/org/apache/lucene/analysis/phonetic/BeiderMorseFilterFactory.html Example of how to configure a Solr field type to use the BeiderMorseFilterFactory for phonetic analysis. This setup includes a standard tokenizer and the BeiderMorseFilter with specific phonetic rule configurations. ```xml ``` -------------------------------- ### Solr Field Type with TypeAsSynonymFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/miscellaneous/TypeAsSynonymFilterFactory.html Example of how to configure TypeAsSynonymFilterFactory within a Solr field type definition. This setup uses UAX29URLEmailTokenizerFactory and specifies a prefix, synonym flags mask, and ignored types for the TypeAsSynonymFilter. ```xml ``` -------------------------------- ### Running the Dynamic Range Facets Example Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/DynamicRangeFacetsExample.html Main method to execute the dynamic range facets example. It indexes documents, runs the search to compute ranges, and then prints the results including min, max, centroid, count, and weight for each range. ```Java System.out.println("Dynamic range facets example:"); System.out.println("-----------------------"); DynamicRangeFacetsExample example = new DynamicRangeFacetsExample(); List results = example.runSearch(); for (DynamicRangeUtil.DynamicRangeInfo range : results) { System.out.printf( Locale.ROOT, "min: %d max: %d centroid: %f count: %d weight: %d%n", range.min(), range.max(), range.centroid(), range.count(), range.weight()); } ``` -------------------------------- ### Solr Field Type with CJKWidthFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/cjk/CJKWidthFilterFactory.html Example of configuring CJKWidthFilterFactory within a Solr field type definition. This setup includes standard tokenization, CJK width normalization, lowercasing, and CJK bigram filtering. ```xml ``` -------------------------------- ### Initialize IndexSearcher and TaxonomyReader Source: https://lucene.apache.org/core/10_4_0/demo/src-html/org/apache/lucene/demo/facet/AssociationsFacetsExample.html Opens the index and taxonomy readers to prepare for searching and facet collection. This setup is required before executing facet queries. ```java DirectoryReader indexReader = DirectoryReader.open(indexDir); IndexSearcher searcher = new IndexSearcher(indexReader); TaxonomyReader taxoReader = new DirectoryTaxonomyReader(taxoDir); ``` -------------------------------- ### Sample Class Constructor Source: https://lucene.apache.org/core/10_4_0/benchmark/org/apache/lucene/benchmark/byTask/programmatic/Sample.html Initializes a new instance of the Sample class. This constructor is used when creating benchmark instances programmatically. ```Java public Sample() ``` -------------------------------- ### Solr Field Type with EdgeNGramFilterFactory Source: https://lucene.apache.org/core/10_4_0/analysis/common/org/apache/lucene/analysis/ngram/EdgeNGramFilterFactory.html Example of configuring EdgeNGramFilterFactory within a Solr field type. This setup uses WhitespaceTokenizerFactory and specifies minimum and maximum gram sizes, along with an option to preserve the original token. ```xml ``` -------------------------------- ### Example of Interval Function Clauses Source: https://lucene.apache.org/core/10_4_0/queryparser/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.html Demonstrates the usage of interval functions for expressing contiguous text fragment relationships. These functions start with the 'fn:' prefix and can be used for ordered sequences or proximity searches within a specified width. ```lucene fn:ordered(quick brown fox) ``` ```lucene title:fn:maxwidth(5 fn:atLeast(2 quick brown fox)) ``` -------------------------------- ### FST Construction Example Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/fst/package-summary.html Demonstrates how to build a minimal FST from sorted input values and their corresponding outputs. Ensure input values are provided in Unicode code point sorted order to avoid exceptions. ```java // Input values (keys). These must be provided to Builder in Unicode code point (UTF8 or UTF32) sorted order. // Note that sorting by Java's String.compareTo, which is UTF16 sorted order, is not correct and can lead to // exceptions while building the FST: String inputValues[] = {"cat", "dog", "dogs"}; long outputValues[] = {5, 7, 12}; PositiveIntOutputs outputs = PositiveIntOutputs.getSingleton(); FSTCompiler fstCompiler = new FSTCompiler(INPUT_TYPE.BYTE1, outputs); BytesRefBuilder scratchBytes = new BytesRefBuilder(); IntsRefBuilder scratchInts = new IntsRefBuilder(); for (int i = 0; i < inputValues.length; i++) { scratchBytes.copyChars(inputValues[i]); fstCompiler.add(Util.toIntsRef(scratchBytes.toBytesRef(), scratchInts), outputValues[i]); } FST fst = FST.fromFSTReader(fstCompiler.compile(), fstCompiler.getFSTReader()); ``` -------------------------------- ### fillSlice Source: https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/util/PagedBytes.Reader.html Gets a slice out of PagedBytes starting at a given offset with a specified length. This method handles slices that span across block borders by allocating resources and copying data. It does not support slices spanning more than two blocks. ```APIDOC ## fillSlice ### Description Gets a slice out of `PagedBytes` starting at _start_ with a given length. Iff the slice spans across a block border this method will allocate sufficient resources and copy the paged data. Slices spanning more than two blocks are not supported. ### Method public void fillSlice(BytesRef b, long start, int length) ### Parameters #### Path Parameters - **b** (BytesRef) - Description: The BytesRef to fill with data. - **start** (long) - Description: The starting offset in PagedBytes. - **length** (int) - Description: The length of the slice to retrieve. ### Note This API is for internal purposes only and might change in incompatible ways in the next release. ```