<?xml version="1.0"?>
<document>
<header>
<title>
Apache Lucene - Basic Demo Sources Walk-through
</title>
</header>
<properties>
<author email="acoliver@apache.org">Andrew C. Oliver</author>
</properties>
<body>
<section id="About the Code"><title>About the Code</title>
<p>
In this section we walk through the sources behind the command-line Lucene demo: where to find them,
their parts and their function. This section is intended for Java developers wishing to understand
how to use Lucene in their applications.
</p>
</section>
<section id="Location of the source"><title>Location of the source</title>
<p>
NOTE: to examine the sources, you need to download and extract a source checkout of
Lucene: (lucene-{version}-src.zip).
</p>
<p>
Relative to the directory created when you extracted Lucene, you
the Java sources for the demo live.
</p>
<p>
Bring it up in <code>vi</code> or your editor of choice and let's take a look at it.
</p>
</section>
<section id="IndexFiles"><title>IndexFiles</title>
<p>
As we discussed in the previous walk-through, the <a
href="api/contrib-demo/org/apache/lucene/demo/IndexFiles.html">IndexFiles</a> class creates a Lucene
Index. Let's take a look at how it does this.
</p>
<p>
The <code>main()</code> method parses the command-line parameters, then in preparation for
>StandardAnalyzer</a> and
</p>
<p>
The value of the <code>-index</code> command-line parameter is the name of the filesystem directory
where all index information should be stored. If <code>IndexFiles</code> is invoked with a
relative path given in the <code>-index</code> command-line parameter, or if the <code>-index</code>
command-line parameter is not given, causing the default relative index path "<code>index</code>"
to be used, the index path will be created as a subdirectory of the current working directory
(if it does not already exist). On some platforms, the index path may be created in a different
directory (such as the user's home directory).
</p>
<p>
The <code>-docs</code> command-line parameter value is the location of the directory containing
files to be indexed.
</p>
<p>
The <code>-update</code> command-line parameter tells <code>IndexFiles</code> not to delete the
index if it already exists. When <code>-update</code> is not given, <code>IndexFiles</code> will
first wipe the slate clean before indexing any documents.
</p>
<p>
<code>IndexWriter</code> to store information in the index. In addition to the
<a href="api/core/org/apache/lucene/store/FSDirectory.html">FSDirectory</a> implementation we are using,
there are several other <code>Directory</code> subclasses that can write to RAM, to databases, etc.
</p>
<p>
Lucene <a href="api/core/org/apache/lucene/analysis/Analyzer.html">Analyzer</a>s are processing pipelines
that break up text into indexed tokens, a.k.a. terms, and optionally perform other operations on these
tokens, e.g. downcasing, synonym insertion, filtering out unwanted tokens, etc. The <code>Analyzer</code>
we are using is <code>StandardAnalyzer</code>, which creates tokens using the Word Break rules from the
Unicode Text Segmentation algorithm specified in <a href="http://unicode.org/reports/tr29/">Unicode
Standard Annex #29</a>; converts tokens to lowercase; and then filters out stopwords. Stopwords are
common language words such as articles (a, an, the, etc.) and other tokens that may have less value for
searching. It should be noted that there are different rules for every language, and you should use the
proper analyzer for each. Lucene currently provides Analyzers for a number of different languages (see
the javadocs under
</p>
<p>
The <code>IndexWriterConfig</code> instance holds all configuration for <code>IndexWriter</code>. For
example, we set the <code>OpenMode</code> to use here based on the value of the <code>-update</code>
command-line parameter.
</p>
<p>
Looking further down in the file, after <code>IndexWriter</code> is instantiated, you should see the
<code>indexDocs()</code> code. This recursive function crawls the directories and creates
<code>Document</code> is simply a data object to represent the text content from the file as well as
its creation time and location. These instances are added to the <code>IndexWriter</code>. If
the <code>-update</code> command-line parameter is given, the <code>IndexWriter</code>
adding documents to the index, the <code>IndexWriter</code> will <strong>update</strong> them
in the index by attempting to find an already-indexed document with the same identifier (in our
case, the file path serves as the identifier); deleting it from the index if it exists; and then
adding the new document to the index.
</p>
</section>
<section id="Searching Files"><title>Searching Files</title>
<p>
quite simple. It primarily collaborates with an
>StandardAnalyzer</a> (which is used in the
query parser is constructed with an analyzer used to interpret your query text in the same way the
documents are interpreted: finding word boundaries, downcasing, and removing useless words like
object contains the results from the
to the searcher. Note that it's also possible to programmatically construct a rich
parser. The query parser just enables decoding the <a href="queryparsersyntax.html">Lucene query
object.
</p>
<p>
The results are printed in pages, sorted by score (i.e. relevance).
</p>
</section>
</body>
</document>