Searched defs:StandardTokenizer (Results 1 - 1 of 1) sorted by relevance

/lucene-3.6.0/lucene/core/src/java/org/apache/lucene/analysis/standard/
H A DStandardTokenizer.java44 * compatibility when creating StandardTokenizer:
49 * <li> As of 3.1, StandardTokenizer implements Unicode text segmentation.
55 public final class StandardTokenizer extends Tokenizer { class in inherits:Tokenizer
125 * Creates a new instance of the {@link org.apache.lucene.analysis.standard.StandardTokenizer}. Attaches
132 public StandardTokenizer(Version matchVersion, Reader input) { method in class:StandardTokenizer
138 * Creates a new StandardTokenizer with a given {@link AttributeSource}.
140 public StandardTokenizer(Version matchVersion, AttributeSource source, Reader input) { method in class:StandardTokenizer
146 * Creates a new StandardTokenizer with a given {@link org.apache.lucene.util.AttributeSource.AttributeFactory}
148 public StandardTokenizer(Version matchVersion, AttributeFactory factory, Reader input) { method in class:StandardTokenizer
200 if (tokenType == StandardTokenizer
[all...]

Completed in 11 milliseconds