public class JapaneseAnalyzer extends StopwordAnalyzerBase
JapaneseTokenizer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
stopwords
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
Constructor and Description |
---|
JapaneseAnalyzer() |
JapaneseAnalyzer(UserDictionary userDict,
JapaneseTokenizer.Mode mode,
CharArraySet stopwords,
java.util.Set<java.lang.String> stoptags) |
Modifier and Type | Method and Description |
---|---|
protected Analyzer.TokenStreamComponents |
createComponents(java.lang.String fieldName)
Creates a new
Analyzer.TokenStreamComponents instance for this analyzer. |
static CharArraySet |
getDefaultStopSet() |
static java.util.Set<java.lang.String> |
getDefaultStopTags() |
protected TokenStream |
normalize(java.lang.String fieldName,
TokenStream in)
Wrap the given
TokenStream in order to apply normalization filters. |
getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSet
attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReader, initReaderForNormalization, normalize, setVersion, tokenStream, tokenStream
public JapaneseAnalyzer()
public JapaneseAnalyzer(UserDictionary userDict, JapaneseTokenizer.Mode mode, CharArraySet stopwords, java.util.Set<java.lang.String> stoptags)
public static CharArraySet getDefaultStopSet()
public static java.util.Set<java.lang.String> getDefaultStopTags()
protected Analyzer.TokenStreamComponents createComponents(java.lang.String fieldName)
Analyzer
Analyzer.TokenStreamComponents
instance for this analyzer.createComponents
in class Analyzer
fieldName
- the name of the fields content passed to the
Analyzer.TokenStreamComponents
sink as a readerAnalyzer.TokenStreamComponents
for this analyzer.protected TokenStream normalize(java.lang.String fieldName, TokenStream in)
Analyzer
TokenStream
in order to apply normalization filters.
The default implementation returns the TokenStream
as-is. This is
used by Analyzer.normalize(String, String)
.Copyright © 2000–2019 The Apache Software Foundation. All rights reserved.