public abstract class LookaheadTokenFilter<T extends LookaheadTokenFilter.Position> extends TokenFilter
Modifier and Type | Class and Description |
---|---|
protected static class |
LookaheadTokenFilter.Position
Holds all state for a single position; subclass this
to record other state at each position.
|
AttributeSource.State
Modifier and Type | Field and Description |
---|---|
protected boolean |
end |
protected int |
inputPos |
protected OffsetAttribute |
offsetAtt |
protected int |
outputPos |
protected PositionIncrementAttribute |
posIncAtt |
protected RollingBuffer<T> |
positions |
protected PositionLengthAttribute |
posLenAtt |
input
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
Modifier | Constructor and Description |
---|---|
protected |
LookaheadTokenFilter(TokenStream input) |
Modifier and Type | Method and Description |
---|---|
protected void |
afterPosition()
This is called when all input tokens leaving a given
position have been returned.
|
protected void |
insertToken()
Call this only from within afterPosition, to insert a new
token.
|
protected abstract T |
newPosition() |
protected boolean |
nextToken()
Call this when you are done looking ahead; it will set
the next token to return.
|
protected boolean |
peekToken()
Returns true if there is a new token.
|
void |
reset()
This method is called by a consumer before it begins consumption using
TokenStream.incrementToken() . |
close, end
incrementToken
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
protected final PositionIncrementAttribute posIncAtt
protected final PositionLengthAttribute posLenAtt
protected final OffsetAttribute offsetAtt
protected int inputPos
protected int outputPos
protected boolean end
protected final RollingBuffer<T extends LookaheadTokenFilter.Position> positions
protected LookaheadTokenFilter(TokenStream input)
protected void insertToken() throws java.io.IOException
java.io.IOException
protected void afterPosition() throws java.io.IOException
java.io.IOException
protected abstract T newPosition()
protected boolean peekToken() throws java.io.IOException
java.io.IOException
protected boolean nextToken() throws java.io.IOException
java.io.IOException
public void reset() throws java.io.IOException
TokenFilter
TokenStream.incrementToken()
.
Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.
If you override this method, always call super.reset()
, otherwise
some internal state will not be correctly reset (e.g., Tokenizer
will
throw IllegalStateException
on further usage).
NOTE:
The default implementation chains the call to the input TokenStream, so
be sure to call super.reset()
when overriding this method.
reset
in class TokenFilter
java.io.IOException
Copyright © 2000–2019 The Apache Software Foundation. All rights reserved.