public class TokenSources
extends java.lang.Object
TokenStream
for use with the Highlighter
- can obtain from
term vectors with offsets and positions or from an Analyzer re-parsing the stored content.TokenStreamFromTermVector
Modifier and Type | Method and Description |
---|---|
static TokenStream |
getAnyTokenStream(IndexReader reader,
int docId,
java.lang.String field,
Analyzer analyzer)
Deprecated.
|
static TokenStream |
getAnyTokenStream(IndexReader reader,
int docId,
java.lang.String field,
Document document,
Analyzer analyzer)
Deprecated.
|
static TokenStream |
getTermVectorTokenStreamOrNull(java.lang.String field,
Fields tvFields,
int maxStartOffset)
Get a token stream by un-inverting the term vector.
|
static TokenStream |
getTokenStream(Document doc,
java.lang.String field,
Analyzer analyzer)
Deprecated.
|
static TokenStream |
getTokenStream(IndexReader reader,
int docId,
java.lang.String field,
Analyzer analyzer)
Deprecated.
|
static TokenStream |
getTokenStream(java.lang.String field,
Fields tvFields,
java.lang.String text,
Analyzer analyzer,
int maxStartOffset)
Get a token stream from either un-inverting a term vector if possible, or by analyzing the text.
|
static TokenStream |
getTokenStream(java.lang.String field,
java.lang.String contents,
Analyzer analyzer)
Deprecated.
|
static TokenStream |
getTokenStream(Terms tpv)
Deprecated.
|
static TokenStream |
getTokenStream(Terms vector,
boolean tokenPositionsGuaranteedContiguous)
Deprecated.
|
static TokenStream |
getTokenStreamWithOffsets(IndexReader reader,
int docId,
java.lang.String field)
Deprecated.
|
public static TokenStream getTokenStream(java.lang.String field, Fields tvFields, java.lang.String text, Analyzer analyzer, int maxStartOffset) throws java.io.IOException
field
- The field to either get term vectors from or to analyze the text from.tvFields
- from IndexReader.getTermVectors(int)
. Possibly null. For performance, this instance should
be re-used for the same document (e.g. when highlighting multiple fields).text
- the text to analyze, failing term vector un-inversionanalyzer
- the analyzer to analyze text
with, failing term vector un-inversionmaxStartOffset
- Terms with a startOffset greater than this aren't returned. Use -1 for no limit.
Suggest using Highlighter.getMaxDocCharsToAnalyze()
- 1.java.io.IOException
public static TokenStream getTermVectorTokenStreamOrNull(java.lang.String field, Fields tvFields, int maxStartOffset) throws java.io.IOException
tvFields
is null
or if the field has no term vector, or if the term vector doesn't have offsets. Positions are recommended on the
term vector but it isn't strictly required.field
- The field to get term vectors from.tvFields
- from IndexReader.getTermVectors(int)
. Possibly null. For performance, this instance should
be re-used for the same document (e.g. when highlighting multiple fields).maxStartOffset
- Terms with a startOffset greater than this aren't returned. Use -1 for no limit.
Suggest using Highlighter.getMaxDocCharsToAnalyze()
- 1java.io.IOException
@Deprecated public static TokenStream getAnyTokenStream(IndexReader reader, int docId, java.lang.String field, Document document, Analyzer analyzer) throws java.io.IOException
TokenStreamFromTermVector
for the
specified docId, then, falls back to using the passed in
Document
to retrieve the TokenStream.
This is useful when you already have the document, but would prefer to use
the vector first.reader
- The IndexReader
to use to try
and get the vector fromdocId
- The docId to retrieve.field
- The field to retrieve on the documentdocument
- The document to fall back onanalyzer
- The analyzer to use for creating the TokenStream if the
vector doesn't existTokenStream
for the
IndexableField
on the
Document
java.io.IOException
- if there was an error loading@Deprecated public static TokenStream getAnyTokenStream(IndexReader reader, int docId, java.lang.String field, Analyzer analyzer) throws java.io.IOException
java.io.IOException
- If there is a low-level I/O error@Deprecated public static TokenStream getTokenStream(Terms vector, boolean tokenPositionsGuaranteedContiguous) throws java.io.IOException
getTokenStream(org.apache.lucene.index.Terms)
now.java.io.IOException
@Deprecated public static TokenStream getTokenStream(Terms tpv) throws java.io.IOException
Terms
. This
can be used to feed the highlighter with a pre-parsed token
stream. The Terms
must have offsets available. If there are no positions available,
all tokens will have position increments reflecting adjacent tokens, or coincident when terms
share a start offset. If there are stopwords filtered from the index, you probably want to ensure
term vectors have positions so that phrase queries won't match across stopwords.java.lang.IllegalArgumentException
- if no offsets are availablejava.io.IOException
@Deprecated public static TokenStream getTokenStreamWithOffsets(IndexReader reader, int docId, java.lang.String field) throws java.io.IOException
TokenStream
with positions and offsets constructed from
field termvectors. If the field has no termvectors or offsets
are not included in the termvector, return null. See getTokenStream(org.apache.lucene.index.Terms)
for an explanation of what happens when positions aren't present.reader
- the IndexReader
to retrieve term vectors fromdocId
- the document to retrieve termvectors forfield
- the field to retrieve termvectors forTokenStream
, or null if offsets are not availablejava.io.IOException
- If there is a low-level I/O errorgetTokenStream(org.apache.lucene.index.Terms)
@Deprecated public static TokenStream getTokenStream(IndexReader reader, int docId, java.lang.String field, Analyzer analyzer) throws java.io.IOException
java.io.IOException
@Deprecated public static TokenStream getTokenStream(Document doc, java.lang.String field, Analyzer analyzer)
@Deprecated public static TokenStream getTokenStream(java.lang.String field, java.lang.String contents, Analyzer analyzer)
Copyright © 2000–2019 The Apache Software Foundation. All rights reserved.