Package com.ibm.icu.impl.data
Class TokenIterator
java.lang.Object
com.ibm.icu.impl.data.TokenIterator
An iterator class that returns successive string tokens from some
source. String tokens are, in general, separated by Pattern_White_Space
in the source test. Furthermore, they may be delimited by
either single or double quotes (opening and closing quotes must
match). Escapes are processed using standard ICU unescaping.
2015-sep-03 TODO: Only used in com.ibm.icu.dev.test.format, move there.
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate StringBuffer
private boolean
private int
private String
private int
private ResourceReader
-
Constructor Summary
ConstructorsConstructorDescriptionConstruct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#'). -
Method Summary
Modifier and TypeMethodDescriptionReturn a string description of the position of the last line returned by readLine() or readLineSkippingComments().int
Return the one-based line number of the line of the last token returned by next().next()
Return the next token from this iterator, or null if the last token has been returned.private int
nextToken
(int position) Read the next token from 'this.line' and append it to 'this.buf'.
-
Field Details
-
reader
-
line
-
buf
-
done
private boolean done -
pos
private int pos -
lastpos
private int lastpos
-
-
Constructor Details
-
TokenIterator
Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#'). Note that trailing comments on a line, beginning with the first unquoted '#', are recognized.
-
-
Method Details
-
next
Return the next token from this iterator, or null if the last token has been returned.- Throws:
IOException
-
getLineNumber
public int getLineNumber()Return the one-based line number of the line of the last token returned by next(). Should only be called after a call to next(); otherwise the return value is undefined. -
describePosition
Return a string description of the position of the last line returned by readLine() or readLineSkippingComments(). -
nextToken
private int nextToken(int position) Read the next token from 'this.line' and append it to 'this.buf'. Tokens are separated by Pattern_White_Space. Tokens may also be delimited by double or single quotes. The closing quote must match the opening quote. If a '#' is encountered, the rest of the line is ignored, unless it is backslash-escaped or within quotes.- Parameters:
position
- the offset into the string- Returns:
- offset to the next character to read from line, or if the end of the line is reached without scanning a valid token, -1
-