Inheritance diagram for HybridToken:
Public Member Functions | |
this (ITokenizer tk, IBuffer buffer) | |
this (ITokenizer tk, IConduit conduit) | |
IBuffer | getHost () |
bool | next () |
Private Member Functions | |
this (ITokenizer tk) | |
IBuffer | getBuffer () |
bool | next (IBuffer buf) |
bool | get () |
Static Private Member Functions | |
this () | |
Private Attributes | |
IBuffer | host |
Compare this to usage of a CompositeToken:
// open a file for reading FileConduit fc = new FileConduit ("test.txt"); // create a Token and bind it to both the file and a line-tokenizer HybridToken line = new HybridToken (Tokenizers.line, fc); // now create a reader upon the token Reader reader = new Reader (line.getHost); // read file a line at a time. Method get() returns false when no more // tokens are found. while (line.get) { int x, y; // reader is now bound to the content of the current line reader.get(x).get(y); Stdout.put(x).put(y).cr(); }
You can use the same mechanism to bind subordinate Tokens:
// open a file for reading FileConduit fc = new FileConduit ("test.txt"); // create a Token and bind it to both the file and a line-tokenizer HybridToken line = new HybridToken (Tokenizers.line, fc); // now create a subordinate Token that splits on whitespace CompositeToken word = new CompositeToken (Tokenizers.space, line.getHost); // read file a line at a time. Method get() returns false when no more // tokens are found. while (line.get) // extract space delimited tokens from each line while (word.get) Stdout.put(word).cr();
Definition at line 355 of file TokenEx.d.
|
Set this token to use the provided Tokenizer, and bind it to the given buffer. Reimplemented from CompositeToken. Definition at line 366 of file TokenEx.d. References IBuffer::create(). |
|
Set this token to use the provided Tokenizer, and bind it to the buffer associated with the given conduit. Reimplemented from CompositeToken. |
|
Return the associated host buffer. The host should be used for purposes of binding a subordinate Token or Reader onto the content of this token. Each call to next() will update this content appropriately, which is also reflected within said host buffer. That is, token.toString() == token.getHost.toString(). |
|
Extract the next token. Returns true if a token was isolated, false if no more tokens were found. Note that one last token may still be present when this return false; this may happen if (for example) the last delimiter is missing before an Eof condition is seen. Check token.getLength() when this method returns false. For example:
Reimplemented from CompositeToken. Definition at line 424 of file TokenEx.d. References IBuffer::setValidContent(), and Token::toString(). |
|
Reimplemented in ReaderToken. Definition at line 81 of file TokenEx.d. References BoundToken::tk. |
|
Construct a static exception instance. Definition at line 141 of file Token.d. References Token::Error. |
|
Return the associated buffer |
|
Extract the next token from the provided buffer. Returns true if a token was isolated, false if no more tokens were found. Note that one last token may still be present when this return false; this may happen if (for example) the last delimiter is missing before an EOF condition is seen. Check token.getLength() when this method returns false. For example:
Definition at line 118 of file TokenEx.d. References ITokenizer::next(), and BoundToken::tk. Referenced by StartLine::VersionUseTokenizer::parse(), HttpParams::parse(), HttpHeaders::parse(), ResponseLine::parse(), testFile2(), and testToken1(). |
|
Extract the next token, taking Eof into consideration. If next() returns false, then this function will still return true as long as there's some content available. For example:
Definition at line 293 of file TokenEx.d. References Token::getLength(), and CompositeToken::next(). Referenced by Properties::load(), testFile1(), testHttpClient(), and testLineRegex(). |
|
|