Inheritance diagram for HybridToken:
Public Member Functions | |
this (ITokenizer tk, IBuffer buffer) | |
this (ITokenizer tk, IConduit conduit) | |
IBuffer | getHost () |
bool | next () |
Private Member Functions | |
this (ITokenizer tk) | |
IBuffer | getBuffer () |
bool | next (IBuffer buf) |
bool | get () |
Private Attributes | |
IBuffer | host |
Compare this to usage of a CompositeToken:
// open a file for reading FileConduit fc = new FileConduit ("test.txt"); // create a Token and bind it to both the file and a line-tokenizer HybridToken line = new HybridToken (Tokenizers.line, fc); // now create a reader upon the token Reader input = new Reader (line.getHost); // read file a line at a time. Method get() returns false when no more // tokens are found. while (line.get) { int x, y; // reader is now bound to the content of the current line input (x) (y); Stdout (x) (y) (CR); }
You can use the same mechanism to bind subordinate Tokens:
// open a file for reading FileConduit fc = new FileConduit ("test.txt"); // create a Token and bind it to both the file and a line-tokenizer HybridToken line = new HybridToken (Tokenizers.line, fc); // now create a subordinate Token that splits on whitespace CompositeToken word = new CompositeToken (Tokenizers.space, line.getHost); // read file a line at a time. Method get() returns false when no more // tokens are found. while (line.get) // extract space delimited tokens from each line while (word.get) Stdout (word) (CR);
Definition at line 572 of file Token.d.
|
Set this token to use the provided Tokenizer, and bind it to the given buffer. Reimplemented from CompositeToken. Definition at line 583 of file Token.d. References IBuffer::create(), and host. |
|
Set this token to use the provided Tokenizer, and bind it to the buffer associated with the given conduit. Reimplemented from CompositeToken. |
|
Return the associated host buffer. The host should be used for purposes of binding a subordinate Token or Reader onto the content of this token. Each call to next() will update this content appropriately, which is also reflected within said host buffer. That is, token.toString == token.getHost.toString. Definition at line 615 of file Token.d. References host. |
|
Extract the next token. Returns true if a token was isolated, false if no more tokens were found. Note that one last token may still be present when this return false; this may happen if (for example) the last delimiter is missing before an Eof condition is seen. Check token.getLength() when this method returns false. For example:
while (token.next || token.getLength) // do something Reimplemented from CompositeToken. Definition at line 641 of file Token.d. References host, and IBuffer::setValidContent(). |
|
Reimplemented in ReaderToken. Definition at line 298 of file Token.d. References BoundToken::tk. |
|
Return the associated buffer Definition at line 463 of file Token.d. References CompositeToken::buffer. |
|
Extract the next token from the provided buffer. Returns true if a token was isolated, false if no more tokens were found. Note that one last token may still be present when this return false; this may happen if (for example) the last delimiter is missing before an EOF condition is seen. Check token.getLength() when this method returns false. For example:
while (token.next() || token.getLength()) // do something Definition at line 335 of file Token.d. References ITokenizer::next(), and BoundToken::tk. Referenced by StartLine::VersionUseTokenizer::parse(), HttpParams::parse(), HttpHeaders::parse(), ResponseLine::parse(), testFile2(), and testToken1(). |
|
Extract the next token, taking Eof into consideration. If next() returns false, then this function will still return true as long as there's some content available. For example:
while (token.get) // do something Definition at line 510 of file Token.d. References CompositeToken::next(). Referenced by Properties::load(), testFile1(), and testHttpClient(). |
|
|