Main Page | Class Hierarchy | Alphabetical List | Class List | File List | Class Members | File Members | Related Pages

CompositeToken Class Reference

Inheritance diagram for CompositeToken:

BoundToken HybridToken List of all members.

Public Member Functions

 this (ITokenizer tk, IBuffer buffer)
 this (ITokenizer tk, IConduit conduit)
IBuffer getBuffer ()
bool next ()
bool get ()

Private Member Functions

 this (ITokenizer tk)
ITokenizer getTokenizer ()
bool next (IBuffer buf)

Static Private Member Functions

 this ()

Private Attributes

IBuffer buffer

Detailed Description

Another subclass of BoundToken that combines both a Tokenizer and an input buffer. This is simply a convenience wrapper than takes care of details that would otherwise clutter the client code.

Compare this to usage of a basic Token:

// open a file for reading FileConduit fc = new FileConduit ("test.txt"); // create a Token and bind it to both the file and a line-tokenizer CompositeToken line = new CompositeToken (Tokenizers.line, fc); // read file a line at a time. Method get() returns false when no more // tokens are found. while (line.get) Stdout.put(line).cr();

You might also consider a HybridToken for further processing of token content.

Definition at line 211 of file TokenEx.d.


Member Function Documentation

this ITokenizer  tk,
IBuffer  buffer
[inline]
 

Set this token to use the provided Tokenizer, and bind it to the given buffer.

Reimplemented in HybridToken.

Definition at line 222 of file TokenEx.d.

this ITokenizer  tk,
IConduit  conduit
[inline]
 

Set this token to use the provided Tokenizer, and bind it to the buffer associated with the given conduit.

Reimplemented in HybridToken.

Definition at line 235 of file TokenEx.d.

IBuffer getBuffer  )  [inline]
 

Return the associated buffer

Definition at line 246 of file TokenEx.d.

bool next  )  [inline]
 

Extract the next token.

Returns true if a token was isolated, false if no more tokens were found. Note that one last token may still be present when this return false; this may happen if (for example) the last delimiter is missing before an Eof condition is seen. Check token.getLength() when this method returns false.

For example:

while (token.next || token.getLength) // do something

Reimplemented in HybridToken.

Definition at line 272 of file TokenEx.d.

Referenced by get(), HttpClient::open(), HttpRequest::readHeaders(), and wordRegex().

bool get  )  [inline]
 

Extract the next token, taking Eof into consideration. If next() returns false, then this function will still return true as long as there's some content available.

For example:

while (token.get) // do something

Definition at line 293 of file TokenEx.d.

References Token::getLength(), and next().

Referenced by Properties::load(), testFile1(), testHttpClient(), and testLineRegex().

this ITokenizer  tk  )  [inline, inherited]
 

Reimplemented in ReaderToken.

Definition at line 81 of file TokenEx.d.

References BoundToken::tk.

this  )  [inline, static, inherited]
 

Construct a static exception instance.

Definition at line 141 of file Token.d.

References Token::Error.

ITokenizer getTokenizer  )  [inline, inherited]
 

Return the associated tokenizer

Definition at line 92 of file TokenEx.d.

References BoundToken::tk.

bool next IBuffer  buf  )  [inline, inherited]
 

Extract the next token from the provided buffer.

Returns true if a token was isolated, false if no more tokens were found. Note that one last token may still be present when this return false; this may happen if (for example) the last delimiter is missing before an EOF condition is seen. Check token.getLength() when this method returns false.

For example:

while (token.next() || token.getLength()) // do something

Definition at line 118 of file TokenEx.d.

References ITokenizer::next(), and BoundToken::tk.

Referenced by StartLine::VersionUseTokenizer::parse(), HttpParams::parse(), HttpHeaders::parse(), ResponseLine::parse(), testFile2(), and testToken1().


Member Data Documentation

IBuffer buffer [private]
 

Definition at line 213 of file TokenEx.d.


The documentation for this class was generated from the following file:
Generated on Sun Nov 7 19:07:01 2004 for Mango by doxygen 1.3.6