Main Page | Class Hierarchy | Alphabetical List | Class List | Directories | File List | Class Members | File Members | Related Pages

BufferTokenizer.d File Reference

Go to the source code of this file.

Functions

class BufferTokenizerTemplate (T)
typedef BufferTokenizerTemplate (char) BufferTokenizer

Variables

module mango io BufferTokenizer
import mango text Token
import mango io Buffer
private import mango io model IBuffer
private import mango io model
mango io model 
IConduit


Detailed Description

Copyright (c) 2004 Kris Bell

This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for damages of any kind arising from the use of this software.

Permission is hereby granted to anyone to use this software for any purpose, including commercial applications, and to alter it and/or redistribute it freely, subject to the following restrictions:

1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment within documentation of said product would be appreciated but is not required.

2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.

3. This notice may not be removed or altered from any distribution of the source.

4. Derivative works are permitted, but they must carry this notice in full and credit the original source.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Version:
Initial version, December 2005
Author:
Kris

Definition in file BufferTokenizer.d.


Function Documentation

class BufferTokenizerTemplate  ) 
 

Tokenize input from a buffer or conduit

All input is tokenized from the associated buffer, and exposed as a freachable property via method opApply(). Empty token are passed to the caller ~ this may happen if, for example, an empty line is seen by a line-tokenizer.

Construct a BufferTokenizer on the provided buffer, using the specified Tokenizer instead of the default one.

Construct a BufferTokenizer upon the buffer associated with the given conduit.

Iterate over the set of tokens

Refill the token content from our buffer. Returns false upon reaching EOF

Definition at line 60 of file BufferTokenizer.d.

References Buffer, RawCodec1::buffer, buffer, BufferTokenizerTemplate(), and TokenTemplate().

Referenced by BufferTokenizerTemplate().

typedef BufferTokenizerTemplate char   ) 
 


Variable Documentation

module mango io BufferTokenizer
 

Definition at line 40 of file BufferTokenizer.d.

import mango text Token
 

Definition at line 42 of file BufferTokenizer.d.

import mango io Buffer
 

Definition at line 44 of file BufferTokenizer.d.

private import mango io model IBuffer
 

Definition at line 46 of file BufferTokenizer.d.

private import mango io model mango io model IConduit
 

Definition at line 46 of file BufferTokenizer.d.


Generated on Sat Dec 24 17:28:34 2005 for Mango by  doxygen 1.4.0