Static Public Member Functions | |
this () | |
Static Public Attributes | |
LineScanner | eol |
LineTokenizer | line |
SpaceTokenizer | space |
PunctTokenizer | punct |
SimpleTokenizer | comma |
Note that these Tokenizers do not maintain any state of their own. Thus they are all thread-safe.
Definition at line 481 of file Tokenizer.d.
|
Make a few common tokenizers available as singletons Definition at line 495 of file Tokenizer.d. |
|
Definition at line 483 of file Tokenizer.d. Referenced by this(). |
|
Definition at line 484 of file Tokenizer.d. |
|
Definition at line 485 of file Tokenizer.d. Referenced by this(). |
|
Definition at line 486 of file Tokenizer.d. Referenced by this(). |
|
Definition at line 487 of file Tokenizer.d. Referenced by this(). |