Main Page | Class Hierarchy | Alphabetical List | Class List | Directories | File List | Class Members | File Members | Related Pages

Reader.d

Go to the documentation of this file.
00001 /*******************************************************************************
00002 
00003         @file Reader.d
00004         
00005         Copyright (c) 2004 Kris Bell
00006         
00007         This software is provided 'as-is', without any express or implied
00008         warranty. In no event will the authors be held liable for damages
00009         of any kind arising from the use of this software.
00010         
00011         Permission is hereby granted to anyone to use this software for any 
00012         purpose, including commercial applications, and to alter it and/or 
00013         redistribute it freely, subject to the following restrictions:
00014         
00015         1. The origin of this software must not be misrepresented; you must 
00016            not claim that you wrote the original software. If you use this 
00017            software in a product, an acknowledgment within documentation of 
00018            said product would be appreciated but is not required.
00019 
00020         2. Altered source versions must be plainly marked as such, and must 
00021            not be misrepresented as being the original software.
00022 
00023         3. This notice may not be removed or altered from any distribution
00024            of the source.
00025 
00026         4. Derivative works are permitted, but they must carry this notice
00027            in full and credit the original source.
00028 
00029 
00030                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00031 
00032 
00033         @version        Initial version, October 2004      
00034 
00035         @author         Kris
00036 
00037 
00038 *******************************************************************************/
00039 
00040 module mango.io.Reader;
00041 
00042 private import  mango.convert.Type;
00043 
00044 private import  mango.io.Exception,
00045                 mango.io.ArrayAllocator;
00046 
00047 public  import  mango.io.Buffer;
00048 
00049 public  import  mango.io.model.IReader,
00050                 mango.io.model.IBuffer,
00051                 mango.io.model.IConduit;
00052 
00053 
00054 /*******************************************************************************
00055 
00056         Reader base-class. Each reader operates upon an IBuffer, which is
00057         provided at construction time. Said buffer is intended to remain 
00058         consistent over the reader lifetime.
00059 
00060         All readers support the full set of native data types, plus a full
00061         selection of array types. The latter can be configured to produce
00062         either a copy (.dup) of the buffer content, or a slice. See class
00063         SimpleAllocator, BufferAllocator and SliceAllocator for more on 
00064         this topic.
00065 
00066         Readers support a C++ iostream type syntax, along with Java-esque 
00067         get() notation. However, the Mango style is to place IO elements 
00068         within their own parenthesis, like so:
00069         
00070                 int count;
00071                 char[] verse;
00072 
00073                 read (verse) (count);
00074 
00075         Note that each element is distict; this enables "strong typing", 
00076         which should catch any typo errors at compile-time. The style is
00077         affectionately called "whisper".
00078 
00079         The code below illustrates basic operation upon a memory buffer:
00080         
00081         @code
00082         Buffer buf = new Buffer (256);
00083 
00084         // map same buffer into both reader and writer
00085         IReader read = new Reader (buf);
00086         IWriter write = new Writer (buf);
00087 
00088         int i = 10;
00089         long j = 20;
00090         double d = 3.14159;
00091         char[] c = "fred";
00092 
00093         // write data types out
00094         write (c) (i) (j) (d);
00095 
00096         // read them back again
00097         read (c) (i) (j) (d);
00098 
00099         // reset
00100         buf.clear();
00101 
00102         // same thing again, but using iostream syntax instead
00103         write << c << i << j << d;
00104 
00105         // read them back again
00106         read >> c >> i >> j >> d;
00107 
00108         // reset
00109         buf.clear();
00110 
00111         // same thing again, but using put() syntax instead
00112         write.put(c).put(i).put(j).put(d);
00113         read.get(c).get(i).get(j).get(d);
00114 
00115         @endcode
00116 
00117         Note that certain Readers, such as the basic binary implementation, 
00118         expect to retrieve the number of array elements from the source. 
00119         For example; when reading an array from a file, the number of elements 
00120         is read from the file also. If the content is not arranged in such a 
00121         manner, you may specify how many elements to read via a second argument:
00122 
00123         @code
00124                 read (myArray, 11);
00125         @endcode
00126 
00127         Readers may also be used with any class implementing the IReadable
00128         interface. See PickleReader for an example of how this can be put
00129         to good use.
00130 
00131         Lastly, each Reader may be configured with a text decoder. These
00132         decoders convert between an internal text representation, and the
00133         char/wchar/dchar representaion. BufferCodec.d contains classes for
00134         handling utf8, utf16, and utf32. The icu.UMango module has support
00135         for a wide variety of converters.
00136         
00137 *******************************************************************************/
00138 
00139 class Reader : IReader, IArrayAllocator 
00140 {       
00141         // the buffer associated with this reader. Note that this
00142         // must not change for the lifetime of the reader, since
00143         // it is assumed to be immutable elsewhere 
00144         protected IBuffer               buffer;         
00145 
00146         // memory-manager for array requests
00147         private IArrayAllocator         memory;
00148 
00149         // string decoder
00150         private IBuffer.Converter       textDecoder;
00151 
00152         /***********************************************************************
00153         
00154                 Construct a Reader upon the provided buffer
00155 
00156         ***********************************************************************/
00157 
00158         this (IBuffer buffer)
00159         {
00160                 this.buffer = buffer;
00161                 textDecoder = &read;
00162 
00163                 setAllocator (this);
00164 
00165                 Buffer.Style s = buffer.getStyle;
00166                 if (s != Buffer.Mixed)
00167                     if ((s == Buffer.Text) ^ isTextBased())
00168                          buffer.error ("text/binary mismatch between Reader and Buffer");
00169         }
00170 
00171         /***********************************************************************
00172                 
00173                 Construct a Reader upon the buffer associated with the
00174                 given conduit.
00175 
00176         ***********************************************************************/
00177 
00178         this (IConduit conduit)
00179         {
00180                 this (new Buffer(conduit));
00181         }
00182 
00183         /***********************************************************************
00184         
00185                 Return the buffer associated with this reader
00186 
00187         ***********************************************************************/
00188 
00189         final IBuffer getBuffer ()
00190         {
00191                 return buffer;
00192         }
00193         
00194         /***********************************************************************
00195         
00196                 Is this Reader text oriented?
00197 
00198         ***********************************************************************/
00199 
00200         bool isTextBased()
00201         {
00202                 return false;
00203         }
00204 
00205         /***********************************************************************
00206         
00207                 Return the allocator associated with this reader. See 
00208                 ArrayAllocator for more information.
00209 
00210         ***********************************************************************/
00211 
00212         final IArrayAllocator getAllocator ()
00213         {
00214                 return memory;
00215         }
00216 
00217         /***********************************************************************
00218         
00219                 Set the allocator to use for array management. Arrays are
00220                 always allocated by the IReader. That is, you cannot read
00221                 data into an array slice (for example). Instead, a number
00222                 of IArrayAllocator classes are available to manage memory
00223                 allocation when reading array content. 
00224 
00225                 By default, an IReader will allocate each array from the 
00226                 heap. You can change that behavior by calling this method
00227                 with an IArrayAllocator of choice. For instance, there 
00228                 is a BufferAllocator which will slice an array directly 
00229                 from the buffer where possible. Also available is the 
00230                 record-oriented SliceAllocator, which slices memory from 
00231                 within a pre-allocated heap area, and should be reset by
00232                 the client code after each record has been read (to avoid 
00233                 unnecessary growth).
00234 
00235                 See ArrayAllocator for more information.
00236 
00237         ***********************************************************************/
00238 
00239         final void setAllocator (IArrayAllocator memory) 
00240         {
00241                 memory.bind (this);
00242                 this.memory = memory;
00243         }
00244 
00245         /***********************************************************************
00246         
00247                 Bind an IDecoder to the writer. Decoders are intended to
00248                 be used as a conversion mechanism between various character
00249                 representations (encodings).
00250 
00251         ***********************************************************************/
00252 
00253         final void setDecoder (AbstractDecoder d) 
00254         {
00255                 d.bind (buffer);
00256                 this.textDecoder = &d.decoder;
00257         }
00258 
00259         /***********************************************************************
00260         
00261                 Wait for something to arrive in the buffer. This may stall
00262                 the current thread forever, although usage of SocketConduit 
00263                 will take advantage of the timeout facilities provided there.
00264 
00265         ***********************************************************************/
00266 
00267         final void wait ()
00268         {       
00269                 buffer.get (1, false);
00270         }
00271 
00272         /***********************************************************************
00273         
00274                 Extract a readable class from the current read-position
00275                 
00276         ***********************************************************************/
00277 
00278         final IReader get (IReadable x) 
00279         {
00280                 assert (x);
00281                 x.read (this); 
00282                 return this;
00283         }
00284 
00285         /***********************************************************************
00286 
00287                 Extract a boolean value from the current read-position  
00288                 
00289         ***********************************************************************/
00290 
00291         IReader get (inout bool x)
00292         {
00293                 read (&x, x.sizeof, Type.Bool);
00294                 return this;
00295         }
00296 
00297         /***********************************************************************
00298 
00299                 Extract an unsigned byte value from the current read-position   
00300                                 
00301         ***********************************************************************/
00302 
00303         IReader get (inout ubyte x) 
00304         {       
00305                 read (&x, x.sizeof, Type.UByte);
00306                 return this;
00307         }
00308 
00309         /***********************************************************************
00310         
00311                 Extract a byte value from the current read-position
00312                 
00313         ***********************************************************************/
00314 
00315         IReader get (inout byte x)
00316         {
00317                 read (&x, x.sizeof, Type.Byte);
00318                 return this;
00319         }
00320 
00321         /***********************************************************************
00322         
00323                 Extract an unsigned short value from the current read-position
00324                 
00325         ***********************************************************************/
00326 
00327         IReader get (inout ushort x)
00328         {
00329                 read (&x, x.sizeof, Type.UShort);
00330                 return this;
00331         }
00332 
00333         /***********************************************************************
00334         
00335                 Extract a short value from the current read-position
00336                 
00337         ***********************************************************************/
00338 
00339         IReader get (inout short x)
00340         {
00341                 read (&x, x.sizeof, Type.Short);
00342                 return this;
00343         }
00344 
00345         /***********************************************************************
00346         
00347                 Extract a unsigned int value from the current read-position
00348                 
00349         ***********************************************************************/
00350 
00351         IReader get (inout uint x)
00352         {
00353                 read (&x, x.sizeof, Type.UInt);
00354                 return this;
00355         }
00356 
00357         /***********************************************************************
00358         
00359                 Extract an int value from the current read-position
00360                 
00361         ***********************************************************************/
00362 
00363         IReader get (inout int x)
00364         {
00365                 read (&x, x.sizeof, Type.Int);
00366                 return this;
00367         }
00368 
00369         /***********************************************************************
00370         
00371                 Extract an unsigned long value from the current read-position
00372                 
00373         ***********************************************************************/
00374 
00375         IReader get (inout ulong x)
00376         {
00377                 read (&x, x.sizeof, Type.ULong);
00378                 return this;
00379         }
00380 
00381         /***********************************************************************
00382         
00383                 Extract a long value from the current read-position
00384                 
00385         ***********************************************************************/
00386 
00387         IReader get (inout long x)
00388         {
00389                 read (&x, x.sizeof, Type.Long);
00390                 return this;
00391         }
00392 
00393         /***********************************************************************
00394         
00395                 Extract a float value from the current read-position
00396                 
00397         ***********************************************************************/
00398 
00399         IReader get (inout float x)
00400         {
00401                 read (&x, x.sizeof, Type.Float);
00402                 return this;
00403         }
00404 
00405         /***********************************************************************
00406         
00407                 Extract a double value from the current read-position
00408                 
00409         ***********************************************************************/
00410 
00411         IReader get (inout double x)
00412         {
00413                 read (&x, x.sizeof, Type.Double);
00414                 return this;
00415         }
00416 
00417         /***********************************************************************
00418         
00419                 Extract a real value from the current read-position
00420                 
00421         ***********************************************************************/
00422 
00423         IReader get (inout real x)
00424         {
00425                 read (&x, x.sizeof, Type.Real);
00426                 return this;
00427         }
00428 
00429         /***********************************************************************
00430         
00431                 Extract a char value from the current read-position
00432                 
00433         ***********************************************************************/
00434 
00435         IReader get (inout char x)
00436         {
00437                 return decode (&x, x.sizeof, Type.Utf8);
00438         }
00439 
00440         /***********************************************************************
00441         
00442                 Extract a wide char value from the current read-position
00443                 
00444         ***********************************************************************/
00445 
00446         IReader get (inout wchar x)
00447         {
00448                 return decode (&x, x.sizeof, Type.Utf16);
00449         }
00450 
00451         /***********************************************************************
00452         
00453                 Extract a double char value from the current read-position
00454                 
00455         ***********************************************************************/
00456 
00457         IReader get (inout dchar x)
00458         {
00459                 return decode (&x, x.sizeof, Type.Utf32);
00460         }
00461 
00462         /***********************************************************************
00463 
00464                 Extract an unsigned byte array from the current read-position   
00465                                 
00466         ***********************************************************************/
00467 
00468         IReader get (inout ubyte[] x, uint elements = uint.max) 
00469         {
00470                 memory.allocate (cast(void[]*) &x, count(elements)*ubyte.sizeof, 
00471                                  ubyte.sizeof, Type.UByte, &read);
00472                 return this;
00473         }
00474 
00475         /***********************************************************************
00476         
00477                 Extract a byte array from the current read-position
00478                 
00479         ***********************************************************************/
00480 
00481         IReader get (inout byte[] x, uint elements = uint.max)
00482         {
00483                 memory.allocate (cast(void[]*) &x, count(elements)*byte.sizeof, 
00484                                  byte.sizeof, Type.Byte, &read);
00485                 return this;
00486         }
00487 
00488         /***********************************************************************
00489         
00490                 Extract an unsigned short array from the current read-position
00491                 
00492         ***********************************************************************/
00493 
00494         IReader get (inout ushort[] x, uint elements = uint.max)
00495         {
00496                 memory.allocate (cast(void[]*) &x, count(elements)*ushort.sizeof, 
00497                                  ushort.sizeof, Type.UShort, &read);
00498                 return this;
00499         }
00500 
00501         /***********************************************************************
00502         
00503                 Extract a short array from the current read-position
00504                 
00505         ***********************************************************************/
00506 
00507         IReader get (inout short[] x, uint elements = uint.max)
00508         {
00509                 memory.allocate (cast(void[]*) &x, count(elements)*short.sizeof, 
00510                                  short.sizeof, Type.Short, &read);
00511                 return this;
00512         }
00513 
00514         /***********************************************************************
00515         
00516                 Extract a unsigned int array from the current read-position
00517                 
00518         ***********************************************************************/
00519 
00520         IReader get (inout uint[] x, uint elements = uint.max)
00521         {
00522                 memory.allocate (cast(void[]*) &x, count(elements)*uint.sizeof, 
00523                                  uint.sizeof, Type.UInt, &read);
00524                 return this;
00525         }
00526 
00527         /***********************************************************************
00528         
00529                 Extract an int array from the current read-position
00530                 
00531         ***********************************************************************/
00532 
00533         IReader get (inout int[] x, uint elements = uint.max)
00534         {
00535                 memory.allocate (cast(void[]*) &x, count(elements)*int.sizeof, 
00536                                  int.sizeof, Type.Int, &read);
00537                 return this;
00538         }
00539 
00540         /***********************************************************************
00541         
00542                 Extract an unsigned long array from the current read-position
00543                 
00544         ***********************************************************************/
00545 
00546         IReader get (inout ulong[] x, uint elements = uint.max)
00547         {
00548                 memory.allocate (cast(void[]*) &x, count(elements)*ulong.sizeof, 
00549                                  ulong.sizeof, Type.ULong, &read);
00550                 return this;
00551         }
00552 
00553         /***********************************************************************
00554         
00555                 Extract a long array from the current read-position
00556                 
00557         ***********************************************************************/
00558 
00559         IReader get (inout long[] x, uint elements = uint.max)
00560         {
00561                 memory.allocate (cast(void[]*) &x, count(elements)*long.sizeof, 
00562                                  long.sizeof, Type.Long, &read);
00563                 return this;
00564         }
00565 
00566         /***********************************************************************
00567         
00568                 Extract a float array from the current read-position
00569                 
00570         ***********************************************************************/
00571 
00572         IReader get (inout float[] x, uint elements = uint.max)
00573         {
00574                 memory.allocate (cast(void[]*) &x, count(elements)*float.sizeof, 
00575                                  float.sizeof, Type.Float, &read);
00576                 return this;
00577         }
00578 
00579         /***********************************************************************
00580         
00581                 Extract a double array from the current read-position
00582                 
00583         ***********************************************************************/
00584 
00585         IReader get (inout double[] x, uint elements = uint.max)
00586         {
00587                 memory.allocate (cast(void[]*) &x, count(elements)*double.sizeof, 
00588                                  double.sizeof, Type.Double, &read);
00589                 return this;
00590         }
00591 
00592         /***********************************************************************
00593         
00594                 Extract a real array from the current read-position
00595                 
00596         ***********************************************************************/
00597 
00598         IReader get (inout real[] x, uint elements = uint.max)
00599         {
00600                 memory.allocate (cast(void[]*) &x, count(elements)*real.sizeof, 
00601                                  real.sizeof, Type.Real, &read);
00602                 return this;
00603         }
00604 
00605         /***********************************************************************
00606         
00607         ***********************************************************************/
00608 
00609         IReader get (inout char[] x, uint elements = uint.max)
00610         {
00611                 return decodeArray (cast(void[]*) &x, count(elements)*char.sizeof, 
00612                                     char.sizeof, Type.Utf8);
00613         }
00614 
00615         /***********************************************************************
00616         
00617         ***********************************************************************/
00618 
00619         IReader get (inout wchar[] x, uint elements = uint.max)
00620         {
00621                 return decodeArray (cast(void[]*) &x, count(elements)*wchar.sizeof, 
00622                                     wchar.sizeof, Type.Utf16);
00623         }
00624 
00625         /***********************************************************************
00626         
00627         ***********************************************************************/
00628 
00629         IReader get (inout dchar[] x, uint elements = uint.max)
00630         {
00631                 return decodeArray (cast(void[]*) &x, count(elements)*dchar.sizeof, 
00632                                     dchar.sizeof, Type.Utf32);
00633         }
00634 
00635         /***********************************************************************
00636         
00637         ***********************************************************************/
00638 
00639         protected IReader decode (void* x, uint bytes, uint type)
00640         {
00641                 textDecoder (x, bytes, type);
00642                 return this;
00643         }
00644 
00645         /***********************************************************************
00646         
00647         ***********************************************************************/
00648 
00649         protected IReader decodeArray (void[]* x, uint bytes, uint width, uint type)
00650         {
00651                 memory.allocate (x, bytes, width, type, textDecoder);
00652                 return this;
00653         }
00654 
00655         /***********************************************************************
00656         
00657         ***********************************************************************/
00658 
00659         protected uint read (void *dst, uint bytes, uint type)
00660         {
00661                 uint i = bytes;
00662                 while (i)
00663                       {
00664                       // get as much as there is available in the buffer
00665                       uint available = buffer.readable();
00666                       
00667                       // cap bytes read
00668                       if (available > i)
00669                           available = i;
00670 
00671                       // copy them over
00672                       dst[0..available] = buffer.get (available);
00673 
00674                       // bump counters
00675                       dst += available;
00676                       i -= available;
00677 
00678                       // if we need more, prime the input by reading
00679                       if (i)
00680                           if (buffer.fill () == IConduit.Eof)
00681                               buffer.error ("end of input");
00682                       }
00683                 return bytes;
00684         }
00685 
00686         /***********************************************************************
00687         
00688                 Read and return an integer from the input stream. This is
00689                 used to extract the element count of a subsequent array.
00690 
00691         ***********************************************************************/
00692 
00693         protected uint count (uint elements)
00694         {
00695                 if (elements == uint.max)
00696                     get (elements);
00697                 return elements;
00698         }
00699 
00700         /***********************************************************************
00701         
00702                 IArrayAllocator method
00703                                         
00704         ***********************************************************************/
00705 
00706         protected final void reset ()
00707         {
00708         }
00709 
00710         /***********************************************************************
00711         
00712                 IArrayAllocator method
00713                                         
00714         ***********************************************************************/
00715 
00716         protected final void bind (IReader reader)
00717         {
00718         }
00719 
00720         /***********************************************************************
00721         
00722                 IArrayAllocator method
00723                                         
00724         ***********************************************************************/
00725 
00726         protected final bool isMutable (void* x)
00727         {
00728                 return true;
00729         }
00730         
00731         /***********************************************************************
00732         
00733                 IArrayAllocator method
00734                                         
00735         ***********************************************************************/
00736 
00737         protected final void allocate (void[]* x, uint bytes, uint width, uint type, IBuffer.Converter decoder)
00738         {       
00739                 void[] tmp = new void [bytes];
00740                 *x = tmp [0 .. decoder (tmp, bytes, type) / width];
00741         }
00742 }

Generated on Sat Dec 24 17:28:33 2005 for Mango by  doxygen 1.4.0