1N/Aperliol - C API for Perl's implementation of IO in Layers.
1N/A /* Defining a layer ... */
1N/AThis document describes the behavior and implementation of the PerlIO
1N/Aabstraction described in L<perlapio> when C<USE_PERLIO> is defined (and
1N/A=head2 History and Background
1N/AThe PerlIO abstraction was introduced in perl5.003_02 but languished as
1N/Ajust an abstraction until perl5.7.0. However during that time a number
1N/Aof perl extensions switched to using it, so the API is mostly fixed to
1N/Amaintain (source) compatibility.
1N/AThe aim of the implementation is to provide the PerlIO API in a flexible
1N/Aand platform neutral manner. It is also a trial of an "Object Oriented
1N/AC, with vtables" approach which may be applied to perl6.
1N/A=head2 Basic Structure
1N/APerlIO is a stack of layers.
1N/AThe low levels of the stack work with the low-level operating system
1N/Acalls (file descriptors in C) getting bytes in and out, the higher
1N/Alayers of the stack buffer, filter, and otherwise manipulate the I/O,
1N/Aand return characters (or bytes) to Perl. Terms I<above> and I<below>
1N/Aare used to refer to the relative positioning of the stack layers.
1N/AA layer contains a "vtable", the table of I/O operations (at C level
1N/Aa table of function pointers), and status flags. The functions in the
1N/Avtable implement operations like "open", "read", and "write".
1N/AWhen I/O, for example "read", is requested, the request goes from Perl
1N/Afirst down the stack using "read" functions of each layer, then at the
1N/Abottom the input is requested from the operating system services, then
1N/Athe result is returned up the stack, finally being interpreted as Perl
1N/AThe requests do not necessarily go always all the way down to the
1N/Aoperating system: that's where PerlIO buffering comes into play.
1N/AWhen you do an open() and specify extra PerlIO layers to be deployed,
1N/Athe layers you specify are "pushed" on top of the already existing
1N/Adefault stack. One way to see it is that "operating system is
1N/Aon the left" and "Perl is on the right".
1N/AWhat exact layers are in this default stack depends on a lot of
1N/Athings: your operating system, Perl version, Perl compile time
1N/Aconfiguration, and Perl runtime configuration. See L<PerlIO>,
1N/Abinmode() operates similarly to open(): by default the specified
1N/Alayers are pushed on top of the existing stack.
1N/AHowever, note that even as the specified layers are "pushed on top"
1N/Afor open() and binmode(), this doesn't mean that the effects are
1N/Alimited to the "top": PerlIO layers can be very 'active' and inspect
1N/Aand affect layers also deeper in the stack. As an example there
1N/Ais a layer called "raw" which repeatedly "pops" layers until
1N/Ait reaches the first layer that has declared itself capable of
1N/Ahandling binary data. The "pushed" layers are processed in left-to-right
1N/Asysopen() operates (unsurprisingly) at a lower level in the stack than
1N/Aopen(). For example in UNIX or UNIX-like systems sysopen() operates
1N/Adirectly at the level of file descriptors: in the terms of PerlIO
1N/Alayers, it uses only the "unix" layer, which is a rather thin wrapper
1N/Aon top of the UNIX file descriptors.
1N/A=head2 Layers vs Disciplines
1N/AInitial discussion of the ability to modify IO streams behaviour used
1N/Athe term "discipline" for the entities which were added. This came (I
1N/Abelieve) from the use of the term in "sfio", which in turn borrowed it
1N/Afrom "line disciplines" on Unix terminals. However, this document (and
1N/Athe C code) uses the term "layer".
1N/AThis is, I hope, a natural term given the implementation, and should
1N/Aavoid connotations that are inherent in earlier uses of "discipline"
1N/Afor things which are rather different.
1N/A=head2 Data Structures
1N/AThe basic data structure is a PerlIOl:
1N/A typedef struct _PerlIO PerlIOl;
1N/A typedef struct _PerlIO_funcs PerlIO_funcs;
1N/A typedef PerlIOl *PerlIO;
1N/A PerlIOl * next; /* Lower layer */
1N/A PerlIO_funcs * tab; /* Functions for this layer */
1N/A IV flags; /* Various flags for state */
1N/AA C<PerlIOl *> is a pointer to the struct, and the I<application>
1N/Alevel C<PerlIO *> is a pointer to a C<PerlIOl *> -
i.e. a pointer
1N/Ato a pointer to the struct. This allows the application level C<PerlIO *>
1N/Ato remain constant while the actual C<PerlIOl *> underneath
1N/Achanges. (Compare perl's C<SV *> which remains constant while its
1N/AC<sv_any> field changes as the scalar's type changes.) An IO stream is
1N/Athen in general represented as a pointer to this linked-list of
1N/AIt should be noted that because of the double indirection in a C<PerlIO *>,
1N/Aa C<< &(perlio->next) >> "is" a C<PerlIO *>, and so to some degree
1N/Aat least one layer can use the "standard" API on the next layer down.
1N/AA "layer" is composed of two parts:
1N/AThe functions and attributes of the "layer class".
1N/AThe per-instance data for a particular handle.
1N/A=head2 Functions and Attributes
1N/AThe functions and attributes are accessed via the "tab" (for table)
1N/Amember of C<PerlIOl>. The functions (methods of the layer "class") are
1N/Afixed, and are defined by the C<PerlIO_funcs> type. They are broadly the
1N/Asame as the public C<PerlIO_xxxxx> functions:
1N/A struct _PerlIO_funcs
1N/A IV (*Pushed)(pTHX_ PerlIO *f,const char *mode,SV *arg, PerlIO_funcs *tab);
1N/A IV (*Popped)(pTHX_ PerlIO *f);
1N/A PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
1N/A int fd, int imode, int perm,
1N/A int narg, SV **args);
1N/A IV (*Binmode)(pTHX_ PerlIO *f);
1N/A SV * (*Getarg)(pTHX_ PerlIO *f, CLONE_PARAMS *param, int flags)
1N/A IV (*Fileno)(pTHX_ PerlIO *f);
1N/A PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags)
1N/A /* Unix-like functions - cf sfio line disciplines */
1N/A SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
1N/A SSize_t (*Unread)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
1N/A SSize_t (*Write)(pTHX_ PerlIO *f, const void *vbuf, Size_t count);
1N/A IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
1N/A Off_t (*Tell)(pTHX_ PerlIO *f);
1N/A IV (*Close)(pTHX_ PerlIO *f);
1N/A /* Stdio-like buffered IO functions */
1N/A IV (*Flush)(pTHX_ PerlIO *f);
1N/A IV (*Fill)(pTHX_ PerlIO *f);
1N/A IV (*Eof)(pTHX_ PerlIO *f);
1N/A IV (*Error)(pTHX_ PerlIO *f);
1N/A void (*Clearerr)(pTHX_ PerlIO *f);
1N/A void (*Setlinebuf)(pTHX_ PerlIO *f);
1N/A /* Perl's snooping functions */
1N/A STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
1N/A Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
1N/A STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
1N/A SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
1N/A void (*Set_ptrcnt)(pTHX_ PerlIO *f,STDCHAR *ptr,SSize_t cnt);
1N/AThe first few members of the struct give a function table size for
1N/Acompatibility check "name" for the layer, the size to C<malloc> for the per-instance data,
1N/Aand some flags which are attributes of the class as whole (such as whether it is a buffering
1N/Alayer), then follow the functions which fall into four basic groups:
1N/AOpening and setup functions
1N/AStdio class buffering options.
1N/AFunctions to support Perl's traditional "fast" access to the buffer.
1N/AA layer does not have to implement all the functions, but the whole
1N/Atable has to be present. Unimplemented slots can be NULL (which will
1N/Aresult in an error when called) or can be filled in with stubs to
1N/A"inherit" behaviour from a "base class". This "inheritance" is fixed
1N/Afor all instances of the layer, but as the layer chooses which stubs
1N/Ato populate the table, limited "multiple inheritance" is possible.
1N/A=head2 Per-instance Data
1N/AThe per-instance data are held in memory beyond the basic PerlIOl
1N/Astruct, by making a PerlIOl the first member of the layer's struct
1N/A struct _PerlIO base; /* Base "class" info */
1N/A STDCHAR * buf; /* Start of buffer */
1N/A STDCHAR * end; /* End of valid part of buffer */
1N/A STDCHAR * ptr; /* Current position in buffer */
1N/A Off_t posn; /* Offset of buf into the file */
1N/A Size_t bufsiz; /* Real size of buffer */
1N/A IV oneword; /* Emergency buffer */
1N/AIn this way (as for perl's scalars) a pointer to a PerlIOBuf can be
1N/Atreated as a pointer to a PerlIOl.
1N/A=head2 Layers in action.
1N/A +-----------+ +----------+ +--------+
1N/A PerlIO ->| |--->| next |--->| NULL |
1N/A +-----------+ +----------+ +--------+
1N/A | | | buffer | | fd |
1N/A +-----------+ | | +--------+
1N/AThe above attempts to show how the layer scheme works in a simple case.
1N/AThe application's C<PerlIO *> points to an entry in the table(s)
1N/Arepresenting open (allocated) handles. For example the first three slots
1N/Ain the table correspond to C<stdin>,C<stdout> and C<stderr>. The table
1N/Ain turn points to the current "top" layer for the handle - in this case
1N/Aan instance of the generic buffering layer "perlio". That layer in turn
1N/Apoints to the next layer down - in this case the lowlevel "unix" layer.
1N/AThe above is roughly equivalent to a "stdio" buffered stream, but with
1N/Amuch more flexibility:
1N/AIf Unix level C<read>/C<write>/C<lseek> is not appropriate for (say)
1N/Asockets then the "unix" layer can be replaced (at open time or even
1N/Adynamically) with a "socket" layer.
1N/ADifferent handles can have different buffering schemes. The "top"
1N/Alayer could be the "mmap" layer if reading disk files was quicker
1N/Ausing C<mmap> than C<read>. An "unbuffered" stream can be implemented
1N/Asimply by not having a buffer layer.
1N/AExtra layers can be inserted to process the data as it flows through.
1N/AThis was the driving need for including the scheme in perl 5.7.0+ - we
1N/Aneeded a mechanism to allow data to be translated between perl's
1N/Ainternal encoding (conceptually at least Unicode as UTF-8), and the
1N/A"native" format used by the system. This is provided by the
1N/A":encoding(xxxx)" layer which typically sits above the buffering layer.
1N/AA layer can be added that does "\n" to CRLF translation. This layer
1N/Acan be used on any platform, not just those that normally do such
1N/A=head2 Per-instance flag bits
1N/AThe generic flag bits are a hybrid of C<O_XXXXX> style flags deduced
1N/Afrom the mode string passed to C<PerlIO_open()>, and state bits for
1N/Atypical buffer layers.
1N/A=item PERLIO_F_CANWRITE
1N/AWrites are permitted,
i.e. opened as "w" or "r+" or "a", etc.
1N/A=item PERLIO_F_CANREAD
1N/AReads are permitted
i.e. opened "r" or "w+" (or even "a+" - ick).
1N/AAn error has occurred (for C<PerlIO_error()>).
1N/A=item PERLIO_F_TRUNCATE
1N/ATruncate file suggested by open mode.
1N/A=item PERLIO_F_APPEND
1N/AAll writes should be appends.
1N/ALayer is performing Win32-like "\n" mapped to CR,LF for output and CR,LF
1N/Amapped to "\n" for input. Normally the provided "crlf" layer is the only
1N/Alayer that need bother about this. C<PerlIO_binmode()> will mess with this
1N/Aflag rather than
add/remove layers if the C<PERLIO_K_CANCRLF> bit is set
1N/Afor the layers class.
1N/AData written to this layer should be UTF-8 encoded; data provided
1N/Aby this layer should be considered UTF-8 encoded. Can be set on any layer
1N/Aby ":utf8" dummy layer. Also set on ":encoding" layer.
1N/ALayer is unbuffered -
i.e. write to next layer down should occur for
1N/Aeach write to this layer.
1N/AThe buffer for this layer currently holds data written to it but not sent
1N/AThe buffer for this layer currently holds unconsumed data read from
1N/A=item PERLIO_F_LINEBUF
1N/ALayer is line buffered. Write data should be passed to next layer down
1N/Awhenever a "\n" is seen. Any data beyond the "\n" should then be
1N/AFile has been C<unlink()>ed, or should be deleted on C<close()>.
1N/A=item PERLIO_F_FASTGETS
1N/AThis instance of this layer supports the "fast C<gets>" interface.
1N/ANormally set based on C<PERLIO_K_FASTGETS> for the class and by the
1N/Aexistence of the function(s) in the table. However a class that
1N/Anormally provides that interface may need to avoid it on a
1N/Aparticular instance. The "pending" layer needs to do this when
1N/Ait is pushed above a layer which does not support the interface.
1N/A(Perl's C<sv_gets()> does not expect the streams fast C<gets> behaviour
1N/Ato change during one "get".)
1N/A=head2 Methods in Detail
1N/ASize of the function table. This is compared against the value PerlIO
1N/Acode "knows" as a compatibility check. Future versions I<may> be able
1N/Ato tolerate layers compiled against an old version of the headers.
1N/AThe name of the layer whose open() method Perl should invoke on
1N/Aopen(). For example if the layer is called APR, you will call:
1N/A open $fh, ">:APR", ...
1N/Aand Perl knows that it has to invoke the PerlIOAPR_open() method
1N/Aimplemented by the APR layer.
1N/AThe size of the per-instance data structure,
e.g.:
1N/AIf this field is zero then C<PerlIO_pushed> does not malloc anything
1N/Aand assumes layer's Pushed function will do any required layer stack
1N/AIf the field is non-zero it must be at least the size of C<PerlIOl>,
1N/AC<PerlIO_pushed> will allocate memory for the layer's data structures
1N/Aand link new layer onto the stream's stack. (If the layer's Pushed
1N/Amethod returns an error indication the layer is popped again.)
1N/A=item * PERLIO_K_BUFFERED
1N/AThe layer is buffered.
1N/AThe layer is acceptable to have in a binmode(FH) stack -
i.e. it does not
1N/A(or will configure itself not to) transform bytes passing through it.
1N/A=item * PERLIO_K_CANCRLF
1N/ALayer can translate between "\n" and CRLF line ends.
1N/A=item * PERLIO_K_FASTGETS
1N/ALayer allows buffer snooping.
1N/A=item * PERLIO_K_MULTIARG
1N/AUsed when the layer's open() accepts more arguments than usual. The
1N/Aextra arguments should come not before the C<MODE> argument. When this
1N/Aflag is used it's up to the layer to validate the args.
1N/A IV (*Pushed)(pTHX_ PerlIO *f,const char *mode, SV *arg);
1N/AThe only absolutely mandatory method. Called when the layer is pushed
1N/Aonto the stack. The C<mode> argument may be NULL if this occurs
1N/Apost-open. The C<arg> will be non-C<NULL> if an argument string was
1N/Apassed. In most cases this should call C<PerlIOBase_pushed()> to
1N/Aconvert C<mode> into the appropriate C<PERLIO_F_XXXXX> flags in
1N/Aaddition to any actions the layer itself takes. If a layer is not
1N/Aexpecting an argument it need neither save the one passed to it, nor
1N/Aprovide C<Getarg()> (it could perhaps C<Perl_warn> that the argument
1N/AReturns 0 on success. On failure returns -1 and should set errno.
1N/A IV (*Popped)(pTHX_ PerlIO *f);
1N/ACalled when the layer is popped from the stack. A layer will normally
1N/Abe popped after C<Close()> is called. But a layer can be popped
1N/Awithout being closed if the program is dynamically managing layers on
1N/Athe stream. In such cases C<Popped()> should free any resources
1N/A(buffers, translation tables, ...) not held directly in the layer's
1N/Astruct. It should also C<Unread()> any unconsumed data that has been
1N/Aread and buffered from the layer below back to that layer, so that it
1N/Acan be re-provided to what ever is now above.
1N/AReturns 0 on success and failure. If C<Popped()> returns I<true> then
1N/AI<
perlio.c> assumes that either the layer has popped itself, or the
1N/Alayer is super special and needs to be retained for other reasons.
1N/AIn most cases it should return I<false>.
1N/A PerlIO * (*Open)(...);
1N/AThe C<Open()> method has lots of arguments because it combines the
1N/Afunctions of perl's C<open>, C<PerlIO_open>, perl's C<sysopen>,
1N/AC<PerlIO_fdopen> and C<PerlIO_reopen>. The full prototype is as
1N/A PerlIO * (*Open)(pTHX_ PerlIO_funcs *tab,
1N/A int fd, int imode, int perm,
1N/A int narg, SV **args);
1N/AOpen should (perhaps indirectly) call C<PerlIO_allocate()> to allocate
1N/Aa slot in the table and associate it with the layers information for
1N/Athe opened file, by calling C<PerlIO_push>. The I<layers> AV is an
1N/Aarray of all the layers destined for the C<PerlIO *>, and any
1N/Aarguments passed to them, I<n> is the index into that array of the
1N/Alayer being called. The macro C<PerlIOArg> will return a (possibly
1N/AC<NULL>) SV * for the argument passed to the layer.
1N/AThe I<mode> string is an "C<fopen()>-like" string which would match
1N/Athe regular expression C</^[I#]?[rwa]\+?[bt]?$/>.
1N/AThe C<'I'> prefix is used during creation of C<stdin>
..C<stderr> via
1N/Aspecial C<PerlIO_fdopen> calls; the C<'#'> prefix means that this is
1N/AC<sysopen> and that I<imode> and I<perm> should be passed to
1N/AC<PerlLIO_open3>; C<'r'> means B<r>ead, C<'w'> means B<w>rite and
1N/AC<'a'> means B<a>ppend. The C<'+'> suffix means that both reading and
1N/Abe binary, and C<'t'> means it is text. (Almost all layers should do
1N/Athe IO in binary mode, and ignore the b/t bits. The C<:crlf> layer
1N/Ashould be pushed to handle the distinction.)
1N/AIf I<old> is not C<NULL> then this is a C<PerlIO_reopen>. Perl itself
1N/Adoes not use this (yet?) and semantics are a little vague.
1N/AIf I<fd> not negative then it is the numeric file descriptor I<fd>,
1N/Awhich will be open in a manner compatible with the supplied mode
1N/Astring, the call is thus equivalent to C<PerlIO_fdopen>. In this case
1N/AI<nargs> will be zero.
1N/AIf I<nargs> is greater than zero then it gives the number of arguments
1N/Apassed to C<open>, otherwise it will be 1 if for example
1N/AC<PerlIO_open> was called. In simple cases SvPV_nolen(*args) is the
1N/AHaving said all that translation-only layers do not need to provide
1N/AC<Open()> at all, but rather leave the opening to a lower level layer
1N/Aand wait to be "pushed". If a layer does provide C<Open()> it should
1N/Anormally call the C<Open()> method of next layer down (if any) and
1N/Athen push itself on top if that succeeds.
1N/AIf C<PerlIO_push> was performed and open has failed, it must
1N/AC<PerlIO_pop> itself, since if it's not, the layer won't be removed
1N/Aand may cause bad problems.
1N/AReturns C<NULL> on failure.
1N/A IV (*Binmode)(pTHX_ PerlIO *f);
1N/AOptional. Used when C<:raw> layer is pushed (explicitly or as a result
1N/Aof binmode(FH)). If not present layer will be popped. If present
1N/Ashould configure layer as binary (or pop itself) and return 0.
1N/AIf it returns -1 for error C<binmode> will fail with layer
1N/A SV * (*Getarg)(pTHX_ PerlIO *f,
1N/A CLONE_PARAMS *param, int flags);
1N/AOptional. If present should return an SV * representing the string
1N/Aargument passed to the layer when it was
1N/Apushed.
e.g. ":encoding(ascii)" would return an SvPV with value
1N/A"ascii". (I<param> and I<flags> arguments can be ignored in most
1N/AC<Dup> uses C<Getarg> to retrieve the argument originally passed to
1N/AC<Pushed>, so you must implement this function if your layer has an
1N/Aextra argument to C<Pushed> and will ever be C<Dup>ed.
1N/A IV (*Fileno)(pTHX_ PerlIO *f);
1N/AReturns the
Unix/Posix numeric file descriptor for the handle. Normally
1N/AC<PerlIOBase_fileno()> (which just asks next layer down) will suffice
1N/AReturns -1 on error, which is considered to include the case where the
1N/Alayer cannot provide such a file descriptor.
1N/A PerlIO * (*Dup)(pTHX_ PerlIO *f, PerlIO *o,
1N/A CLONE_PARAMS *param, int flags);
1N/AXXX: Needs more docs.
1N/AUsed as part of the "clone" process when a thread is spawned (in which
1N/Acase param will be non-NULL) and when a stream is being duplicated via
1N/ASimilar to C<Open>, returns PerlIO* on success, C<NULL> on failure.
1N/A SSize_t (*Read)(pTHX_ PerlIO *f, void *vbuf, Size_t count);
1N/ABasic read operation.
1N/ATypically will call C<Fill> and manipulate pointers (possibly via the
1N/AAPI). C<PerlIOBuf_read()> may be suitable for derived classes which
1N/Aprovide "fast gets" methods.
1N/AReturns actual bytes read, or -1 on an error.
1N/A SSize_t (*Unread)(pTHX_ PerlIO *f,
1N/A const void *vbuf, Size_t count);
1N/AA superset of stdio's C<ungetc()>. Should arrange for future reads to
1N/Asee the bytes in C<vbuf>. If there is no obviously better implementation
1N/Athen C<PerlIOBase_unread()> provides the function by pushing a "fake"
1N/A"pending" layer above the calling layer.
1N/AReturns the number of unread chars.
1N/A SSize_t (*Write)(PerlIO *f, const void *vbuf, Size_t count);
1N/ABasic write operation.
1N/AReturns bytes written or -1 on an error.
1N/A IV (*Seek)(pTHX_ PerlIO *f, Off_t offset, int whence);
1N/APosition the file pointer. Should normally call its own C<Flush>
1N/Amethod and then the C<Seek> method of next layer down.
1N/AReturns 0 on success, -1 on failure.
1N/A Off_t (*Tell)(pTHX_ PerlIO *f);
1N/AReturn the file pointer. May be based on layers cached concept of
1N/Aposition to avoid overhead.
1N/AReturns -1 on failure to get the file pointer.
1N/A IV (*Close)(pTHX_ PerlIO *f);
1N/AClose the stream. Should normally call C<PerlIOBase_close()> to flush
1N/Aitself and close layers below, and then deallocate any data structures
1N/A(buffers, translation tables, ...) not held directly in the data
1N/AReturns 0 on success, -1 on failure.
1N/A IV (*Flush)(pTHX_ PerlIO *f);
1N/AShould make stream's state consistent with layers below. That is, any
1N/Abuffered write data should be written, and file position of lower layers
1N/Aadjusted for data read from below but not actually consumed.
1N/A(Should perhaps C<Unread()> such data to the lower layer.)
1N/AReturns 0 on success, -1 on failure.
1N/A IV (*Fill)(pTHX_ PerlIO *f);
1N/AThe buffer for this layer should be filled (for read) from layer
1N/Abelow. When you "subclass" PerlIOBuf layer, you want to use its
1N/AI<_read> method and to supply your own fill method, which fills the
1N/AReturns 0 on success, -1 on failure.
1N/A IV (*Eof)(pTHX_ PerlIO *f);
1N/AReturn end-of-file indicator. C<PerlIOBase_eof()> is normally sufficient.
1N/AReturns 0 on end-of-file, 1 if not end-of-file, -1 on error.
1N/A IV (*Error)(pTHX_ PerlIO *f);
1N/AReturn error indicator. C<PerlIOBase_error()> is normally sufficient.
1N/AReturns 1 if there is an error (usually when C<PERLIO_F_ERROR> is set,
1N/A void (*Clearerr)(pTHX_ PerlIO *f);
1N/AClear end-of-file and error indicators. Should call C<PerlIOBase_clearerr()>
1N/Ato set the C<PERLIO_F_XXXXX> flags, which may suffice.
1N/A void (*Setlinebuf)(pTHX_ PerlIO *f);
1N/AMark the stream as line buffered. C<PerlIOBase_setlinebuf()> sets the
1N/APERLIO_F_LINEBUF flag and is normally sufficient.
1N/A STDCHAR * (*Get_base)(pTHX_ PerlIO *f);
1N/AAllocate (if not already done so) the read buffer for this layer and
1N/Areturn pointer to it. Return NULL on failure.
1N/A Size_t (*Get_bufsiz)(pTHX_ PerlIO *f);
1N/AReturn the number of bytes that last C<Fill()> put in the buffer.
1N/A STDCHAR * (*Get_ptr)(pTHX_ PerlIO *f);
1N/AReturn the current read pointer relative to this layer's buffer.
1N/A SSize_t (*Get_cnt)(pTHX_ PerlIO *f);
1N/AReturn the number of bytes left to be read in the current buffer.
1N/A void (*Set_ptrcnt)(pTHX_ PerlIO *f,
1N/A STDCHAR *ptr, SSize_t cnt);
1N/AAdjust the read pointer and count of bytes to match C<ptr>
and/or C<cnt>.
1N/AThe application (or layer above) must ensure they are consistent.
1N/A(Checking is allowed by the paranoid.)
1N/ATo ask for the next layer down use PerlIONext(PerlIO *f).
1N/ATo check that a PerlIO* is valid use PerlIOValid(PerlIO *f). (All
1N/Athis does is really just to check that the pointer is non-NULL and
1N/Athat the pointer behind that is non-NULL.)
1N/APerlIOBase(PerlIO *f) returns the "Base" pointer, or in other words,
1N/Athe C<PerlIOl*> pointer.
1N/APerlIOSelf(PerlIO* f, type) return the PerlIOBase cast to a type.
1N/APerl_PerlIO_or_Base(PerlIO* f, callback, base, failure, args) either
1N/Acalls the I<callback> from the functions of the layer I<f> (just by
1N/Athe name of the IO function, like "Read") with the I<args>, or if
1N/Athere is no such callback, calls the I<base> version of the callback
1N/Awith the same args, or if the f is invalid, set errno to EBADF and
1N/APerl_PerlIO_or_fail(PerlIO* f, callback, failure, args) either calls
1N/Athe I<callback> of the functions of the layer I<f> with the I<args>,
1N/Aor if there is no such callback, set errno to EINVAL. Or if the f is
1N/Ainvalid, set errno to EBADF and return I<failure>.
1N/APerl_PerlIO_or_Base_void(PerlIO* f, callback, base, args) either calls
1N/Athe I<callback> of the functions of the layer I<f> with the I<args>,
1N/Aor if there is no such callback, calls the I<base> version of the
1N/Acallback with the same args, or if the f is invalid, set errno to
1N/APerl_PerlIO_or_fail_void(PerlIO* f, callback, args) either calls the
1N/AI<callback> of the functions of the layer I<f> with the I<args>, or if
1N/Athere is no such callback, set errno to EINVAL. Or if the f is
1N/Ainvalid, set errno to EBADF.
1N/A=head2 Implementing PerlIO Layers
1N/AIf you find the implementation document unclear or not sufficient,
1N/Alook at the existing PerlIO layer implementations, which include:
1N/A=item * C implementations
1N/A"unix", "perlio", "stdio", "crlf", "utf8", "byte", "raw", "pending"
1N/Alayers, and also the "mmap" and "win32" layers if applicable.
1N/A(The "win32" is currently unfinished and unused, to see what is used
1N/Ainstead in Win32, see L<PerlIO/"Querying the layers of filehandles"> .)
1N/APerlIO::encoding, PerlIO::scalar, PerlIO::via in the Perl core.
1N/APerlIO::gzip and APR::PerlIO (mod_perl 2.0) on CPAN.
1N/A=item * Perl implementations
1N/APerlIO::via::QuotedPrint in the Perl core and PerlIO::via::* on CPAN.
1N/AIf you are creating a PerlIO layer, you may want to be lazy, in other
1N/Awords, implement only the methods that interest you. The other methods
1N/Ayou can either replace with the "blank" methods
1N/A PerlIOBase_noop_fail
1N/A(which do nothing, and return zero and -1, respectively) or for
1N/Acertain methods you may assume a default behaviour by using a NULL
1N/Amethod. The Open method looks for help in the 'parent' layer.
1N/AThe following table summarizes the behaviour:
1N/A method behaviour with NULL
1N/A Clearerr PerlIOBase_clearerr
1N/A Close PerlIOBase_close
1N/A Error PerlIOBase_error
1N/A Fileno PerlIOBase_fileno
1N/A Read PerlIOBase_read
1N/A Setlinebuf PerlIOBase_setlinebuf
1N/A Unread PerlIOBase_unread
1N/A FAILURE Set errno (to EINVAL in UNIXish, to LIB$_INVARG in VMS) and
1N/A return -1 (for numeric return values) or NULL (for pointers)
1N/A INHERITED Inherited from the layer below
1N/A SUCCESS Return 0 (for numeric return values) or a pointer
1N/AA basic non-buffered layer which calls
Unix/POSIX C<read()>, C<write()>,
1N/AC<lseek()>, C<close()>. No buffering. Even on platforms that distinguish
1N/Abetween O_TEXT and O_BINARY this layer is always O_BINARY.
1N/AA very complete generic buffering layer which provides the whole of
1N/APerlIO API. It is also intended to be used as a "base class" for other
1N/Alayers. (For example its C<Read()> method is implemented in terms of
1N/Athe C<Get_cnt()>/C<Get_ptr()>/C<Set_ptrcnt()> methods).
1N/A"perlio" over "unix" provides a complete replacement for stdio as seen
1N/Avia PerlIO API. This is the default for USE_PERLIO when system's stdio
1N/Adoes not permit perl's "fast gets" access, and which do not
1N/Adistinguish between C<O_TEXT> and C<O_BINARY>.
1N/AA layer which provides the PerlIO API via the layer scheme, but
1N/Aimplements it by calling system's stdio. This is (currently) the default
1N/Aif system's stdio provides sufficient access to allow perl's "fast gets"
1N/Aaccess and which do not distinguish between C<O_TEXT> and C<O_BINARY>.
1N/AA layer derived using "perlio" as a base class. It provides Win32-like
1N/A"\n" to CR,LF translation. Can either be applied above "perlio" or serve
1N/Aas the buffer layer itself. "crlf" over "unix" is the default if system
1N/Adistinguishes between C<O_TEXT> and C<O_BINARY> opens. (At some point
1N/A"unix" will be replaced by a "native" Win32 IO layer on that platform,
1N/Aas Win32's
read/write layer has various drawbacks.) The "crlf" layer is
1N/Aa reasonable model for a layer which transforms data in some way.
1N/AIf Configure detects C<mmap()> functions this layer is provided (with
1N/A"perlio" as a "base") which does "read" operations by mmap()ing the
1N/Afile. Performance improvement is marginal on modern systems, so it is
1N/Amainly there as a proof of concept. It is likely to be unbundled from
1N/Athe core at some point. The "mmap" layer is a reasonable model for a
1N/Aminimalist "derived" layer.
1N/AAn "internal" derivative of "perlio" which can be used to provide
1N/AUnread() function for layers which have no buffer or cannot be
1N/Abothered. (Basically this layer's C<Fill()> pops itself off the stack
1N/Aand so resumes reading from layer below.)
1N/AA dummy layer which never exists on the layer stack. Instead when
1N/A"pushed" it actually pops the stack removing itself, it then calls
1N/ABinmode function table entry on all the layers in the stack - normally
1N/Athis (via PerlIOBase_binmode) removes any layers which do not have
1N/AC<PERLIO_K_RAW> bit set. Layers can modify that behaviour by defining
1N/Atheir own Binmode entry.
1N/AAnother dummy layer. When pushed it pops itself and sets the
1N/AC<PERLIO_F_UTF8> flag on the layer which was (and now is once more)
1N/Athe top of the stack.
1N/AIn addition F<
perlio.c> also provides a number of C<PerlIOBase_xxxx()>
1N/Afunctions which are intended to be used in the table slots of classes
1N/Awhich do not need to do anything special for a particular method.
1N/A=head2 Extension Layers
1N/ALayers can made available by extension modules. When an unknown layer
1N/Ais encountered the PerlIO code will perform the equivalent of :
1N/AWhere I<layer> is the unknown layer. F<
PerlIO.pm> will then attempt to:
1N/A require PerlIO::layer;
1N/AIf after that process the layer is still not defined then the C<open>
1N/AThe following extension layers are bundled with perl:
1N/Amakes this layer available, although F<
PerlIO.pm> "knows" where to
1N/Afind it. It is an example of a layer which takes an argument as it is
1N/A open( $fh, "<:encoding(iso-8859-7)", $pathname );
1N/AProvides support for reading data from and writing data to a scalar.
1N/A open( $fh, "+<:scalar", \$scalar );
1N/AWhen a handle is so opened, then reads get bytes from the string value
1N/Aof I<$scalar>, and writes change the value. In both cases the position
1N/Ain I<$scalar> starts as zero but can be altered via C<seek>, and
1N/Adetermined via C<tell>.
1N/APlease note that this layer is implied when calling open() thus:
1N/A open( $fh, "+<", \$scalar );
1N/AProvided to allow layers to be implemented as Perl code. For instance:
1N/A use PerlIO::via::StripHTML;
1N/ASee L<PerlIO::via> for details.
1N/AThings that need to be done to improve this document.
1N/AExplain how to make a valid fh without going through open()(
i.e. apply
1N/Aa layer). For example if the file is not opened through perl, but we
1N/Awant to get back a fh, like it was opened by Perl.
1N/AHow PerlIO_apply_layera fits in, where its docs, was it made public?
1N/ACurrently the example could be something like this:
1N/A PerlIO *foo_to_PerlIO(pTHX_ char *mode, ...)
1N/A char *mode; /* "w", "r", etc */
1N/A const char *layers = ":APR"; /* the layer name */
1N/A PerlIO *f = PerlIO_allocate(aTHX);
1N/A PerlIO_apply_layers(aTHX_ f, mode, layers);
1N/A PerlIOAPR *st = PerlIOSelf(f, PerlIOAPR);
1N/A /* fill in the st struct, as in _open() */
1N/A PerlIOBase(f)->flags |= PERLIO_F_OPEN;
1N/AThe handling of errors by the layer is not specified.
e.g. when $!
1N/Ashould be set explicitly, when the error handling should be just
1N/Adelegated to the top layer.
1N/AProbably give some hints on using SETERRNO() or pointers to where they
1N/AI think it would help to give some concrete examples to make it easier
1N/Ato understand the API. Of course I agree that the API has to be
1N/Aconcise, but since there is no second document that is more of a
1N/Aguide, I think that it'd make it easier to start with the doc which is
1N/Aan API, but has examples in it in places where things are unclear, to
1N/Aa person who is not a PerlIO guru (yet).