1N/AperlXStut - Tutorial for writing XSUBs
1N/AThis tutorial will educate the reader on the steps involved in creating
1N/Aa Perl extension. The reader is assumed to have access to L<perlguts>,
1N/AL<perlapi> and L<perlxs>.
1N/AThis tutorial starts with very simple examples and becomes more complex,
1N/Awith each new example adding new features. Certain concepts may not be
1N/Acompletely explained until later in the tutorial in order to slowly ease
1N/Athe reader into building extensions.
1N/AThis tutorial was written from a Unix point of view. Where I know them
1N/Ato be otherwise different for other platforms (
e.g. Win32), I will list
1N/Athem. If you find something that was missed, please let me know.
1N/AThis tutorial assumes that the make program that Perl is configured to
1N/Ause is called C<make>. Instead of running "make" in the examples that
1N/Afollow, you may have to substitute whatever make program Perl has been
1N/Aconfigured to use. Running B<perl -V:make> should tell you what it is.
1N/A=head2 Version caveat
1N/AWhen writing a Perl extension for general consumption, one should expect that
1N/Athe extension will be used with versions of Perl different from the
1N/Aversion available on your machine. Since you are reading this document,
1N/Athe version of Perl on your machine is probably 5.005 or later, but the users
1N/Aof your extension may have more ancient versions.
1N/ATo understand what kinds of incompatibilities one may expect, and in the rare
1N/Acase that the version of Perl on your machine is older than this document,
1N/Asee the section on "Troubleshooting these Examples" for more information.
1N/AIf your extension uses some features of Perl which are not available on older
1N/Areleases of Perl, your users would appreciate an early meaningful warning.
1N/AYou would probably put this information into the F<README> file, but nowadays
1N/Ainstallation of extensions may be performed automatically, guided by F<
CPAN.pm>
1N/Amodule or other tools.
1N/Aopportunity to perform version checks. One can put something like this
1N/A eval { require 5.007 }
1N/A ### This module uses frobnication framework which is not available before
1N/A ### version 5.007 of Perl. Upgrade your Perl before installing Kara::Mba.
1N/A=head2 Dynamic Loading versus Static Loading
1N/AIt is commonly thought that if a system does not have the capability to
1N/Adynamically load a library, you cannot build XSUBs. This is incorrect.
1N/AYou I<can> build them, but you must link the XSUBs subroutines with the
1N/Arest of Perl, creating a new executable. This situation is similar to
1N/AThis tutorial can still be used on such a system. The XSUB build mechanism
1N/Awill check the system and build a dynamically-loadable library if possible,
1N/Aor else a static library and then, optionally, a new statically-linked
1N/Aexecutable with that static library linked in.
1N/AShould you wish to build a statically-linked executable on a system which
1N/Acan dynamically load libraries, you may, in all the following examples,
1N/Awhere the command "C<make>" with no arguments is executed, run the command
1N/A"C<make perl>" instead.
1N/AIf you have generated such a statically-linked executable by choice, then
1N/Ainstead of saying "C<make test>", you should say "C<make test_static>".
1N/AOn systems that cannot build dynamically-loadable libraries at all, simply
1N/Asaying "C<make test>" is sufficient.
1N/ANow let's go on with the show!
1N/AOur first extension will be very simple. When we call the routine in the
1N/Aextension, it will print out a well-known message and return.
1N/ARun "C<h2xs -A -n Mytest>". This creates a directory named Mytest,
1N/Apossibly under ext/ if that directory exists in the current working
1N/Adirectory. Several files will be created in the Mytest dir, including
1N/AThe MANIFEST file contains the names of all the files just created in the
1N/A use ExtUtils::MakeMaker;
1N/A # the contents of the Makefile that is written.
1N/A DEFINE => '', #
e.g., '-DHAVE_SOMETHING'
1N/A our @ISA = qw(Exporter DynaLoader);
1N/A # Items to export into callers namespace by default. Note: do not export
1N/A # names by default without a very good reason. Use EXPORT_OK instead.
1N/A our $VERSION = '0.01';
1N/A bootstrap Mytest $VERSION;
1N/A # Preloaded methods go here.
1N/A # Autoload methods go after __END__, and are processed by the autosplit program.
1N/A # Below is the stub of documentation for your module. You better edit it!
1N/AThe rest of the .pm file contains sample code for providing documentation for
1N/A MODULE = Mytest PACKAGE = Mytest
1N/ALet's edit the .xs file by adding this to the end of the file:
1N/A printf("Hello, world!\n");
1N/AIt is okay for the lines starting at the "CODE:" line to not be indented.
1N/AHowever, for readability purposes, it is suggested that you indent CODE:
1N/Aone level and the lines following one more level.
1N/Awhich make needs. Its output looks something like:
1N/A Checking if your kit is complete...
1N/A Writing Makefile for Mytest
1N/ANow, running make will produce output that looks something like this (some
1N/Along lines have been shortened for clarity and some extraneous lines have
1N/A Please specify prototyping behavior for
Mytest.xs (see perlxs manual)
1N/A Running Mkbootstrap for Mytest ()
1N/AYou can safely ignore the line about "prototyping behavior" - it is
1N/Aexplained in the section "The PROTOTYPES: Keyword" in L<perlxs>.
1N/AIf you are on a Win32 system, and the build process fails with linker
1N/Aerrors for functions in the C library, check if your Perl is configured
1N/Ato use PerlCRT (running B<perl -V:libc> should show you if this is the
1N/Acase). If Perl is configured to use PerlCRT, you have to make sure
1N/Aso that the compiler can find it on its own.
msvcrt.lib is usually
1N/APerl has its own special way of easily writing test scripts, but for this
1N/Aexample only, we'll create our own test script. Create a file called hello
1N/Athat looks like this:
1N/A use ExtUtils::testlib;
1N/ANow we make the script executable (C<chmod +x hello>), run the script
1N/Aand we should see the following output:
1N/ANow let's add to our extension a subroutine that will take a single numeric
1N/Aargument as input and return 0 if the number is even or 1 if the number
1N/A RETVAL = (input % 2 == 0);
1N/AThere does not need to be white space at the start of the "C<int input>"
1N/Aline, but it is useful for improving readability. Placing a semi-colon at
1N/Athe end of that line is also optional. Any amount and kind of white space
1N/Amay be placed between the "C<int>" and "C<input>".
1N/ANow re-run make to rebuild our new shared library.
1N/ANow perform the same steps as before, generating a Makefile from the
1N/AIn order to test that our extension works, we now need to look at the
1N/Afile
test.pl. This file is set up to imitate the same kind of testing
1N/Astructure that Perl itself has. Within the test script, you perform a
1N/Anumber of tests to confirm the behavior of the extension, printing "ok"
1N/Awhen the test is correct, "not ok" when it is not. Change the print
1N/Astatement in the BEGIN block to print "1..4", and add the following code
1N/Ato the end of the file:
1N/A print &Mytest::is_even(0) == 1 ? "ok 2" : "not ok 2", "\n";
1N/A print &Mytest::is_even(1) == 0 ? "ok 3" : "not ok 3", "\n";
1N/A print &Mytest::is_even(2) == 1 ? "ok 4" : "not ok 4", "\n";
1N/AWe will be calling the test script through the command "C<make test>". You
1N/Ashould see output that looks something like this:
1N/A=head2 What has gone on?
1N/AThe program h2xs is the starting point for creating extensions. In later
1N/Aexamples we'll see how we can use h2xs to read header files and generate
1N/Atemplates to connect to C routines.
1N/Ah2xs creates a number of files in the extension directory. The file
1N/AMakefile.PL is a perl script which will generate a true Makefile to build
1N/Athe extension. We'll take a closer look at it later.
1N/AThe .pm and .xs files contain the meat of the extension. The .xs file holds
1N/Athe C routines that make up the extension. The .pm file contains routines
1N/Athat tell Perl how to load your extension.
1N/AGenerating the Makefile and running C<make> created a directory called blib
1N/A(which stands for "build library") in the current working directory. This
1N/Adirectory will contain the shared library that we will build. Once we have
1N/Atested it, we can install it into its final location.
1N/AInvoking the test script via "C<make test>" did something very important.
1N/AIt invoked perl with all those C<-I> arguments so that it could find the
1N/Avarious files that are part of the extension. It is I<very> important that
1N/Awhile you are still testing extensions that you use "C<make test>". If you
1N/Atry to run the test script all by itself, you will get a fatal error.
1N/AAnother reason it is important to use "C<make test>" to run your test
1N/Ascript is that if you are testing an upgrade to an already-existing version,
1N/Ausing "C<make test>" insures that you will test your new extension, not the
1N/Aalready-existing version.
1N/AWhen Perl sees a C<use extension;>, it searches for a file with the same name
1N/Aas the C<use>'d extension that has a .pm suffix. If that file cannot be found,
1N/APerl dies with a fatal error. The default search path is contained in the
1N/AIn our case,
Mytest.pm tells perl that it will need the Exporter and Dynamic
1N/ALoader extensions. It then sets the C<@ISA> and C<@EXPORT> arrays and the
1N/AC<$VERSION> scalar; finally it tells perl to bootstrap the module. Perl
1N/Awill call its dynamic loader routine (if there is one) and load the shared
1N/AThe two arrays C<@ISA> and C<@EXPORT> are very important. The C<@ISA>
1N/Aarray contains a list of other packages in which to search for methods (or
1N/Asubroutines) that do not exist in the current package. This is usually
1N/Aonly important for object-oriented extensions (which we will talk about
1N/Amuch later), and so usually doesn't need to be modified.
1N/AThe C<@EXPORT> array tells Perl which of the extension's variables and
1N/Asubroutines should be placed into the calling package's namespace. Because
1N/Ayou don't know if the user has already used your variable and subroutine
1N/Anames, it's vitally important to carefully select what to export. Do I<not>
1N/Aexport method or variable names I<by default> without a good reason.
1N/AAs a general rule, if the module is trying to be object-oriented then don't
1N/Aexport anything. If it's just a collection of functions and variables, then
1N/Ayou can export them via another array, called C<@EXPORT_OK>. This array
1N/Adoes not automatically place its subroutine and variable names into the
1N/Anamespace unless the user specifically requests that this be done.
1N/ASee L<perlmod> for more information.
1N/AThe C<$VERSION> variable is used to ensure that the .pm file and the shared
1N/Alibrary are "in sync" with each other. Any time you make changes to
1N/Athe .pm or .xs files, you should increment the value of this variable.
1N/A=head2 Writing good test scripts
1N/AThe importance of writing good test scripts cannot be overemphasized. You
1N/Ashould closely follow the "
ok/not ok" style that Perl itself uses, so that
1N/Ait is very easy and unambiguous to determine the outcome of each test case.
1N/AWhen you find and fix a bug, make sure you add a test case for it.
1N/ABy running "C<make test>", you ensure that your
test.pl script runs and uses
1N/Athe correct version of your extension. If you have many test cases, you
1N/Amight want to copy Perl's test style. Create a directory named "t" in the
1N/Aextension's directory and append the suffix ".t" to the names of your test
1N/Afiles. When you run "C<make test>", all of these test files will be executed.
1N/AOur third extension will take one argument as its input, round off that
1N/Avalue, and set the I<argument> to the rounded value.
1N/A arg = floor(arg + 0.5);
1N/A } else if (arg < 0.0) {
1N/A arg = ceil(arg - 0.5);
1N/AEdit the
Makefile.PL file so that the corresponding line looks like this:
1N/AGenerate the Makefile and run make. Change the BEGIN block to print
1N/A $i = -1.5; &Mytest::round($i); print $i == -2.0 ? "ok 5" : "not ok 5", "\n";
1N/A $i = -1.1; &Mytest::round($i); print $i == -1.0 ? "ok 6" : "not ok 6", "\n";
1N/A $i = 0.0; &Mytest::round($i); print $i == 0.0 ? "ok 7" : "not ok 7", "\n";
1N/A $i = 0.5; &Mytest::round($i); print $i == 1.0 ? "ok 8" : "not ok 8", "\n";
1N/A $i = 1.2; &Mytest::round($i); print $i == 1.0 ? "ok 9" : "not ok 9", "\n";
1N/ARunning "C<make test>" should now print out that all nine tests are okay.
1N/ANotice that in these new test cases, the argument passed to round was a
1N/Ascalar variable. You might be wondering if you can round a constant or
1N/Aliteral. To see what happens, temporarily add the following line to
test.pl:
1N/ARun "C<make test>" and notice that Perl dies with a fatal error. Perl won't
1N/Alet you change the value of constants!
1N/A=head2 What's new here?
1N/Aextra library to be linked into the extension's shared library, the math
1N/Alibrary libm in this case. We'll talk later about how to write XSUBs that
1N/Acan call every routine in a library.
1N/AThe value of the function is not being passed back as the function's return
1N/Avalue, but by changing the value of the variable that was passed into the
1N/Afunction. You might have guessed that when you saw that the return value
1N/Aof round is of type "void".
1N/A=head2 Input and Output Parameters
1N/AYou specify the parameters that will be passed into the XSUB on the line(s)
1N/Aafter you declare the function's return value and name. Each input parameter
1N/Aline starts with optional white space, and may have an optional terminating
1N/AThe list of output parameters occurs at the very end of the function, just
1N/Abefore after the OUTPUT: directive. The use of RETVAL tells Perl that you
1N/Awish to send this value back as the return value of the XSUB function. In
1N/AExample 3, we wanted the "return value" placed in the original variable
1N/Awhich we passed in, so we listed it (and not RETVAL) in the OUTPUT: section.
1N/A=head2 The XSUBPP Program
1N/AThe B<xsubpp> program takes the XS code in the .xs file and translates it into
1N/AC code, placing it in a file whose suffix is .c. The C code created makes
1N/Aheavy use of the C functions within Perl.
1N/A=head2 The TYPEMAP file
1N/AThe B<xsubpp> program uses rules to convert from Perl's data types (scalar,
1N/Aarray, etc.) to C's data types (int, char, etc.). These rules are stored
1N/AThe first section maps various C data types to a name, which corresponds
1N/Asomewhat with the various Perl types. The second section contains C code
1N/Awhich B<xsubpp> uses to handle input parameters. The third section contains
1N/AC code which B<xsubpp> uses to handle output parameters.
1N/ALet's take a look at a portion of the .c file created for our extension.
1N/A croak("Usage: Mytest::round(arg)");
1N/A double arg = (double)SvNV(ST(0)); /* XXXXX */
1N/A arg = floor(arg + 0.5);
1N/A } else if (arg < 0.0) {
1N/A arg = ceil(arg - 0.5);
1N/A sv_setnv(ST(0), (double)arg); /* XXXXX */
1N/ANotice the two lines commented with "XXXXX". If you check the first section
1N/Aof the typemap file, you'll see that doubles are of type T_DOUBLE. In the
1N/AINPUT section, an argument that is T_DOUBLE is assigned to the variable
1N/Aarg by calling the routine SvNV on something, then casting it to double,
1N/Athen assigned to the variable arg. Similarly, in the OUTPUT section,
1N/Aonce arg has its final value, it is passed to the sv_setnv function to
1N/Abe passed back to the calling subroutine. These two functions are explained
1N/Ain L<perlguts>; we'll talk more later about what that "ST(0)" means in the
1N/Asection on the argument stack.
1N/A=head2 Warning about Output Arguments
1N/AIn general, it's not a good idea to write extensions that modify their input
1N/Aparameters, as in Example 3. Instead, you should probably return multiple
1N/Avalues in an array and let the caller handle them (we'll do this in a later
1N/Aexample). However, in order to better accommodate calling pre-existing C
1N/Aroutines, which often do modify their input parameters, this behavior is
1N/AIn this example, we'll now begin to write XSUBs that will interact with
1N/Apre-defined C libraries. To begin with, we will build a small library of
1N/Aour own, then let h2xs write our .pm and .xs files for us.
1N/ACreate a new directory called Mytest2 at the same level as the directory
1N/AMytest. In the Mytest2 directory, create another directory called mylib,
1N/Aand cd into that directory.
1N/AHere we'll create some files that will generate a test library. These will
1N/Ainclude a C source file and a header file. We'll also create a
Makefile.PL 1N/Ain this directory. Then we'll make sure that running make at the Mytest2
1N/Alevel will automatically run this
Makefile.PL file and the resulting Makefile.
1N/AIn the mylib directory, create a file
mylib.h that looks like this:
1N/A extern double foo(int, long, const char*);
1N/A foo(int a, long b, const char *c)
1N/A return (a + b + atof(c) + TESTVAL);
1N/A use ExtUtils::MakeMaker;
1N/A NAME => 'Mytest2::mylib',
1N/A SKIP => [qw(all static static_lib dynamic dynamic_lib)],
1N/A clean => {'FILES' => 'libmylib$(LIB_EXT)'},
1N/A sub MY::top_targets {
1N/A static :: libmylib$(LIB_EXT)
1N/A libmylib$(LIB_EXT): $(O_FILES)
1N/A $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
1N/A $(RANLIB) libmylib$(LIB_EXT)
1N/AMake sure you use a tab and not spaces on the lines beginning with "$(AR)"
1N/Aand "$(RANLIB)". Make will not function properly if you use spaces.
1N/AIt has also been reported that the "cr" argument to $(AR) is unnecessary
1N/AWe will now create the main top-level Mytest2 files. Change to the directory
1N/Aabove Mytest2 and run the following command:
1N/AThis will print out a warning about overwriting Mytest2, but that's okay.
1N/AThe normal
Makefile.PL that h2xs generates doesn't know about the mylib
1N/Adirectory. We need to tell it that there is a subdirectory and that we
1N/Awill be generating a library in it. Let's add the argument MYEXTLIB to
1N/Athe WriteMakefile call so that it looks like this:
1N/A 'NAME' => 'Mytest2',
1N/A 'DEFINE' => '', #
e.g., '-DHAVE_SOMETHING'
1N/Aand then at the end add a subroutine (which will override the pre-existing
1N/Asubroutine). Remember to use a tab character to indent the line beginning
1N/A cd mylib && $(MAKE) $(PASSTHRU)
1N/ALet's also fix the MANIFEST file so that it accurately reflects the contents
1N/Aof our extension. The single line that says "mylib" should be replaced by
1N/Athe following three lines:
1N/ATo keep our namespace nice and unpolluted, edit the .pm file and change
1N/Athe variable C<@EXPORT> to C<@EXPORT_OK>. Finally, in the
1N/A.xs file, edit the #include line to read:
1N/AAnd also add the following function definition to the end of the .xs file:
1N/ANow we also need to create a typemap file because the default Perl doesn't
1N/Acurrently support the const char * type. Create a file called typemap in
1N/Athe Mytest2 directory and place the following in it:
1N/AMakefile in the mylib directory. Run make and watch that it does cd into
1N/Athe mylib directory and run make in there as well.
1N/ANow edit the
test.pl script and change the BEGIN block to print "1..4",
1N/Aand add the following lines to the end of the script:
1N/A print &Mytest2::foo(1, 2, "Hello, world!") == 7 ? "ok 2\n" : "not ok 2\n";
1N/A print &Mytest2::foo(1, 2, "0.0") == 7 ? "ok 3\n" : "not ok 3\n";
1N/A print abs(&Mytest2::foo(0, 0, "-3.4") - 0.6) <= 0.01 ? "ok 4\n" : "not ok 4\n";
1N/A(When dealing with floating-point comparisons, it is best to not check for
1N/Aequality, but rather that the difference between the expected and actual
1N/Aresult is below a certain amount (called epsilon) which is 0.01 in this case)
1N/ARun "C<make test>" and all should be well.
1N/A=head2 What has happened here?
1N/AUnlike previous examples, we've now run h2xs on a real include file. This
1N/Ahas caused some extra goodies to appear in both the .pm and .xs files.
1N/AIn the .xs file, there's now a #include directive with the absolute path to
1N/Athe
mylib.h header file. We changed this to a relative path so that we
1N/Acould move the extension directory if we wanted to.
1N/AThere's now some new C code that's been added to the .xs file. The purpose
1N/Aof the C<constant> routine is to make the values that are #define'd in the
1N/Aheader file accessible by the Perl script (by calling either C<TESTVAL> or
1N/AC<&Mytest2::TESTVAL>). There's also some XS code to allow calls to the
1N/AThe .pm file originally exported the name C<TESTVAL> in the C<@EXPORT> array.
1N/AThis could lead to name clashes. A good rule of thumb is that if the #define
1N/Ais only going to be used by the C routines themselves, and not by the user,
1N/Athey should be removed from the C<@EXPORT> array. Alternately, if you don't
1N/Amind using the "fully qualified name" of a variable, you could move most
1N/Aor all of the items from the C<@EXPORT> array into the C<@EXPORT_OK> array.
1N/AIf our include file had contained #include directives, these would not have
1N/Abeen processed by h2xs. There is no good solution to this right now.
1N/AWe've also told Perl about the library that we built in the mylib
1N/Asubdirectory. That required only the addition of the C<MYEXTLIB> variable
1N/Ato the WriteMakefile call and the replacement of the postamble subroutine
1N/Alibrary is a bit more complicated, but not excessively so. Again we
1N/Areplaced the postamble subroutine to insert our own code. This code
1N/Asimply specified that the library to be created here was a static archive
1N/Alibrary (as opposed to a dynamically loadable library) and provided the
1N/Acommands to build it.
1N/A=head2 Anatomy of .xs file
1N/AThe .xs file of L<"EXAMPLE 4"> contained some new elements. To understand
1N/Athe meaning of these elements, pay attention to the line which reads
1N/A MODULE = Mytest2 PACKAGE = Mytest2
1N/AAnything before this line is plain C code which describes which headers
1N/Ato include, and defines some convenience functions. No translations are
1N/Aperformed on this part, apart from having embedded POD documentation
1N/Askipped over (see L<perlpod>) it goes into the generated output C file as is.
1N/AAnything after this line is the description of XSUB functions.
1N/AThese descriptions are translated by B<xsubpp> into C code which
1N/Aimplements these functions using Perl calling conventions, and which
1N/Amakes these functions visible from Perl interpreter.
1N/APay a special attention to the function C<constant>. This name appears
1N/Atwice in the generated .xs file: once in the first part, as a static C
1N/Afunction, then another time in the second part, when an XSUB interface to
1N/Athis static C function is defined.
1N/AThis is quite typical for .xs files: usually the .xs file provides
1N/Aan interface to an existing C function. Then this C function is defined
1N/Asomewhere (either in an external library, or in the first part of .xs file),
1N/Aand a Perl interface to this function (
i.e. "Perl glue") is described in the
1N/Asecond part of .xs file. The situation in L<"EXAMPLE 1">, L<"EXAMPLE 2">,
1N/Aand L<"EXAMPLE 3">, when all the work is done inside the "Perl glue", is
1N/Asomewhat of an exception rather than the rule.
1N/A=head2 Getting the fat out of XSUBs
1N/AIn L<"EXAMPLE 4"> the second part of .xs file contained the following
1N/Adescription of an XSUB:
1N/ANote that in contrast with L<"EXAMPLE 1">, L<"EXAMPLE 2"> and L<"EXAMPLE 3">,
1N/Athis description does not contain the actual I<code> for what is done
1N/Ais done during a call to Perl function foo(). To understand what is going
1N/Aon here, one can add a CODE section to this XSUB:
1N/A RETVAL = foo(a,b,c);
1N/AHowever, these two XSUBs provide almost identical generated C code: B<xsubpp>
1N/Acompiler is smart enough to figure out the C<CODE:> section from the first
1N/Atwo lines of the description of XSUB. What about C<OUTPUT:> section? In
1N/Afact, that is absolutely the same! The C<OUTPUT:> section can be removed
1N/Aas well, I<as far as C<CODE:> section or C<PPCODE:> section> is not
1N/Aspecified: B<xsubpp> can see that it needs to generate a function call
1N/Asection, and will autogenerate the OUTPUT section too. Thus one can
1N/Ashortcut the XSUB to become:
1N/ACan we do the same with an XSUB
1N/A RETVAL = (input % 2 == 0);
1N/Aof L<"EXAMPLE 2">? To do this, one needs to define a C function C<int
1N/Ais_even(int input)>. As we saw in L<Anatomy of .xs file>, a proper place
1N/Afor this definition is in the first part of .xs file. In fact a C function
1N/A return (arg % 2 == 0);
1N/Ais probably overkill for this. Something as simple as a C<#define> will
1N/A #define is_even(arg) ((arg) % 2 == 0)
1N/AAfter having this in the first part of .xs file, the "Perl glue" part becomes
1N/AThis technique of separation of the glue part from the workhorse part has
1N/Aobvious tradeoffs: if you want to change a Perl interface, you need to
1N/Achange two places in your code. However, it removes a lot of clutter,
1N/Aand makes the workhorse part independent from idiosyncrasies of Perl calling
1N/Aconvention. (In fact, there is nothing Perl-specific in the above description,
1N/Aa different version of B<xsubpp> might have translated this to TCL glue or
1N/APython glue as well.)
1N/A=head2 More about XSUB arguments
1N/AWith the completion of Example 4, we now have an easy way to simulate some
1N/Areal-life libraries whose interfaces may not be the cleanest in the world.
1N/AWe shall now continue with a discussion of the arguments passed to the
1N/AWhen you specify arguments to routines in the .xs file, you are really
1N/Apassing three pieces of information for each argument listed. The first
1N/Apiece is the order of that argument relative to the others (first, second,
1N/Aetc). The second is the type of argument, and consists of the type
1N/Adeclaration of the argument (
e.g., int, char*, etc). The third piece is
1N/Athe calling convention for the argument in the call to the library function.
1N/AWhile Perl passes arguments to functions by reference,
1N/AC passes arguments by value; to implement a C function which modifies data
1N/Aof one of the "arguments", the actual argument of this C function would be
1N/Aa pointer to the data. Thus two C functions with declarations
1N/A int string_length(char *s);
1N/A int upper_case_char(char *cp);
1N/Amay have completely different semantics: the first one may inspect an array
1N/Aof chars pointed by s, and the second one may immediately dereference C<cp>
1N/Aand manipulate C<*cp> only (using the return value as, say, a success
1N/Aindicator). From Perl one would use these functions in
1N/Aa completely different manner.
1N/AOne conveys this info to B<xsubpp> by replacing C<*> before the
1N/Aargument by C<&>. C<&> means that the argument should be passed to a library
1N/Afunction by its address. The above two function may be XSUB-ified as
1N/AFor example, consider:
1N/AThe first Perl argument to this function would be treated as a char and assigned
1N/Ato the variable a, and its address would be passed into the function foo.
1N/AThe second Perl argument would be treated as a string pointer and assigned to the
1N/Avariable b. The I<value> of b would be passed into the function foo. The
1N/Aactual call to the function foo that B<xsubpp> generates would look like this:
1N/AB<xsubpp> will parse the following function argument lists identically:
1N/AHowever, to help ease understanding, it is suggested that you place a "&"
1N/Anext to the variable name and away from the variable type), and place a
1N/A"*" near the variable type, but away from the variable name (as in the
1N/Acall to foo above). By doing so, it is easy to understand exactly what
1N/Awill be passed to the C function -- it will be whatever is in the "last
1N/AYou should take great pains to try to pass the function the type of variable
1N/Ait wants, when possible. It will save you a lot of trouble in the long run.
1N/A=head2 The Argument Stack
1N/AIf we look at any of the C code generated by any of the examples except
1N/Aexample 1, you will notice a number of references to ST(n), where n is
1N/Ausually 0. "ST" is actually a macro that points to the n'th argument
1N/Aon the argument stack. ST(0) is thus the first argument on the stack and
1N/Atherefore the first argument passed to the XSUB, ST(1) is the second
1N/AWhen you list the arguments to the XSUB in the .xs file, that tells B<xsubpp>
1N/Awhich argument corresponds to which of the argument stack (
i.e., the first
1N/Aone listed is the first argument, and so on). You invite disaster if you
1N/Ado not list them in the same order as the function expects them.
1N/AThe actual values on the argument stack are pointers to the values passed
1N/Ain. When an argument is listed as being an OUTPUT value, its corresponding
1N/Avalue on the stack (
i.e., ST(0) if it was the first argument) is changed.
1N/AYou can verify this by looking at the C code generated for Example 3.
1N/AThe code for the round() XSUB routine contains lines that look like this:
1N/A double arg = (double)SvNV(ST(0));
1N/A /* Round the contents of the variable arg */
1N/A sv_setnv(ST(0), (double)arg);
1N/AThe arg variable is initially set by taking the value from ST(0), then is
1N/Astored back into ST(0) at the end of the routine.
1N/AXSUBs are also allowed to return lists, not just scalars. This must be
1N/Adone by manipulating stack values ST(0), ST(1), etc, in a subtly
1N/Adifferent way. See L<perlxs> for details.
1N/AXSUBs are also allowed to avoid automatic conversion of Perl function arguments
1N/Ato C function arguments. See L<perlxs> for details. Some people prefer
1N/Amanual conversion by inspecting C<ST(i)> even in the cases when automatic
1N/Aconversion will do, arguing that this makes the logic of an XSUB call clearer.
1N/ACompare with L<"Getting the fat out of XSUBs"> for a similar tradeoff of
1N/Aa complete separation of "Perl glue" and "workhorse" parts of an XSUB.
1N/AWhile experts may argue about these idioms, a novice to Perl guts may
1N/Aprefer a way which is as little Perl-guts-specific as possible, meaning
1N/Aautomatic conversion and automatic call generation, as in
1N/AL<"Getting the fat out of XSUBs">. This approach has the additional
1N/Abenefit of protecting the XSUB writer from future changes to the Perl API.
1N/A=head2 Extending your Extension
1N/ASometimes you might want to provide some extra methods or subroutines
1N/Ato assist in making the interface between Perl and your extension simpler
1N/Aor easier to understand. These routines should live in the .pm file.
1N/AWhether they are automatically loaded when the extension itself is loaded
1N/Aor only loaded when called depends on where in the .pm file the subroutine
1N/Adefinition is placed. You can also consult L<AutoLoader> for an alternate
1N/Away to store and load your extra subroutines.
1N/A=head2 Documenting your Extension
1N/AThere is absolutely no excuse for not documenting your extension.
1N/ADocumentation belongs in the .pm file. This file will be fed to pod2man,
1N/Aand the embedded documentation will be converted to the manpage format,
1N/Athen placed in the blib directory. It will be copied to Perl's
1N/Amanpage directory when the extension is installed.
1N/AYou may intersperse documentation and Perl code within the .pm file.
1N/AIn fact, if you want to use method autoloading, you must do this,
1N/Aas the comment inside the .pm file explains.
1N/ASee L<perlpod> for more information about the pod format.
1N/A=head2 Installing your Extension
1N/AOnce your extension is complete and passes all its tests, installing it
1N/Ais quite simple: you simply run "make install". You will either need
1N/Ato have write permission into the directories where Perl is installed,
1N/Aor ask your system administrator to run the make for you.
1N/AAlternately, you can specify the exact directory to place the extension's
1N/A(or in between the make and install if you have a brain-dead version of make).
1N/AThis can be very useful if you are building an extension that will eventually
1N/Abe distributed to multiple systems. You can then just archive the files in
1N/Athe destination directory and distribute them to your destination systems.
1N/AIn this example, we'll do some more work with the argument stack. The
1N/Aprevious examples have all returned only a single value. We'll now
1N/Acreate an extension that returns an array.
1N/AThis extension is very Unix-oriented (struct statfs and the statfs system
1N/Acall). If you are not running on a Unix system, you can substitute for
1N/Astatfs any other function that returns multiple values, you can hard-code
1N/Avalues to be returned to the caller (although this will be a bit harder
1N/Ato test the error case), or you can simply not do this example. If you
1N/Achange the XSUB, be sure to fix the test cases to match the changes.
1N/AReturn to the Mytest directory and add the following code to the end of
1N/A i = statfs(path, &buf);
1N/A XPUSHs(sv_2mortal(newSVnv(errno)));
1N/AYou'll also need to add the following code to the top of the .xs file, just
1N/AAlso add the following code segment to
test.pl while incrementing the "1..9"
1N/Astring in the BEGIN block to "1..11":
1N/A @a = &Mytest::statfs("/blech");
1N/A print ((scalar(@a) == 1 && $a[0] == 2) ? "ok 10\n" : "not ok 10\n");
1N/A @a = &Mytest::statfs("/");
1N/A print scalar(@a) == 9 ? "ok 11\n" : "not ok 11\n";
1N/A=head2 New Things in this Example
1N/AThis example added quite a few new concepts. We'll take them one at a time.
1N/AThe INIT: directive contains code that will be placed immediately after
1N/Athe argument stack is decoded. C does not allow variable declarations at
1N/Aarbitrary locations inside a function,
1N/Aso this is usually the best way to declare local variables needed by the XSUB.
1N/A(Alternatively, one could put the whole C<PPCODE:> section into braces, and
1N/Aput these declarations on top.)
1N/AThis routine also returns a different number of arguments depending on the
1N/Asuccess or failure of the call to statfs. If there is an error, the error
1N/Anumber is returned as a single-element array. If the call is successful,
1N/Athen a 9-element array is returned. Since only one argument is passed into
1N/Athis function, we need room on the stack to hold the 9 values which may be
1N/AWe do this by using the PPCODE: directive, rather than the CODE: directive.
1N/AThis tells B<xsubpp> that we will be managing the return values that will be
1N/Aput on the argument stack by ourselves.
1N/AWhen we want to place values to be returned to the caller onto the stack,
1N/Awe use the series of macros that begin with "XPUSH". There are five
1N/Adifferent versions, for placing integers, unsigned integers, doubles,
1N/Astrings, and Perl scalars on the stack. In our example, we placed a
1N/APerl scalar onto the stack. (In fact this is the only macro which
1N/Acan be used to return multiple values.)
1N/AThe XPUSH* macros will automatically extend the return stack to prevent
1N/Ait from being overrun. You push values onto the stack in the order you
1N/Awant them seen by the calling program.
1N/AThe values pushed onto the return stack of the XSUB are actually mortal SV's.
1N/AThey are made mortal so that once the values are copied by the calling
1N/Aprogram, the SV's that held the returned values can be deallocated.
1N/AIf they were not mortal, then they would continue to exist after the XSUB
1N/Aroutine returned, but would not be accessible. This is a memory leak.
1N/AIf we were interested in performance, not in code compactness, in the success
1N/Abranch we would not use C<XPUSHs> macros, but C<PUSHs> macros, and would
1N/Apre-extend the stack before pushing the return values:
1N/AThe tradeoff is that one needs to calculate the number of return values
1N/Ain advance (though overextending the stack will not typically hurt
1N/Aanything but memory consumption).
1N/ASimilarly, in the failure branch we could use C<PUSHs> I<without> extending
1N/Athe stack: the Perl function reference comes to an XSUB on the stack, thus
1N/Athe stack is I<always> large enough to take one return value.
1N/AIn this example, we will accept a reference to an array as an input
1N/Aparameter, and return a reference to an array of hashes. This will
1N/Ademonstrate manipulation of complex Perl data types from an XSUB.
1N/AThis extension is somewhat contrived. It is based on the code in
1N/Athe previous example. It calls the statfs function multiple times,
1N/Aaccepting a reference to an array of filenames as input, and returning
1N/Aa reference to an array of hashes containing the data for each of the
1N/AReturn to the Mytest directory and add the following code to the end of
1N/A || (SvTYPE(SvRV(paths)) != SVt_PVAV)
1N/A || ((numpaths = av_len((AV *)SvRV(paths))) < 0))
1N/A results = (AV *)sv_2mortal((SV *)newAV());
1N/A for (n = 0; n <= numpaths; n++) {
1N/A char * fn = SvPV(*av_fetch((AV *)SvRV(paths), n, 0), l);
1N/A i = statfs(fn, &buf);
1N/A av_push(results, newSVnv(errno));
1N/A rh = (HV *)sv_2mortal((SV *)newHV());
1N/A av_push(results, newRV((SV *)rh));
1N/A RETVAL = newRV((SV *)results);
1N/AAnd add the following code to
test.pl, while incrementing the "1..11"
1N/Astring in the BEGIN block to "1..13":
1N/A $results = Mytest::multi_statfs([ '/', '/blech' ]);
1N/A print ((ref $results->[0]) ? "ok 12\n" : "not ok 12\n");
1N/A print ((! ref $results->[1]) ? "ok 13\n" : "not ok 13\n");
1N/A=head2 New Things in this Example
1N/AThere are a number of new concepts introduced here, described below:
1N/AThis function does not use a typemap. Instead, we declare it as accepting
1N/Aone SV* (scalar) parameter, and returning an SV* value, and we take care of
1N/Apopulating these scalars within the code. Because we are only returning
1N/Aone value, we don't need a C<PPCODE:> directive - instead, we use C<CODE:>
1N/Aand C<OUTPUT:> directives.
1N/AWhen dealing with references, it is important to handle them with caution.
1N/AThe C<INIT:> block first checks that
1N/AC<SvROK> returns true, which indicates that paths is a valid reference. It
1N/Athen verifies that the object referenced by paths is an array, using C<SvRV>
1N/Ato dereference paths, and C<SvTYPE> to discover its type. As an added test,
1N/Ait checks that the array referenced by paths is non-empty, using the C<av_len>
1N/Afunction (which returns -1 if the array is empty). The XSRETURN_UNDEF macro
1N/Ais used to abort the XSUB and return the undefined value whenever all three of
1N/Athese conditions are not met.
1N/AWe manipulate several arrays in this XSUB. Note that an array is represented
1N/Ainternally by an AV* pointer. The functions and macros for manipulating
1N/Aarrays are similar to the functions in Perl: C<av_len> returns the highest
1N/Aindex in an AV*, much like $#array; C<av_fetch> fetches a single scalar value
1N/Afrom an array, given its index; C<av_push> pushes a scalar value onto the
1N/Aend of the array, automatically extending the array as necessary.
1N/ASpecifically, we read pathnames one at a time from the input array, and
1N/Astore the results in an output array (results) in the same order. If
1N/Astatfs fails, the element pushed onto the return array is the value of
1N/Aerrno after the failure. If statfs succeeds, though, the value pushed
1N/Aonto the return array is a reference to a hash containing some of the
1N/Ainformation in the statfs structure.
1N/AAs with the return stack, it would be possible (and a small performance win)
1N/Ato pre-extend the return array before pushing data into it, since we know
1N/Ahow many elements we will return:
1N/A av_extend(results, numpaths);
1N/AWe are performing only one hash operation in this function, which is storing
1N/Aa new scalar under a key using C<hv_store>. A hash is represented by an HV*
1N/Apointer. Like arrays, the functions for manipulating hashes from an XSUB
1N/Amirror the functionality available from Perl. See L<perlguts> and L<perlapi>
1N/ATo create a reference, we use the C<newRV> function. Note that you can
1N/Acast an AV* or an HV* to type SV* in this case (and many others). This
1N/Aallows you to take references to arrays, hashes and scalars with the same
1N/Afunction. Conversely, the C<SvRV> function always returns an SV*, which may
1N/Aneed to be cast to the appropriate type if it is something other than a
1N/Ascalar (check with C<SvTYPE>).
1N/AAt this point, xsubpp is doing very little work - the differences between
1N/A=head2 EXAMPLE 7 (Coming Soon)
1N/AXPUSH args AND set RETVAL AND assign return value to array
1N/A=head2 EXAMPLE 8 (Coming Soon)
1N/A=head2 EXAMPLE 9 Passing open files to XSes
1N/AYou would think passing files to an XS is difficult, with all the
1N/Atypeglobs and stuff. Well, it isn't.
1N/ASuppose that for some strange reason we need a wrapper around the
1N/Astandard C library function C<fputs()>. This is all we need:
1N/A #define PERLIO_NOT_STDIO 0
1N/AThe real work is done in the standard typemap.
1N/AB<But> you loose all the fine stuff done by the perlio layers. This
1N/Acalls the stdio function C<fputs()>, which knows nothing about them.
1N/AThe standard typemap offers three variants of PerlIO *:
1N/AC<InputStream> (T_IN), C<InOutStream> (T_INOUT) and C<OutputStream>
1N/A(T_OUT). A bare C<PerlIO *> is considered a T_INOUT. If it matters
1N/Ain your code (see below for why it might) #define or typedef
1N/Aone of the specific names and use that as the argument or result
1N/Atype in your XS file.
1N/AThe standard typemap does not contain PerlIO * before perl 5.7,
1N/Abut it has the three stream variants. Using a PerlIO * directly
1N/Ais not backwards compatible unless you provide your own typemap.
1N/AFor streams coming I<from> perl the main difference is that
1N/AC<OutputStream> will get the output PerlIO * - which may make
1N/Aa difference on a socket. Like in our example...
1N/AFor streams being handed I<to> perl a new file handle is created
1N/A(
i.e. a reference to a new glob) and associated with the PerlIO *
1N/Aprovided. If the
read/write state of the PerlIO * is not correct then you
1N/Amay get errors or warnings from when the file handle is used.
1N/ASo if you opened the PerlIO * as "w" it should really be an
1N/AC<OutputStream> if open as "r" it should be an C<InputStream>.
1N/ANow, suppose you want to use perlio layers in your XS. We'll use the
1N/Aperlio C<PerlIO_puts()> function as an example.
1N/AIn the C part of the XS file (above the first MODULE line) you
1N/A #define OutputStream PerlIO *
1N/A typedef PerlIO * OutputStream;
1N/AAnd this is the XS code:
1N/A perlioputs(s, stream)
1N/A RETVAL = PerlIO_puts(stream, s);
1N/AWe have to use a C<CODE> section because C<PerlIO_puts()> has the arguments
1N/Areversed compared to C<fputs()>, and we want to keep the arguments the same.
1N/AWanting to explore this thoroughly, we want to use the stdio C<fputs()>
1N/Aon a PerlIO *. This means we have to ask the perlio system for a stdio
1N/A perliofputs(s, stream)
1N/A FILE *fp = PerlIO_findFILE(stream);
1N/A if (fp != (FILE*) 0) {
1N/A RETVAL = fputs(s, fp);
1N/ANote: C<PerlIO_findFILE()> will search the layers for a stdio
1N/Alayer. If it can't find one, it will call C<PerlIO_exportFILE()> to
1N/Agenerate a new stdio C<FILE>. Please only call C<PerlIO_exportFILE()> if
1N/Ayou want a I<new> C<FILE>. It will generate one on each call and push a
1N/Anew stdio layer. So don't call it repeatedly on the same
1N/Afile. C<PerlIO()>_findFILE will retrieve the stdio layer once it has been
1N/Agenerated by C<PerlIO_exportFILE()>.
1N/AThis applies to the perlio system only. For versions before 5.7,
1N/AC<PerlIO_exportFILE()> is equivalent to C<PerlIO_findFILE()>.
1N/A=head2 Troubleshooting these Examples
1N/AAs mentioned at the top of this document, if you are having problems with
1N/Athese example extensions, you might see if any of these help you.
1N/AIn versions of 5.002 prior to the gamma version, the test script in Example
1N/A1 will not function properly. You need to change the "use lib" line to
1N/AIn versions of 5.002 prior to version 5.002b1h, the
test.pl file was not
1N/Aautomatically created by h2xs. This means that you cannot say "make test"
1N/Ato run the test script. You will need to add the following line before the
1N/A"use extension" statement:
1N/AIn versions 5.000 and 5.001, instead of using the above line, you will need
1N/Ato use the following line:
1N/A BEGIN { unshift(@INC, "./blib") }
1N/AThis document assumes that the executable named "perl" is Perl version 5.
1N/ASome systems may have installed Perl version 5 as "perl5".
1N/AFor more information, consult L<perlguts>, L<perlapi>, L<perlxs>, L<perlmod>,
1N/AJeff Okamoto <F<okamoto@corp.hp.com>>
1N/AReviewed and assisted by Dean Roehrich, Ilya Zakharevich, Andreas Koenig,
1N/APerlIO material contributed by Lupe Christoph, with some clarification