1N/A.ds DT July 9, 1993 \" use troff -mm
1N/A.SA 1 \" right justified
1N/A.TL "311466-6713" "49059-6" \" charging case filing case
1N/AGuidelines for writing \f5ksh-93\fP built-in commands
1N/A.AU "David G. Korn" DGK FP 11267 8062 D-237 "(research!dgk)"
1N/A.TM 11267-930???-93 \" technical memo + TM numbers
1N/A.AS 2 \" abstract start for TM
1N/AOne of the features of \f5ksh93\fP, the latest version of \f5ksh\fP,
1N/Ais the ability to add built-in commands at run time.
1N/AThis feature only works on operating systems that have the ability
1N/Ato load and link code into the current process at run time.
1N/ASome examples of the systems that have this feature
1N/Aare System V Release 4, Solaris, Sun OS, HP-UX Release 8 and above,
1N/AAIX 3.2 and above, and Microsoft Windows systems.
1N/AThis memo describes how to write and compile programs
1N/Ato can be loaded into \f5ksh\fP at run time as built-in
1N/A.OK Shell "Command interpreter" Language UNIX \" keyword
1N/AA built-in command is executed without creating a separate process.
1N/AInstead, the command is invoked as a C function by \f5ksh\fP.
1N/AIf this function has no side effects in the shell process,
1N/Athen the behavior of this built-in is identical to that of
1N/Athe equivalent stand-alone command. The primary difference
1N/Ain this case is performance. The overhead of process creation
1N/Ais eliminated. For commands of short duration, the effect
1N/Acan be dramatic. For example, on SUN OS 4.1, the time do
1N/Arun \f5wc\fP on a small file of about 1000 bytes, runs
1N/Aabout 50 times faster as a built-in command.
1N/AIn addition, built-in commands that have side effects on the
1N/Ashell environment can be written.
1N/AThis is usually done to extend the application domain for
1N/Ashell programming. For example, an X-windows extension
1N/Athat makes heavy use of the shell variable namespace
1N/Awas added as a group of built-ins commands that
1N/Aare added at run time.
1N/AThe result is a windowing shell that can be used to write
1N/AX-windows applications.
1N/AWhile there are definite advantages to adding built-in
1N/Acommands, there are some disadvantages as well.
1N/ASince the built-in command and \f5ksh\fP share the same
1N/Aaddress space, a coding error in the built-in program
1N/Amay affect the behavior of \f5ksh\fP; perhaps causing
1N/Ait to core dump or hang.
1N/ADebugging is also more complex since your code is now
1N/Aa part of a larger entity.
1N/AThe isolation provided by a separate process
1N/Aguarantees that all resources used by the command
1N/Awill be freed when the command completes.
1N/AAlso, since the address space of \f5ksh\fP will be larger,
1N/Athis may increase the time it takes \f5ksh\fP to fork() and
1N/Aexec() a non-builtin command.
1N/AIt makes no sense to add a built-in command that takes
1N/Aa long time to run or that is run only once, since the performance
1N/Abenefits will be negligible.
1N/ABuilt-ins that have side effects in the current shell
1N/Aenvironment have the disadvantage of increasing the
1N/Acoupling between the built-in and \f5ksh\fP making
1N/Athe overall system less modular and more monolithic.
1N/ADespite these drawbacks, in many cases extending
1N/A\f5ksh\fP by adding built-in
1N/Acommands makes sense and allows reuse of the shell
1N/Ascripting ability in an application specific domain.
1N/AThis memo describes how to write \f5ksh\fP extensions.
1N/A.H 1 "WRITING BUILT-IN COMMANDS"
1N/AThere is a development kit available for writing \f5ksh\fP
1N/Abuilt-ins. The development kit has three directories,
1N/A\f5include\fP, \f5lib\fP, and \f5bin\fP.
1N/AThe \f5include\fP directory contains a sub-directory
1N/Anamed \f5ast\fP that contains interface prototypes
1N/Afor functions that you can call from built-ins. The \f5lib\fP
1N/Adirectory contains the \fBast\fP library\*F
1N/A\fBast\fP stands for Advanced Software Technology
1N/Aand a library named \fBlibcmd\fP that contains a version
1N/Aof several of the standard POSIX\*(Rf
1N/A.I "POSIX \- Part 2: Shell and Utilities,"
1N/Autilities that can be made run time built-ins.
1N/AIt is best to set the value of the environment variable
1N/A\fB\s-1PACKAGE_\s+1ast\fP to the pathname of the directory
1N/Acontaining the development kit.
1N/AUsers of \f5nmake\fP\*(Rf
1N/ANmake reference needed
1N/A2.3 and above will then be able to
1N/Ain their makefiles and not have to specify any \f5-I\fP switches
1N/AA built-in command has a calling convention similar to
1N/Athe \f5main\fP function of a program,
1N/A\f5int main(int argc, char *argv[])\fP.
1N/AHowever, instead of \f5main\fP, you must use the function name
1N/A\f5b_\fP\fIname\fP, where \fIname\fP is the name
1N/Aof the built-in you wish to define.
1N/AThe built-in function takes a third
1N/A\f5void*\fP argument which you can define as \f5NULL\fP.
1N/AInstead of \f5exit\fP, you need to use \f5return\fP
1N/Ato terminate your command.
1N/AThe return value, will become the exit status of the command.
1N/AThe steps necessary to create and add a run time built-in are
1N/Aillustrated in the following simple example.
1N/ASuppose, you wish to add a built-in command named \f5hello\fP
1N/Awhich requires one argument and prints the word hello followed
1N/Aby its argument. First, write the following program in the file
1N/Aint b_hello(int argc, char *argv[], void *context)
1N/A fprintf(stderr,"Usage: hello arg\en");
1N/A printf("hello %s\en",argv[1]);
1N/ANext, the program needs to be compiled.
1N/AOn some systems it is necessary to specify a compiler
1N/Aoption to produce position independent code
1N/AIf you do not compile with \f5nmake\fP
1N/Ait is important to specify the a special include directory
1N/Awhen compiling built-ins.
1N/Ain the development kit is required.
1N/Ayou must build a shared library instead.
1N/AUnfortunately, the method for generating a shared library
1N/Adiffers with operating system.
1N/AHowever, if you are building with the AT\&T \f5nmake\fP
1N/Aprogram you can use the \f5:LIBRARY:\fP rule to specify
1N/Athis in a system independent fashion.
1N/AIn addition, if you have several built-ins, it is desirable
1N/Ato build a shared library that contains them all.
1N/AThe final step is using the built-in.
1N/AThis can be done with the \f5ksh\fP command \f5builtin\fP.
1N/Athe built-in \f5hello\fP, invoke the command,
1N/A\f5builtin -f hello hello\fP
1N/AThe suffix for the shared library can be omitted in
1N/Awhich case the shell will add an appropriate suffix
1N/Afor the system that it is loading from.
1N/AOnce this command has been invoked, you can invoke \f5hello\fP
1N/Aas you do any other command.
1N/AIt is often desirable to make a command \fIbuilt-in\fP
1N/Athe first time that it is referenced. The first
1N/Atime \f5hello\fP is invoked, \f5ksh\fP should load and execute it,
1N/Awhereas for subsequent invocations \f5ksh\fP should just execute the built-in.
1N/AThis can be done by creating a file named \f5hello\fP
1N/Awith the following contents:
1N/A builtin -f hello hello
1N/AThis file \f5hello\fP needs to be placed in a directory that is
1N/Ain your \fB\s-1FPATH\s+1\fP variable. In addition, the full
1N/Aso that the run time loader will be able to find this shared library
1N/Ano matter where the command \f5hello\fP is invoked.
1N/A.H 1 "CODING REQUIREMENTS AND CONVENTIONS"
1N/AAs mentioned above, the entry point for built-ins must be of
1N/Athe form \f5b_\fP\fIname\fP.
1N/AYour built-ins can call functions from the standard C library,
1N/Athe \fBast\fP library, interface functions provided by \f5ksh\fP,
1N/Aand your own functions.
1N/AYou should avoid using any global symbols beginning with
1N/Asince these are used by \f5ksh\fP itself.
1N/AIn addition, \f5#define\fP constants in \f5ksh\fP interface
1N/Afiles, use symbols beginning with \fBSH_\fP to that you should
1N/Aavoid using names beginning with \fBSH_\fP.
1N/AThe development kit provides a portable interface
1N/Ato the C library and to libast.
1N/AThe header files in the development kit are compatible with
1N/ABrian W. Kernighan and Dennis M. Ritchie,
1N/A.IR "The C Programming Language" ,
1N/AAmerican National Standard for Information Systems \- Programming
1N/ALanguage \- C, ANSI X3.159-1989.
1N/AThe best thing to do is to include the header file \f5<
shell.h>\fP.
1N/AThis header file causes the \f5<
ast.h>\fP header, the
1N/Aheader to be included as well as defining prototypes
1N/Afor functions that you can call to get shell
1N/Aservices for your builtins.
1N/Aprovides prototypes for many \fBlibast\fP functions
1N/Aand all the symbol and function definitions from the
1N/AIt also provides all the symbols and definitions for the
1N/A.I "POSIX \- Part 1: System Application Program Interface,"
1N/AYou should include \f5<
ast.h>\fP instead of one or more of
1N/AThe \f5<
error.h>\fP header provides the interface to the error
1N/Aand option parsing routines defined below.
1N/AThe \f5<
stak.h>\fP header provides the interface to the memory
1N/Aallocation routines described below.
1N/Ashould include the file \f5<
ls.h>\fP instead.
1N/AThis provides the complete POSIX interface to \f5stat()\fP
1N/Arelated functions even on non-POSIX systems.
1N/A\f5ksh\fP uses \fBsfio\fP,
1N/ADavid Korn and Kiem-Phong Vo,
1N/AProceedings of the Summer Usenix,
1N/Ato perform all I/O operations.
1N/AThe \fBsfio\fP library, which is part of \fBlibast\fP,
1N/Aprovides a superset of the functionality provided by the standard
1N/AI/O library defined in ANSI-C.
1N/AIf none of the additional functionality is required,
1N/Aand if you are not familiar with \fBsfio\fP and
1N/Ayou do not want to spend the time learning it,
1N/Athen you can use \fBsfio\fP via the \fBstdio\fP library
1N/Ainterface. The development kit contains the header \f5<
stdio.h>\fP
1N/Awhich maps \fBstdio\fP calls to \fBsfio\fP calls.
1N/AIn most instances the mapping is done
1N/Aby macros or inline functions so that there is no overhead.
1N/AThe man page for the \fBsfio\fP library is in an Appendix.
1N/AHowever, there are some very nice extensions and
1N/Aperformance improvements in \fBsfio\fP
1N/Aand if you plan any major extensions I recommend
1N/Athat you use it natively.
1N/A.H 2 "Error Handling"
1N/AFor error messages it is best to use the \fBast\fP library
1N/Afunction \f5errormsg()\fP rather that sending output to
1N/A\f5stderr\fP or the equivalent \f5sfstderr\fP directly.
1N/AUsing \f5errormsg()\fP will make error message appear
1N/Amore uniform to the user.
1N/AFurthermore, using \f5errormsg()\fP should make it easier
1N/Ato do error message translation for other locales
1N/Ain future versions of \f5ksh\fP.
1N/AThe first argument to
1N/A\f5errormsg()\fP specifies the dictionary in which the string
1N/Awill be searched for translation.
1N/AThe second argument to \f5errormsg()\fP contains that error type
1N/Aand value. The third argument is a \fIprintf\fP style format
1N/Aand the remaining arguments are arguments to be printed
1N/Aas part of the message. A new-line is inserted at the
1N/Aend of each message and therefore, should not appear as
1N/Apart of the format string.
1N/AThe second argument should be one of the following:
1N/A.LI \f5ERROR_exit(\fP\fIn\fP\f5)\fP:
1N/AIf \fIn\fP is not-zero, the builtin will exit value \fIn\fP after
1N/Aprinting the message.
1N/A.LI \f5ERROR_system(\fP\fIn\fP\f5)\fP:
1N/AExit builtin with exit value \fIn\fP after printing the message.
1N/AThe message will display the message corresponding to \f5errno\fP
1N/Aenclosed within \f5[\ ]\fP at the end of the message.
1N/A.LI \f5ERROR_usage(\fP\fIn\fP\f5)\fP:
1N/AWill generate a usage message and exit. If \fIn\fP is non-zero,
1N/Athe exit value will be 2. Otherwise the exit value will be 0.
1N/A.LI \f5ERROR_debug(\fP\fIn\fP\f5)\fP:
1N/AWill print a level \fIn\fP debugging message and will then continue.
1N/A.LI \f5ERROR_warn(\fP\fIn\fP\f5)\fP:
1N/APrints a warning message. \fIn\fP is ignored.
1N/A.H 2 "Option Parsing"
1N/AThe first thing that a built-in should do is to check
1N/Athe arguments for correctness and to print any usage
1N/Amessages on standard error.
1N/AFor consistency with the rest of \f5ksh\fP, it is best
1N/Ato use the \f5libast\fP functions \f5optget()\fP and
1N/A\f5optusage()\fPfor this
1N/AThe \f5optget()\fP function is similar to the
1N/ASystem V C library function \f5getopt()\fP,
1N/Abut provides some additional capabilities.
1N/ABuilt-ins that use \f5optget()\fP provide a more
1N/Aconsistent user interface.
1N/AThe \f5optget()\fP function is invoked as
1N/A\f5int optget(char *argv[], const char *optstring)\fP
1N/Awhere \f5argv\fP is the argument list and \f5optstring\fP
1N/Ais a string that specifies the allowable arguments and
1N/Aadditional information that is used to format \fIusage\fP
1N/AIn fact a complete man page in \f5troff\fP or \f5html\fP
1N/Acan be generated by passing a usage string as described
1N/Aby the \f5getopts\fP command.
1N/Asingle letter options are represented by the letter itself,
1N/Aand options that take a string argument are followed by the \f5:\fP
1N/AOption strings have the following special characters:
1N/AUsed after a letter option to indicate that the option
1N/Atakes an option argument.
1N/Avalue after the given argument is encountered.
1N/AUsed after a letter option to indicate that the option
1N/Acan only take a numerical value.
1N/Avalue after the given argument is encountered.
1N/AUsed after a \f5:\fP or \f5#\fP (and after the optional \f5?\fP)
1N/Apreceding option argument is not required.
1N/A.LI \f5[\fP...\f5]\fP
1N/AAfter a \f5:\fP or \f5#\fP, the characters contained
1N/Ainside the brackets are used to identify the option
1N/Aargument when generating a \fIusage\fP message.
1N/AThe remainder of the string will only be used when generating
1N/AThe \f5optget()\fP function returns the matching option letter if
1N/Aone of the legal option is matched.
1N/AOtherwise, \f5optget()\fP returns
1N/Acontains the error string.
1N/AIndicates the end of options.
1N/AA usage message has been required.
1N/AYou normally call \f5optusage()\fP to generate and display
1N/AThe following is an example of the option parsing portion
1N/Aof the \f5wc\fP utility.
1N/Awhile(1) switch(n=optget(argv,"xf:[file]"))
1N/A.H 2 "Storage Management"
1N/AIt is important that any memory used by your built-in
1N/Abe returned. Otherwise, if your built-in is called frequently,
1N/A\f5ksh\fP will eventually run out of memory.
1N/AYou should avoid using \f5malloc()\fP for memory that must
1N/Abe freed before returning from you built-in, because by default,
1N/A\f5ksh\fP will terminate you built-in in the event of an
1N/Ainterrupt and the memory will not be freed.
1N/AThe best way to to allocate variable sized storage is
1N/Athrough calls to the \fBstak\fP library
1N/Awhich is included in \fBlibast\fP
1N/Aand which is used extensively by \f5ksh\fP itself.
1N/AObjects allocated with the \f5stakalloc()\fP
1N/Afunction are freed when you function completes
1N/AThe \fBstak\fP library provides a convenient way to
1N/Abuild variable length strings and other objects dynamically.
1N/AThe man page for the \fBstak\fP library is contained
1N/ABefore \f5ksh\fP calls each built-in command, it saves
1N/Athe current stack location and restores it after
1N/AIt is not necessary to save and restore the stack
1N/Alocation in the \f5b_\fP entry function,
1N/Abut you may want to write functions that use this stack
1N/Aare restore it when leaving the function.
1N/AThe following coding convention will do this in
1N/A\fIyourfunction\fP\f5()
1N/A if(saveoffset=staktell())
1N/A savebase = stakfreeze(0);
1N/A stakset(savebase,saveoffset);
1N/A.H 1 "CALLING \f5ksh\fP SERVICES"
1N/ASome of the more interesting applications are those that extend
1N/Athe functionality of \f5ksh\fP in application specific directions.
1N/AA prime example of this is the X-windows extension which adds
1N/Abuiltins to create and delete widgets.
1N/AThe \fBnval\fP library is used to interface with the shell
1N/AThe \fBshell\fP library is used to access other shell services.
1N/A.H 2 "The nval library"
1N/AA great deal of power is derived from the ability to use
1N/Aportions of the hierarchal variable namespace provided by \f5ksh-93\fP
1N/Aand turn these names into active objects.
1N/AThe \fBnval\fP library is used to interface with shell
1N/AA man page for this file is provided in an Appendix.
1N/Ato access the functions defined in the \fBnval\fP library.
1N/AAll the functions provided by the \fBnval\fP library begin
1N/Awith the prefix \f5nv_\fP.
1N/AEach shell variable is an object in an associative table
1N/Athat is referenced by name.
1N/AThe type \f5Namval_t*\fP is pointer to a shell variable.
1N/ATo operate on a shell variable, you first get a handle
1N/Ato the variable with the \f5nv_open()\fP function
1N/Aand then supply the handle returned as the first
1N/Aargument of the function that provides an operation
1N/AYou must call \f5nv_close()\fP when you are finished
1N/Ausing this handle so that the space can be freed once
1N/AThe two most frequent operations are to get the value of
1N/Athe variable, and to assign value to the variable.
1N/AThe \f5nv_getval()\fP returns a pointer the the
1N/Avalue of the variable.
1N/AIn some cases the pointer returned is to a region that
1N/Awill be overwritten by the next \f5nv_getval()\fP call
1N/Aso that if the value isn't used immediately, it should
1N/AMany variables can also generate a numeric value.
1N/AThe \f5nv_getnum()\fP function returns a numeric
1N/Avalue for the given variable pointer, calling the
1N/Aarithmetic evaluator if necessary.
1N/AThe \f5nv_putval()\fP function is used to assign a new
1N/Avalue to a given variable.
1N/AThe second argument to \f5putval()\fP is the value
1N/Aand the third argument is a \fIflag\fP which
1N/Ais used in interpreting the second argument.
1N/AEach shell variable can have one or more attributes.
1N/AThe \f5nv_isattr()\fP is used to test for the existence
1N/Aof one or more attributes.
1N/ASee the appendix for a complete list of attributes.
1N/ABy default, each shell variable passively stores the string you
1N/Agive with with \f5nv_putval()\fP, and returns the value
1N/Awith \f5getval()\fP. However, it is possible to turn
1N/Aany node into an active entity by assigning functions
1N/Ato it that will be called whenever \f5nv_putval()\fP
1N/AIn fact there are up to five functions that can
1N/Aassociated with each variable to override the
1N/AThe type \f5Namfun_t\fP is used to define these functions.
1N/AOnly those that are non-\f5NULL\fP override the
1N/ATo override the default actions, you must allocate an
1N/Ainstance of \f5Namfun_t\fP, and then assign
1N/Athe functions that you wish to override.
1N/Afunction is called by the \f5nv_putval()\fP function.
1N/AA \f5NULL\fP for the \fIvalue\fP argument
1N/Aindicates a request to unset the variable.
1N/AThe \fItype\fP argument might contain the \f5NV_INTEGER\fP
1N/Abit so you should be prepared to do a conversion if
1N/Afunction is called by \f5nv_getval()\fP
1N/Avalue and must return a string.
1N/Afunction is called by by the arithmetic evaluator
1N/Aand must return double.
1N/AIf omitted, then it will call \f5nv_getval()\fP and
1N/Aconvert the result to a number.
1N/AThe functionality of a variable can further be increased
1N/Aby adding discipline functions that
1N/Acan be associated with the variable.
1N/AA discipline function allows a script that uses your
1N/Avariable to define functions whose name is
1N/A\fIvarname\fP\f5.\fP\fIdiscname\fP
1N/Awhere \fIvarname\fP is the name of the variable, and \fIdiscname\fP
1N/Ais the name of the discipline.
1N/AWhen the user defines such a function, the \f5settrap()\fP
1N/Afunction will be called with the name of the discipline and
1N/Aa pointer to the parse tree corresponding to the discipline
1N/AThe application determines when these functions are actually
1N/ABy default, \f5ksh\fP defines \f5get\fP,
1N/A\f5set\fP, and \f5unset\fP as discipline functions.
1N/AIn addition, it is possible to provide a data area that
1N/Awill be passed as an argument to
1N/Aeach of these functions whenever any of these functions are called.
1N/ATo have private data, you need to define and allocate a structure
1N/A\f5struct \fIyours\fP
1N/A \fIyour_data_fields\fP;
1N/A.H 2 "The shell library"
1N/AThere are several functions that are used by \f5ksh\fP itself
1N/Athat can also be called from built-in commands.
1N/AThe man page for these routines are in the Appendix.
1N/AThe \f5sh_addbuiltin()\fP function can be used to add or delete
1N/Abuiltin commands. It takes the name of the built-in, the
1N/Aaddress of the function that implements the built-in, and
1N/Aa \f5void*\fP pointer that will be passed to this function
1N/Aas the third agument whenever it is invoked.
1N/AIf the function address is \f5NULL\fP, the specified built-in
1N/Awill be deleted. However, special built-in functions cannot
1N/Abe deleted or modified.
1N/AThe \f5sh_fmtq()\fP function takes a string and returns
1N/Aa string that is quoted as necessary so that it can
1N/Abe used as shell input.
1N/AThis function is used to implement the \f5%q\fP option
1N/Aof the shell built-in \f5printf\fP command.
1N/AThe \f5sh_parse()\fP function returns a parse tree corresponding
1N/Ato a give file stream. The tree can be executed by supplying
1N/Ait as the first argument to
1N/Athe \f5sh_trap()\fP function and giving a value of \f51\fP as the
1N/AAlternatively, the \f5sh_trap()\fP function can parse and execute
1N/Aa string by passing the string as the first argument and giving \f50\fP
1N/Aas the second argument.
1N/AThe \f5sh_isoption()\fP function can be used to set to see whether one
1N/Aor more of the option settings is enabled.