<?xml version="1.0"?>

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V5.0//EN" "http://www.oasis-open.org/docbook/xml/5.0b5/dtd/docbook.dtd" [

<!ENTITY tag_bourneonly '<inlinemediaobject><imageobject><imagedata fileref="images/tag_bourne.png"></imagedata></imageobject><textobject><phrase>[Bourne]</phrase></textobject></inlinemediaobject> '>

<!ENTITY tag_kshonly '<inlinemediaobject><imageobject><imagedata fileref="images/tag_ksh.png"></imagedata></imageobject><textobject><phrase>[ksh]</phrase></textobject></inlinemediaobject> '>

<!ENTITY tag_ksh88only '<inlinemediaobject><imageobject><imagedata fileref="images/tag_ksh88.png"></imagedata></imageobject><textobject><phrase>[ksh88]</phrase></textobject></inlinemediaobject> '>

<!ENTITY tag_ksh93only '<inlinemediaobject><imageobject><imagedata fileref="images/tag_ksh93.png"></imagedata></imageobject><textobject><phrase>[ksh93]</phrase></textobject></inlinemediaobject> '>

<!ENTITY tag_performance '<inlinemediaobject><imageobject><imagedata fileref="images/tag_perf.png"></imagedata></imageobject><textobject><phrase>[perf]</phrase></textobject></inlinemediaobject> '>

<!ENTITY tag_i18n '<inlinemediaobject><imageobject><imagedata fileref="images/tag_i18n.png"></imagedata></imageobject><textobject><phrase>[i18n]</phrase></textobject></inlinemediaobject> '>

<!ENTITY tag_l10n '<inlinemediaobject><imageobject><imagedata fileref="images/tag_l10n.png"></imagedata></imageobject><textobject><phrase>[l10n]</phrase></textobject></inlinemediaobject> '>

]>

<article

xmlns:xlink="http://www.w3.org/1999/xlink"

xmlns="http://docbook.org/ns/docbook"

xml:lang="en">

<!-- xmlns:xi="http://www.w3.org/2001/XInclude" -->

<info>

<title><emphasis>[DRAFT]</emphasis> Bourne/Korn Shell Coding Conventions</title>

<!-- subtitle abuse -->

<subtitle>

This page is currently work-in-progress until it is approved by the OS/Net community. Please send any comments to

<email>shell-discuss@opensolaris.org</email>.

</subtitle>

<authorgroup>

<author><orgname>OpenSolaris.org</orgname></author>

</authorgroup>

</info>

<section xml:id="intro">

<title>Intro</title>

<para>This document describes the shell coding style used for all the SMF script changes integrated into (Open)Solaris.</para>

<para>All new SMF shell code should conform to this coding standard, which is intended to match our existing C coding standard.</para>

<para>When in doubt, think "what would be the C-Style equivalent ?" and "What does the POSIX (shell) standard say ?"</para>

</section><!-- end of intro -->

<section xml:id="rules">

<title>Rules</title>

<section xml:id="general">

<title>General</title>

<section xml:id="basic_format">

<title>Basic Format</title>

<para>Similar to <literal>cstyle</literal>, the basic format is that all

lines are indented by TABs or eight spaces, and continuation lines (which

in the shell end with "\") are indented by an equivalent number of TABs

and then an additional four spaces, e.g.

<programlisting>

cp foo bar

cp some_realllllllllllllllly_realllllllllllllly_long_path \

to_another_really_long_path

</programlisting>

</para>

<para>The encoding used for the shell scripts is either <literal>ASCII</literal>

or <literal>UTF-8</literal>, alternative encodings are only allowed when the

application requires this.</para>

</section>

<section xml:id="commenting">

<title>Commenting</title>

<para>Shell comments are preceded by the '<literal>#</literal>' character. Place

single-line comments in the right-hand margin. Use an extra '<literal>#</literal>'

above and below the comment in the case of multi-line comments:

<programlisting>

cp foo bar # Copy foo to bar

#

# Modify the permissions on bar. We need to set them to root/sys

# in order to match the package prototype.

#

chown root bar

chgrp sys bar

</programlisting>

</para>

</section>

<section xml:id="interpreter_magic">

<title>Interpreter magic</title>

<para>The proper interpreter magic for your shell script should be one of these:

<programlisting>

#!/bin/sh Standard Bourne shell script

#!/bin/ksh -p Standard Korn shell 88 script. You should always write ksh

scripts with -p so that ${ENV} (if set by the user) is not

sourced into your script by the shell.

#!/bin/ksh93 Standard Korn shell 93 script (-p is not needed since ${ENV} is

only used for interactive shell sessions).

</programlisting>

</para>

</section>

<section xml:id="harden_your_script_against_unexpected_input">

<title>Harden the script against unexpected (user) input</title>

<para>Harden your script against unexpected (user) input, including

command line options, filenames with blanks (or other special

characters) in the name, or file input</para>

</section>

<section xml:id="use_builtin_commands">

<title>&tag_kshonly;&tag_performance;Use builtin commands if the shell provides them</title>

<para>

Use builtin commands if the shell provides them. For example ksh93s+

(ksh93, version 's+') delivered with Solaris (as defined by PSARC 2006/550)

supports the following builtins:

<simplelist type="inline">

<member>basename</member>

<member>cat</member>

<member>chgrp</member>

<member>chmod</member>

<member>chown</member>

<member>cmp</member>

<member>comm</member>

<member>cp</member>

<member>cut</member>

<member>date</member>

<member>dirname</member>

<member>expr</member>

<member>fds</member>

<member>fmt</member>

<member>fold</member>

<member>getconf</member>

<member>head</member>

<member>id</member>

<member>join</member>

<member>ln</member>

<member>logname</member>

<member>mkdir</member>

<member>mkfifo</member>

<member>mv</member>

<member>paste</member>

<member>pathchk</member>

<member>rev</member>

<member>rm</member>

<member>rmdir</member>

<member>stty</member>

<member>tail</member>

<member>tee</member>

<member>tty</member>

<member>uname</member>

<member>uniq</member>

<member>wc</member>

<member>sync</member>

</simplelist>

Those builtins can be enabled via <literal>$ builtin name_of_builtin #</literal> in shell

scripts (note that ksh93 builtins implement exact POSIX behaviour - some

commands in Solaris <filename>/usr/bin/</filename> directory implement pre-POSIX behaviour.

Add <literal>/usr/xpg6/bin/:/usr/xpg4/bin</literal> before

<filename>/usr/bin/</filename> in <envar>${PATH}</envar> to test whether your script works with

the XPG6/POSIX versions)

</para>

</section>

<section xml:id="use_blocks_not_subshells">

<title>&tag_performance;Use blocks and not subshells if possible</title>

<para>Use blocks and not subshells if possible, e.g. use

<literal>$ { print "foo" ; print "bar" ; }</literal> instead of

<literal>$ (print "foo" ; print "bar") #</literal> - blocks are

faster since they do not require to save the subshell context (ksh93) or

trigger a shell child process (Bourne shell, bash, ksh88 etc.)

</para>

</section>

<section xml:id="use_long_options_for_set_builtin">

<title>&tag_kshonly; use long options for "<literal>set</literal>"</title>

<para>use long options for "<literal>set</literal>", for example instead of <literal>$ set -x #</literal>

use <literal>$ set -o xtrace #</literal> to make the code more readable.</para>

</section>

<section xml:id="use_posix_command_substitutions_syntax">

<title>&tag_kshonly; Use <literal>$(...)</literal> instead of <literal>`...`</literal> command substitutions</title>

<para>Use <literal>$(...)</literal> instead of <literal>`...`</literal> - <literal>`...`</literal>

is an obsolete construct in ksh+POSIX sh scripts and <literal>$(...)</literal>.is a cleaner design,

requires no escaping rules, allows easy nesting etc.</para>

<note><title>&tag_ksh93only; <literal>${ ...;}</literal>-style command substitutions</title>

<para>ksh93 has support for an alternative version of command substitutions with the

syntax <literal>${ ...;}</literal> which do not run in a subshell.

</para></note>

</section>

<section xml:id="put_command_substitution_result_in_quotes">

<title>&tag_kshonly; Always put the result of a <literal>$(...)</literal> or

<literal>$( ...;)</literal> command substitution in quotes</title>

<para>Always put the result of <literal>$( ... )</literal> or <literal>$( ...;)</literal> in

quotes (e.g. <literal>foo="$( ... )"</literal> or <literal>foo="$( ...;)"</literal>) unless

there is a very good reason for not doing it</para>

</section>

<section xml:id="always_set_path">

<title>Scripts should always set their <envar>PATH</envar></title>

<para>Scripts should always set their <envar>PATH</envar> to make sure they do not use

alternative commands by accident (unless the value of <envar>PATH</envar> is well-known

and guaranteed to be set by the caller)</para>

</section>

<section xml:id="make_sure_commands_are_available">

<title>Make sure that commands from other packages/applications are really installed on the machine</title>

<para>Scripts should make sure that commands in optional packages are really

there, e.g. add a "precheck" block in scipts to avoid later failure when

doing the main job</para>

</section>

<section xml:id="check_usage_of_boolean_variables">

<title>Check how boolean values are used/implemented in your application</title>

<para>Check how boolean values are used in your application.</para>

<para>For example:

<programlisting>

mybool=0

# do something

if [ $mybool -eq 1 ] ; then do_something_1 ; fi

</programlisting>

could be rewritten like this:

<programlisting>

mybool=false # (valid values are "true" or "false", pointing

# to the builtin equivalents of /bin/true or /bin/false)

# do something

if ${mybool} ; then do_something_1 ; fi

</programlisting>

or

<programlisting>

integer mybool=0 # values are 0 or 1

# do something

if (( mybool==1 )) ; then do_something_1 ; fi

</programlisting>

</para>

</section>

<section xml:id="shell_uses_characters_not_bytes">

<title>&tag_i18n;The shell always operates on <emphasis>characters</emphasis> not bytes</title>

<para>Shell scripts operate on characters and <emphasis>not</emphasis> bytes.

Some locales use multiple bytes (called "multibyte locales") to represent one character</para>

<note><para>ksh93 has support for binary variables which explicitly

operate on bytes, not characters. This is the <emphasis>only</emphasis> allowed

exception.</para></note>

</section>

<section xml:id="multibyte_locale_input">

<title>&tag_i18n;Multibyte locales and input</title>

<para>Think about whether your application has to handle file names or

variables in multibyte locales and make sure all commands used in your

script can handle such characters (e.g. lots of commands in Solaris's

<filename>/usr/bin/</filename> are <emphasis>not</emphasis> able to handle such values - either use ksh93

builtin constructs (which are guaranteed to be multibyte-aware) or

commands from <filename>/usr/xpg4/bin/</filename> and/or <filename>/usr/xpg6/bin</filename>)

</para>

</section>

<section xml:id="use_external_filters_only_for_large_datasets">

<title>&tag_performance;Only use external filters like <literal>grep</literal>/<literal>sed</literal>/<literal>awk</literal>/etc.

if you want to process lots of data with them</title>

<para>Only use external filters like <literal>grep</literal>/<literal>sed</literal>/<literal>awk</literal>/etc.

if a significant amount of data is processed by the filter or if

benchmarking shows that the use of builtin commands is significantly slower

(otherwise the time and resources needed to start the filter are

far greater then the amount of data being processed,

creating a performance problem).</para>

<para>For example:

<programlisting>

if [ "$(echo "$x" | egrep '.*foo.*')" != "" ] ; then

do_something ;

done

</programlisting>

can be re-written using ksh93 builtin constructs, saving several

<literal>|fork()|+|exec()|</literal>'s:

<programlisting>

if [[ "${x}" == ~(E).*foo.* ]] ; then

do_something ;

done

</programlisting>

</para>

</section>

<section xml:id="use_dashdash_if_first_arg_is_variable">

<title>If the first operand of a command is a variable, use <literal>--</literal></title>

<para>If the first operand of a command is a variable, use <literal>--</literal>

for any command that accepts this as end of argument to

avoid problems if the variable expands to a value starting with <literal>-</literal>.

</para>

<note><para>

At least

<simplelist type="inline">

<member>print</member>

<member>/usr/bin/fgrep</member><member>/usr/xpg4/bin/fgrep</member>

<member>/usr/bin/grep</member> <member>/usr/xpg4/bin/grep</member>

<member>/usr/bin/egrep</member><member>/usr/xpg4/bin/egrep</member>

</simplelist>

support <literal>--</literal> as "end of arguments"-terminator.

</para></note>

</section>

<section xml:id="use_export">

<title>&tag_kshonly;&tag_performance;Use <literal>$ export FOOBAR=val #</literal> instead of

<literal>$ FOOBAR=val ; export FOOBAR #</literal></title>

<para>Use <literal>$ export FOOBAR=val # instead of $ FOOBAR=val ; export FOOBAR #</literal> -

this is much faster.</para>

</section>

<section xml:id="use_subshell_around_set_dashdash_usage">

<title>Use a subshell (e.g. <literal>$ ( mycmd ) #</literal>) around places which use

<literal>set -- $(mycmd)</literal> and/or <literal>shift</literal></title>

<para>Use a subshell (e.g. <literal>$ ( mycmd ) #</literal>) around places which use

<literal>set -- $(mycmd)</literal> and/or <literal>shift</literal> unless the variable

affected is either a local one or if it's guaranteed that this variable will no longer be used

(be careful for loadable functions, e.g. ksh/ksh93's <literal>autoload</literal> !!!!)

</para>

</section>

<section xml:id="be_careful_with_tabs_in_script_code">

<title>Be careful with using TABS in script code, they are not portable

between editors or platforms</title>

<para>Be careful with using TABS in script code, they are not portable

between editors or platforms.</para>

<para>If you use ksh93 use <literal>$'\t'</literal> to include TABs in sources, not the TAB character itself.</para>

</section>

<section xml:id="centralise_error_exit">

<title>If you have multiple points where your application exits with an error

message create a central function for this purpose</title>

<para>If you have multiple points where your application exits with an error

message create a central function for this, e.g.

<programlisting>

if [ -z "$tmpdir" ] ; then

print -u2 "mktemp failed to produce output; aborting."

exit 1

fi

if [ ! -d $tmpdir ] ; then

print -u2 "mktemp failed to create a directory; aborting."

exit 1

fi

</programlisting>

should be replaced with

<programlisting>

function fatal_error

{

print -u2 "${progname}: $*"

exit 1

}

# do something (and save ARGV[0] to variable "progname")

if [ -z "$tmpdir" ] ; then

fatal_error "mktemp failed to produce output; aborting."

fi

if [ ! -d "$tmpdir" ] ; then

fatal_error "mktemp failed to create a directory; aborting."

fi

</programlisting>

</para>

</section>

<section xml:id="use_set_o_nounset">

<title>&tag_kshonly; Think about using <literal>$ set -o nounset #</literal> by default</title>

<para>Think about using <literal>$ set -o nounset #</literal> by default (or at least during the

script's development phase) to catch errors where variables are used

when they are not set (yet), e.g.

<screen>

$ <userinput>(set -o nounset ; print ${foonotset})</userinput>

<computeroutput>/bin/ksh93: foonotset: parameter not set</computeroutput>

</screen>

</para>

</section>

<section xml:id="avoid_eval_builtin">

<title>Avoid using <literal>eval</literal> unless absolutely necessary</title>

<para>Avoid using <literal>eval</literal> unless absolutely necessary. Subtle things

can happen when a string is passed back through the shell

parser. You can use name references to avoid uses such as

<literal>eval $name="$value"</literal>.

</para>

</section>

<section xml:id="use_concatenation_operator">

<title>&tag_ksh93only;Use the string/array concatenation operator <literal>+=</literal></title>

<para>Use <literal>+=</literal> instead of manually adding strings/array elements, e.g.

<programlisting>

foo=""

foo="${foo}a"

foo="${foo}b"

foo="${foo}c"

</programlisting>

should be replaced with

<programlisting>

foo=""

foo+="a"

foo+="b"

foo+="c"

</programlisting>

</para>

</section>

<section xml:id="use_source_not_dot">

<title>&tag_ksh93only;Use <literal>source</literal> instead of '<literal>.</literal> '(dot)

to include other shell script fragments</title>

<para>Use <literal>source</literal> instead of '<literal>.</literal>'

(dot) to include other shell script fragments - the new form is much

more readable than the tiny dot and a failure can be caught within the script.</para>

</section>

<section xml:id="use_builtin_localisation_support">

<title>&tag_ksh93only;&tag_performance;&tag_l10n;Use <literal>$"..."</literal> instead of

<literal>gettext ... "..."</literal> for strings that need to be localized for different locales</title>

<para>Use $"..." instead of <literal>gettext ... "..."</literal> for strings that need to be

localized for different locales. <literal>gettext</literal> will require a

<literal>fork()+exec()</literal> and

reads the whole catalog each time it's called, creating a huge overhead for localisation

(and the <literal>$"..."</literal> is easier to use, e.g. you only have to put a

<literal>$</literal> in front of the catalog and the string will be localised).

</para>

</section>

<section xml:id="use_set_o_noglob">

<title>&tag_kshonly;&tag_performance;Use <literal>set -o noglob</literal> if you do not need to expand files</title>

<para>If you don't expect to expand files, you can do set <literal>-f</literal>

(<literal>set -o noglob</literal>) as well. This way the need to use <literal>""</literal> is

greatly reduced.</para>

</section>

<section xml:id="use_empty_ifs_to_handle_spaces">

<title>&tag_ksh93only;Use <literal>IFS=</literal> to avoid problems with spaces in filenames</title>

<para>Unless you want to do word splitting, put <literal>IFS=</literal>

at the beginning of a command. This way spaces in

file names won't be a problem. You can do

<literal>IFS='delims' read -r</literal> line

to override <envar>IFS</envar> just for the <literal>read</literal> command. However,

you can't do this for the <literal>set</literal> builtin.</para>

</section>

<section xml:id="set_locale_when_comparing_against_localised_output">

<title>Set the message locale if you process output of tools which may be localised</title>

<para>Set the message locale (<envar>LC_MESSAGES</envar>) if you process output of tools which may be localised</para>

<example><title>Set <envar>LC_MESSAGES</envar> when testing for specific outout of the <filename>/usr/bin/file</filename> utility:</title>

<programlisting>

# set french as default message locale

export LC_MESSAGES=fr_FR.UTF-8

...

# test whether the file "/tmp" has the filetype "directory" or not

# we set LC_MESSAGES to "C" to ensure the returned message is in english

if [[ "$(LC_MESSAGES=C file /tmp)" = *directory ]] ; then

print "is a directory"

fi

</programlisting>

<note><para>The environment variable <envar>LC_ALL</envar> always

overrides any other <envar>LC_*</envar> environment variables

(and <envar>LANG</envar>, too),

including <envar>LC_MESSAGES</envar>.

if there is the chance that <envar>LC_ALL</envar> may be set

replace <envar>LC_MESSAGES</envar> with <envar>LC_ALL</envar>

in the example above.</para></note>

</example>

</section>

<section xml:id="cleanup_after_yourself">

<title>Cleanup after yourself.</title>

<para>Cleanup after yourself. For example ksh/ksh93 have an <literal>EXIT</literal> trap which

is very useful for this.

</para>

<note><para>

Note that the <literal>EXIT</literal> trap is executed for a subshell and each subshell

level can run it's own <literal>EXIT</literal> trap, for example

<screen>

$ <userinput>(trap "print bam" EXIT ; (trap "print snap" EXIT ; print "foo"))</userinput>

<computeroutput>foo

snap

bam</computeroutput>

</screen>

</para></note>

</section>

<section xml:id="use_proper_exit_code">

<title>Use a proper <literal>exit</literal> code</title>

<para>Explicitly set the exit code of a script, otherwise the exit code

from the last command executed will be used which may trigger problems

if the value is unexpected.</para>

</section>

<section xml:id="shell_lint">

<title>&tag_ksh93only;Use <literal>shcomp -n scriptname.sh /dev/null</literal> to check for common errors</title>

<para>Use <literal>shcomp -n scriptname.sh /dev/null</literal> to

check for common problems (such as insecure, depreciated or ambiguous constructs) in shell scripts.</para>

</section>

</section><!-- end of general -->

<section xml:id="functions">

<title>Functions</title>

<section xml:id="use_functions">

<title>Use functions to break up your code</title>

<para>Use functions to break up your code into smaller, logical blocks.</para>

</section>

<section xml:id="do_not_reserved_keywords_for_function_names">

<title>Do not use function names which are reserved keywords in C/C++/JAVA or the POSIX shell standard</title>

<para>Do not use function names which are reserved keywords (or function names) in C/C++/JAVA or the POSIX shell standard

(to avoid confusion and/or future changes/updates to the shell language).

</para>

</section>

<section xml:id="use_ksh_style_function_syntax">

<title>&tag_kshonly;&tag_performance;Use ksh-style <literal>function</literal></title>

<para>It is <emphasis>highly</emphasis> recommended to use ksh style functions

(<literal>function foo { ... }</literal>) instead

of Bourne-style functions (<literal>foo() { ... }</literal>) if possible

(and local variables instead of spamming the global namespace).</para>

<warning><para>

The difference between old-style Bourne functions and ksh functions is one of the major differences

between ksh88 and ksh93 - ksh88 allowed variables to be local for Bourne-style functions while ksh93

conforms to the POSIX standard and will use a function-local scope for variables declared in

Bourne-style functions.</para>

<para>Example (note that "<literal>integer</literal>" is an alias for "<literal>typeset -li</literal>"):

<programlisting>

# new style function with local variable

$ ksh93 -c 'integer x=2 ; function foo { integer x=5 ; } ; print "x=$x"

; foo ; print "x=$x" ;'

x=2

x=2

# old style function with an attempt to create a local variable

$ ksh93 -c 'integer x=2 ; foo() { integer x=5 ; } ; print "x=$x" ; foo ;

print "x=$x" ;'

x=2

x=5

</programlisting>

<uri xlink:href="http://www.opensolaris.org/os/project/ksh93-integration/docs/ksh93r/general/compatibility/">usr/src/lib/libshell/common/COMPATIBILITY</uri>

says about this issue:

<blockquote><para>

Functions, defined with name() with ksh-93 are compatible with

the POSIX standard, not with ksh-88. No local variables are

permitted, and there is no separate scope. Functions defined

with the function name syntax, maintain compatibility.

This also affects function traces.

</para></blockquote>

(this issue also affects <filename>/usr/xpg4/bin/sh</filename> in Solaris 10 because it is based on ksh88. This is a bug.).

</para></warning>

</section>

<section xml:id="use_proper_return_code">

<title>Use a proper <literal>return</literal> code</title>

<para>Explicitly set the return code of a function - otherwise the exit code

from the last command executed will be used which may trigger problems

if the value is unexpected.</para>

<para>The only allowed exception is if a function uses the shell's <literal>errexit</literal> mode to leave

a function, subshell or the script if a command returns a non-zero exit code.

</para>

</section>

<section xml:id="use_fpath_to_load_common_code">

<title>&tag_kshonly;Use <envar>FPATH</envar> to load common functions, not <literal>source</literal></title>

<para>

Use the ksh <envar>FPATH</envar> (function path) feature to load functions which are shared between scripts

and not <literal>source</literal> - this allows to load such a function on demand and not all at once.</para>

</section>

</section><!-- end of functions -->

<section xml:id="if_for_while">

<title><literal>if</literal>, <literal>for</literal> and <literal>while</literal></title>

<section xml:id="if_for_while_format">

<title>Format</title>

<para>To match <literal>cstyle</literal>, the shell token equivalent to the <literal>C</literal>

"<literal>{</literal>" should appear on the same line, separated by a

"<literal>;</literal>", as in:

<programlisting>

if [ "$x" = "hello" ] ; then

echo $x

fi

if [[ "$x" = "hello" ]] ; then

print $x

fi

for i in 1 2 3; do

echo $i

done

for ((i=0 ; i &lt; 3 ; i++)); do

print $i

done

while [ $# -gt 0 ]; do

echo $1

shift

done

while (( $# &gt; 0 )); do

print $1

shift

done

</programlisting>

</para>

</section>

<section xml:id="test_builtin">

<title><literal>test</literal> Builtin</title>

<para>DO NOT use the test builtin. Sorry, executive decision.</para>

<para>In our Bourne shell, the <literal>test</literal> built-in is the same as the "["

builtin (if you don't believe me, try "type test" or refer to <filename>usr/src/cmd/sh/msg.c</filename>).</para>

<para>

So please do not write:

<programlisting>

if test $# -gt 0 ; then

</programlisting>

instead use:

<programlisting>

if [ $# -gt 0 ] ; then

</programlisting>

</para>

</section>

<section xml:id="use_ksh_test_syntax">

<title>&tag_kshonly;&tag_performance;Use "<literal>[[ expr ]]</literal>" instead of "<literal>[ expr ]</literal>"</title>

<para>Use "<literal>[[ expr ]]</literal>" instead of "<literal>[ expr ]</literal>" if possible

since it avoids going through the whole pattern expansion/etc. machinery and

adds additional operators not available in the Bourne shell, such as short-circuit

<literal>&amp;&amp;</literal> and <literal>||</literal>.

</para>

</section>

<section xml:id="use_posix_arithmetic_expressions">

<title>&tag_kshonly; Use "<literal>(( ... ))</literal>" for arithmetic expressions</title>

<para>Use "<literal>(( ... ))</literal>" instead of "<literal>[ expr ]</literal>"

or "<literal>[[ expr ]]</literal>" expressions.

</para>

<para>

Example: Replace

<programlisting>

i=5

# do something

if [ $i -gt 5 ] ; then

</programlisting>

with

<programlisting>

i=5

# do something

if (( i &gt; 5 )) ; then

</programlisting>

</para>

</section>

<section xml:id="compare_exit_code_using_math">

<title>&tag_kshonly;&tag_performance;Compare exit code using arithmetic expressions expressions</title>

<para>Use POSIX arithmetic expressions to test for exit/return codes of commands and functions.

For example turn

<programlisting>

if [ $? -gt 0 ] ; then

</programlisting>

into

<programlisting>

if (( $? &gt; 0 )) ; then

</programlisting>

</para>

</section>

<section xml:id="use_builtin_commands_in_loops">

<title>&tag_bourneonly; Use builtin commands in conditions for <literal>while</literal> endless loops</title>

<para>Make sure that your shell has a "<literal>true</literal>" builtin (like ksh93) when

executing endless loops like <literal>$ while true ; do do_something ; done #</literal> -

otherwise each loop cycle runs a <literal>|fork()|+|exec()|</literal>-cycle to run

<filename>/bin/true</filename>

</para>

</section>

<section xml:id="single_line_if_statements">

<title>Single-line if-statements</title>

<para>It is permissible to use <literal>&amp;&amp;</literal> and <literal>||</literal> to construct

shorthand for an "<literal>if</literal>" statement in the case where the if statement has a

single consequent line:

<programlisting>

[ $# -eq 0 ] &amp;&amp; exit 0

</programlisting>

instead of the longer:

<programlisting>

if [ $# -eq 0 ]; then

exit 0

fi

</programlisting>

</para>

</section>

<section xml:id="exit_status_and_if_for_while">

<title>Exit Status and <literal>if</literal>/<literal>while</literal> statements</title>

<para>Recall that "<literal>if</literal>" and "<literal>while</literal>"

operate on the exit status of the statement

to be executed. In the shell, zero (0) means true and non-zero means false.

The exit status of the last command which was executed is available in the $?

variable. When using "<literal>if</literal>" and "<literal>while</literal>",

it is typically not necessary to use

<literal>$?</literal> explicitly, as in:

<programlisting>

grep foo /etc/passwd &gt;/dev/null 2>&amp;1

if [ $? -eq 0 ]; then

echo "found"

fi

</programlisting>

Instead, you can more concisely write:

<programlisting>

if grep foo /etc/passwd &gt;/dev/null 2>&amp;1; then

echo "found"

fi

</programlisting>

Or, when appropriate:

<programlisting>

grep foo /etc/passwd &gt;/dev/null 2>&amp;1 &amp;&amp; echo "found"

</programlisting>

</para>

</section>

</section><!-- end of if/for/while -->

<section xml:id="variables">

<title>Variable types, naming and usage</title>

<section xml:id="names_should_be_lowercase">

<title>Names of local, non-environment, non-constant variables should be lowercase</title>

<para>Names of variables local to the current script which are not exported to the environment

should be lowercase while variable names which are exported to the

environment should be uppercase.</para>

<para>The only exception are global constants (=global readonly variables,

e.g. <literal>$ float -r M_PI=3.14159265358979323846 #</literal> (taken from &lt;math.h&gt;))

which may be allowed to use uppercase names, too.

</para>

<warning><para>

Uppercase variable names should be avoided because there is a good chance

of naming collisions with either special variable names used by the shell

(e.g. <literal>PWD</literal>, <literal>SECONDS</literal> etc.).

</para></warning>

</section>

<section xml:id="do_not_reserved_keywords_for_variable_names">

<title>Do not use variable names which are reserved keywords/variable names in C/C++/JAVA or the POSIX shell standard</title>

<para>Do not use variable names which are reserved keywords in C/C++/JAVA or the POSIX shell standard

(to avoid confusion and/or future changes/updates to the shell language).

</para>

<note>

<para>The Korn Shell and the POSIX shell standard have many more

reserved variable names than the original Bourne shell. All

these reserved variable names are spelled uppercase.

</para>

</note>

</section>

<section xml:id="use_brackets_around_long_names">

<title>Always use <literal>'{'</literal>+<literal>'}'</literal> when using variable

names longer than one character</title>

<para>Always use <literal>'{'</literal>+<literal>'}'</literal> when using

variable names longer than one character unless a simple variable name is

followed by a blank, <literal>/</literal>, <literal>;</literal>, or <literal>$</literal>

character (to avoid problems with array,

compound variables or accidental misinterpretation by users/shell)

<programlisting>

print "$foo=info"

</programlisting>

should be rewritten to

<programlisting>

print "${foo}=info"

</programlisting>

</para>

</section>

<section xml:id="quote_variables_containing_filenames_or_userinput">

<title><emphasis>Always</emphasis> put variables into quotes when handling filenames or user input</title>

<para><emphasis>Always</emphasis> put variables into quotes when handling filenames or user input, even if

the values are hardcoded or the values appear to be fixed. Otherwise at

least two things may go wrong:

<itemizedlist>

<listitem><para>a malicious user may be able to exploit a script's inner working to

infect his/her own code</para></listitem>

<listitem><para>a script may (fatally) misbehave for unexpected input (e.g. file names

with blanks and/or special symbols which are interpreted by the shell)</para></listitem>

</itemizedlist>

</para>

<note><para>

As alternative a script may set <literal>IFS='' ; set -o noglob</literal> to turn off the

interpretation of any field seperators and the pattern globbing.

</para></note>

</section>

<section xml:id="use_typed_variables">

<title>&tag_kshonly;&tag_performance;Use typed variables if possible.</title>

<para>For example the following is very

inefficient since it transforms the integer values to strings and back

several times:

<programlisting>

a=0

b=1

c=2

# more code

if [ $a -lt 5 -o $b -gt c ] ; then do_something ; fi

</programlisting>

This could be rewritten using ksh constructs:

<programlisting>

integer a=0

integer b=1

integer c=2

# more code

if (( a &lt; 5 || b &gt; c )) ; then do_something ; fi

</programlisting>

</para>

</section>

<section xml:id="store_lists_in_arrays">

<title>&tag_ksh93only; Store lists in arrays or associative arrays</title>

<para>Store lists in arrays or associative arrays - this is usually easier

to manage.</para>

<para>

For example:

<programlisting>

x="

/etc/foo

/etc/bar

/etc/baz

"

echo $x

</programlisting>

can be replaced with

<programlisting>

typeset -a mylist

mylist[0]="/etc/foo"

mylist[1]="/etc/bar"

mylist[2]="/etc/baz"

print "${mylist[@]}"

</programlisting>

or (ksh93-style append entries to a normal (non-associative) array)

<programlisting>

typeset -a mylist

mylist+=( "/etc/foo" )

mylist+=( "/etc/bar" )

mylist+=( "/etc/baz" )

print "${mylist[@]}"

</programlisting>

</para>

<note>

<title>Difference between expanding arrays with mylist[@] and mylist[*] subscript operators</title>

<para>

Arrays may be expanded using two similar subscript operators, @ and *. These subscripts

differ only when the variable expansion appears within double quotes. If the variable expansion

is between double-quotes, "${mylist[*]}" expands to a single string with the value of each array

member separated by the first character of the <envar>IFS</envar> variable, and "${mylist[@]}"

expands each element of name to a separate string.

</para>

<example><title>Difference between [@] and [*] when expanding arrays</title>

<programlisting>

typeset -a mylist

mylist+=( "/etc/foo" )

mylist+=( "/etc/bar" )

mylist+=( "/etc/baz" )

IFS=","

printf "mylist[*]={ 0=|%s| 1=|%s| 2=|%s| 3=|%s| }\n" "${mylist[*]}"

printf "mylist[@]={ 0=|%s| 1=|%s| 2=|%s| 3=|%s| }\n" "${mylist[@]}"

</programlisting>

<para>will print:</para>

<screen>

<computeroutput>mylist[*]={ 0=|/etc/foo,/etc/bar,/etc/baz| 1=|| 2=|| 3=|| }

mylist[@]={ 0=|/etc/foo| 1=|/etc/bar| 2=|/etc/baz| 3=|| }

</computeroutput>

</screen>

</example>

</note>

</section>

<section xml:id="use_compound_variables_or_lists_for_grouping">

<title>&tag_ksh93only; Use compound variables or associative arrays to group similar variables together</title>

<para>Use compound variables or associative arrays to group similar variables together.</para>

<para>

For example:

<programlisting>

box_width=56

box_height=10

box_depth=19

echo "${box_width} ${box_height} ${box_depth}"

</programlisting>

could be rewritten to ("associative array"-style)

<programlisting>

typeset -A -E box=( [width]=56 [height]=10 [depth]=19 )

print -- "${box[width]} ${box[height]} ${box[depth]}"

</programlisting>

or ("compound variable"-style

<programlisting>

box=(

float width=56

float height=10

float depth=19

)

print -- "${box.width} ${box.height} ${box.depth}"

</programlisting>

</para>

</section>

</section><!-- end of variables -->

<section xml:id="io">

<title>I/O</title>

<section xml:id="avoid_echo">

<title>Avoid using the "<literal>echo</literal>" command for output</title>

<para>The behaviour of "<literal>echo</literal>" is not portable

(e.g. System V, BSD, UCB and ksh93/bash shell builtin versions all

slightly differ in functionality) and should be avoided if possible.

POSIX defines the "<literal>printf</literal>" command as replacement

which provides more flexible and portable behaviour.</para>

<note>

<title>&tag_kshonly;Use "<literal>print</literal>" and not "<literal>echo</literal>" in Korn Shell scripts</title>

<para>Korn shell scripts should prefer the "<literal>print</literal>"

builtin which was introduced as replacement for "<literal>echo</literal>".</para>

<caution>

<para>Use <literal>$ print -- ${varname}" #</literal> when there is the slightest chance that the

variable "<literal>varname</literal>" may contain symbols like "-". Or better use "<literal>printf</literal>"

instead, for example

<programlisting>

integer fx

# do something

print $fx

</programlisting>

may fail if "f" contains a negative value. A better way may be to use

<programlisting>

integer fx

# do something

printf "%d\n" fx

</programlisting>

</para>

</caution>

</note>

</section>

<section xml:id="use_redirect_not_exec_to_open_files">

<title>&tag_ksh93only;Use <literal>redirect</literal> and not <literal>exec</literal> to open files</title>

<para>Use <literal>redirect</literal> and not <literal>exec</literal> to open files - <literal>exec</literal>

will terminate the current function or script if an error occurs while <literal>redirect</literal>

just returns a non-zero exit code which can be caught.</para>

<para>Example:

<programlisting>

if redirect 5&lt;/etc/profile ; then

print "file open ok"

head &lt;&amp;5

else

print "could not open file"

fi

</programlisting>

</para>

</section>

<section xml:id="group_identical_redirections_together">

<title>&tag_performance;Avoid redirections per command when the output goes into the same file,

e.g. <literal>$ echo "foo" &gt;xxx ; echo "bar" &gt;&gt;xxx ; echo "baz" &gt;&gt;xxx #</literal></title>

<para>Each of the redirections above trigger an

<literal>|open()|,|write()|,|close()|</literal>-sequence. It is much

more efficient (and faster) to group the rediction into a block,

e.g. <literal>{ echo "foo" ; echo "bar" ; echo "baz" } &gt;xxx #</literal></para>

</section>

<section xml:id="avoid_using_temporary_files">

<title>&tag_performance;Avoid the creation of temporary files and store the values in variables instead</title>

<para>Avoid the creation of temporary files and store the values in variables instead if possible</para>

<para>

Example:

<programlisting>

ls -1 &gt;xxx

for i in $(cat xxx) ; do

do_something ;

done

</programlisting>

can be replaced with

<programlisting>

x="$(ls -1)"

for i in ${x} ; do

do_something ;

done

</programlisting>

</para>

<note><para>ksh93 supports binary variables (e.g. <literal>typeset -b varname</literal>) which can hold any value.</para></note>

</section>

<section xml:id="create_subdirs_for_multiple_temporary_files">

<title>If you create more than one temporary file create an unique subdir</title>

<para>If you create more than one temporary file create an unique subdir for

these files and make sure the dir is writable. Make sure you cleanup

after yourself (unless you are debugging).

</para>

</section>

<section xml:id="use_dynamic_file_descriptors">

<title>&tag_ksh93only;Use {n}&lt;file instead of fixed file descriptor numbers</title>

<para>When opening a file use {n}&lt;file, where <envar>n</envar> is an

integer variable rather than specifying a fixed descriptor number.</para>

<para>This is highly recommended in functions to avoid that fixed file

descriptor numbers interfere with the calling script.</para>

<example><title>Open a network connection and store the file descriptor number in a variable</title>

<programlisting>

function cat_http

{

integer netfd

...

# open TCP channel

redirect {netfd}&lt;&gt;"/dev/tcp/${host}/${port}"

# send HTTP request

request="GET /${path} HTTP/1.1\n"

request+="Host: ${host}\n"

request+="User-Agent: demo code/ksh93 (2007-08-30; $(uname -s -r -p))\n"

request+="Connection: close\n"

print "${request}\n" &gt;&amp;${netfd}

# collect response and send it to stdout

cat &lt;&amp;${netfd}

# close connection

exec {netfd}&lt;&amp;-

...

}

</programlisting>

</example>

</section>

<section xml:id="use_inline_here_documents">

<title>&tag_ksh93only;&tag_performance;Use inline here documents

instead of <literal>echo "$x" | command</literal></title>

<para>Use inline here documents, for example

<programlisting>

command &lt;&lt;&lt; $x

</programlisting>

rather than

<programlisting>

print -r -- "$x" | command

</programlisting>

</para>

</section>

<section xml:id="use_read_r">

<title>&tag_ksh93only;Use the <literal>-r</literal> option of <literal>read</literal> to read a line</title>

<para>Use the <literal>-r</literal> option of <literal>read</literal> to read a line.

You never know when a line will end in <literal>\</literal> and without a

<literal>-r</literal> multiple

lines can be read.</para>

</section>

<section xml:id="print_compound_variables_using_print_C">

<title>&tag_ksh93only;Print compound variables using <literal>print -C varname</literal> or <literal>print -v varname</literal></title>

<para>Print compound variables using <literal>print -C varname</literal> or

<literal>print -v varname</literal> to make sure that non-printable characters

are correctly encoded.</para>

<example><title>Print compound variable with non-printable characters</title>

<programlisting>

compound x=(

a=5

b="hello"

c=(

d=9

e="$(printf "1\v3")" <co xml:id="co.vertical_tab1" />

)

)

print -v x

</programlisting>

<para>will print:</para>

<screen>

<computeroutput>(

a=5

b=hello

c=(

d=9

e=$'1\0133' <co xml:id="co.vertical_tab2" />

)

)</computeroutput>

</screen>

<calloutlist>

<callout arearefs="co.vertical_tab1 co.vertical_tab2">

<para>vertical tab, <literal>\v</literal>, octal=<literal>\013</literal>.</para>

</callout>

</calloutlist>

</example>

</section>

<section xml:id="command_name_before_redirections">

<title>Put the command name and arguments before redirections</title>

<para>Put the command name and arguments before redirections.

You can legally do <literal>$ &gt; file date</literal> instead of <literal>date &gt; file</literal>

but don't do it.</para>

</section>

<section xml:id="enable_gmacs_editor_mode_for_user_prompts">

<title>&tag_ksh93only;Enable the <literal>gmacs</literal> editor

mode when reading user input using the <literal>read</literal> builtin</title>

<para>Enable the <literal>gmacs</literal>editor mode before reading user

input using the <literal>read</literal> builtin to enable the use of

cursor+backspace+delete keys in the edit line</para>

<example><title>Prompt user for a string with gmacs editor mode enabled</title>

<programlisting>

set -o gmacs <co xml:id="co.enable_gmacs" />

typeset inputstring="default value"

...

read -v<co xml:id="co.read_v" /> inputstring<co xml:id="co.readvar" />?"Please enter a string: "<co xml:id="co.prompt" />

...

printf "The user entered the following string: '%s'\n" "${inputstring}"

...

</programlisting>

<calloutlist>

<callout arearefs="co.enable_gmacs">

<para>Enable gmacs editor mode.</para>

</callout>

<callout arearefs="co.read_v">

<para>The value of the variable is displayed and used as a default value.</para>

</callout>

<callout arearefs="co.readvar">

<para>Variable used to store the result.</para>

</callout>

<callout arearefs="co.prompt">

<para>Prompt string which is displayed in stderr.</para>

</callout>

</calloutlist>

</example>

</section>

</section><!-- end of I/O -->

<section xml:id="math">

<title>Math</title>

<section xml:id="use_builtin_arithmetic_expressions">

<title>&tag_kshonly;&tag_performance;Use builtin arithmetic expressions instead of external applications</title>

<para>Use builtin (POSIX shell) arithmetic expressions instead of

<filename>expr</filename>,

<filename>bc</filename>,

<filename>dc</filename>,

<filename>awk</filename>,

<filename>nawk</filename> or

<filename>perl</filename>.

</para>

<note>

<para>ksh93 supports C99-like floating-point arithmetic including special values

such as

<simplelist type="inline">

<member>+Inf</member>

<member>-Inf</member>

<member>+NaN</member>

<member>-NaN</member>

</simplelist>.

</para>

</note>

</section>

<section xml:id="use_floating_point_arithmetic_expressions">

<title>&tag_ksh93only; Use floating-point arithmetic expressions if

calculations may trigger a division by zero or other exceptions</title>

<para>Use floating-point arithmetic expressions if calculations may

trigger a division by zero or other exceptions - floating point arithmetic expressions in

ksh93 support special values such as <literal>+Inf</literal>/<literal>-Inf</literal> and

<literal>+NaN</literal>/<literal>-NaN</literal> which can greatly simplify testing for

error conditions, e.g. instead of a <literal>trap</literal> or explicit

<literal>if ... then... else</literal> checks for every sub-expression

you can check the results for such special values.

</para>

<para>Example:

<screen>

$ <userinput>ksh93 -c 'integer i=0 j=5 ; print -- "x=$((j/i)) "'</userinput>

<computeroutput>ksh93: line 1: j/i: divide by zero</computeroutput>

$ <userinput>ksh93 -c 'float i=0 j=-5 ; print -- "x=$((j/i)) "'</userinput>

<computeroutput>x=-Inf</computeroutput>

</screen>

</para>

</section>

<section xml:id="use_printf_a_for_passing_float_values">

<title>&tag_ksh93only; Use <literal>printf "%a"</literal> when passing floating-point values</title>

<para>Use <literal>printf "%a"</literal> when passing floating-point values between scripts or

as output of a function to avoid rounding errors when converting between

bases.</para>

<para>

Example:

<programlisting>

function xxx

{

float val

(( val=sin(5.) ))

printf "%a\n" val

}

float out

(( out=$(xxx) ))

xxx

print -- $out

</programlisting>

This will print:

<programlisting>

-0.9589242747

</programlisting>

</para>

</section>

<section xml:id="put_constants_into_readonly_variables">

<title>&tag_kshonly;&tag_performance;Put constant values into readonly variables</title>

<para>Put constant values into readonly variables</para>

<para>For example:

<programlisting>

float -r M_PI=3.14159265358979323846

</programlisting>

or

<programlisting>

float M_PI=3.14159265358979323846

readonly M_PI

</programlisting>

</para>

</section>

<section xml:id="avoid_unnecessary_string_number_conversions">

<title>&tag_kshonly;&tag_performance;Avoid string to number

(and/or number to string) conversions in arithmetic expressions

expressions</title>

<para>Avoid string to number and/or number to string conversions in

arithmetic expressions expressions to avoid performance degradation

and rounding errors.</para>

<example><title>(( x=$x*2 )) vs. (( x=x*2 ))</title>

<programlisting>

float x

...

(( x=$x*2 ))

</programlisting>

<para>

will convert the variable "x" (stored in the machine's native

<literal>|long double|</literal> datatype) to a string value in base10 format,

apply pattern expansion (globbing), then insert this string into the

arithmetic expressions and parse the value which converts it into the internal |long double| datatype format again.

This is both slow and generates rounding errors when converting the floating-point value between

the internal base2 and the base10 representation of the string.

</para>

<para>

The correct usage would be:

</para>

<programlisting>

float x

...

(( x=x*2 ))

</programlisting>

<para>

e.g. omit the '$' because it's (at least) redundant within arithmetic expressions.

</para>

</example>

<example><title>x=$(( y+5.5 )) vs. (( x=y+5.5 ))</title>

<programlisting>

float x

float y=7.1

...

x=$(( y+5.5 ))

</programlisting>

<para>

will calculate the value of <literal>y+5.5</literal>, convert it to a

base-10 string value amd assign the value to the floating-point variable

<literal>x</literal> again which will convert the string value back to the

internal |long double| datatype format again.

</para>

<para>

The correct usage would be:

</para>

<programlisting>

float x

float y=7.1

...

(( x=y+5.5 ))

</programlisting>

<para>

i.e. this will save the string conversions and avoid any base2--&gt;base10--&gt;base2-conversions.

</para>

</example>

</section>

<section xml:id="set_lc_numeric_when_using_floating_point">

<title>&tag_ksh93only;Set <envar>LC_NUMERIC</envar> when using floating-point constants</title>

<para>Set <envar>LC_NUMERIC</envar> when using floating-point constants to avoid problems with radix-point

representations which differ from the representation used in the script, for example the <literal>de_DE.*</literal> locale

use ',' instead of '.' as default radix point symbol.</para>

<para>For example:

<programlisting>

# Make sure all math stuff runs in the "C" locale to avoid problems with alternative

# radix point representations (e.g. ',' instead of '.' in de_DE.*-locales). This

# needs to be set _before_ any floating-point constants are defined in this script)

if [[ "${LC_ALL}" != "" ]] ; then

export \

LC_MONETARY="${LC_ALL}" \

LC_MESSAGES="${LC_ALL}" \

LC_COLLATE="${LC_ALL}" \

LC_CTYPE="${LC_ALL}"

unset LC_ALL

fi

export LC_NUMERIC=C

...

float -r M_PI=3.14159265358979323846

</programlisting>

</para>

<note><para>The environment variable <envar>LC_ALL</envar> always overrides all other <envar>LC_*</envar> variables,

including <envar>LC_NUMERIC</envar>. The script should always protect itself against custom <envar>LC_NUMERIC</envar> and

<envar>LC_ALL</envar> values as shown in the example above.

</para></note>

</section>

</section><!-- end of math -->

<section xml:id="misc">

<title>Misc</title>

<section xml:id="debug_use_lineno_in_ps4">

<title>Put <literal>[${LINENO}]</literal> in your <envar>PS4</envar></title>

<para>Put <literal>[${LINENO}]</literal> in your <envar>PS4</envar> prompt so that you will get line

numbers with you run with <literal>-x</literal>. If you are looking at performance

issues put <literal>$SECONDS</literal> in the <envar>PS4</envar> prompt as well.</para>

</section>

</section><!-- end of misc -->

</section><!-- end of RULES -->

</article>