thread_safety.html.en revision 8b7e19de6d547ab1ad4256316fbf0d2497f724f8
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head><!--
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen This file is generated from xml source: DO NOT EDIT
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<title>Apache HTTP Server 2.x Thread Safety Issues - Apache HTTP Server</title>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<link href="/style/css/manual.css" rel="stylesheet" media="all" type="text/css" title="Main stylesheet" />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<link href="/style/css/manual-loose-100pc.css" rel="alternate stylesheet" media="all" type="text/css" title="No Sidebar - Default font size" />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<link href="/style/css/manual-print.css" rel="stylesheet" media="print" type="text/css" />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<link href="/images/favicon.ico" rel="shortcut icon" /></head>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<p class="menu"><a href="/mod/">Modules</a> | <a href="/mod/directives.html">Directives</a> | <a href="/faq/">FAQ</a> | <a href="/glossary.html">Glossary</a> | <a href="/sitemap.html">Sitemap</a></p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<div class="up"><a href="./"><img title="<-" alt="<-" src="/images/left.gif" /></a></div>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<a href="http://www.apache.org/">Apache</a> > <a href="http://httpd.apache.org/">HTTP Server</a> > <a href="http://httpd.apache.org/docs/">Documentation</a> > <a href="../">Version 2.5</a> > <a href="./">Developer Documentation</a></div><div id="page-content"><div id="preamble"><h1>Apache HTTP Server 2.x Thread Safety Issues</h1>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<p><span>Available Languages: </span><a href="/en/developer/thread_safety.html" title="English"> en </a></p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>When using any of the threaded mpms in the Apache HTTP Server 2.x it is important
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen that every function called from Apache be thread safe. When linking in 3rd
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen party extensions it can be difficult to determine whether the resulting
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen server will be thread safe. Casual testing generally won't tell you this
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen either as thread safety problems can lead to subtle race conditons that
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen may only show up in certain conditions under heavy load.</p>
268200a84e420471f8a30824b4f3e371b601d089slive<div id="quickview"><ul id="toc"><li><img alt="" src="/images/down.gif" /> <a href="#variables">Global and static variables</a></li>
268200a84e420471f8a30824b4f3e371b601d089slive<li><img alt="" src="/images/down.gif" /> <a href="#errno">errno</a></li>
268200a84e420471f8a30824b4f3e371b601d089slive<li><img alt="" src="/images/down.gif" /> <a href="#functions">Common standard troublesome functions</a></li>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<li><img alt="" src="/images/down.gif" /> <a href="#commonlibs">Common 3rd Party Libraries</a></li>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<li><img alt="" src="/images/down.gif" /> <a href="#liblist">Library List</a></li>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<div class="top"><a href="#page-header"><img alt="top" src="/images/up.gif" /></a></div>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<h2><a name="variables" id="variables">Global and static variables</a></h2>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>When writing your module or when trying to determine if a module or
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen 3rd party library is thread safe there are some common things to keep in
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>First, you need to recognize that in a threaded model each individual
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen thread has its own program counter, stack and registers. Local variables
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen live on the stack, so those are fine. You need to watch out for any
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen static or global variables. This doesn't mean that you are absolutely not
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen allowed to use static or global variables. There are times when you
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen actually want something to affect all threads, but generally you need to
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen avoid using them if you want your code to be thread safe.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>In the case where you have a global variable that needs to be global and
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen accessed by all threads, be very careful when you update it. If, for
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen example, it is an incrementing counter, you need to atomically increment
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen it to avoid race conditions with other threads. You do this using a mutex
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen (mutual exclusion). Lock the mutex, read the current value, increment it
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen and write it back and then unlock the mutex. Any other thread that wants
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen to modify the value has to first check the mutex and block until it is
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen cleared.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>If you are using <a href="http://apr.apache.org/">APR</a>, have a look
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen at the <code>apr_atomic_<var>*</var></code> functions and the
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>apr_thread_mutex_<var>*</var></code> functions.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen</div><div class="top"><a href="#page-header"><img alt="top" src="/images/up.gif" /></a></div>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>This is a common global variable that holds the error number of the
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen last error that occurred. If one thread calls a low-level function that
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen sets errno and then another thread checks it, we are bleeding error
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen numbers from one thread into another. To solve this, make sure your
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen module or library defines <code>_REENTRANT</code> or is compiled with
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>-D_REENTRANT</code>. This will make errno a per-thread variable
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen and should hopefully be transparent to the code. It does this by doing
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen something like this:</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen #define errno (*(__errno_location()))
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>which means that accessing errno will call
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>__errno_location()</code> which is provided by the libc. Setting
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>_REENTRANT</code> also forces redefinition of some other functions
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen to their <code><var>*</var>_r</code> equivalents and sometimes changes
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen the common <code>getc</code>/<code>putc</code> macros into safer function
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen calls. Check your libc documentation for specifics. Instead of, or in
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen addition to <code>_REENTRANT</code> the symbols that may affect this are
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>_POSIX_C_SOURCE</code>, <code>_THREAD_SAFE</code>,
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>_SVID_SOURCE</code>, and <code>_BSD_SOURCE</code>.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen</div><div class="top"><a href="#page-header"><img alt="top" src="/images/up.gif" /></a></div>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<h2><a name="functions" id="functions">Common standard troublesome functions</a></h2>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>Not only do things have to be thread safe, but they also have to be
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen reentrant. <code>strtok()</code> is an obvious one. You call it the first
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen time with your delimiter which it then remembers and on each subsequent
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen call it returns the next token. Obviously if multiple threads are
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen calling it you will have a problem. Most systems have a reentrant version
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive of of the function called <code>strtok_r()</code> where you pass in an
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen extra argument which contains an allocated <code>char *</code> which the
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen function will use instead of its own static storage for maintaining
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen the tokenizing state. If you are using <a href="http://apr.apache.org/">APR</a> you can use <code>apr_strtok()</code>.</p>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive <p><code>crypt()</code> is another function that tends to not be reentrant,
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen so if you run across calls to that function in a library, watch out. On
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen some systems it is reentrant though, so it is not always a problem. If
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen your system has <code>crypt_r()</code> chances are you should be using
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen that, or if possible simply avoid the whole mess by using md5 instead.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen</div><div class="top"><a href="#page-header"><img alt="top" src="/images/up.gif" /></a></div>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<h2><a name="commonlibs" id="commonlibs">Common 3rd Party Libraries</a></h2>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>The following is a list of common libraries that are used by 3rd party
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen Apache modules. You can check to see if your module is using a potentially
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen unsafe library by using tools such as <code>ldd(1)</code> and
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>nm(1)</code>. For <a href="http://www.php.net/">PHP</a>, for example,
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen try this:</p>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive libsablot.so.0 => /usr/local/lib/libsablot.so.0 (0x401f6000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libexpat.so.0 => /usr/lib/libexpat.so.0 (0x402da000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libsnmp.so.0 => /usr/lib/libsnmp.so.0 (0x402f9000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libpdf.so.1 => /usr/local/lib/libpdf.so.1 (0x40353000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libz.so.1 => /usr/lib/libz.so.1 (0x403e2000)<br />
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive libpng.so.2 => /usr/lib/libpng.so.2 (0x403f0000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libmysqlclient.so.11 => /usr/lib/libmysqlclient.so.11 (0x40411000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libming.so => /usr/lib/libming.so (0x40449000)<br />
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x404a8000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0x404e7000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libcrypt.so.1 => /lib/libcrypt.so.1 (0x40505000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libssl.so.2 => /lib/libssl.so.2 (0x40532000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libcrypto.so.2 => /lib/libcrypto.so.2 (0x40560000)<br />
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive libresolv.so.2 => /lib/libresolv.so.2 (0x40624000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libnsl.so.1 => /lib/libnsl.so.1 (0x40637000)<br />
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>In addition to these libraries you will need to have a look at any
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen libraries linked statically into the module. You can use <code>nm(1)</code>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive to look for individual symbols in the module.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen</div><div class="top"><a href="#page-header"><img alt="top" src="/images/up.gif" /></a></div>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<h2><a name="liblist" id="liblist">Library List</a></h2>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <p>Please drop a note to <a href="http://httpd.apache.org/lists.html#http-dev">dev@httpd.apache.org</a>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive if you have additions or corrections to this list.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <table class="bordered"><tr class="header"><th>Library</th><th>Version</th><th>Thread Safe?</th><th>Notes</th></tr>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr><td><a href="http://aspell.sourceforge.net/">ASpell/PSpell</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://www.sleepycat.com/">Berkeley DB</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>Be careful about sharing a connection across threads.</td></tr>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://sources.redhat.com/bzip2/index.html">bzip2</a></td>
c31b0f4fe27c68b46a051eb0b48a485c7bc3a875slive <td>Both low-level and high-level APIs are thread-safe. However,
c31b0f4fe27c68b46a051eb0b48a485c7bc3a875slive high-level API requires thread-safe access to errno.</td></tr>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr class="odd"><td><a href="http://cr.yp.to/cdb.html">cdb</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr><td><a href="http://www.washington.edu/imap/">C-Client</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>gethostbyname()</code> which are not thread-safe on most C
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen library implementations. c-client's static data is meant to be shared
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <code>gethostbyname()</code> are thread-safe on your OS, c-client
393f9c667e58b8cd6083ce03d5e03cc9f7b0dfaabnicholes<tr class="odd"><td><a href="http://www.ijg.org/files/">libcrypt</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr><td><a href="http://expat.sourceforge.net/">Expat</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>Need a separate parser instance per thread</td></tr>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr class="odd"><td><a href="http://www.freetds.org/">FreeTDS</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr><td><a href="http://www.freetype.org/">FreeType</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr class="odd"><td><a href="http://www.boutell.com/gd/">GD 1.8.x</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://www.boutell.com/gd/">GD 2.0.x</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://www.gnu.org/software/gdbm/gdbm.html">gdbm</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>Errors returned via a static <code>gdbm_error</code>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://www.imagemagick.org/">ImageMagick</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>ImageMagick docs claim it is thread safe since version 5.2.2 (see <a href="http://www.imagemagick.com/www/changelog.html">Change log</a>).
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://www.enlightenment.org/p.php?p=about/efl&l=en">Imlib2</a></td>
6ec3b97fc9c9f163cfaae67fc448eea3f887f34dslive<tr><td><a href="http://www.ijg.org/files/">libjpeg</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://mysql.com">libmysqlclient</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive <td>Use mysqlclient_r library variant to ensure thread-safety. For
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive more information, please read <a href="http://dev.mysql.com/doc/mysql/en/Threaded_clients.html">http://dev.mysql.com/doc/mysql/en/Threaded_clients.html</a>.</td></tr>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr><td><a href="http://www.opaque.net/ming/">Ming</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://net-snmp.sourceforge.net/">Net-SNMP</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://www.openldap.org/">OpenLDAP</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>Use <code>ldap_r</code> library variant to ensure
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://www.openssl.org/">OpenSSL</a></td>
3a9b96be347a9900a5ed381d2d7dba0129685b38yoshiki <td>Requires proper usage of <code>CRYPTO_num_locks</code>,
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://www.oracle.com/">liboci8 (Oracle 8+)</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr class="odd"><td><a href="http://pdflib.com/">pdflib</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>PDFLib docs claim it is thread safe; changes.txt indicates it
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen has been partially thread-safe since V1.91: <a href="http://www.pdflib.com/products/pdflib-family/pdflib/">http://www.pdflib.com/products/pdflib-family/pdflib/</a>.</td></tr>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://www.libpng.org/pub/png/libpng.html">libpng</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr class="odd"><td><a href="http://www.libpng.org/pub/png/libpng.html">libpng</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<tr><td><a href="http://www.postgresql.org/docs/8.4/static/libpq-threading.html">libpq (PostgreSQL)</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen <td>Don't share connections across threads and watch out for
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr class="odd"><td><a href="http://www.gingerall.com/charlie/ga/xml/p_sab.xml">Sablotron</a></td>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<tr><td><a href="http://www.gzip.org/zlib/">zlib</a></td>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive <td>Relies upon thread-safe zalloc and zfree functions Default is to
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen use libc's calloc/free which are thread-safe.</td></tr>
519e249ff843c8805d5ce2fdc299caacf7ad9e43slive<p><span>Available Languages: </span><a href="/en/developer/thread_safety.html" title="English"> en </a></p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<p class="apache">Copyright 2012 The Apache Software Foundation.<br />Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.</p>
57d0156f7bbd9ea3a72342cf9912aba61d118702rbowen<p class="menu"><a href="/mod/">Modules</a> | <a href="/mod/directives.html">Directives</a> | <a href="/faq/">FAQ</a> | <a href="/glossary.html">Glossary</a> | <a href="/sitemap.html">Sitemap</a></p></div>