perf-tuning.html revision fbf96f0f97e556a0c5203e6d511b07a3d386c1fe
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<!-- Background white, links blue (unvisited), navy (visited), red (active) -->
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi BGCOLOR="#FFFFFF"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi TEXT="#000000"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi LINK="#0000FF"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi VLINK="#000080"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi ALINK="#FF0000"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Author: Dean Gaudet
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Apache is a general webserver, which is designed to be correct first, and
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifast second. Even so, it's performance is quite satisfactory. Most
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisites have less than 10Mbits of outgoing bandwidth, which Apache can
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifill using only a low end Pentium-based webserver. In practice sites
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwith more bandwidth require more than one machine to fill the bandwidth
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidue to other constraints (such as CGI or database transaction overhead).
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiFor these reasons the development focus has been mostly on correctness
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand configurability.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Unfortunately many folks overlook these facts and cite raw performance
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinumbers as if they are some indication of the quality of a web server
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiproduct. There is a bare minimum performance that is acceptable, beyond
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithat extra speed only caters to a much smaller segment of the market.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiBut in order to avoid this hurdle to the acceptance of Apache in some
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimarkets, effort was put into Apache 1.3 to bring performance up to a
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchipoint where the difference with other high-end webservers is minimal.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Finally there are the folks who just plain want to see how fast something
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchican go. The author falls into this category. The rest of this document
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis dedicated to these folks who want to squeeze every last bit of
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiperformance out of Apache's current model, and want to understand why
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiit does some things which slow it down.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Note that this is tailored towards Apache 1.3 on Unix. Some of it applies
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito Apache on NT. Apache on NT has not been tuned for performance yet,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin fact it probably performs very poorly because NT performance requires
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchia different programming model.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<H3>Hardware and Operating System Issues</H3>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>The single biggest hardware issue affecting webserver performance
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis RAM. A webserver should never ever have to swap, swapping increases
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe latency of each request beyond a point that users consider "fast
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchienough". This causes users to hit stop and reload, further increasing
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe load. You can, and should, control the <CODE>MaxClients</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisetting so that your server does not spawn so many children it starts
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Beyond that the rest is mundane: get a fast enough CPU, a fast enough
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinetwork card, and fast enough disks, where "fast enough" is something
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithat needs to be determined by experimentation.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Operating system choice is largely a matter of local concerns. But
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchia general guideline is to always apply the latest vendor TCP/IP patches.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiHTTP serving completely breaks many of the assumptions built into Unix
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchikernels up through 1994 and even 1995. Good choices include
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchirecent FreeBSD, and Linux.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Prior to Apache 1.3, <CODE>HostnameLookups</CODE> defaulted to On.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis adds latency
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito every request because it requires a DNS lookup to complete before
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe request is finished. In Apache 1.3 this setting defaults to Off.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiHowever (1.3 or later), if you use any <CODE>allow from domain</CODE> or
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>deny from domain</CODE> directives then you will pay for a
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidouble reverse DNS lookup (a reverse, followed by a forward to make sure
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithat the reverse is not being spoofed). So for the highest performance
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiavoid using these directives (it's fine to use IP addresses rather than
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidomain names).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Note that it's possible to scope the directives, such as within
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchia <CODE><Location /server-status></CODE> section. In this
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicase the DNS lookups are only performed on requests matching the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicriteria. Here's an example which disables
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilookups except for .html and .cgi files:
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiHostnameLookups off
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<Files ~ "\.(html|cgi)$>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi HostnameLookups on
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi</Files>
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiBut even still, if you just need DNS names
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin some CGIs you could consider doing the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>gethostbyname</CODE> call in the specific CGIs that need it.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<H4>FollowSymLinks and SymLinksIfOwnerMatch</H4>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Wherever in your URL-space you do not have an
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>Options FollowSymLinks</CODE>, or you do have an
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>Options SymLinksIfOwnerMatch</CODE> Apache will have to
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiissue extra system calls to check up on symlinks. One extra call per
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifilename component. For example, if you had:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<Directory />
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi Options SymLinksIfOwnerMatch
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi</Directory>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand a request is made for the URI <CODE>/index.html</CODE>.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThen Apache will perform <CODE>lstat(2)</CODE> on <CODE>/www</CODE>,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>/www/htdocs</CODE>, and <CODE>/www/htdocs/index.html</CODE>. The
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiresults of these <CODE>lstats</CODE> are never cached,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiso they will occur on every single request. If you really desire the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisymlinks security checking you can do something like this:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<Directory />
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi Options FollowSymLinks
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi</Directory>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi Options -FollowSymLinks +SymLinksIfOwnerMatch
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi</Directory>
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis at least avoids the extra checks for the <CODE>DocumentRoot</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchipath. Note that you'll need to add similar sections if you have any
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>Alias</CODE> or <CODE>RewriteRule</CODE> paths outside of your
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidocument root. For highest performance, and no symlink protection,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiset <CODE>FollowSymLinks</CODE> everywhere, and never set
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Wherever in your URL-space you allow overrides (typically
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>.htaccess</CODE> files) Apache will attempt to open
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>.htaccess</CODE> for each filename component. For example,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<Directory />
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi AllowOverride all
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi</Directory>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand a request is made for the URI <CODE>/index.html</CODE>. Then
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiApache will attempt to open <CODE>/.htaccess</CODE>,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>/www/.htaccess</CODE>, and <CODE>/www/htdocs/.htaccess</CODE>.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThe solutions are similar to the previous case of <CODE>Options
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiFollowSymLinks</CODE>. For highest performance use
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>AllowOverride None</CODE> everywhere in your filesystem.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>If at all possible, avoid content-negotiation if you're really
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiinterested in every last ounce of performance. In practice the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibenefits of negotiation outweigh the performance penalties. There's
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchione case where you can speed up the server. Instead of using
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchia wildcard such as:
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiDirectoryIndex index
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiUse a complete list of options:
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiDirectoryIndex index.cgi index.pl index.shtml index.html
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwhere you list the most common choice first.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Prior to Apache 1.3 the <CODE>MinSpareServers</CODE>,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>MaxSpareServers</CODE>, and <CODE>StartServers</CODE> settings
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiall had drastic effects on benchmark results. In particular, Apache
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchirequired a "ramp-up" period in order to reach a number of children
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisufficient to serve the load being applied. After the initial
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchispawning of <CODE>StartServers</CODE> children, only one child per
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisecond would be created to satisfy the <CODE>MinSpareServers</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisetting. So a server being accessed by 100 simultaneous clients,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiusing the default <CODE>StartServers</CODE> of 5 would take on
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe order 95 seconds to spawn enough children to handle the load. This
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiworks fine in practice on real-life servers, because they aren't restarted
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifrequently. But does really poorly on benchmarks which might only run
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifor ten minutes.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>The one-per-second rule was implemented in an effort to avoid
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiswamping the machine with the startup of new children. If the machine
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis busy spawning children it can't service requests. But it has such
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchia drastic effect on the perceived performance of Apache that it had
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito be replaced. As of Apache 1.3,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe code will relax the one-per-second rule. It
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwill spawn one, wait a second, then spawn two, wait a second, then spawn
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifour, and it will continue exponentially until it is spawning 32 children
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiper second. It will stop whenever it satisfies the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>This appears to be responsive enough that it's
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchialmost unnecessary to twiddle the <CODE>MinSpareServers</CODE>,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>MaxSpareServers</CODE> and <CODE>StartServers</CODE> knobs. When
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimore than 4 children are spawned per second, a message will be emitted
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito the <CODE>ErrorLog</CODE>. If you see a lot of these errors then
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconsider tuning these settings. Use the <CODE>mod_status</CODE> output
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Related to process creation is process death induced by the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>MaxRequestsPerChild</CODE> setting. By default this is 30, which
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis probably far too low unless your server is using a module such as
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>mod_perl</CODE> which causes children to have bloated memory
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiimages. If your server is serving mostly static pages then consider
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiraising this value to something like 10000. The code is robust enough
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithat this shouldn't be a problem.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>When keep-alives are in use, children will be kept busy
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidoing nothing waiting for more requests on the already open
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconnection. The default <CODE>KeepAliveTimeout</CODE> of
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi15 seconds attempts to minimize this effect. The tradeoff
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchihere is between network bandwidth and server resources.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiIn no event should you raise this above about 60 seconds, as
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.research.digital.com/wrl/techreports/abstracts/95.4.html"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi>most of the benefits are lost</A>.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand you also set <CODE>Rule STATUS=yes</CODE> when building
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiApache, then on every request Apache will perform two calls to
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>gettimeofday(2)</CODE> (or <CODE>times(2)</CODE> depending
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchion your operating system), and (pre-1.3) several extra calls to
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>time(2)</CODE>. This is all done so that the status report
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicontains timing indications. For highest performance, set <CODE>Rule
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiSTATUS=no</CODE>.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<H4>accept Serialization - multiple sockets</H4>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>This discusses a shortcoming in the Unix socket API.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiweb server uses multiple <CODE>Listen</CODE> statements to listen on
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchieither multiple ports or multiple addresses. In order to test each
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisocket to see if a connection is ready Apache uses <CODE>select(2)</CODE>.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>select(2)</CODE> indicates that a socket has <EM>zero</EM> or
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<EM>at least one</EM> connection waiting on it. Apache's model includes
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimultiple children, and all the idle ones test for new connections at the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisame time. A naive implementation looks something like this
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(these examples do not match the code, they're contrived for
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchipedagogical purposes):
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi fd_set accept_fds;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi FD_ZERO (&accept_fds);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi for (i = first_socket; i <= last_socket; ++i) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi FD_SET (i, &accept_fds);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi rc = select (last_socket+1, &accept_fds, NULL, NULL, NULL);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (rc < 1) continue;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi new_connection = -1;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi for (i = first_socket; i <= last_socket; ++i) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (FD_ISSET (i, &accept_fds)) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi new_connection = accept (i, NULL, NULL);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (new_connection != -1) break;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (new_connection != -1) break;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi process the new_connection;
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiBut this naive implementation has a serious starvation problem. Recall
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithat multiple children execute this loop at the same time, and so multiple
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchichildren will block at <CODE>select</CODE> when they are in between
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchirequests. All those blocked children will awaken and return from
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>select</CODE> when a single request appears on any socket
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(the number of children which awaken varies depending on the operating
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisystem and timing issues).
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThey will all then fall down into the loop and try to <CODE>accept</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe connection. But only one will succeed (assuming there's still only
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchione connection ready), the rest will be <EM>blocked</EM> in
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis effectively locks those children into serving requests from that
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchione socket and no other sockets, and they'll be stuck there until enough
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinew requests appear on that socket to wake them all up.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis starvation problem was first documented in
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://bugs.apache.org/index/full/467">PR#467</A>. There
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiare at least two solutions.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>One solution is to make the sockets non-blocking. In this case the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>accept</CODE> won't block the children, and they will be allowed
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito continue immediately. But this wastes CPU time. Suppose you have
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiten idle children in <CODE>select</CODE>, and one connection arrives.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThen nine of those children will wake up, try to <CODE>accept</CODE> the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconnection, fail, and loop back into <CODE>select</CODE>, accomplishing
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinothing. Meanwhile none of those children are servicing requests that
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchioccurred on other sockets until they get back up to the <CODE>select</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiagain. Overall this solution does not seem very fruitful unless you
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchihave as many idle CPUs (in a multiprocessor box) as you have idle children,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinot a very likely situation.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Another solution, the one used by Apache, is to serialize entry into
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe inner loop. The loop looks like this (differences highlighted):
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi fd_set accept_fds;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi FD_ZERO (&accept_fds);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi for (i = first_socket; i <= last_socket; ++i) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi FD_SET (i, &accept_fds);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi rc = select (last_socket+1, &accept_fds, NULL, NULL, NULL);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (rc < 1) continue;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi new_connection = -1;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi for (i = first_socket; i <= last_socket; ++i) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (FD_ISSET (i, &accept_fds)) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi new_connection = accept (i, NULL, NULL);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (new_connection != -1) break;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (new_connection != -1) break;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi process the new_connection;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>accept_mutex_on</CODE> and <CODE>accept_mutex_off</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiimplement a mutual exclusion semaphore. Only one child can have the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimutex at any time. There are several choices for implementing these
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimutexes. The choice is defined in <CODE>src/conf.h</CODE> (pre-1.3) or
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>src/main/conf.h</CODE> (1.3 or later). Some architectures
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchido not have any locking choice made, on these architectures it is unsafe
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito use multiple <CODE>Listen</CODE> directives.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DT><CODE>USE_FLOCK_SERIALIZED_ACCEPT</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DD>This method uses the <CODE>flock(2)</CODE> system call to lock a
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilock file (located by the <CODE>LockFile</CODE> directive).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DT><CODE>USE_FCNTL_SERIALIZED_ACCEPT</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DD>This method uses the <CODE>fcntl(2)</CODE> system call to lock a
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilock file (located by the <CODE>LockFile</CODE> directive).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DT><CODE>USE_SYSVSEM_SERIALIZED_ACCEPT</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DD>(1.3 or later) This method uses SysV-style semaphores to implement the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimutex. Unfortunately SysV-style semaphores have some bad side-effects.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiOne is that it's possible Apache will die without cleaning up the semaphore
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(see the <CODE>ipcs(8)</CODE> man page). The other is that the semaphore
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiAPI allows for a denial of service attack by any CGIs running under the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisame uid as the webserver (i.e. all CGIs unless you use something
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilike suexec or cgiwrapper). For these reasons this method is not used
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchion any architecture except IRIX (where the previous two are prohibitively
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiexpensive on most IRIX boxes).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DT><CODE>USE_USLOCK_SERIALIZED_ACCEPT</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DD>(1.3 or later) This method is only available on IRIX, and uses
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>usconfig(2)</CODE> to create a mutex. While this method avoids
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe hassles of SysV-style semaphores, it is not the default for IRIX.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis is because on single processor IRIX boxes (5.3 or 6.2) the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiuslock code is two orders of magnitude slower than the SysV-semaphore
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicode. On multi-processor IRIX boxes the uslock code is an order of magnitude
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifaster than the SysV-semaphore code. Kind of a messed up situation.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiSo if you're using a multiprocessor IRIX box then you should rebuild your
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwebserver with <CODE>-DUSE_USLOCK_SERIALIZED_ACCEPT</CODE> on the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DT><CODE>USE_PTHREAD_SERIALIZED_ACCEPT</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<DD>(1.3 or later) This method uses POSIX mutexes and should work on
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiany architecture implementing the full POSIX threads specification,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchihowever appears to only work on Solaris (2.5 or later), and even then
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchionly in certain configurations. If you experiment with this you should
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwatch out for your server hanging and not responding. Static content
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchionly servers may work just fine.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>If your system has another method of serialization which isn't in the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiabove list then it may be worthwhile adding code for it (and submitting
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchia patch back to Apache).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Another solution that has been considered but never implemented is
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito partially serialize the loop -- that is, let in a certain number
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiof processes. This would only be of interest on multiprocessor boxes
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwhere it's possible multiple children could run simultaneously, and the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiserialization actually doesn't take advantage of the full bandwidth.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis is a possible area of future investigation, but priority remains
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilow because highly parallel web servers are not the norm.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Ideally you should run servers without multiple <CODE>Listen</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchistatements if you want the highest performance. But read on.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<H4>accept Serialization - single socket</H4>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>The above is fine and dandy for multiple socket servers, but what
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiabout single socket servers? In theory they shouldn't experience
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiany of these same problems because all children can just block in
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>accept(2)</CODE> until a connection arrives, and no starvation
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiresults. In practice this hides almost the same "spinning" behaviour
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidiscussed above in the non-blocking solution. The way that most TCP
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchistacks are implemented, the kernel actually wakes up all processes blocked
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin <CODE>accept</CODE> when a single connection arrives. One of those
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiprocesses gets the connection and returns to user-space, the rest spin in
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe kernel and go back to sleep when they discover there's no connection
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifor them. This spinning is hidden from the user-land code, but it's
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithere nonetheless. This can result in the same load-spiking wasteful
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibehaviour that a non-blocking solution to the multiple sockets case can.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>For this reason we have found that many architectures behave more
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi"nicely" if we serialize even the single socket case. So this is
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiactually the default in almost all cases. Crude experiments under
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiLinux (2.0.30 on a dual Pentium pro 166 w/128Mb RAM) have shown that
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe serialization of the single socket case causes less than a 3%
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidecrease in requests per second over unserialized single-socket.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiBut unserialized single-socket showed an extra 100ms latency on
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchieach request. This latency is probably a wash on long haul lines,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand only an issue on LANs. If you want to override the single socket
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiserialization you can define <CODE>SINGLE_LISTEN_UNSERIALIZED_ACCEPT</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand then single-socket servers will not serialize at all.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>As discussed in
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi HREF="ftp://ds.internic.net/internet-drafts/draft-ietf-http-connection-00.txt"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi>draft-ietf-http-connection-00.txt</A> section 8,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin order for an HTTP server to <STRONG>reliably</STRONG> implement the protocol
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiit needs to shutdown each direction of the communication independently
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(recall that a TCP connection is bi-directional, each half is independent
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiof the other). This fact is often overlooked by other servers, but
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis correctly implemented in Apache as of 1.2.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>When this feature was added to Apache it caused a flurry of
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiproblems on various versions of Unix because of a shortsightedness.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThe TCP specification does not state that the FIN_WAIT_2 state has a
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitimeout, but it doesn't prohibit it. On systems without the timeout,
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiApache 1.2 induces many sockets stuck forever in the FIN_WAIT_2 state.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiIn many cases this can be avoided by simply upgrading to the latest
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiTCP/IP patches supplied by the vendor, in cases where the vendor has
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinever released patches (i.e. SunOS4 -- although folks with a source
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilicense can patch it themselves) we have decided to disable this feature.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>There are two ways of accomplishing this. One is the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisocket option <CODE>SO_LINGER</CODE>. But as fate would have it,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithis has never been implemented properly in most TCP/IP stacks. Even
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchion those stacks with a proper implementation (i.e. Linux 2.0.31) this
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimethod proves to be more expensive (cputime) than the next solution.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>For the most part, Apache implements this in a function called
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>lingering_close</CODE> (in <CODE>http_main.c</CODE>). The
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifunction looks roughly like this:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi void lingering_close (int s)
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi char junk_buffer[2048];
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi /* shutdown the sending side */
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi shutdown (s, 1);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi signal (SIGALRM, lingering_death);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi select (s for reading, 2 second timeout);
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (error) break;
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi if (s is ready for reading) {
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi read (s, junk_buffer, sizeof (junk_buffer));
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi /* just toss away whatever is here */
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis naturally adds some expense at the end of a connection, but it
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis required for a reliable implementation. As HTTP/1.1 becomes more
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiprevalent, and all connections are persistent, this expense will be
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiamortized over more requests. If you want to play with fire and
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidisable this feature you can define <CODE>NO_LINGCLOSE</CODE>, but
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithis is not recommended at all. In particular, as HTTP/1.1 pipelined
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchipersistent connections come into use <CODE>lingering_close</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiis an absolute necessity (and
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.w3.org/Protocols/HTTP/Performance/Pipeline.html">
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchipipelined connections are faster</A>, so you
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwant to support them).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Apache's parent and children communicate with each other through
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisomething called the scoreboard. Ideally this should be implemented
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin shared memory. For those operating systems that we either have
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiaccess to, or have been given detailed ports for, it typically is
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiimplemented using shared memory. The rest default to using an
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchion-disk file. The on-disk file is not only slow, but it is unreliable
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(and less featured). Peruse the <CODE>src/main/conf.h</CODE> file
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifor your architecture and look for either <CODE>USE_MMAP_SCOREBOARD</CODE> or
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>USE_SHMGET_SCOREBOARD</CODE>. Defining one of those two (as
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwell as their companions <CODE>HAVE_MMAP</CODE> and <CODE>HAVE_SHMGET</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchirespectively) enables the supplied shared memory code. If your system has
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchianother type of shared memory, edit the file <CODE>src/main/http_main.c</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand add the hooks necessary to use it in Apache. (Send us back a patch
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Historical note: The Linux port of Apache didn't start to use
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchishared memory until version 1.2 of Apache. This oversight resulted
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin really poor and unreliable behaviour of earlier versions of Apache
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>If you have no intention of using dynamically loaded modules
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(you probably don't if you're reading this and tuning your
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiserver for every last ounce of performance) then you should add
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>-DDYNAMIC_MODULE_LIMIT=0</CODE> when building your server.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis will save RAM that's allocated only for supporting dynamically
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiloaded modules.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<H3>Appendix: Detailed Analysis of a Trace</H3>
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiHere is a system call trace of Apache 1.3 running on Linux. The run-time
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconfiguration file is essentially the default plus:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<Directory />
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi AllowOverride none
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi Options FollowSymLinks
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi</Directory>
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThe file being requested is a static 6K file of no particular content.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiTraces of non-static requests or requests with content negotiation
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilook wildly different (and quite ugly in some cases). First the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchientire trace, then we'll examine details. (This was generated by
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe <CODE>strace</CODE> program, other similar programs include
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>truss</CODE>, <CODE>ktrace</CODE>, and <CODE>par</CODE>.)
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiaccept(15, {sin_family=AF_INET, sin_port=htons(22283), sin_addr=inet_addr("127.0.0.1")}, [16]) = 3
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiflock(18, LOCK_UN) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {SIG_IGN}, {0x8059954, [], SA_INTERRUPT}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigetsockname(3, {sin_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisetsockopt(3, IPPROTO_TCP1, [1], 4) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiread(3, "GET /6k HTTP/1.0\r\nUser-Agent: "..., 4096) = 60
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {SIG_IGN}, {SIG_IGN}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitime(NULL) = 873959960
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigettimeofday({873959960, 404935}, NULL) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchistat("/home/dgaudet/ap/apachen/htdocs/6k", {st_mode=S_IFREG|0644, st_size=6144, ...}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiopen("/home/dgaudet/ap/apachen/htdocs/6k", O_RDONLY) = 4
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimmap(0, 6144, PROT_READ, MAP_PRIVATE, 4, 0) = 0x400ee000
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwritev(3, [{"HTTP/1.1 200 OK\r\nDate: Thu, 11"..., 245}, {"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 6144}], 2) = 6389
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitime(NULL) = 873959960
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwrite(17, "127.0.0.1 - - [10/Sep/1997:23:39"..., 71) = 71
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigettimeofday({873959960, 417742}, NULL) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitimes({tms_utime=5, tms_stime=0, tms_cutime=0, tms_cstime=0}) = 446747
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchishutdown(3, 1 /* send */) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchioldselect(4, [3], NULL, [3], {2, 0}) = 1 (in [3], left {2, 0})
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiread(3, "", 2048) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {0x8059954, [], SA_INTERRUPT}, {SIG_IGN}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimunmap(0x400ee000, 6144) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiflock(18, LOCK_EX) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Notice the accept serialization:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiflock(18, LOCK_UN) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiflock(18, LOCK_EX) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThese two calls can be removed by defining
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>SINGLE_LISTEN_UNSERIALIZED_ACCEPT</CODE> as described earlier.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Notice the <CODE>SIGUSR1</CODE> manipulation:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {SIG_IGN}, {0x8059954, [], SA_INTERRUPT}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {SIG_IGN}, {SIG_IGN}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {0x8059954, [], SA_INTERRUPT}, {SIG_IGN}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis is caused by the implementation of graceful restarts. When the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiparent receives a <CODE>SIGUSR1</CODE> it sends a <CODE>SIGUSR1</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchito all of its children (and it also increments a "generation counter"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiin shared memory). Any children that are idle (between connections)
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwill immediately die
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchioff when they receive the signal. Any children that are in keep-alive
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconnections, but are in between requests will die off immediately. But
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiany children that have a connection and are still waiting for the first
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchirequest will not die off immediately.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>To see why this is necessary, consider how a browser reacts to a closed
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconnection. If the connection was a keep-alive connection and the request
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibeing serviced was not the first request then the browser will quietly
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchireissue the request on a new connection. It has to do this because the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiserver is always free to close a keep-alive connection in between requests
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(i.e. due to a timeout or because of a maximum number of requests).
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiBut, if the connection is closed before the first response has been
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchireceived the typical browser will display a "document contains no data"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidialogue (or a broken image icon). This is done on the assumption that
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithe server is broken in some way (or maybe too overloaded to respond
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiat all). So Apache tries to avoid ever deliberately closing the connection
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibefore it has sent a single response. This is the cause of those
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Note that it is theoretically possible to eliminate all three of
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithese calls. But in rough tests the gain proved to be almost unnoticeable.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>In order to implement virtual hosts, Apache needs to know the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilocal socket address used to accept the connection:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigetsockname(3, {sin_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiIt is possible to eliminate this call in many situations (such as when
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithere are no virtual hosts, or when <CODE>Listen</CODE> directives are
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiused which do not have wildcard addresses). But no effort has yet been
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimade to do these optimizations.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Apache turns off the Nagle algorithm:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisetsockopt(3, IPPROTO_TCP1, [1], 4) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibecause of problems described in
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.isi.edu/~johnh/PAPERS/Heidemann97a.html">a
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchipaper by John Heidemann</A>.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitime(NULL) = 873959960
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitime(NULL) = 873959960
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiOne of these occurs at the beginning of the request, and the other occurs
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchias a result of writing the log. At least one of these is required to
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiproperly implement the HTTP protocol. The second occurs because the
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiCommon Log Format dictates that the log record include a timestamp of the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiend of the request. A custom logging module could eliminate one of the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicalls. Or you can use a method which moves the time into shared memory,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisee the <A HREF="#patches">patches section below</A>.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>As described earlier, <CODE>Rule STATUS=yes</CODE> causes two
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>gettimeofday</CODE> calls and a call to <CODE>times</CODE>:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigettimeofday({873959960, 404935}, NULL) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigettimeofday({873959960, 417742}, NULL) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitimes({tms_utime=5, tms_stime=0, tms_cutime=0, tms_cstime=0}) = 446747
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThese can be removed by either removing <CODE>mod_status</CODE> or
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>It might seem odd to call <CODE>stat</CODE>:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchistat("/home/dgaudet/ap/apachen/htdocs/6k", {st_mode=S_IFREG|0644, st_size=6144, ...}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThis is part of the algorithm which calculates the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>PATH_INFO</CODE> for use by CGIs. In fact if the request had been
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifor the URI <CODE>/cgi-bin/printenv/foobar</CODE> then there would be
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitwo calls to <CODE>stat</CODE>. The first for
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>/home/dgaudet/ap/apachen/cgi-bin/printenv/foobar</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwhich does not exist, and the second for
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>/home/dgaudet/ap/apachen/cgi-bin/printenv</CODE>, which does exist.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiRegardless, at least one <CODE>stat</CODE> call is necessary when
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiserving static files because the file size and modification times are
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiused to generate HTTP headers (such as <CODE>Content-Length</CODE>,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>Last-Modified</CODE>) and implement protocol features (such
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchias <CODE>If-Modified-Since</CODE>). A somewhat more clever server
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicould avoid the <CODE>stat</CODE> when serving non-static files,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchihowever doing so in Apache is very difficult given the modular structure.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>All static files are served using <CODE>mmap</CODE>:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimmap(0, 6144, PROT_READ, MAP_PRIVATE, 4, 0) = 0x400ee000
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimunmap(0x400ee000, 6144) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiOn some architectures it's slower to <CODE>mmap</CODE> small
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifiles than it is to simply <CODE>read</CODE> them. The define
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>MMAP_THRESHOLD</CODE> can be set to the minimum
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisize required before using <CODE>mmap</CODE>. By default
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiit's set to 0 (except on SunOS4 where experimentation has
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchishown 8192 to be a better value). Using a tool such as <A
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiHREF="http://www.bitmover.com/lmbench/">lmbench</A> you
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchican determine the optimal setting for your environment.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>You may also wish to experiment with <CODE>MMAP_SEGMENT_SIZE</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(default 32768) which determines the maximum number of bytes that
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwill be written at a time from mmap()d files. Apache only resets the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiclient's <CODE>Timeout</CODE> in between write()s. So setting this
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchilarge may lock out low bandwidth clients unless you also increase the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>It may even be the case that <CODE>mmap</CODE> isn't
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiused on your architecture, if so then defining <CODE>USE_MMAP_FILES</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand <CODE>HAVE_MMAP</CODE> might work (if it works then report back to us).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Apache does its best to avoid copying bytes around in memory. The
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchifirst write of any request typically is turned into a <CODE>writev</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwhich combines both the headers and the first hunk of data:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwritev(3, [{"HTTP/1.1 200 OK\r\nDate: Thu, 11"..., 245}, {"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 6144}], 2) = 6389
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiWhen doing HTTP/1.1 chunked encoding Apache will generate up to four
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchielement <CODE>writev</CODE>s. The goal is to push the byte copying
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiinto the kernel, where it typically has to happen anyhow (to assemble
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinetwork packets). On testing, various Unixes (BSDI 2.x, Solaris 2.5,
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiLinux 2.0.31+) properly combine the elements into network packets.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiPre-2.0.31 Linux will not combine, and will create a packet for
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchieach element, so upgrading is a good idea. Defining <CODE>NO_WRITEV</CODE>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwill disable this combining, but result in very poor chunked encoding
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>The log write:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwrite(17, "127.0.0.1 - - [10/Sep/1997:23:39"..., 71) = 71
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchican be deferred by defining <CODE>BUFFERED_LOGS</CODE>. In this case
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiup to <CODE>PIPE_BUF</CODE> bytes (a POSIX defined constant) of log entries
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiare buffered before writing. At no time does it split a log entry
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiacross a <CODE>PIPE_BUF</CODE> boundary because those writes may not
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibe atomic. (i.e. entries from multiple children could become mixed together).
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThe code does it best to flush this buffer when a child dies.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>The lingering close code causes four system calls:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchishutdown(3, 1 /* send */) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchioldselect(4, [3], NULL, [3], {2, 0}) = 1 (in [3], left {2, 0})
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiread(3, "", 2048) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwhich were described earlier.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Let's apply some of these optimizations:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>-DSINGLE_LISTEN_UNSERIALIZED_ACCEPT -DBUFFERED_LOGS</CODE> and
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<CODE>Rule STATUS=no</CODE>. Here's the final trace:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiaccept(15, {sin_family=AF_INET, sin_port=htons(22286), sin_addr=inet_addr("127.0.0.1")}, [16]) = 3
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {SIG_IGN}, {0x8058c98, [], SA_INTERRUPT}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchigetsockname(3, {sin_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisetsockopt(3, IPPROTO_TCP1, [1], 4) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiread(3, "GET /6k HTTP/1.0\r\nUser-Agent: "..., 4096) = 60
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {SIG_IGN}, {SIG_IGN}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitime(NULL) = 873961916
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchistat("/home/dgaudet/ap/apachen/htdocs/6k", {st_mode=S_IFREG|0644, st_size=6144, ...}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiopen("/home/dgaudet/ap/apachen/htdocs/6k", O_RDONLY) = 4
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimmap(0, 6144, PROT_READ, MAP_PRIVATE, 4, 0) = 0x400e3000
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiwritev(3, [{"HTTP/1.1 200 OK\r\nDate: Thu, 11"..., 245}, {"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 6144}], 2) = 6389
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchitime(NULL) = 873961916
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchishutdown(3, 1 /* send */) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchioldselect(4, [3], NULL, [3], {2, 0}) = 1 (in [3], left {2, 0})
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiread(3, "", 2048) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisigaction(SIGUSR1, {0x8058c98, [], SA_INTERRUPT}, {SIG_IGN}) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimunmap(0x400e3000, 6144) = 0
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThat's 19 system calls, of which 4 remain relatively easy to remove,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibut don't seem worth the effort.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<H3><A NAME="patches">Appendix: Patches Available</A></H3>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.arctic.org/~dgaudet/apache/1.3/">
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiseveral performance patches available for 1.3.</A> But they may
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibe slightly out of date by the time Apache 1.3.0 has been released,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiit shouldn't be difficult for someone with a little C knowledge to
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiupdate them. In particular:
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.arctic.org/~dgaudet/apache/1.3/shared_time.patch"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi>patch</A> to remove all <CODE>time(2)</CODE> system calls.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.arctic.org/~dgaudet/apache/1.3/mod_include_speedups.patch"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi>patch</A> to remove various system calls from <CODE>mod_include</CODE>,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithese calls are used by few sites but required for backwards compatibility.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.arctic.org/~dgaudet/apache/1.3/top_fuel.patch"
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi>patch</A> which integrates the above two plus a few other speedups at the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicost of removing some functionality.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Apache (on Unix) is a <EM>pre-forking</EM> model server. The
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<EM>parent</EM> process is responsible only for forking <EM>child</EM>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiprocesses, it does not serve any requests or service any network
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchisockets. The child processes actually process connections, they serve
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchimultiple connections (one at a time) before dying.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiThe parent spawns new or kills off old
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchichildren in response to changes in the load on the server (it does so
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiby monitoring a scoreboard which the children keep up to date).
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>This model for servers offers a robustness that other models do
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchinot. In particular, the parent code is very simple, and with a high
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidegree of confidence the parent will continue to do its job without
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchierror. The children are complex, and when you add in third party
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicode via modules, you risk segmentation faults and other forms of
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchicorruption. Even should such a thing happen, it only affects one
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiconnection and the server continues serving requests. The parent
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiquickly replaces the dead child.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Pre-forking is also very portable across dialects of Unix.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiHistorically this has been an important goal for Apache, and it continues
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>The pre-forking model comes under criticism for various
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiperformance aspects. Of particular concern are the overhead
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiof forking a process, the overhead of context switches between
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiprocesses, and the memory overhead of having multiple processes.
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiFurthermore it does not offer as many opportunities for data-caching
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchibetween requests (such as a pool of <CODE>mmapped</CODE> files).
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiVarious other models exist and extensive analysis can be found in the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.cs.wustl.edu/~jxh/research/research.html"> papers
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiof the JAWS project</A>. In practice all of these costs vary drastically
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchidepending on the operating system.
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<P>Apache's core code is already multithread aware, and Apache version
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi1.3 is multithreaded on NT. There have been at least two other experimental
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiimplementations of threaded Apache, one using the 1.3 code base on DCE,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiand one using a custom user-level threads package and the 1.0 code base,
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchineither are available publically. There is also an experimental port of
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiApache 1.3 to <A HREF="http://www.mozilla.org/docs/refList/refNSPR/">
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiNetscape's Portable Run Time</A>, which
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://www.arctic.org/~dgaudet/apache/2.0/">is available</A>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi(but you're encouraged to join the
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchi<A HREF="http://dev.apache.org/mailing-lists">new-httpd mailing list</A>
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiif you intend to use it).
d14abf155341d55053c76eeec58b787a456b753bRobert MustacchiPart of our redesign for version 2.0
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchiof Apache will include abstractions of the server model so that we
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchican continue to support the pre-forking model, and also support various
d14abf155341d55053c76eeec58b787a456b753bRobert Mustacchithreaded models.