mpm_{worker,prefork}: save some cycles by not copying the listener's pollfds for each pollset operation. We don't need a copy when poll()ing if those are allocated with the correct lifetime (the listener thread) at the very beginning.

mpm_{prefork,worker}: follow up to r1635521. Don't crash when there is no accept mutex (one listener => apr_proc_mutex_name(NULL)).

mpm_event(opt), mpm_worker, mpm_prefork: follow up to r1635521, r1640161. Retain num_buckets and max_buckets accross restarts so that we can determine whether new buckets were allocated and set their idle_spawn_rate at the same level as the existing ones (max). Also, adjust ap_daemons_limit and ap_daemons_to_start lower bounds at mpm_run() time, once num_buckets is available for the current generation (previously done at check_config time, hence before num_buckets is computed, and then with the previous generation's value).

mpm_event(opt), mpm_worker: follow up to r1635521. Allocate retained idle_spawn_rate vector on the process pool.

mpm_worker, mpm_prefork: follow up to r1635521. Use one dedicated accept mutex per listeners bucket.

MPMs, core: make duplicated listeners (SO_REUSEPORT) introduced in r1599531 less intrusive. Submitted by: Yingqi Lu <yingqi.lu@intel.com> Modified/Committed by: ylavic Add ListenCoresBucketsRatio which is a configurable ratio between the number of CPU cores (online) and the number of listeners buckets to create, defaulting to zero (so that listeners buckets become an opt-in, ie. ncpus / ratio > 1). This could also be made an opt-out by using the previous hardcoded value (8) as default. Make ap_close_listeners() act on all the listeners (including duplicated ones), since the function is also called externally (eg. mod_cgid, mod_ssl_ct and possibly any third party module) to cleanup opened descriptors when a process is forked (the duplicated listeners are kept in a scoped/static variable). Add ap_close_listeners_ex() to close a single bucket of listeners, used by the children to close unused duplicates and internally by ap_close_listeners(). Make ap_duplicate_listeners() compute the number of buckets to be used, instead of each MPM. This number is now based on the above ratio and will not change unless asked to (given *num_buckets < 1, that is when the MPM does not run in one-process mode nor after a graceful restart). Remove some global variables (mpm_listen, enable_default_listeners) previously used to communicate between MPMs and ap_listen, since ap_duplicate_listeners() API can now be used to do so. Also rename num_buckets as ap_num_listen_buckets, and prefix have_so_reuseport with ap_ (both printed by ap_log_common(), hence kept global). Detect ap_have_so_reuseport once only at startup. Restore dummy_connection() as before r1599531 since sending POD signals should not depend on the number of listeners buckets (there is still one single socket receiving the connections). For each MPM (concerned), move the bucket data (pod, listeners and eventually accept mutex) into a struct and instanciate an array of them (sized by the number of buckets), for each child to use its own data according to its bucket index, and the parent to maintain the whole.

core: follow up to r1629909: fix min_spare_threads lower bound and check wrt num_buckets.

core: ensure that MPMs return an error on runtime failure and hence that httpd's main process also exits with an error.

mpms: enforce assertion that unreachable code is not reached.

core: Use process scoreboard to store each child's listener bucket, and silently adjust the configured number of processes/threads to be above the computed number of listener buckets (depending on the CPU cores).

worker MPM: Fix null-dereference crash when using -DFOREGROUND with worker MPM

Mark all unused/duplicated/closed child listeners as inactive. Fix unused "ap_listen_rec *lr;" in worker_run().

-MPMs prefork & worker: avoid segfault with SAFE_ACCEPT in ONE_PROCESS mode.

fix restart issues

Break out common code to be share-able

Now allow MPMs to call ap_log_common()

Optimize w/ duplicated listeners and use of SO_REUSEPORT where available.

Fix some comment typos

worker MPM: Don't forcibly kill worker threads if the child process is exiting gracefully. Submitted by: Oracle, via trawick This modification was made some years ago for Oracle HTTP Server by an Oracle employee.

Log a warning if ThreadStackSize has an inappropriate value. PR : 54311 Submitted by: Tianyin Xu <tixu cs ucsd edu> Only event.c has been compiled on my system. The 2 others are just cut and paste. I have left the same value for APLOGNO in event and eventopt has for the other logged messages. I don't know if it is the right way to do.

missed one

no longer needed

Pull out the "extended" pod functions used by event and worker to core, since it will be used by Simple and Lean and likely other MPMs. Avoid duplication.

Follow up to r1363557: Refactor with some existing code to slightly optimize thread tracking. Fix an implementation problem affecting statically linked MPMs which would break the ability to fail restart if a resource shortage was found immediately. Found by: rjung

mpm_event, mpm_worker: Remain active amidst prevalent child process resource shortages. This is a somewhat different direction than r168182 ("transient thread creation errors shouldn't take down the whole server"). r168182: If APEXIT_CHILDSICK is received and there aren't any active children at the time, exit. Now: If APEXIT_CHILDSICK is received and we never successfully initialized a child, exit. The issue seen with the r168182 handling is that it is rather easy to be left with no active child processes (which causes the server to exit completely) during a resource shortage that lasts for some measurable period of time, as contrasted with a resource shortage that results in only a handful of allocation failures. Now the server will remain active, though as long as the resource shortage exists children may continually fail and the parent will try once per second to create a replacement. The existing logic to reduce the spawn rate after such errors will prevent the parent from trying to create children more rapidly.

mpm_event, mpm_worker: Fix cases where the spawn rate wasn't reduced after child process resource shortages. The broken scenario: child X exits with APEXIT_CHILDSICK another child Y is created and reuses child X's scoreboard slot child X's exit status is processed The case that worked is when child X's scoreboard slot hadn't been reused by the time that its exit status was processed.

Add lots of unique tags to error log messages

revert accidental commit 1200129

clean up unused ap_my_generation variable (presence is odd, since this variable as an API was supposedly removed long ago, and a similarly named static variable was removed in r1078286)

If a child is created just before graceful restart and then exits because of a missing lock file, don't shutdown the whole server. PR: 39311 Submitted by: Shawn Michael <smichael rightnow com>

Cleanup effort in prep for GA push: Trim trailing whitespace... no func change

Add wrappers for malloc, calloc, realloc that check for out of memory situations. Use them in most places where malloc, and friends are used. This results in clean error messages in an out of memory situation instead of segfaulting or silently malfunctioning. In some places, it just allows to remove some logging code. PR 51568, PR 51569, PR 51571.

Install signal handlers for crashes before detaching. Previously, there was a time span where crashes would be neither visible on the console nor logged in the error log.

Reduce period during startup and restart where a successive signal may be lost PR: 43696 Submitted by: Arun Bhalla <arun shme net>

Log the deprecation warning at level info, not warn

Fix search'n'replace error: Actually warn if MaxClients is used

Rename MaxClients to MaxRequestWorkers which describes more accurately what it does.

Add child_status hook for tracking creation/termination of MPM child processes. Add end_generation hook for notification when the last MPM child of a generation exits. end_generation is implemented completely by core using the child_status hook run by the MPM. simple and mpmt_os2 MPMs don't currently run the child_status hook, so neither hook is invoked with those MPMs.

Simplify the interface to ap_reclaim_child_processes() and ap_relieve_child_processes(): instead of requiring the MPM to implement an otherwise-useless hook, just use a callback function. As I don't expect third-party MPM devs are following our day to day progress, the API changes are considered part of yesterday's MMN change.

mpm_common.c implements a handful of config directives; the related state has to be re-initialized to the default values at pre-config time, but that was handled inconsistently by the MPMs, resulting in unexpected behavior when some of the directives were removed across restart move that necessary initialization from the MPMs to common code run from core's pre-config; MPMs that need to override defaults can do so by running after core's pre-config (the NetWare MPM now does that) the DEFAULT_MAX_REQUESTS_PER_CHILD compile setting wasn't useful enough to keep the simple MPM wasn't consistent in which of these directives were respected and which weren't, and that hasn't changed (see procmgr.max_requests_per_child vs. ap_max_requests_per_child)

consolidate logic to remove the pidfile in a new API, ap_remove_pid(), related to existing ap_log_pid() and ap_read_pid() presumably this is useful to third-party MPMs as well

* server/mpm/prefork/prefork.c (set_signals): Ignore SIGXFSZ; have a write() beyond 2GB fail with E2BIG rather than killing the process in the now hopefully rare case of non-LFS file usage on LFS systems. * server/mpm/worker/worker.c (set_signals): Likewise. * server/mpm/event/event.c (set_signals): Likewise.

worker, event, prefork: Correct several issues when built as DSOs; most notably, the scoreboard was reinitialized during graceful restart, such that processes of the previous generation were not observable. Other noticeable ones: * the generation number did not increment properly * worker's check-config hook didn't detect startup properly, so messages about inconsistent directives could have been sent to the wrong place max_daemons_limit wasn't tracked across restart, but as the scoreboard was always cleared this was a non-issue. Reinitialization of idle_spawn_rate, hold_off_on_exponential_spawning, and maxclients-message-reported may or may not have been features, but the big picture is that the behaviour shouldn't have differed. These changes should have been part of r758185.

zap needless reinitialization of is_graceful on the 2nd call of the open-logs hook

sanity check use of volatile in prefork, worker, and event

worker: child_main() never returns, so remove some dead code after one of the calls

Rename MaxRequestsPerChild to MaxConnectionsPerChild, which describes more accurately what the directive does. The old name still works but logs a warning.

Use the new APLOG_USE_MODULE/AP_DECLARE_MODULE macros everywhere to take advantage of per-module loglevels

Silence compiler warning.

fix compile failure in r940981

Catch up with ap_[proc|global]_mutex_create api change

all signs indicate that status value is always -1 on error

reduce the warning level in the bindprocessor() call used to make sure child processes aren't bound to the CPU the parent is running on. In recent levels of AIX, the parent isn't implicitly bound to a CPU (so the children don't inherit it), and when the AIX WPAR feature is enabled this call can return EPERM, even though the child process is already unbound.

Log command line on startup, so there's a record of command line arguments like -f. Suggested by Shaya Potter. [Dan Poirier] PR: 48752

change the preprocessor define ap_accept_mutex_type to upper case

worker: don't report server has reached MaxClients until it does. Add warning when within MinSpareThreads. PR: 46996

string constness

Replace AcceptMutex, LockFile, RewriteLock, SSLMutex, SSLStaplingMutex, and WatchdogMutexPath with a single Mutex directive. Add APIs to simplify setup and user customization of APR proc and global mutexes. (See util_mutex.h.) Build-time setting DEFAULT_LOCKFILE is no longer respected; set DEFAULT_REL_RUNTIMEDIR instead. Some existing modules, such as mod_ldap and mod_auth_digest gain configurability for their mutexes.

Enhance ap_hook_monitor to pass along a server_rec (in general the ap_server_conf) and tuck away some storage in there which may be useful as an opaque data pointer.

* prevent a fork bomb if the accept mutex is clobbered by exiting with APEXIT_CHILDSICK if the worker process hasn't served any requests * move the accept mutex error leg into its own function to take advantage of common code and to keep rarely executed instructions out of the mainline instruction cache.

* Do better checking of pollset operations in various places to avoid segfaults. PR: 46467 Submitted by: Stefan Fritsch <sf sfritsch.de> Reviewed by: rpluem

no longer amusing. I'll still pay off if somebody does it legitimately, but I haven't been to Apachecon in ages so we'll have to work out a different delivery mechanism. With the stability of the 2.0.x and 2.2.x releases and Moore's Law in effect for the last decade, it might be possible...

main() can use ap_run_mpm() directly, so axe the old ap_mpm_run() function change the mpm hooks to return OK/DONE instead of 0/1

mpm-query hook: distinguish between DECLINED and APR_ENOTIMPL so that . ap_mpm_query() can return APR_EGENERAL if called too early (for debugging a module) . some hypothetical module which implements the mpm-query hook can bypass the MPM with APR_ENOTIMPL

rename the new retained-data APIs to ap_retained_data_create()/ap_retained_data_get(), for better consistency with existing APIs (e.g., slotmem) don't bother changing the MMN, which doesn't reflect recent -dev changes anyway

the mpm_get_child_pid hook is unnecessary, as was the per-MPM MPM_CHILD_PID() macro which it replaced axe this new hook, and use ap_get_scoreboard_process() instead

Core can specify a platform-specific rewrite args hook. Use that on Unix. Windows continues to specify this in the MPM, at least until someone can separate out the MPM-specific bits.

Register rewrite_args hook. Now, dunno if that was intended, but at least allow to start the httpd

prefork, worker, event: restore expected accept mutex mechanism and allocator limit support (the related AP_MPM_WANT_foo defines no longer exist)

resurrect the old (pre-r757867) compiled-in MPM_NAME value, but via a hook for dynamically linked MPMs

The rewrite-args hook isn't available to MPM DSOs. Piggy-back on the linked-in mod_unixd.c to handle that. (The set of MPMs modified is the same set for which mod_unixd is enabled, which is approximately equivalent to the set of Unix MPMs which anybody has tried to maintain in recent memory.)

prefork, worker, and event MPMs: use retained-data API to maintain information across reconfigs, to allow these MPMs to work as DSOs

Introduce a new set of APIs to allow MPMs to be proper modules instead of integral parts which share global variables, functions, and macros with the rest of httpd. Converted now: prefork, worker, event, simple, WinNT* *WinNT hasn't been built or tested, and relies on a hack to include the WinNT mpm.h to disable Unixy MPM support routines in mpm_common.c

Add conn_rec::current_thread.

API Cleanup in preperation for 2.4.x, make sure all exported functions or variables contain an ap_ prefix.

Further unixd hacks to remove duplication between old-unixd and mod_unixd, and get it working with old MPMS[1] + mod_unixd. It's still an uneasy split, as some modules (mod_cgid, suexec)[2] also use unixd. More thinking+hacking due. [1] Should be prefork/worker/event, but only worker is tested. [2] cgid is OK, suexec is untested.

Switch all unix MPMs to use drop_privileges hook (mod_unixd) for startup and add a flag to prevent running without any module taking responsibility for managing system privileges!

worker/event MPMs: graceful-stop doesn't work if the listener thread is waiting for an available worker thread. PR 43081 Reported by: Paul Querna Debugged by: Takashi Sato [takashi lans-tv.com]

Remove all references to CORE_PRIVATE.

Remove ap_graceful_stop_signalled from all MPMs.

the cost of synchronized SMP proof updates far outweighs the value of having 100% accurate MaxRequestsPerChild. * it doesn't even count requests, it counts connections. * all it is is a number someone pulls out of a hat in an attempt to deal with resource leaks. i.e., no strong need for accuracy. atomic operations are relatively expensive and this is mainline code.

Add alternative fixes for CVE-2007-3304: * configure.in: Check for getpgid. * include/mpm_common.h (ap_mpm_safe_kill): New prototype. * server/mpm_common.c (reclaim_one_pid): Ensure pid validity before calling apr_proc_wait(). (ap_mpm_safe_kill): New function. * server/mpm/prefork/prefork.c, server/mpm/worker/worker.c, server/mpm/experimental/event/event.c: Use ap_mpm_safe_kill() on pids from the scoreboard, throughout. * include/ap_mmn.h: Minor bump.

Revert r547987 ("svn merge -c -547987 .")

PID table impl: parent process keeps a local table store of Apache child process PIDs and uses that to check validity of what's in the scoreboard.

remove a bit of useless code from worker and event MPMs PR: 41193 Submitted by: Davi Arnaut <davi haxent.com.br> Reviewed by: trawick

* Remove an unused variable and thus fix a compiler warning.

added APR_THREAD_FUNC on listener_thread() in event and worker MPMs to match definition of apr_thread_start_t

Replace ap_get_server_version with ap_get_server_banner() and ap_get_server_description(). High-level summary: The full server version information is now included in the error log at startup as well as server status reports, irrespective of the setting of the ServerTokens directive. Third-party modules must now use ap_get_server_banner() or ap_get_server_description() in place of ap_get_server_version().

Introduce a check_config phase between pre_config and open_logs, to allow modules to review interdependent configuration directive values and adjust them while messages can still be logged to the console. The open_logs phase is already used somewhat for this purpose by certain MPMs (winnt, prefork, worker, and event) but only by forcing their functions ahead of the core ap_open_logs() function, and since this phase runs after the ap_signal_server function during startup, it can not be used to generate messages on the console when restarting. Add the check_config phase to mod_info and mod_example. Handle relevant MPM directives during this phase and format messages for both the console and the error log, as appropriate. Bounds and sanity checks on the values of the MPM directives are handled in sequence in this phase instead of in the various directive handling functions, since those functions (e.g., set_max_clients()) may not be called at all if their directives do not appear in the configuration files, and even if they are called, there is no guarantee that this will occur in any particular order. Remove from the worker and event MPMs the code in the pre_config phase that alters the configuration node tree by re-ordering ThreadsPerChild ahead of MaxClients. This code is effective but insufficient; for example, if ServerLimit follows MaxClients, the test against server_limit in set_max_clients() is invalid. (In practice, this only results in incorrect or absent warnings on the console, because server_limit is set to its configured value when the main loop re-runs the configuration process.) Prevent ap_threads_per_child from exceeding thread_limit in the winnt, worker, and event MPMs. This situation could occur if ThreadsPerChild was not specified in the configuration files and ThreadLimit was set to a value smaller than DEFAULT_THREADS_PER_CHILD, because set_threads_per_child() would never be called and therefore its bounds check against thread_limit would not be performed. Remove from the winnt, prefork, worker, and event MPMs the changed_limit_at_restart flag. Set the first_server_limit and first_thread_limit values during the first execution of the check_config function, and use them to detect changes to ServerLimit and ThreadLimit across restarts and issue appropriately formatted warnings. Remove the comments about the error log being a "bit bucket"; this was true when the code was originally committed in r92530 but that was due to a bug fixed in r92769. Be consistent about setting all MPM configuration directive values in the pre_config phase. Rephrase and reformat the console and log file messages relating to MPM configuration directives to be consistent across all MPMs. Use briefer messages when logging to the error log than to the console. Update miscellaneous stale comments and messages (e.g., reference to daemons_min_free in worker and event MPMs, "prefork open_logs" in winnt MPM, and StartServers in netware MPM). The winnt, netware, beos, and mpmt_os2 MPMs should be tested by developers with access to those platforms, especially the winnt MPM, which has unique logic with respect to distinguishing between parent and child processes during the configuration phases. Update the English documentation for the worker MPM's ThreadsPerChild directive, which no longer needs to precede other MPM directives in the configuration files if it has a non-default value. The German (.de) and Japanese (.ja) translations should be updated by developers fluent in those languages.

update license header text

On graceless shutdown or restart, send AP_SIG_GRACEFUL to all worker threads so that any that are polling on Keep-Alive connections will wake up and exit promptly. Otherwise, on Linux, they wait until poll()'s timeout interval completes, which is often too long for the parent process; the parent issues SIGKILL before the child's main thread manages to re-join all the worker threads. PR 38737. On Solaris, by contrast, the close_worker_sockets() function successfully alerts all worker threads that they should exit quickly. On Linux, this side-effect of one thread closing another's socket doesn't occur; see: http://bugme.osdl.org/show_bug.cgi?id=546 Also, expand on the placeholder in CHANGES regarding previous cleanups to scoreboard.c and scoreboard.h in r404848 and r404849.

Make the worker and event MPMs not touch the scoreboard when handling a fork() failure. The previous behaviour appears to have been inherited from the prefork MPM, where is it appropriate. The prefork MPM sets thread_limit to 1 and therefore each child process has a single worker_score structure in the scoreboard's array, i.e., ap_scoreboard_image->servers[slot][0]. In make_child(), it sets this structure's status to SERVER_STARTING, and then does a fork(); if the fork() fails, it resets the status to SERVER_DEAD. The worker and event MPMs, by constrast, obviously use multiple worker_score structures per child process. They may also be in use by worker threads from a previous generation at any particular moment. Therefore make_child() and the parent process in general doesn't normally update them; make_child() doesn't set them all to SERVER_STARTING before doing fork(), for example. So, make_child() shouldn't set them to SERVER_DEAD if fork() fails (and even if it should, it certainly shouldn't be just updating the first one).

Initialise server generation (Chris Darroch)

PR 39275. don't fork MaxClients worth of processes if child_init or fork() are slow.

Update the copyright year in all .c, .h and .xml files

decouple the two scoreboard full messages

MaxClients isn't the only reason the scoreboard can fill up. another reason should be fixed soon. also add a comment to explain an obscure line of code.

Make the worker and event MPMs support GracefulShutdownTimeout just like the prefork MPM does. Submitted by: Chris Darroch <chrisd pearsoncmg.com> Ported to event MPM by: rooneg Issue number: 38621 * server/mpm/worker/worker.c (ap_mpm_run): Make the graceful shutdown log message refer to the right signal, and respect the graceful shutdown timeout if there is one. * server/mpm/experimental/event/event.c (ap_mpm_run): Ditto. * CHANGES: Note change.

No functional Change: Removing trailing whitespace. This also means that "blank" lines consisting of just spaces or tabs are now really blank lines

No functional change: simple detabbing of indented code.

use Greg's cleaner fix for CAN-2005-2970

SECURITY: CAN-2005-2970 (cve.mitre.org) worker MPM: Fix a memory leak which can occur after an aborted connection in some limited circumstances.

Revert one part. I should have tested seperately. This line isn't neccessary.

Actually allow the admin to set thread_limit higher than the default. Also, initialise the first_ variables, since they are tested in the set_ functions.

More getpid format fixes

*how* did I mis-spell that.

Move the "GracefulShutdownTimeout" command definition into mpm_common

Implement a graceful-stop for the worker MPM. We close our listeners, and then ask each child process to do the same. We then monitor until all children have exited. The change to ap_start_shutdown(void) to ap_start_shutdown(int) may look like an external API change, but the function is defined static, and used only in one place.

Implement "de-listening" in the worker MPM. Fixes PR28167. Comments in CHANGES file also cover r239710-1.

downgrade the log level of a proc_mutex_unlock error message when it occurs during restart; it isn't at all uncommon then when SysV semaphores are used for the accept mutex same logic already in place for two other mutex calls

* server/mpm/worker/worker.c (listener_thread): Create ptrans pool as a child of pconf, to fix APR pool lifetime checks in apr_tables.c. Submitted by: Joe Schaefer

missed a couple.

worker and event mpms: transient thread creation errors shouldn't take down the whole server.

the style police have been sleeping

worker MPM/mod_status: Support per-worker tracking of pid and generation in the scoreboard so that mod_status can accurately represent workers in processes which are gracefully terminating. New child processes with worker MPM can take over slots of individual threads within gracefully terminating processes. Sometimes this is a problem (too many of these gracefully terminating processes), so it is helpful to have mod_status provide the information required to recognize these processes.

Update copyright year to 2005 and standardize on current copyright owner line.

worker MPM: Fix a problem which could cause httpd processes to remain active after shutdown. The problem occurred when a scoreboard entry currently in use by an exiting child process was used for a new child process. At that point, the MPM forgot about the exiting child process, so ap_reclaim_child_processes() wouldn't be able to forceably terminate it. (An exiting child process may *never* exit due to a stuck or long-running request being handled on one of the threads.)

use existing directive name ThreadStackSize instead of WorkerStackSize as made obvious by: Brad Nicholes

Threaded MPMs for Unix and Win32: Add WorkerStackSize directive to override default thread stack size for threads which handle client connections. Required for some third-party modules on platforms with small default thread stack size. This is also useful for trimming back the stack size on platforms with relatively large default stack size in order to conserve address space for supporting more threads per child.

fix name of The Apache Software Foundation

fix copyright dates according to the first checkin

apply Apache License, Version 2.0

worker MPM: fix stack overlay bug that could cause the parent process to crash.

Unix MPMs: Stop dropping connections when the file descriptor is at least FD_SETSIZE. That isn't a problem for most users (since poll() is widespread). On old boxes where select() must be used by APR, APR needs to fail operations that don't work with such fds.

update license to 2004.

remove inadvertent commit of clumsy gdb workaround (sorry!)

update leader, threadpool, and worker MPMs to return MPM state from ap_mpm_query()

remove calls to the apr_proc_other_child_read function that no longer exists

Switch over to the new pollset API. * server/mpm/worker/worker.c: (listener_thread): create and add sockets to the pollset using the new APIs. rearrange the round-robin a little bit to work with the new pollset return values. cleaning: get rid of an extraneous status variable. get rid of obsoleted round-robin code.

Lower the severity of the "listener thread didn't exit" message to debug, as it is of interest only to developers. PR: 9011

Fix a misleading message from the some of the threaded MPMs when MaxClients has to be lowered due to the setting of ServerLimit.

Updated the various MPM's to use the new bucket_alloc_create_ex API when necessary. Which is to say that it's necessary in all cases except for prefork, where the change to apr-util to have it use the allocator from the pool passed in is already sufficient. Reviewed by: Jean-Jacques Clar, Sander Striker, Brad Nicholes

Make sure that the global ap_max_mem_free is initialized along with the other MPM globals whenever HTTPD is started or restarted.

consolidate code related to sig_coredump a note on perchild: perchild exception handling was busted before this and is still busted the problem I noticed was that a client that exposes a segfault never gets its connection dropped and the parent never notices that the child process that segfaulted hasexited

finished that boring job: update license to 2003. Happy New Year! ;-))

Adjust the descriptions of these directives, avoid the child/process/thread confusion. PR: Obtained from: Submitted by: Reviewed by:

Minor error log typo

Bring Apache in line with APR's new poll implementation. This isn't optimal, because it still uses the compatibility functions to make it work, but it does work. The next step will be to take the memory for the pollsets under Apache's control.

replaced APR_USEC_PER_SEC division with the new time macros

Add a new directive: MaxMemFree. MaxMemFree makes it possible to configure the maximum amount of memory the allocators will hold on to for reuse. Anything over the MaxMemFree threshold will be free()d. This directive is usefull when uncommon large peaks occur in memory usage. It should _not_ be used to mask defective modules' memory use.

Now that we have the functionality in apr, we can now have 2.0's startup message regarding AcceptMutex (what we are using and what the default is) "fully" match what is present in 1.3. PR: Obtained from: Submitted by: Reviewed by:

Catch up with the apr_allocator_set_owner -> apr_allocator_owner_set renames in APR.

Fix for the worker deadlock problem

Add "-k start|startssl|restart|graceful|stop" support to httpd for the Unix MPMs. These have semantics very similar to the old apachectl commands of the same name. The use of stderr/stdout and exit status for error conditions needs to be revisited. For now it matches apachectl behavior. Justin Erenkrantz got the ball rolling with this feature. Some of his support code was used unchanged. Other code was shuffled around and modified or rewritten.

stop using APLOG_NOERRNO in calls to ap_log_?error()

wait for worker threads to exit even in graceless termination (we already have code to make them exit sooner now)

Defer starting the listener until we have at least tried to start all the worker threads. We want to get the workers in the pool of available threads ASAP to keep perform_idle_server_maintenance from thrashing and starting too many processes prematurely. The code before this patch would dribble workers into the worker pool over an extended period of time and this prevented perform_idle_server_maintenance from accurately deciding when new processes were needed.

Close sockets on worker MPM when doing a graceless restart. This should resolve some segfaults see when doing such restarts. (Justin tweaked the palloc/memset in favor of calloc.) Submitted by: Aaron Bannert Reviewed by: Greg Ames, Sander Striker, Justin Erenkrantz

Detect APR_EINTR from ap_queue_pop() and avoid calling ap_queue_info_set_idle() more than once at a time per worker thread. This fixes an assert coredump. Submitted by: Aaron Bannert Reviewed by: Brian Pane

Moved the recycled pool list from the queue to the queue_info structure. The advantage of doing this is that it enables us to guarantee that the number of ptrans pools in existence at once is no greater than the number of worker threads, and that we'll never have to delete ptrans pools.

Add a "queue_info" structure to the worker MPM. This is used to prevent the listener thread from accept()ing more connections than there are available workers. This prevents long-running requests from starving connections that have been accepted but not yet processed. The queue_info is a simple counter, mutex, and condition variable. Only the listener thread blocks on the condition, and only when there are no idle workers. In the fast path there is a mutex lock, integer decrement, and and unlock (among a few conditionals). The worker threads each notify the queue_info when they are about to block on the normal worker_queue waiting for some connection to process, which wakes up any sleeping listener thread to go perform another accept() in parallel.

back out ill-conceived attempt to fix a segfault during graceless termination

use an independent pool for threads so that when we abandon them during graceless termination the cleanups on pchild won't mess with stuff they are still referencing

worker MPM: Fix a situation where a child exited without releasing the accept mutex. Depending on the OS and mutex mechanism this could result in a hang.

Remind the admin about the User and Group directives when we are unable to set permissions on a semaphore. PR: 7812

quiet a bogus gcc warning that crops up with -O2

worker MPM: add -DFOREGROUND option to use when you want the parent process to run in the foreground (NO_DETACH is a special mode for running under programs like daemontools) the existing flag -DDEBUG turns on FOREGROUND now instead of NO_DETACH

worker MPM: Don't create a listener thread until we have a worker thread. Otherwise, in situations where we'll have to wait a while to take over scoreboard slots from a previous generation, we'll be accepting connections we can't process yet. Don't let the listener thread clobber the scoreboard entry of the first worker thread.

Allow worker MPM to build on systems without pthread_kill(). Submitted by: Pier Fumagalli (and mangled by Jeff)

The old, legacy (and unused) code in which the scoreboard was totally and completely contained in a file (SCOREBOARD_FILE) has been removed. This does not affect scoreboards which are *mapped* to files using named-shared-memory at all. This implies that scoreboards must be based, at some level, on native shared memory (mmap, shm_open, shmget, whatever), but the code has assumed that for quite awhile now. Having the scoreboard be *based* on a file makes no sense today. PR: Obtained from: Submitted by: Reviewed by:

BUCKET FREELISTS Add an allocator-passing mechanism throughout the bucket brigades API. From Apache's standpoint, the apr_bucket_alloc_t* used throughout a given connection is stored in the conn_rec by the create_connection hook. That means it's the MPM's job to optimize recycling of apr_bucket_alloc_t's -- the MPM must ensure that no two threads can ever use the same one at the same time, for instance.

worker MPM: get MaxRequestsPerChild to work again by allowing the main thread of a child to be interrupted by one of the other threads in the process this should get graceful termination to work after encountering one of the various possible error conditions in the listener and worker threads

document yet another worker MPM glitch

remove some unnecessary, commented-out code; add a comment about a use of signals

Don't drop connections during graceful restart. Previously, worker threads could exit even though there were connections waiting in the queue. Now, for a graceful restart the worker threads won't exit until they are told that the queue has been drained and no more connections will ever be added.

graceful shutdown (no dropped connections) requires that the listener go away before the workers... introduce separate XXX_may_exit flags for our different categories of threads so that a future fix for graceful shutdown can terminate them in the right order rename signal_workers() to signal_threads() and give it a parameter so it knows whether or not termination should be graceful this commit doesn't change the behavior in any noticeable way; the flags used to tell threads to go away are still set at about the same time

document a glitch that prevents complete graceful termination of the process when one of the threads encounters a critical error

make the comment for the call to apr_signal_thread() a little more correct about what it is going to do

don't log a failure of ap_queue_pop() at termination; it is expected

in a worker thread, don't dare try to process a connection if ap_queue_pop() failed... csd is not cleared by ap_queue_pop() so checking csd is not helpful

Fold the compiled-in knowledge of the allowable AcceptMutex settings into its help entry. Requires the use of a extern string rather than a function call for the initialization to be valid in the macro (Thx to Jeff!). In the meantime, bump down the error logging until we deal with true default and configured setting information ala 1.3. PR: Obtained from: Submitted by: Reviewed by:

no need to kill the scoreboard cleanup anymore on graceful restart; the scoreboard is longer-lived PR: Obtained from: Submitted by: Reviewed by:

write a debug message to the log when we're stuck in the sicko state of trying to take over scoreboard slots that aren't going to be released (we could also be stalled while taking over slots if a thread in child gracefully terminating is serving a long-running request) update a comment describing the sicko state to remove any information I'm not absolutely sure of

Bring 2.0 up to parity, a bit, with how much info we provide to the admin regarding valid values for AcceptMutex. Should also tell 'em what "default" actually maps to, but that can wait. PR: Obtained from: Submitted by: Reviewed by:

The pre_mpm hook creates server-lifetime objects (or at least, for the generations across graceful restarts.) They should use the process pool.

Make the listener thread stop accepting new connections and exit cleanly at graceful restart time. This is a basic requirement of reliable graceful restarts (the kind that won't drop connections). This allows a future fix to make worker threads hang around until they service all connections previously accepted by the listener thread. The old mechanism of doing a dummy connection to wake up the listener thread in each old child process didn't work. It didn't guarantee that (in the main thread) the byte was read from the pod and global variables were set before the listener thread grabbed the connection. It didn't guarantee that a child process in the new generation didn't get some of the dummy connections. Rather than burn extra syscalls adding a unique socket or pipe to the poll set (and breaking single listen unserialized accept in the same change), this uses a signal sent from the main thread to the listener thread to break it out of the poll or accept. (We don't worry about breaking it out of the optional mutex because the child process holding the mutex will break out of poll/accept and release the mutex, allowing a child blocked in the mutex to get it. Eventually all children blocked in the mutex will come out.) Since the listener thread now exits reliably, the main thread joins it. PR: Obtained from: Submitted by: Reviewed by:

get the start thread to exit when we have reached termination but it is still trying to create worker threads previously, after a non-graceful restart followed by a terminate you could see a bunch of log messages showing the parent repeatedly sending SIGTERM and finally SIGKILL to one or more children... with this change, the sequence of messages should stop very soon add a comment to start_threads() describing a current problem

be a little more specific in a thread-create error message it is helpful to distinguish between a failure creating the first thread (listener) vs. a failure creating one of n worker threads

"apachectl stop" shouldn't send the gracefully-terminate char down the pod... the child processes need to know that it isn't a graceful termination and they shouldn't wait for old connections to finish

get rid of unnecessary code to track the number of worker threads

prefork and worker: after allocating a new scoreboard, make sure the right generation is stored in the global score

Simplify s/APR_OS_PROC_T_FMT/APR_PID_T_FMT/, apr_os_foo entities aren't for external consumption.

Update the mpms to call the new allocator/pools API

Update our copyright for this year.

failures on the accept mutex are common at restart time, so be smart about the log level and use APLOG_DEBUG if we're restarting

fix a typo in a log message... the message claimed that apr_proc_mutex_lock() failed when actually it was apr_proc_mutex_unlock() which failed

skip the lock release if the lock obtain failed

restart fixes for worker: fix a segfault and a window in which we could miss joining newly-created threads we can't try to signal workers if the worker queue hasn't been initialized (or we segfault) make sure the start thread is done creating threads before we try to join; otherwise we can just miss some of them and not be able to clean them up properly

Implement apr_proc_detach changes and allow -DNO_DETACH in the multi-process mode to not "daemonize" while detaching from the controlling terminal. This is necessary for Apache to work with process-management tools like AIX's "System Resource Controller" as well as Dan Bernstein's "daemontools". What this means is apache must _NOT_ be a process group leader if called with the -DNO_DETACH flag. Submitted from: Jos Backus <josb@cncdsl.com> Edited/Reviewed by: Aaron Bannert

Convert the ap_queue_foo routines to return apr_status_t as appropriate.

zap an incorrect comment

Implement new ScoreBoardFile directive logic. This affects how we create the scoreboard's shared memory segment. We now have the best of both worlds: if config specifies ScoreBoardFile create name-based shared memory, errors are fatal else /* we get to choose */ create anonymous shared memory if ENOTIMPL create name-based shared memory from DEFAULT_SCOREBOARD else errors are fatal This gives us the flexibility to have anonymous shared memory (on platforms that support it) as well as name-based shared memory when third-party programs want access to our scoreboard. The ap_scoreboard_fname static variable is now owned by the scoreboard.c file, and no longer by the MPMs. The MPMs MUST NOT set ap_scoreboard_fname to a default, since that will override the default creation logic and only allow name-based segments. Submitted by: Aaron Bannert Reviewed by: Justin Erenkrantz

Retain signal handling in the worker MPM for the one_process case (httpd with -DDEBUG, -X, or -DONE_PROCESS). Fix -X, -DNO_DETACH, -DONE_PROCESS, etc. flags. Tested on solaris w/ start/stop, restart, graceful, and with the above debugging flags.

Performance optimization: updated the worker MPM to recycle per-transaction pools instead of destroying them. Based on Ian's benchmark testing, this reduces CPU utilization by about 1% on Solaris.

if a child detects a resource shortage on accept(), limit the rate of fork()s to 1 per second until the situation clears up. Inspired by: Martin Kraemer

Remove all signal handling from the worker MPM's child processes. Instead, we use the pipe of death for all communication between parent and child.

Some compatibility fixes. Built yet untested, read three times through. Please review and raise the flag if any of these introduce bad mojo.

Not being able to bind to a socket is a fatal error. This makes all MPMs treat it as such. We now print a message to the console, and return a non-zero status code.

Reintroduce the create_connection hook. This hook is required to enable modules to completely take over all network i/o from the core.

Change the Windows MPM to only use the pre_mpm phase in the parent process. The child processes use the child_init phase to reattach to the shared memory. This makes Windows work like Unix, which should make it easier for module authors to write portable modules.

get rid of a bunch of warnings about unused variables

Added logic to the worker MPM to skip the poll before accept if there's only one listener socket (based on the implementation in the prefork MPM). This also enables us to do single-listener unserialized accept

This patch restores most of Ryan's patch (11/12/2001) to remove the client_socket from the conn_rec. Diffs from Ryan's patch include: - rename the create_connection hook to install_transport_filters - move the point of invocation of the hook till after the call to after ap_update_vhost_given_ip to enable the hook to use vhost config info in its decision making.

Take advantage of the new pre_config return value when apr_proc_detach fails. I'll be making some changes to apr_proc_detach, and it will be nice to be able to report runtime errors. Tested on worker but the code changes are identical on other MPMs.

The pre_config hook now takes a return value. This allows modules to cause the server to bail out under error conditions.

zap an unintended (and incorrect) comment that slipped in with the last commit; sorry!

fix the problem where a scoreboard init failure could leave mod_cgid stranded a pre_mpm hook can now return failures, so problems in ap_create_scoreboard percolate back to a place where Apache can exit cleanly

Remove the create_connection hook and put the client_socket back into the conn_rec. The create_connection_hook has a design flaw that prevents it from making decisions based on vhost information.

Fix for a segfault in the worker MPM during graceful shutdown: The per-transaction pools in the worker MPM can't be children of the listener thread's pool, because that pool may go out of scope while some workers are still procesing requests using the transaction pools.

This patch eliminated from the _SHARED_ segment of the scoreboard all pointer math. This is required for portable scoreboards. vhost becomes the 'vhost name string' so it now survives ap_generation clicks. next was apparently never used. This patch also accounts for the changes to the apr_shm api, and gives Win32 the magic of a shared scoreboard. Breakage aplenty on non-win32 platforms, I suspect, but this radical surgery, and culling of unused functions, was really, really needed.

handle the disappearance of apr_lock_create_np() and apr_proc_mutex_create_np() and the new parameter to apr_lock_create() and apr_proc_mutex_create()

mpm_common.c did all this dirty work - cleans up a duplicate symbol error from the OSX.1 build.

Clean up the error checking a little. Make sure we're comparing against APR_SUCCESS instead of just non-zero. Add some new error checking/reporting.

Take advantage of the new usable apr_thread_exit().

Add ap_os_killpg as well as unixd_killpg and beosd_killpg. Move the worker MPM to use ap_os_killpg in place of unixd_killpg as this makes it more portable. Ryan notes that really we should be using the POD mechanism to control death of the child processes, so this is temporary until we have that change in place. Note added to STATUS to that effect.

unixd.h is included in mpm.h so shouldn't need to be included here as well.

This allows platforms without the _NP locking palaver to build the worker MPM.

This fixes a segfault that showed up on BeOS and may catch other systems.

Add directives to worker and preform MPMs so that the admin can set the equivalent of HARD_SERVER_LIMIT/HARD_THREAD_LIMIT at startup.

leaving the pipe-of-death in the ap_listeners list led to segfaults during sighup processing since the pipe-of-death listen_rec was allocated from a short-lived pool simply allocating the pipe-of-death listen_rec from the right pool isn't a complete fix since at that point we no longer get the proper cleanup and we're left with a file descriptor leak the simplest overall fix seems to be to keep the pipe-of-death out of the ap_listeners list for now

Change core code to allow an MPM to set hard thread/server limits at startup.

Use the new APR pool code to reduce pool-related lock contention in the worker MPM. Submitted by: Sander Striker Reviewed by: Brian Pane

Style change. Convert tabs->spaces, shorten long lines, etc, etc, etc.

get worker MPM to respect the LockFile setting

Fix segfault when restarting worker MPM. We can not examine the POD as a normal listener. There was an additional bug than what Ian submitted to fix listen.c (i.e. check sa for null): make_pipe_of_death does not zero out lr->bind_addr (since it uses apr_palloc). So, on Solaris, the first time through, bind_addr was probably 0, but the second time, not (pools!). (This caused Aaron's analysis that the patch worked the first time, but not the second time.) Submitted by: Ian Holsman Reviewed by: Justin Erenkrantz, Aaron Bannert

fix segv in worker MPM following check_pipe_of_death() call

Get mod_cgid killed when a MPM exits due to a fatal error. Presumably other such processes are affected to. Now we give main() a chance to clean up.

Pull lingering close out of the pool cleanup phase. This was causing too many bugs. Instead, it is called where it used to be called. I have abstracted some of the logic out of the lingering close so that it gets the socket from the connection using the core's conn_config vector. This is in anticipation of a change to use a hook for the lingering close, which I hope to do soon.

Get rid of the listensocks array. It was just duplicating the ap_listeners list. No reason to have the same information stored twice.

Port the worker MPM to the new accept model. This has passed every test in the test suite (except Limit tests). We waste a bit of space with the listensocks array, which is the exact same as the ap_listeners list, but we have done that for a while. I'll fix that soon.

Add the server_rec argument back to the create_connection hook. Submitted by: Greg Stein

Remove commas from the end of the macros that define directives that are used by MPMs. Previous to this patch, you would use these macros without commans, which was unlike any other directives. Now, after the macro, you must have a comma. This makes the macros look more like the rest of the directives. I know this is cosmetic, and I was going to leave it alone, but when I found out that it bothered Cliff too, I decided to fix it after all. Submitted by: Ryan Bloom and Cliff Woolley

Remove more network logic from the MPMs. Essentially, each MPM can define AP_MPM_DISABLE_NAGLE_ACCEPTED_SOCK, which tells Apache to disable_nagle on the accepted socket.

Begin to abstract out the underlying transport layer. The first step is to remove the socket from the conn_rec, the server now lives in a context that is passed to the core's input and output filters. This forces us to be very careful when adding calls that use the socket directly, because the socket isn't available in most locations.

Really reset the MaxClients value in worker when the configured value is not a multiple of the number of threads per child. We said we did previously but we forgot to.

Remove ap_lingering_close from all of the MPMs. This is now done as a cleanup registered with the connection_pool. I have also turned ap_lingering_close into a static function, because it is only used in connection.c. This is the next step to consolidating all of the socket function calls. ap_lingering_close will only be added if the core is dealing with a standard socket.

Fix the spelling of the AP_MPMQ_MIN_SPARE_DAEMONS and AP_MPMQ_MAX_REQUESTS_DAEMON macros. Better to do it now rather than later. **WARNING** This will of course break the compile on any third-party MPMs you might have floating around, but it's a really quick change to make.

Fix the reporting for child processes that die. This removes all of the non-portable W* macros from Apache. Submitted by: Jeff Trawick and Ryan Bloom

Missed some error messages in the switch to the new lock system. Added the proc_mutex header for the sake of consistency.

The worker MPM is now completely ported to APR's new lock API. It uses native APR types for thread mutexes, cross-process mutexes, and condition variables. Who's next?

First pass at converting worker MPM to APR's new lock API. This converts the APR_INTRAPROCESS locks into lightweight, non-nested apr_thread_mutex_t locks.

Fix MaxClients in the Worker MPM, so that it specifies the maximum number of clients that can connect at the same time, instead of specifying the maximum number of child processes. Submitted by: Aaron Bannert <aaron@clove.org>

if we're gonna trash the connection due to a queue overflow, at the very least we should close the socket and write a log message (mostly to aid debugging, as this is a showstopper problem) this is no fix; there is a design issue to consider; hopefully this will

I was kinda hoping those (void)some_function() and (request_rec *)NULL casts would go away before this committed, but alas I didn't say anything. :-) This gets rid of them and a few others just like them that I also found in worker.c.

This patch fixes a nasty bug in the worker MPM where the state of the worker threads was not being reported back to the scoreboard, and eventually all the threads running in the children would be reported as being in the "C -- closing connection" state. This would wreak havoc on the idle_server_maintenance() routine. Since these threads would never be counted as idle, the server would spawn children as fast as possible. Submitted by: Aaron Bannert <aaron@clove.org> Reviewed by: Justin Erenkrantz

Switch back to SIGUSR1 for graceful restarts on all platforms that support it. This defines a symbol called AP_SIG_GRACEFUL in ap_config_auto.h which will have the appropriate signal value. All direct references to SIGWINCH have been replaced with AP_SIG_GRACEFUL. On Linux 2.0, use SIGWINCH instead since SIGUSR1 is used by glibc 2.0's user-space threading library to control threads. All later versions of Linux/glibc don't have this problem. (Not to mention the security holes in older Linux versions which make it unsuitable for use as a web server.) If your platform doesn't have SIGUSR1, use the appropriate mojo in configure to define what your graceful restart signal should be. In theory, a configure switch could be added to allow the admin to specify the appropriate signal that should be used. This is left as an exercise to the reader for now. The docs need to be updated. Since the signal is now configurable, just saying SIGUSR1 for graceful restart isn't completely true. Also, the apachectl functionality needs to be moved into httpd - this is what Win32 does and it makes us consistent across platforms. Roy issued a veto against use of SIGWINCH by default, so this should resolve that veto.

Cleanup the worker MPM. We no longer re-use transaction pools. This incurs less overhead than shuffling the pools around so that they can be re-used. Remove one of the queue's condition variables. We just redefined the API to state that you can't try to add more stuff than you allocated segments for. Submitted by: Aaron Bannert <aaron@clove.org>

Add -X functionality back to httpd. - Updates upgrading.html - Reverts Aaron's earlier docco patch (sorry...) - Adds -X to all mpms in the tree

Update the comments regarding -DONE_PROCESS and -DNO_DETACH since -X doesn't exist anymore. Submitted by: Aaron Bannert

Remove the declarations for variables that were moved into the core.

Fix a performance problem with the worker MPM. We now create transaction pools once, and re-use them for each connection. Submitted by: Aaron Bannert <aaron@clove.org>

didt pay attention to the first question; pretty sure answer is no

i think the answer to aaron's question is "a typo". otherwise there is actually 1 less thread available to serve requests than configured. PR: Obtained from: Submitted by: Reviewed by:

Make the worker MPM shutdown and restart cleanly. This also cleans up some race conditions, and gets the worker using pools more cleanly. Submitted by: [Aaron Bannert <aaron@clove.org>]

The Unix MPMs other than perchild now allow child server processes to use the accept mutex when starting as root and using SysV sems for the accept mutex. Previously, this combination would lead to fatal errors in the child server processes. perchild can't use SysV sems because of security issues. translation: steal apache 1.3 code to call semop(IPC_SET) on the semaphore to set permissions and uid/gid

namespace-protect accept_lock_mech fix a bug in the worker MPM which effectively disabled the AcceptMutex directive

Begin to sanitize the MPM configuration directives. Now, all MPMs use the same functions for all common MPM directives. This should make it easier to catch all bugs in these directives once. Everybody should check their favorite MPM to ensure that it still compiles, and that these directives work. This is a big patch, and although it looks good, and things compiled for me, that is no garauntee that it will work on all platforms. :-) Submitted by: Cody Sherr <csherr@covalent.net>

The worker MPM now handles shutdown and restart requests. It definately isn't perfect, but we do stop the servers correctly. The biggest problem right now is that SIGHUP causes the server to just die.

record something in the error log if the parent process seg faults Submitted by: Jeff Trawick

Explain the worker MPM

Get the worker MPM working again. This should fix the serialization problems, and it makes up initialize the queue only once.

Speed up the server's response to a spike in incoming workload or restarts. New processes aren't able to start all their threads right away if other dying processes share the same scoreboard real estate. So give empty process slots in the scoreboard top priority.

cosmetic changes only for a clean build; no serialization problems addressed; I presume those are waiting for an APR-ization of the queue/condition mechanism

fix a problem in the threaded and worker mpm's where a keepalive connection on one thread can prevent the process from exiting. let's rename ap_graceful_restart_signalled to something more appropriate after the next beta.

It helps if we broadcast to the workers that we are trying to shutdown.