mpm_winnt.c revision 10a4cdd68ef1ca0e54af296fe1d08ac00150c90b
* or thrown out entirely... /* To share the semaphores with other processes, we need a NULL ACL * Code from MS KB Q106387 * The Win32 call WaitForMultipleObjects will only allow you to wait for * a maximum of MAXIMUM_WAIT_OBJECTS (current 64). Since the threading * model in the multithreaded version of apache wants to use this call, * we are restricted to a maximum of 64 threads. This is a simplistic * routine that will increase this size. * Signalling Apache on NT. * Under Unix, Apache can be told to shutdown or restart by sending various * signals (HUP, USR, TERM). On NT we don't have easy access to signals, so * we use "events" instead. The parent apache process goes into a loop * where it waits forever for a set of events. Two of those events are * (where PID is the PID of the apache parent process). When one of these * is signalled, the Apache parent performs the appropriate action. The events * can become signalled through internal Apache methods (e.g. if the child * finds a fatal error and needs to kill its parent), via the service * control manager (the control thread will signal the shutdown event when * requested to stop the Apache service), from the -k Apache command line, * or from any external program which finds the Apache PID from the * The signal_parent() function, below, is used to signal one of these events. * It can be called by any child or parent process, since it does not * rely on global variables. * On entry, type gives the event to signal. 0 means shutdown, 1 means /* after updating the shutdown_pending or restart flags, we need * to wake up the parent process so it can see the changes. The * parent will normally be waiting for either a child process * to die, or for a signal on the "spache-signal" event. So set the * "apache-signal" event here. /* Um, problem, can't signal the parent, which means we can't * signal ourselves to die. Ignore for now... /* Same problem as above */ * Initialise the signal names, in the global variables signal_name_prefix, * signal_restart_name and signal_shutdown_name. * Routines that deal with sockets, some are WIN32 specific... /* The Nagle algorithm says that we should delay sending partial * packets in hopes of getting more data. We don't want to do * this; we are not telnet. There are bad interactions between * persistent connections and Nagle's algorithm that have very severe * performance penalties. (Failing to disable Nagle is not much of a * problem with simple HTTP.) * In spite of these problems, failure here is not a shooting offense. "setsockopt: (TCP_NODELAY)");
* Routines to deal with managing the list of listening sockets. /* Setup the listeners */ /* Setup the listeners */ /* Set up a default listener if necessary */ /* Open the pipe to the parent process to receive the inherited socket * data. The sockets have been set to listening in the parent process. "setup_inherited_listeners: Unable to read socket data from parent");
"Child %d: setup_inherited_listener() read = %d bytes of WSAProtocolInfo.",
my_pid);
"Child %d: setup_inherited_listeners(), WSASocket failed to open the inherited socket.",
my_pid);
/* Now, read the AcceptExCompPort from the parent */ /* Associate the open listeners with the completion port. * Bypass the operation for Windows 95/98 /********************************************************************** * Multithreaded implementation * This code is fairly specific to Win32. * The model used to handle requests is a set of threads. One "main" * thread listens for new requests. When something becomes * available, it does a select and places the newly available socket * onto a list of "jobs" (add_job()). Then any one of a fixed number * of "worker" threads takes the top job off the job list with * remove_job() and handles that connection to completion. After * the connection has finished the thread is free to take another * In the code, the "main" thread is running within the child_main() * function. The first thing this function does is create the * worker threads, which operate in the child_sub_main() function. The * main thread then goes into a loop within child_main() where they * do a select() on the listening sockets. The select times out once * per second so that the thread can check for an "exit" signal * from the parent process (see below). If this signal is set, the * thread can exit, but only after it has accepted all incoming * connections already in the listen queue (since Win32 appears * to through away listened but unaccepted connections when a * Because the main and worker threads exist within a single process * they are vulnerable to crashes or memory leaks (crashes can also * be caused within modules, of course). There also needs to be a * mechanism to perform restarts and shutdowns. This is done by * creating the main & worker threads within a subprocess. A * main process (the "parent process") creates one (or more) * processes to do the work, then the parent sits around waiting * for the working process to die, in which case it starts a new * one. The parent process also handles restarts (by creating * a new working process then signalling the previous working process * exit ) and shutdowns (by signalling the working process to exit). * The parent process operates within the master_main() function. This * process also handles requests from the service manager (NT only). * Signalling between the parent and working process uses a Win32 * event. Each child has a unique name for the event, which is * passed to it with the -Z argument when the child is spawned. The * parent sets (signals) this event to tell the child to die. * At present all children do a graceful die - they finish all * current jobs _and_ empty the listen queue before they exit. * A non-graceful die would need a second event. The -Z argument in * the child is also used to create the shutdown and restart events, * since the prefix (apPID) contains the parent process PID. * The code below starts with functions at the lowest level - * worker threads, and works up to the top level - the main() * function of the parent process. * The scoreboard (in process memory) contains details of the worker * threads (within the active working process). There is no shared * "scoreboard" between processes, since only one is ever active * at once (or at most, two, when one has been told to shutdown but * is processes outstanding requests, and a new one has been started). * This is controlled by a "start_mutex" which ensures only one working * process is active at once. **********************************************************************/ * Definition of jobs, shared by main and worker threads. * Globals common to main and worker threads. This structure is not * used by the parent process. /* Windows 9x specific code... * model. A single thread accepts connections and queues the accepted socket * to the accept queue for consumption by a pool of worker threads. * Calls remove_job() to pull a job from the accept queue. All the worker * threads block on remove_job. * accept_and_queue_connections() * The accept threads runs this function, which accepts connections off * the network and calls add_job() to queue jobs to the accept_queue. * Add or remove an accepted socket from the list of sockets * connected to clients. allowed_globals.jobmutex protects * against multiple concurrent access to the linked list of jobs. "Ouch! Out of memory in add_job()!");
/* A "real" error occurred, log it and increment the count of * select errors. This count is used to ensure we don't go into * a busy loop of continuous errors. "select failed with errno %d",
h_errno);
"Too many errors in select loop. Child process exiting.");
/* fetch the native socket descriptor */ "accept: (client socket)");
/* allocate the completion context and the transaction pool */ "win9x_get_connection: apr_pcalloc() failed. Process will exit.");
/* do we NEED_DUPPED_CSD ?? */ * Windows 2000/NT specific code... * create_acceptex_context() * reset_acceptex_context() * drain_acceptex_complport() * TODO: Insert a discussion of 'completion contexts' and what these function do here... "Child %d: - Draining an ABORTED packet off " "the AcceptEx completion port.",
my_pid);
"Child %d: - Draining and discarding an active connection " "off the AcceptEx completion port.",
my_pid);
/* It is only valid to clean-up in the process that initiated the I/O */ /* allocate the completion context */ "create_acceptex_context: apr_pcalloc() failed. Process will exit.");
/* initialize the completion context */ "create_acceptex_context: CreateEvent() failed. Process will exit.");
/* create and initialize the accept socket */ "create_acceptex_context: socket() failed. Process will exit.");
/* SO_UPDATE_ACCEPT_CONTEXT is required for shutdown() to work */ "setsockopt(SO_UPDATE_ACCEPT_CONTEXT) failed.");
/* Not a failure condition. Keep running. */ /* AcceptEx on the completion context. The completion context will be signaled * when a connection is accepted. */ 0,
//context->recv_buf_size, "create_acceptex_context: AcceptEx failed. Process will exit.");
/* recreate and initialize the accept socket if it is not being reused */ "reset_acceptex_context: socket() failed. Process will exit.");
/* SO_UPDATE_ACCEPT_CONTEXT is required for shutdown() to work */ "setsockopt(SO_UPDATE_ACCEPT_CONTEXT) failed.");
/* Not a failure condition. Keep running. */ /* reset the completion context */ /* AcceptEx on the completion context. The completion context will be signaled * when a connection is accepted. */ "reset_acceptex_context: AcceptEx failed for " "listening socket: %d and accept socket: %d",
/* Clean-up the AcceptEx completion context */ /* Prepare the completion context for reuse */ /* Retry once, this time requesting a new socket */ /* Failed again, so give up, but leave the thread up * Should we signal a shutdown now? "Child %d: winnt_get_connection: reset_acceptex_context failed.",
/* May need to atomize the workers_may_exit check with the /* Is this a deadly condition? Hummm... */ "Child %d: - GetQueuedCompletionStatus() failed",
/* Sometimes we catch ERROR_OPERATION_ABORTED completion packets * from the old child process (during a restart). Ignore them. "Child %d: - Draining ERROR_OPERATION_ABORTED packet off " "the completion port.",
my_pid);
/* CompKey == my_pid means this thread was unblocked by * the shutdown code (not by io completion). /* Sometimes we catch shutdown io completion packets * posted by the old child process (during a restart). Ignore them. /* Check to see if we need to create more completion contexts, * but only if we are not in the process of shutting down /* Received a connection */ 0,
//context->recv_buf_size, * worker_main() - this is the main loop for the worker threads * Each thread runs within this function. They wait within remove_job() * for a job to become available, then handle all the requests on that * connection until it is closed, then return to remove_job(). * The worker thread will exit when it removes a job which contains * socket number -1. This provides a graceful thread exit, since * it will never exit during a connection. * This code in this function is basically equivalent to the child_main() * from the multi-process (Unix) environment, except that we * - do not call child_init_modules (child init API phase) * - block in remove_job, and when unblocked we have an already * accepted socket, instead of blocking on a mutex or select(). /* Grab a connection off the network */ "worker_main: attach_socket() failed. Continuing...");
/* Kill the clean-up registered by the iol. We want to leave * the accept socket open because we are about to try to /* Disable lingering close for the moment to fix a seg fault. * All the sendfile code needs some serious work to return * proper error values, handle updating bytes_sent, etc. * I'll enable lingering close after I've fixed the sendfile "Child %d: Thread exiting.",
my_pid);
/* TODO: Add code to clean-up completion contexts here */ "Unable to create an AcceptEx completion context -- process will exit");
* child_main() runs the main control thread for the child process. * - sets up the worker thread pool * - starts the accept thread (Win 9x) * - creates AcceptEx contexts (Win NT) * - waits for exit_event, maintenance_event or maintenance timeout * and does the right thing depending on which event is received. /* This is the child process or we are running in single process /* Single process mode */ /* Initialize the child_events */ * Wait until we have permission to start accepting connections. * start_mutex is used to ensure that only one child ever "Child %d: Failed to acquire the start_mutex. Process will exit.",
my_pid);
"Child %d: Acquired the start mutex.",
my_pid);
/* Create the worker thread pool */ /* Begin accepting connections */ /* Win95/98: Start the accept thread */ /* Windows NT/2000: Create AcceptEx completion contexts */ /* Wait for one of three events: * The exit_event is signaled by the parent process to notify * the child that it is time to exit. * This event is signaled by the worker thread pool to direct * this thread to create more completion contexts. * To do periodic maintenance on the server (check for thread exits, * number of completion contexts, etc.) /* Something serious is wrong */ "Child %d: WAIT_FAILED -- shutting down server");
/* Hey, this cannot happen */ "Child %d: WAIT_TIMEOUT -- shutting down server",
my_pid);
/* Exit event was signaled */ "Child %d: Exit event signaled. Child process is ending.",
my_pid);
/* Child maintenance event signaled */ "Child %d: Child maintenance event signaled.",
my_pid);
/* Setting is_graceful will close keep-alive connections */ /* Shutdown the worker threads */ /* workers_may_exit = 1; Not used on Win9x */ else {
/* Windows NT/2000 */ * Setting shutdown_in_progress prevents new AcceptEx completion * contexts from being queued to the port but allows threads to * continue consuming from the port. This gives the server a * chance to handle any accepted connections. /* Setting workers_may_exit prevents threads from consumimg from the * completion port (especially threads that unblock off of keep-alive /* Unblock threads blocked on the completion port */ /* Cancel any remaining pending AcceptEx completion contexts */ /* Drain the canceled contexts off the port */ /* Release the start_mutex to let the new process (in the restart * scenario) a chance to begin servicing requests "Child %d: Releasing the start mutex",
my_pid);
/* Give busy worker threads a chance to service their connections. * Kill them off if they take too long "Child %d: All worker threads have ended.",
my_pid);
* Spawn a child Apache process. The child process has the command line arguments from * argc and argv[], plus a -Z argument giving the name of an event. The child should * open and poll or wait on this event. When it is signalled, the child should die. * prefix is a prefix string for the event name. * The child_num argument on entry contains a serial number for this child (used to create * a unique event name). On exit, this number will have been incremented by one, ready * On exit, the value pointed to be *ev will contain the event created * to signal the new child process. * The return value is the handle to the child process if successful, else -1. If -1 is * returned the error will already have been logged by ap_log_error(). /********************************************************************** * master_main - this is the parent (main) process. We create a * child process to do the work, then sit around waiting for either * the child to exit, or a restart or exit signal. If the child dies, * we just respawn a new one. If we have a shutdown or graceful restart, * tell the child to die when it is ready. If it is a non-graceful * restart, force the child to die immediately. **********************************************************************/ /* Build the command line. Should look something like this: * First, get the path to the executable... "Parent: Path to Apache process too long");
"Parent: GetModuleFileName() returned NULL for current process.");
/* Build the command line */ /* Build the environment, since Win9x disrespects the active env */ * Win32's CreateProcess call requires that the environment * be passed in an environment block, a null terminated block of * null terminated strings. /* Create a pipe to send socket info to the child */ "Parent: Unable to create pipe to child process.");
/* Give the read end of the pipe (hPipeRead) to the child as stdin. The * parent will write the socket data to the child on this pipe. TRUE,
/* Inherit handles */ "Parent: Not able to create the child process.");
* We must close the handles to the new process and its main thread * to prevent handle and memory leaks. /* Create the exit_event, apCchild_pid */ "Parent: Could not create exit event for child process");
/* Assume the child process lives. Update the process and event tables */ /* We never store the thread's handle, so close it now. */ /* Run the chain of open sockets. For each socket, duplicate it * for the target process then send the WSAPROTOCOL_INFO * (returned by dup socket) to the child */ "Parent: Duplicating socket %d and sending it to child process %d",
nsd,
pi.
dwProcessId);
"Parent: WSADuplicateSocket failed for socket %d.",
lr->
sd );
"Parent: Unable to write duplicated socket %d to the child.",
lr->
sd );
/* Now, send the AcceptEx completion port to the child */ "Parent: Unable to duplicate AcceptEx completion port. Shutting down.");
* Should only be one in this version of Apache for WIN32 "master_main: create child process failed. Exiting.");
/* Wait for shutdown or restart events or for child death */ /* Something serious is wrong */ "master_main: WaitForMultipeObjects WAIT_FAILED -- doing server shutdown");
/* Hey, this cannot happen */ "master_main: WaitForMultipeObjects with INFINITE wait exited with WAIT_TIMEOUT");
/* shutdown_event signalled */ printf(
"shutdown event signaled\n");
"master_main: Shutdown event signaled -- doing server shutdown.");
"ResetEvent(shutdown_event)");
/* restart_event signalled */ "master_main: Restart event signaled. Doing a graceful restart.");
"master_main: ResetEvent(restart_event) failed.");
/* Signal each child process to die * We are making a big assumption here that the child process, once signaled, * will REALLY go away. Since this is a restart, we do not want to hold the * new child process up waiting for the old child to die. Remove the old * child out of the process_handles apr_table_t and hope for the best... "master_main: SetEvent for child process in slot #%d failed", i);
/* A child process must have exited because of a fatal error condition (seg fault, etc.). * Remove the dead process * from the process_handles and process_kill_events apr_table_t and create a new * TODO: Consider restarting the child immediately without looping through http_main * and without rereading the configuration. Will need this if we ever support multiple * children. One option, create a parent thread which waits on child death and restarts it. * Consider, however, that if the user makes httpd.conf invalid, we want to die before * our child tries it... otherwise we have a nasty loop. "master_main: Child process failed. Restarting the child process.");
/* APD2("main_process: child in slot %d died", rv); */ /* restart_child(process_hancles, process_kill_events, cld, ¤t_live_processes); */ /* Drain the AcceptEx completion port of any outstanding I/O pending for the dead /* Signal each child processes to die */ "master_main: SetEvent for child process in slot #%d failed", i);
return 0;
/* Tell the caller we do not want to restart */ return 1;
/* Tell the caller we want a restart */ /* service_nt_main_fn needs to append the StartService() args * outside of our call stack and thread as the service starts... /* Remember service_to_start failures to log and fail in pre_config. * Remember inst_argc and inst_argv for installing or starting the * service after we preflight the config. /* Handle the following SCM aspects in this phase: * -k runservice [transition for WinNT, nothing for Win9x] * -k (!)install [error out if name is not installed] * We can't leave this phase until we know our identity * and modify the command arguments appropriately. /* AP_PARENT_PID is only valid in the child */ /* The parent is responsible for providing the * COMPLETE ARGUMENTS REQUIRED to the child. * No further argument parsing is needed, but * for good measure we will provide a simple * signal string for later testing. /* This is the parent, we have a long way to go :-) */ /* Rewrite process->argv[]; * strip out -k signal into signal_arg * strip out -n servicename into service_name & display_name * add default -d serverroot from the path of this executable * The end result will look like: * The invocation command (%0) * The -d serverroot default from the running executable * The requested service's (-n) registry ConfigArgs * The WinNT SCM's StartService() args /* WARNING: There is an implict assumption here that the * executable resides in the ServerRoot! "Failed to get the running module's file name");
/* Use process->pool so that the rewritten argv * lasts for the lifetime of the server process, * because pconf will be destroyed after the * initial pre-flight of the config parser. /* TODO: warn of depreciated syntax, "use -k install instead" */ /* TODO: warn of depreciated syntax, "use -k uninstall instead" */ /* Set optreset and optind to allow apr_getopt to work correctly /* Track the number of args actually entered by the user */ /* Provide a default 'run' -k arg to simplify signal_arg tests */ /* Start the NT Service _NOW_ because the WinNT SCM is * expecting us to rapidly assume control of our own * process, the SCM will tell us our service name, and * may have extra StartService() command arguments to * Any other process has a console, so we don't to begin * a Win9x service until the configuration is parsed and * any command line errors are reported. * We hold the return value so that we can die in pre_config * after logging begins, and the failure can land in the log. "%s: ConfigArgs are missing from the registry.",
/* Track the args actually entered by the user. * These will be used for the -k install parameters, as well as * for the -k start service override arguments. /* Handle the following SCM aspects in this phase: * -k runservice [WinNT errors logged from rewrite_args] * in these cases we -don't- care if httpd.conf has config errors! "%s: Unable to start the service manager.",
/* Handle the following SCM aspects in this phase: * -k runservice [Win95, only once - after we parsed the config] * because all of these signals are useful _only_ if there * is a valid conf\httpd.conf environment to start. * We reached this phase by avoiding errors that would cause * these options to fail unexpectedly in another process. /* This code should be run once in the parent and not run /* Create the AcceptEx IoCompletionPort once in the parent. * The completion port persists across restarts. 0);
/* CONCURRENT ACTIVE THREADS */ "Parent: Unable to create the AcceptExCompletionPort -- process will exit");
/* Create shutdown event, apPID_shutdown, where PID is the parent * Apache process ID. Shutdown is signaled by 'apache -k shutdown'. /* Create restart event, apPID_restart, where PID is the parent * Apache process ID. Restart is signaled by 'apache -k restart'. /* Now that we are flying at 15000 feet... * wipe out the Win95 service console, * signal the SCM the WinNT service started, or * if not a service, setup console handlers instead. "%s: Unable to start the service manager.",
else /* ! -k runservice */ /* Create the start mutex, apPID, where PID is the parent Apache process ID. * Ths start mutex is used during a restart to prevent more than one * child process from entering the accept loop at once. else /* parent_pid != my_pid */ static int restart = 0;
/* Default is "not a restart" */ /* Running as Child process or in one_process (debug) mode */ "Child %d: Child process is running",
my_pid);
"Child %d: Child process is exiting",
my_pid);
}
/* Child or single process */ else {
/* Parent process */ /* Shutting down. Clean up... */ return "PidFile directive not allowed in <VirtualHost>";
"WARNING: ThreadsPerChild of %d exceeds compile time" " lowering ThreadsPerChild to %d. To increase, please" " see the HARD_THREAD_LIMIT define in %s.",
"WARNING: Require ThreadsPerChild > 0, setting to 1");
" does not exist or is not a directory",
NULL);
/* Stub functions until this MPM supports the connection status API */ "A file for logging the server process ID"},
"Number of threads each child creates" },
"Maximum number of requests a particular child serves before dying." },
"The location of the directory Apache changes to before dumping core" },
NULL,
/* create per-directory config structure */ NULL,
/* merge per-directory config structures */ NULL,
/* create per-server config structure */ NULL,
/* merge per-server config structures */