core: follow up to r1668532: always initialize optional_fn pointers in ap_create_scoreboard().

core: Initialize scoreboard's used optional functions on graceful restarts to avoid a crash when relocation occurs. PR 57177.

* server/scoreboard.c: Check for child_num < 0 also in ap_update_child_status and ap_update_child_status_from_conn. PR 56767.

mod_status should honor remote_ip as documented

SECURITY (CVE-2014-0226): Fix a race condition in scoreboard handling, which could lead to a heap buffer overflow. Thanks to Marek Kroemeke working with HP's Zero Day Initiative for reporting this. * include/scoreboard.h: Add ap_copy_scoreboard_worker. * server/scoreboard.c (ap_copy_scoreboard_worker): New function. * modules/generators/mod_status.c (status_handler): Use it. * modules/lua/lua_request.c (lua_ap_scoreboard_worker): Likewise. Reviewed by: trawick, jorton, covener, jim Submitted by: jorton, covener

Have static analyser tool happy. Point 2 of PR 54936

save a apr_time_now() call for every request.

core: Respect DefaultRuntimeDir/DEFAULT_REL_RUNTIMEDIR for the scoreboard (ScoreBoardFile).

Revert r1294936 (Jeff's revert) and fix NetWare / Win32 build. Added the missing AP_DECLARE* decorations for the function implementations and variables.

Add the port number to the vhost's name in the scoreboard

SECURITY (CVE-2012-0031): Fix possible crash on shutdown if a child changes the sb_type field in the scoreboard. Since unprivileged children should not be able to affect the parent in this way, this is treated as a Low severity security issue. Thanks to "halfdog" <me halfdog.net> for reporting this issue. * include/scoreboard.h (global_score): Remove sb_type field. * include/ap_mmn.h: Bump MMN for above. * server/scoreboard.c (ap_cleanup_scoreboard, ap_create_scoreboard): Use a static global to store store the scoreboard type.

Add lots of unique tags to error log messages

More cleanup: Expand tabs and some more indentation fixes No functional change

Add wrappers for malloc, calloc, realloc that check for out of memory situations. Use them in most places where malloc, and friends are used. This results in clean error messages in an out of memory situation instead of segfaulting or silently malfunctioning. In some places, it just allows to remove some logging code. PR 51568, PR 51569, PR 51571.

s/seperate/separate/

Introduce ap_(get|set)_core_module_config() functions/macros and use them everywhere. We know that the core module has module_index 0. Therefore we can save some pointer operations in ap_get_module_config(cv, &core_module) and ap_set_module_config(cv, &core_module, val). As these are called rather often, this may actually have some (small) measurable effect.

fix some logging calls to include the server_rec (and even the conn_rec in a couple of places)

whitespace

OK... a good commit point (we don't quite compile yet though...) Pull out the worker scoreboard cruft and start moving most worker stuff to shm. Use slotmem for workers and provide space for growth. Redo logic: ap_proxy_define_* ap_proxy_create_* ap_proxy_initialize_* Right now just for workers, but lay framework for balancers as well. The idea is to break out the functional parts to make it easy for dynamics. Defining is simply describing the worker, and tucking that info away. When we create, we go ahead and create the shared memory, etc... Initialize is simply to allow child process to access the shm...

running_gen was only set to an interesting value on a path where it wasn't used before returning from the function; axe it

Use the new APLOG_USE_MODULE/AP_DECLARE_MODULE macros everywhere to take advantage of per-module loglevels

Refactor mod_status out of the ExtendedStatus toggle picture, as other modules can and should be able to use this scoreboard information. Rename suggestions for such a directive are welcomed. Note that mod_status toggles the default, to present the ExtendedStatus for server operators, whenever it is loaded. Docs changes to follow.

* server/scoreboard.c::ap_create_scoreboard Reset the pointer to ap_logio_get_last_bytes each time we create or reinit an existing scoreboard to avoid using an invalid pointer in case mod_logio got reloaded to a different address. Reviewed by: rjung, trawick

Use the new ap_update_child_status_from_conn() to put the client IP into the scoreboard (and show it in mod_status) as soon as we start reading the request. Empty the vhost and request fields in ap_update_child_status_from_conn so that mod_status does not display outdated data if the connection closes without the request being completely read.

mod_logio: introduce new optional function ap_logio_get_last_bytes to get total byte count of last request. core: Use ap_logio_get_last_bytes to report more accurate byte counts in mod_status if mod_logio is loaded. Without mod_logio, don't increment counts for HEAD requests. PR: 25656

refactor child status update functions to accommodate the addition of ap_update_child_status_from_conn() Reviewed by: rpluem

Provide new ap_update_child_status_from_conn() mostly for use with mod_noloris.c Add some logic protection, for NULL ref, which shoulda be there in any case.

Introduce a new set of APIs to allow MPMs to be proper modules instead of integral parts which share global variables, functions, and macros with the rest of httpd. Converted now: prefork, worker, event, simple, WinNT* *WinNT hasn't been built or tested, and relies on a hack to include the WinNT mpm.h to disable Unixy MPM support routines in mpm_common.c

API Cleanup in preperation for 2.4.x, make sure all exported functions or variables contain an ap_ prefix.

* Enable the proxy to keep connections persistent in the HTTPS case. Basicly the persistence is created by keeping the conn_rec structure created for our backend connection (whether http or https) in the connection pool. This required to adjust scoreboard.c in a way that its functions can properly deal with a NULL scoreboard handle by ignoring the call or returning an error code.

* Use the same declaration for ap_time_process_request as in the header file. PR: 43789 Submitted by: Tom Donovan <Tom.Donovan acm.org> Reviewed by: rpluem

Add in SeeRequestTail directive, to handle the shortcoming of only storing 63 bytes of the request, when the requests are longer than that and only vary towards the end; eg: GET /disk1/storage/apache/htdocs/images/image-store1/food/fruits/seeded/apples.jpg GET /disk1/storage/apache/htdocs/images/image-store1/food/fruits/seeded/pears.jpg GET /disk1/storage/apache/htdocs/images/image-store1/food/fruits/seeded/plums.jpg

Actually re-fix Christian's recommendation, someone previously caught this.

MMN major bump required; this API is altogether inconsistent, transparent types should be opaque, opaque types should be transparent. Solve one aspect, follow the _by_indexes() example for ap_get_scoreboard_worker family of functions, and the primary one will now accept the abstracted conn_rec value of sbh to find that connections slot.

PR 40037: try to avoid risk of misleading the lusers when they can't initialise scoreboard after a crash.

Some minor cleanups: 1. Rename worker_size, since this could easily be confused with the actual MPM worker struct stuff. 2. Remove superflous ()s in the || test 3. Add superflous () in the pointer adjusting :) Even though C prec rules make it clear what we are doing, make it painfully obvious. Remove unrequired temp var.

lb_score is bigger proxy_worker_stat. Prevent wasting space.

Reinsert extern int PROXY_DECLARE_DATA proxy_lb_workers into the Proxy API.

Revert r547987 ("svn merge -c -547987 .")

PID table impl: parent process keeps a local table store of Apache child process PIDs and uses that to check validity of what's in the scoreboard.

update license header text

Make the range test legible; in the process, uncover and close a bounds overflow condition.

Typo (..for use with by the scoreboard) changed to "..for use by the scoreboard" (not 100% sure -- or should it be "..for use with the scoreboard"?)

Scoreboard size (Chris Darroch)

Update the copyright year in all .c, .h and .xml files

No functional Change: Removing trailing whitespace. This also means that "blank" lines consisting of just spaces or tabs are now really blank lines

Fix some symbols which should -not- be exported, and decorate real_exit_code with ap_ (this symbol must be exported.) If the others were desired to be public, they should have been ap_xxx decorated.

Update copyright year to 2005 and standardize on current copyright owner line.

Keep track of time taken to process requests again.

veto and revert win64 patch: 64bit changes must percolate from the bottom (APR/system) up -- we can't give the client a 64bit API and then cast it to 32bits internally without introducing security holes on other platforms.

WIN64: API changes to clean up Windows 64bit compile warnings

* server/scoreboard.c (create_namebased_scoreboard): Use apr_shm_remove() rather than the unsufficient apr_file_remove(). PR: 27189

Worker shared data has to be visible across all child processes, so remove creating that data on per-child basis.

Add load balancer support to the scoreboard in preparation for load balancing support in mod_proxy. PR: Obtained from: Submitted by: Mladen Turk <mturk@apache.org> Reviewed by:

fix name of The Apache Software Foundation

fix copyright dates according to the first check in

apply Apache License, Version 2.0

update license to 2004.

On failures, try to add a hint what went wrong

finished that boring job: update license to 2003. Happy New Year! ;-))

stop using APLOG_NOERRNO in calls to ap_log_?error()

Fix a potential size mismatch (apr_off_t might be 64bits on some platforms).

The old, legacy (and unused) code in which the scoreboard was totally and completely contained in a file (SCOREBOARD_FILE) has been removed. This does not affect scoreboards which are *mapped* to files using named-shared-memory at all. This implies that scoreboards must be based, at some level, on native shared memory (mmap, shm_open, shmget, whatever), but the code has assumed that for quite awhile now. Having the scoreboard be *based* on a file makes no sense today. PR: Obtained from: Submitted by: Reviewed by:

ap_create_scoreboard() now initializes the scoreboard if it already exists, so MPMs shouldn't call it for a graceful restart

The source of some ills ... there is no reason to keep going if the scoreboard exists. I suspect this should be a general cleanup as well [at the end of ap_create_scoreboard.] But calling ap_run_pre_mpm with the process->pool should take care of a clobbered scoreboard shm on graceful restart.

Eliminate potential ap_server_root_relative segfaults, with the input of Jeff Trawick's style changes to the first patches. Doesn't include the fixes to ssl [more complex], and we won't trap errors that involve ap_serverroot, since we presume that was normalized on the way in. Therefore, testing ap_server_root_relative(DEFAULT_FOO) cases should never become necessary.

Update our copyright for this year.

Fix the calculation of thread_num in the worker score structure. Submitted by: Stas Bekman Reviewed by: Jeff Trawick

ap_calc_scoreboard_size() needs to be exported for win32

remove a tab

Placing the thread-limit and server-limit values, those which define the size of internal structures in the scoreboard, actually into the scoreboard allow third-party applications to access those structures. Submitted by: Adam Sussman <myddryn@vishnu.vidya.com> Reviewed by: Aaron Bannert

This fixes a bug in mod_status on Windows where restart time was not properly initialized. Move ap_restart_time into the scoreboard global area so the child process on non-forking platforms can have access to it.

Implement new ScoreBoardFile directive logic. This affects how we create the scoreboard's shared memory segment. We now have the best of both worlds: if config specifies ScoreBoardFile create name-based shared memory, errors are fatal else /* we get to choose */ create anonymous shared memory if ENOTIMPL create name-based shared memory from DEFAULT_SCOREBOARD else errors are fatal This gives us the flexibility to have anonymous shared memory (on platforms that support it) as well as name-based shared memory when third-party programs want access to our scoreboard. The ap_scoreboard_fname static variable is now owned by the scoreboard.c file, and no longer by the MPMs. The MPMs MUST NOT set ap_scoreboard_fname to a default, since that will override the default creation logic and only allow name-based segments. Submitted by: Aaron Bannert Reviewed by: Justin Erenkrantz

Moved the #endif of APR_HAS_SHARED_MEMORY to include a reference to the global variable ap_scoreboard_shm. Since this variable was already being excluded by APR_HAS_SHARED_MEMORY at the beginning of the file.

Create the scoreboard (in the parent) in a global pool context, so it survives graceful restarts. This fixes a SEGV during graceful restarts. Children who attach to this scoreboard keep the same pool as before (pchild) since they should detach/unmap when the child process exits.

Change the Windows MPM to only use the pre_mpm phase in the parent process. The child processes use the child_init phase to reattach to the shared memory. This makes Windows work like Unix, which should make it easier for module authors to write portable modules.

fix the problem where a scoreboard init failure could leave mod_cgid stranded a pre_mpm hook can now return failures, so problems in ap_create_scoreboard percolate back to a place where Apache can exit cleanly

axe an unused variable don't pass uninitialized rv to ap_log_error as the error code

No. I don't like this patch. I like what it does [shared scoreboard for Windows, finally] but not how it's implemented. However it works, and is equally crufty to what already exists for win32 listeners and other exposed data within scoreboard.c. To do this right, we need to drop all the external references to data within the scoreboard, and add an accessor for remaining bits (such as Win32's need for direct access to the apr_shm_t.) And within Win32, we need to stack all this _within_ the pre-mpm hook. But those are missions in and of themselves.

A quick hack around Aaron's patch. Win32 'supports' anon shm, but Apache's implementation requires fork().

Just a style update before I work on this thing any more. Removed tabs and lined up parameters and indentation.

Although this patch is technically correct, I'm not happy with the way it gets things done. OTOH, it is a simple enough change to get things working correctly for now. I will come up with the right way to do this in the next couple days. This patch re-enables the use of anonymous shared memory in the scoreboard on platforms that have it.

Added APR_HAS_SHARED_MEMORY to a section of code where it was missing, for those of us that don't have shared memory support.

This patch eliminated from the _SHARED_ segment of the scoreboard all pointer math. This is required for portable scoreboards. vhost becomes the 'vhost name string' so it now survives ap_generation clicks. next was apparently never used. This patch also accounts for the changes to the apr_shm api, and gives Win32 the magic of a shared scoreboard. Breakage aplenty on non-win32 platforms, I suspect, but this radical surgery, and culling of unused functions, was really, really needed.

Make calc_scoreboard_size() and init_scoreboard() usable from MPMs.

fix a horrible bug which caused scoreboard initialation to always exit

The scoreboard is no longer created while we can write to stderr, so call ap_log_error() to note problems allocating the scoreboard. apr_shm_malloc() doesn't set any kind of error code, so don't try to reference errno. setup_shared() now returns an error code to its caller rather than exiting directly. This isn't a complete fix, as ap_create_scoreboard() needs to do the same thing. Currently, when we can't allocate the scoreboard we leave the mod_cgid daemon process stranded.

Clean up a compiler emit for signedness

Change core code to allow an MPM to set hard thread/server limits at startup.

Provide an accessor function for the global_score portion of the scoreboard. Submitted by: Harrie Hazewinkel <harrie@covalent.net> Reviewed by: Aaron Bannert

Export ap_get_parent_scoreboard and ap_get_servers_scoreboard to be usable on Win32 and friends. Submitted by: Harrie Hazewinkel <harrie@covalent.net> Reviewed by: Aaron Bannert

prevent seg faults in mod_status trying to access vhost structures from the former generation, after a restart. It happens mostly when the MPM has threads.

when we bail out due to a fatal error in apr_shm_init(), be sure to print the apr_status_t in numeric form the old message lt-httpd: could not open(create) scoreboard: Unknown resolver error wasn't so useful, but the new, improved message lt-httpd: could not open(create) scoreboard: (22528)Unknown resolver error makes it look like a syscall is returning ENOSPC and that apr shmem is returning (APR_OS_START_SYSERR + errno) instead of simply errno. off to APR land...

adjust to apr_uri_ rename

adjust to UNP_ -> APR_URI_UNP_ rename

clean up the scoreboard properly after threaded processes die gracefully. find_child_by_pid() failed to do its job, because ap_max_daemons_used was too low. There was some recent breakage in threaded that contributes to the problem. However, find_child_by_pid() should not be using the current number of daemons. It could have been recently reduced, which makes it fail to find the exiting process.

Exported symbols for mod_pop

prevent seg faults in mod_status with ExtendedStatus enabled, after a restart. A pointer to the previous generation's vhost server_rec could be left in the worker_score under certain conditions.

Fix ap_create_scoreboard fooness on Win32.

Make scoreboard creation a hook. This allows management modules to have access to the scoreboard at the time that it is created, and at every restart request. Submitted by: Cody Sherr <csherr@covalent.net> Reviewed by: Ryan Bloom

Changed AP_MPMQ_MAX_DAEMONS to refer to MaxClients and added an AP_MPMQ_MAX_DAEMON_USED to refer to the highest daemon index actually used in the scoreboard. I also updated the pertinent calls. Paul J. Reder

Remove a couple fields from the scoreboard that aren't currently used. If we need these, they should be added when we begin to use them.

scratch an old itch - give lingering close its own state in the scoreboard. clean up SERVER_ACCEPTING and SERVER_QUEUEING (never set) while I'm at it.

Add two functions to allow modules to access random parts of the scoreboard. This allows modules compiled for one MPM to access the scoreboard, even if it the server was compiled for another MPM. Submitted by: Harrie Hazewinkel <harrie@covalent.net>

Moved util_uri to apr-util/uri/apr_uri, which means adding the apr_ prefix to all of the uri functions (yuck), changing some includes, and using APR error codes instead of HTTP-specific error codes. Other notes to test this patch: - You need to delete the util_uri.h file - exports picks up on this. - I'd like to remove the apr_uri.h from httpd.h, but that might increase the complexity of this patch even further. Once this patch is accepted (in some form), then I can focus on removing apr_uri.h from httpd.h entirely. I need baby steps (heh) right now. - I imagine that this might break a bunch of stuff in Win32 or other OS builds with foreign dependency files. Any help here is appreciated. This is a start... -- justin Submitted by: Justin Erenkrantz Reviewed by: Roy Fielding

Make first phase changes to the scoreboard data structures in preparation for the rewriting of the scoreboard per my posted design notes. [Paul J. Reder]

fix some generation logic/displays in mod_status by setting the generation field in parent_score

minor scoreboard/status improvements: . get the SS field in extended status output formatted correctly (seconds since beginning of request shouldn't be a huge negative number :) ) . use APR_OS_PID_T_FMT and pid_t where appropriate in mod_status to avoid casting, some of which may have been broken on some architectures

tweak ap_get_remote_host() so that the caller can find out if she got back an IP address mod_access needed to know this, but the old code didn't handle IPv6

Allow modules to query the MPM about it's execution profile. This query API can and should be extended in the future, but for now, max_daemons, and threading or forking is a very good start. Non-Unix MPM's do have the MPM query function, although there is no garauntee that the information is perfect, please check. Submitted by: Jon Travis <jtravis@covalent.net>

*) fix inline handling. we had: apr_inline, APR_INLINE, USE_GNU_INLINE, and INLINE. Now, we just have APR_INLINE and APR_HAS_INLINE. - convert all usage - note that apr_general messed up the defn (compared to apr.h) - simplify the inline decision logic in os/*/os.h - simplify the code in os/*/os-inline.c *) toss ap_checkconv() [no longer used]

Update copyright to 2001

Start getting extended status working again.

*) include the strings stuff the right way (some platforms use string.h; apr_want figures all that out for us) *) include ap_mpm.h to pick up a moved func declaration

Fix a warning.

Clean up some of the includes: - explicitly include apr_lib.h since ap_config.h doesn't - use apr_want.h where possible - use APR_HAVE_ where possible - remove some unneeded includes

*) remove obsolete http_conf_globals.h *) simplify all this #ifdef'ing: just nuke the unixd.h header. it isn't actually needed.

BeOS doesn't have unixd.h either.

Replace reinit_scoreboard with ap_create_scoreboard.

renaming various functions for consistency sake see: http://apr.apache.org/~dougm/apr_rename.pl PR: Obtained from: Submitted by: Reviewed by:

Fix graceful shutdown with the scoreboard changes. Basically, we want to set the sb_type only after we zero out the scoreboard. Before this, we were segfaulting when we tried to free() a shared memory scoreboard.

Get the Unix MPMs restarting again. If we are going to register a cleanup for ap_cleanup_scoreboard, then we have to kill that cleanup with the same function. This also makes ap_cleanup_scoreboard a non-static function, and makes ap_cleanup_shared_mem a static function.

Setup the server start time correctly. This also gets the server uptime to be reported correctly. I have also moved this code into a common location, so that individual MPMs do not need to worry about setting up the start time.

Get OS/2 MPM running again after the reunification of the scoreboard code. mod_status still doesn't work right, I'll look into that next.

cleanups need to return apr_status_t

Ooops, that S and D are right next to each other.

Begin restructuring scoreboard code to enable adding back in the ability to use IPC other than shared memory. Get mod_status working on Windows again. Still to do: Rename some of the function APIs. Replace all calls to reinit_scoreboard with ap_create_scoreboard. Add back in support for scoreboard files.

Bring mod_status for 2.0 back in line with mod_status for 1.3. This is basically a straight port of the 1.3 module to 2.0. The MPMs need to be modified a bit to work with mod_status, but prefork, mpmt_pthread, and dexter have already been changed. I will fix perchild tonight. There is a lot of common code that can be abstracted, and there seems to be a small bug with regard to what mpmt_pthread and dexter report as current connections. ExtendedStatus does work again, although until the bug mentioned above is fixed, it isn't as useful on mpmt_pthread and dexter. Next week, I will look at allowing other modules to add data to the STATUS page and possibly to the scoreboard itself.