Revert r4635428 corresponding to PR41867. The code reverted attempted to restrict comparisons of the r->filename to given DirectoryMatch blocks. r->filename was already a non-directory entity at this point, because we have already fallen out of the } while (thisinfo.filetype == APR_DIR); block above. The addition of r->d_is_directory was redundant. That is what is always returned by ap_get_core_module_config(r->per_dir_config). Note modifying dir_config required an MMN major bump as this commit could have realigned the offset of refs (had it been added to the end, this would correspond to an mmn minor bump) and other fields packed into the same bytes (this is undefined). Bump on revert to prevent unexpected crashes.

core: Do not match files when using DirectoryMatch. PR41867.

core: Support named groups and backreferences within the LocationMatch, DirectoryMatch, FilesMatch and ProxyMatch directives.

Add LogLevelOverride directive that allows to override the loglevel for clients from certain IPs

add dirwalk_stat hook, for use by mpm-itk

ap_sub_req_lookup_dirent() depends on the over-allocation done by ap_make_full_path and ap_escape_uri, so let's document it so that it is not accidentally removed.

spelling fix

core: Add post_perdir_config hook. Submitted by: Steinar Gunderson <sgunderson bigfoot.com> trawick added/fixed include/ pieces

Adjust log message: Condition can be caused by configuration error not only by bugs

add filesystem paths to some common error messages. (slightly modified from dev@ post to not bury the "failed/denied")

Clean up size_t abuse, part 2. ap_malloc/calloc/realloc are explicitly excluded from this cleanup as they must be signature identical to the clib functions, and although the definition of size_t has been flakey, the definition of those functions appears to be generally clean since ANSI C.

Further clarify the naming of the entity that originates the request by calling that entity a useragent instead of a client.

Add lots of unique tags to error log messages

Introduce a per connection "peer_ip" and a per request "client_ip" to distinguish between the raw IP address of the connection and the effective IP address of the request.

Introduce a per request version of the remote IP address, which can be optionally modified by a module when the effective IP of the client is not the same as the real IP of the client (such as a load balancer).

More cleanup: Expand tabs and some more indentation fixes No functional change

Cleanup effort in prep for GA push: Trim trailing whitespace... no func change

Add AllowOverrideList directive and documentation

Introduce ap_(get|set)_core_module_config() functions/macros and use them everywhere. We know that the core module has module_index 0. Therefore we can save some pointer operations in ap_get_module_config(cv, &core_module) and ap_set_module_config(cv, &core_module, val). As these are called rather often, this may actually have some (small) measurable effect.

- Add <ElseIf> and <Else> to complement <If> sections. These are both easier to use and more efficient than using several <If> sections. - Update <If> documentation a bit.

core: AllowEncodedSlashes new option NoDecode to allow encoded slashes in request URL path info but not decode them. Change behavior of option "On" to decode the encoded slashes as 2.0 and 2.2 do. PR 35256, PR 46830.

Initialize the core_dir_config->sec_files and ->sec_if only if needed. This saves some memory and two apr_array_append()s per directory merge.

Merge the <If> sections in a separate step ap_if_walk, after ap_location_walk. This makes <If> apply to all requests, not only to file base requests and it allows to use <If> inside <Directory>, <Location>, and <Files> sections. The merging of <If> sections always happens after the merging of <Location> sections, even if the <If> section is embedded inside a <Directory> or <Files> section.

Save a useless check: ap_get_request_note() cannot fail unless there is a bug somewhere and if prep_walk_cache() returned NULL, we would just segfault later on.

ap_expr related fixes/enhancements: - implement regex backreferences and make them available for setting envvars in SetEnvIfExpr - implement nested function calls in %-syntax: %{func1:%{func2:arg}} - actually implement evaluation of concatenation operator (oops...) - Fix <If ... > treating an internal error as success

Replace ap_expr with a parser derived from mod_ssl's parser. Make mod_ssl use the new parser. Rework ap_expr's public interface and provide hooks for modules to add variables and functions. The Netware and Windows build files still need to be adjusted

PR#45722: if we get EACCESS in the directory walk, give the hint that's in 1.3 about search permissions on intermediate directories. I left the prefix of the message as "denied" instead of "failed" to satisfy web searches. The extended error text was added to 1.3.x but not trunk in http://svn.apache.org/viewvc?view=revision&revision=99781 This plagues users on IRC!

PR49809: Allow DirectoryMatch to match the EOL character ($). and stop applying to subdirectories (that don't also match the regex) implicitly. The manual already uses DirectoryMatch syntax that assumes this in other places!

Add the same r->user == NULL check in the Satisfy Any code path as r965709 did for Satisfy all

Intel's compiler warns about assigning the non-enum value 0 to the enumeration apr_finfo_t.filetype, which is done when forgetting the previously-derived file type use the appropriate enum value APR_NOFILE instead Also change comparisons of the field with 0 to use APR_NOFILE instead, as is the practice of some existing code.

remove more unused variables revove some obsolete comments netware file is untested

Don't risk segfault in authz if r->user is not set PR 42995

fix log messages

The approach for allowing authorization by user or IP introduced in r956387, etc. causes problems because the authentication module calls note_*_auth_failure if authentication fails. This is inappropriate if access is later allowed because of the IP. So, instead of calling the auth_checker hook even if authentication failed, we introduce a new access_checker_ex hook that runs between the access_checker and the check_user_id hooks. If an access_checker_ex functions returns OK, the request will be allowed without authentication. To make use of this, change mod_authz_core to walk the require blocks in the access_checker_ex phase and deny/allow the request if the authz result does not depend on an authenticated user. To distinguish a real AUTHZ_DENIED from an authz provider from an authz provider needing an authenticated user, the latter must return the new AUTHZ_DENIED_NO_USER code.

Don't restore r->user after authn/authz failure. Any authn module that sets r->user even if authentication failed is broken, anyway.

Fix authorization by user or IP/ENV/... Note ap_note_auth_failure() breakage in STATUS

More trace logging

- Add loglevels to request_rec and conn_rec - Introduce per-directory loglevel configuration

Use the new APLOG_USE_MODULE/AP_DECLARE_MODULE macros everywhere to take advantage of per-module loglevels

Fix string constness to get rid of gcc compiler warnings by -Wwrite-strings.

Bring back OS/2 support. Reverses r758929 with a little bit of conflict resolution.

remove OS/2 platform support

API Cleanup in preperation for 2.4.x, make sure all exported functions or variables contain an ap_ prefix.

Determine registered authn/z providers directly in ap_setup_auth_internal(), which allows optional functions that just wrapped ap_list_provider_names() to be removed from authn/z modules. This change requires modules/aaa/mod_auth.h to be included into server/request.c, which necessitates a minor change to configure.in for Unix platforms. I'm unable to tell whether a similar change is necessary for Windows and NetWare builds or not. Could developers with access to those platforms please test and make any needed configuration or build alterations? Thanks!

When testing for slash-terminated configuration paths in ap_location_walk(), don't look past the start of an empty string such as that created by a <Location ""> directive.

mod_request: Insert the KEPT_BODY filter via the insert_filter hook instead of during fixups. Add a safety check to ensure the filters cannot be inserted more than once. [Graham Leggett, Ruediger Pluem]

Move the KeptBodySize directive, kept_body filters and the ap_parse_request_body function out of the http module and into a new module called mod_request, reducing the size of the core.

Remove all references to CORE_PRIVATE.

Make it compile on pre-C99 compilers (Move decls to beginning of func)

Avoid calling access control hooks for internal requests with configurations which match those of the initial request. Revert to the original behaviour (call access control hooks for internal requests with URIs different from the initial request) if any access control hooks or providers are not registered as permitting this optimization. Introduce wrappers for access control hook and provider registration which can accept additional mode and flag data. The configuration walk optimizations were originally proposed a while ago (see http://marc.info/?l=apache-httpd-dev&m=116536713506234&w=2); they have been used since then in production systems and appear to be stable and effective. They permit certain combinations of modules and clients to function efficiently, especially when a deeply recursive series of internal requests, such as those generated by certain WebDAV requests, are all subject to the identical authentication and authorization directives. The major change from the original proposal is a cleaner mechanism for detecting modules which may expect the old behaviour. This has been tested successfully with Subversion's mod_authz_svn, which specifically requires the old behaviour when performing path-based authorization based against its own private access control configuration files.

HTTPD Core: Implement <If> sections for conditional (runtime) configuration. N.B. This is a first pass, and has a way to go!

* lfi is already a pointer in contrast fi which was used before.

core: fix SymlinksIfOwnerMatch checking: (1) Fix Origin checking (PR 36783 - Robert L Mathews) (2) Check ownership if both FollowSymlinks and SymlinksIfOwnerMatch are set

core: Add the option to keep aside a request body up to a certain size that would otherwise be discarded, to be consumed by filters such as mod_include. When enabled for a directory, POST requests to shtml files can be passed through to embedded scripts as POST requests, rather being downgraded to GET requests.

Reinstate location walk for subrequests PR 41960 (Jose Kahan)

Multiple trivial fixes from Christophe JAILLET PR 38699, 39518, 42005, 42006, 42007, 42008, 42009 The patches are all his, and are sufficiently trivial to review at a glance.

re-introduce ap_satisfies API back into core and modify how the access_checker, check_user_id and auth_checker hooks are called so that they respect the precedence that is set through the satisfy ALL/ANY directive. This also restores the directives order, allow, deny, satisfyas supported directives rather than being deprecated. These directives still remain in mod_access_compat however.

Detab

PR#41829 - Torsten Förtsch Fix uninitialised var in server/request.c

* If APR_FINFO_NAME is added to the list of wanted information, apr_stat always returns APR_INCOMPLETE on Unix platforms in the case that the call to the native stat / lstat did not fail. So we can safely assume that the stat / lstat call has been successful in this case.

Improve the error message, include the full path of the file we are actually trying to call stat on.

* Although the lstat call should rarely ever fail, because we stat'ed the same file several lines above, check the return value to be save that nothing bad happens in the case it does fail.

* Fix typo. No functional change.

* Check for symbolic links of the target file in the optimized case that we had already done this specific directory walk for this request. This can happen when we have an internal redirect, like the ones caused by mod_dir (/ -> index.html). See also http://mail-archives.apache.org/mod_mbox/httpd-dev/200607.mbox/%3c44B5521F.8050906@globalvanet.com%3e Reviewed by: wrowe

update license header text

Update the copyright year in all .c, .h and .xml files

Make some error checking more clear by using the defined constants instead of literal values. Patch by: Dan Rall <dlr apache.org> * server/request.c (ap_process_request_internal): Check the return value of hook functions against the constant OK instead of the magic number 0 to improve clarity.

Authz refactoring Merge from branches/authz-dev Basically here is a list of what has been done: - Convert all of the authz modules from hook based to provider based - Remove the ap_requires field from the core_dir_config structure - Remove the function ap_requires() since its functionality is no longer supported or necessary in the refactoring - Remove the calls to ap_some_auth_required() in the core request handling to allow the hooks to be called in all cases. - Add the new module mod_authz_core which will act as the authorization provider vector and contain common authz directives such as 'Require', 'Reject' and '<RequireAlias>' - Add the new module mod_authn_core which will contain common authentication directives such as 'AuthType', 'AuthName' and '<AuthnProviderAlias>' - Move the check for METHOD_MASK out of the authz providers and into the authz_core provider vector - Define the status codes that can be returned by the authz providers as AUTHZ_DENIED, AUTHZ_GRANTED and AUTHZ_GENERAL_ERROR - Remove the 'Satisfy' directive - Implement the '<RequireAll>', '<RequireOne>' block directives to handle the 'and' and 'or' logic for authorization. - Remove the 'AuthzXXXAuthoritative' directives from all of the authz providers - Implement the 'Reject' directive that will deny authorization if the argument is true - Fold the 'Reject' directive into the '<RequireAll>', '<RequireOne>' logic - Reimplement the host based authorization functionality provided by 'allow', 'deny' and 'order' as authz providers - Remove the 'allow', 'deny' and 'order' directives - Merge mod_authn_alias into mod_authn_core - Add '<RequireAlias>' functionality which is similar to '<AuthnProviderAlias>' but specific to authorization aliasing - Remove all of the references to the 'authzxxxAuthoritative' directives from the documentation - Remove the 'Satisfy' directive from the documentation - Remove 'Allow', 'Deny', 'Order' directives from the documentation - Document '<RequireAll>', '<RequireOne>', 'Reject' directives - Reimplement the APIs ap_auth_type(), ap_auth_name() as optional functions and move the actual implementation into mod_authn_core - Reimplement the API ap_some_auth_required() as an optional function and move the actual implementation into mod_authz_core Major Changes: - Added the directives <RequireAll>, <RequireOne>, <RequireAlias>, Reject - Expanded the functionality of the directive 'Require' to handle all authorization and access control - Added the new authz providers 'env', 'ip', 'host', 'all' to handle host-based access control - Removed the directives 'Allow', 'Deny', 'Order', 'Satisfy', 'AuthzXXXAuthoritative' - Removed the ap_require() API - Moved the directives 'AuthType', 'AuthName' out of mod_core and into mod_authn_core - Moved the directive 'Require' out of mod_core and into mod_authz_core - Merged mod_authn_alias into mod_authn_core - Renamed mod_authz_dbm authz providers from 'group' and 'file-group' to 'dbm-group' and 'dbm-file-group' Benefits: - All authorization and access control is now handle through two directives, 'Require' and 'Reject' - Authorization has been expanded to allow for complex 'AND/OR' control logic through the directives '<RequireAll>' and '<RequireOne>' - Configuration is now much simpler and consistent across the board - Other modules like mod_ssl and mod_proxy should be able to plug into and take advantage of the same provider based authorization mechanism by implementing their own providers Issues: - Backwards compatibility between 2.2 and 2.3 configurations will be broken in the area of authorization and access control due to the fact that the directives 'allow', 'deny', 'order' and 'satisfy' have been removed. When moving from 2.2 to 2.3 these directives will have to be changed to 'Require all granted', 'Require all denied' or some variation of the authz host-based providers. - Existing third party authorization modules will have to adapt to the new structure.

No functional Change: Removing trailing whitespace. This also means that "blank" lines consisting of just spaces or tabs are now really blank lines

No functional change: simple detabbing of indented code.

Pay close attention to core_create_req() ... and note that not one other member of the r->vars is initialized herein. Move this initialization elsewhere. (If this is the 'default' - it really aught to be the zero value, for that matter).

Initialise a variable, to avoid a gcc warning.

When doing a subrequest lookup, do not lookup using the quickhandler if next_filter is NULL, since this implies that the subrequest will be internal_fast_redirect'ed or will never be called. See line request.c:1513 onwards (in make_sub_request) also.

* server/request.c (core_opts_merge): When AllowOverride is specified for the directory, ignore the inherited override_opts field. PR: 35330 Submitted by: kabe <kabe sra-tohoku.co.jp>

Any failure in apr_stat on a symlink currently gives "Symbolic link not allowed", which results in much head-scratching if the actual problem is a broken link of some sort. The real fix would be to propogate the correct apr_stat error into the error log, but that would require more refactoring than I'm prepared to do. This change simply expands the error message to include both possibilities. It improves the situation for PR28515 but does not solve it.

Doxygen fixup / cleanup submited by: Neale Ranns neale ranns.org reviewed by: Ian Holsman

Move the POSIX reg* implementations into the ap_* namespace; internalise the ap_reg*<->PCRE wrapper: * configure.in: Add srclib/pcre to the include path. * include/ap_regex.h: Renamed from include/pcreposix.h. Prefix all constants with AP_; prefix all functions and types with ap_. Define AP_DECLARE to nothing if necessary. Remove regcomp error codes. * include/httpd.h: Include ap_regex.h not pcreposix.h. (ap_pregcomp, ap_regexec, ap_regfree): s/regex_t/ap_regex_t/. (ap_regexec, ap_regerror): Prototypes moved to ap_regex.h. * server/util.c (regex_cleanup, ap_pregcomp, ap_pregsub, ap_pregfree): Adjust for ap_ prefixed types. (ap_regexec, ap_regerror): Removed. * server/Makefile.in: Build util_pcre.c. * server/util_pcre.c: Copied from srclib/pcre/pcreposix.c; remove use of PCRE-internals to do error mapping; rename types to add AP_/ap_ prefixes as above. Use APR includes. (ap_regerror): Use apr_snprintf. * srclib/pcre/Makefile.in: Don't build pcreposix.c into libpcre.la. * modules/*: Update to use new type and constant names. PR: 27750 (part one) Submitted by: Andres Salomon <dilinger voxel.net>, Joe Orton

Update copyright year to 2005 and standardize on current copyright owner line.

veto and revert win64 patch: 64bit changes must percolate from the bottom (APR/system) up -- we can't give the client a 64bit API and then cast it to 32bits internally without introducing security holes on other platforms.

WIN64: API changes to clean up Windows 64bit compile warnings

Minor comment fixes, no code changes: - 'sub request' -> 'subrequest' - @retrn -> @return - ... PR: Obtained from: Submitted by: Reviewed by:

use more intuitive variable names ap_sub_req_*_uri to use new_uri ap_sub_req_*_file to use new_file PR: Obtained from: Submitted by: Reviewed by:

Added 'AllowOverride Options=Indexes,MultiViews' to give an admin better control over what options can be used in .htaccess files. PR: 29310 Submitted by: Tom Alsberg <alsbergt-apache cs.huji.ac.il>

fix name of The Apache Software Foundation

fix copyright dates according to the first check in

apply Apache License, Version 2.0

update license to 2004.

spell check

Let mod_autoindex show filenames containing special chars like %. PR: 13598

tag the pools created for requests and subrequests

switch to APR 1.0 API (which is still in flux) because of the changes to the argument lists of apr_mmap_dup and apr_socket_create, 2.1-dev won't build with apr and apr-util's 0.9 branch anymore

Rework of the recursion stopper - collapse recursion counters into one function Reviewed by: Justin Erenkrantz

Prevent the server from crashing when entering infinite loops. The new LimitInternalRecursion directive configures limits of subsequent internal redirects and nested subrequests, after which the request will be aborted. [William Rowe, Jeff Trawick, Andr� Malo] PR: 19753 (and probably others)

finished that boring job: update license to 2003. Happy New Year! ;-))

Fix "Satisfy Any" logic. The access_checker has nothing to do with auth_type. Related PR: 9076

here we go. add a directive that will keep %2f from being decoded into '/', allowing the *_walk to do their magic and return 404 if it's in the path, and allowing it in the path-info.

Make the code simpler to follow, and perhaps clear up the follow-symlink bug reports we have seen on bugzilla. e.g. 14206 etc.

core_opts_merge() needs to be static

Mr. Trawick was dead on, and this revealed a much bigger bug. Factor out the opts/override merging (since we do it three times) and eliminate all the nasty goto's. This bug likely caused all sorts of dir_walk configuration issues including htaccess issues. Also add a few more docs where things aren't so obvious. PR: 14147

no such thing as AP_ASSERT() there is ap_assert() and AP_DEBUG_ASSERT()

Fix a trailing slash/last filename truncation bug observed on Linux, which affected DAV MOVE operations and even general file access. PR 14147, 10687, 10236 [Dan Good <debug@gooddan.com>] I'm accepting Jeff Trawick's suggestion of twisting the test into an assert, since it seems very unlikely (after correctly resetting the flag) that this will fault. Reviewed by: Jeff Trawick, Will Rowe

i hate it when tha happens..

more info is better than less..

Morph DONE result from a sub-request handler to OK as DONE is only relevant when we are the main request. This fixes a problem with mod_include printing out an error message on DAV sub-requests because mod_dav will return DONE instead of OK. This would result in the correct output merged in with an error string.

stop using APLOG_NOERRNO in calls to ap_log_?error()

Optimization: skip cache setup in location_walk() if the vhost config contains no <Location> blocks

A bug affecting any platform that had pre-lstat'ed the file or directory in question. We neglected to clear the FINFO_LINK bit that told us this was lstat()ed, but also causes stat() to follow an lstat() approach. This is part 2 of 2 to correct the NTFS Junction (symlink) bug. Bugz report 8014, identified by Sam Morris <sam@netcity.co.uk>

Formatting irk grinding at me while stepping code

Fix this optimization. We will skip along so long as we aren't at the right number of components, -or- we mismatch on the current component.

Small optimization, if we are behind (at the 5th segment, for example), catch up the segment-count-sorted directory list without string compares. Mostly affects win32 which jumps from seg 0 (root) to 4 for UNC path names.

Add a new parameter to the quick_handler hook to instruct quick handlers to optionally do a lookup rather than actually serve content. This is the first of several changes required fix several problems with how quick handlers work with subrequests.

Allow URIs specifying CGI scripts to include '/' at the end (e.g., /cgi-bin/printenv/) on AIX and Solaris (and other OSs which ignore '/' at the end of the names of non-directories). PR: 10138

Update our copyright for this year.

Move the quick_handler comment to the new quick handler location. Do not call quick handler on a dirent subrequest. This fixes a nasty problem in mod_cache where it was serving up content on a dirent subrequest.

Style Police comming through...

Remove the prev pointer from the filter chain. This removes the complexity of trying to set the filter chain correctly, with the side-effect of forcing us to walk the entire chain whenever we add a filter. Since the filter chains are small, the decrease in complexity is worth it. Reviewed by: Allan Edwards

Only insert net_filter once per request. Initialize the output protocol filters.

This fixes most of the header bug that was committed last night. The server is seg faulting on pipelined requests currently, but I want to get people back to a running server.

Fix the mod_dir/mod_negotiation bug, where redirects and sub requests were not getting the correct filters. This is done by creating a location in the request rec that holds protocol level filters. Protocol level filters survive for one request, from the time the request is received from the user to the time the response is sent. r->output_filters now stores the request level filters, which are only valid for the lifetime of one request_rec. This patch works, but it is not complete. The second half of the problem is that add_any_filter doesn't check where it puts the filters that it adds, so it is possible for filters to be put on this wrong list, and for filters to be lost completely during request processing. That half of the fix will be coming in the next day or so. Submitted by: Will Rowe, Justin Erenkrantz, Ryan Bloom

Fix ap_directory_walk() per-dir merge bug seen when no <Directory /> is present. showstoppers-- Kudos to Jeff for finding it. Kudos to BrianP for leading us in the right direction. Kudos to OtherBill for pointing out the right way to fix this.

Fix resolve_symlink to save the original symlink name if known. We would previously receive APR_INCOMPLETE on symlinks if wanted has FINFO_NAME set because it isn't supported via apr_stat(). Furthermore, we don't care what the real name is anyway (even if it apr_stat returned .name) - we want to call it by the name the symlink says it is.

Which PR? I can't count them all. Get QUERY_STRING and PATH_INFO working again. Also rounds out our fix to work around negotiated directories which Greg Ames fixed; this addition in request.c simply shortcuts all further processing.

optimization: switched to ap_add_output_filter_handle() for installation of subreq filter

ap_sub_req_lookup_dirent: fix mod_negotiation loop with near infinite subrequests this function has been creating bogus subrequest URIs when there is path_info for a long time. They didn't matter until fixup_dir started using them for URI subrequests, which led to a loop with ever growing bogus internal URIs and filenames.

These changes are to allow caching of subrequests via a quick_handler. * Change SUBREQ_CORE so that it is a HTTP_HEADER (20) filter instead of a content filter (10) this allows subrequests to add content filters properly * Change subreq handling of 'handle-include' so that it splits/passes the brigade before the subreq is created. (This allows quick_handler to push content back from this phase)

code police again

sigh. code style police

quick handler now runs on subrequests as well PR: Obtained from: Submitted by: Reviewed by:

Merge a singular path for dir_walk to parse, if we have an r->filename which is an APR_DIR, and path_info contents. Also, al la Mr. Pane, optimize our canonical_filename by simply noting the length of the identity match, and refresh canonical_filename when we are finished.

Eliminate a duplicate absolute path test, and NEVER serve a request for an APR_DIR file from the cache when we have path_info, it is a contradition (APR_DIR always forces dir_walk to gather the next segment from path_info, even if it is APR_NOFILE, until we have no path_info.) So we can't use a predetermined filename/path_info combo, ever, if the filename resolves to an APR_DIR.

Generalized the recent prep_walk_cache optimizations to allow other modules to register "notes" within the array of working data in the core_request_config

Performance fix for prep_walk_cache(): Moved the directory/location/file-walk caches from the request's pool userdata hash table to the core_request_config struct. This change removes about 60% of the processing time from prep_walk_cache().

Move the insert_filter hook from the prepare request phase to the invoke handler phase, since it can't fail, and contributes nothing to the request 'character', but everything to it's invocation.

. unwind MORE redundant code [wasn't I just here six months ago???] . fix a redundant return [how no compilers caught a code-not-reachable is beyond me :]

Style reformat. Tabs->spaces, etc, etc, etc.

Reduce magic levels.

Stat, don't lstat the final target. This means we may double-stat the final target file if check for symlinks reveals it's an APR_LNK, but this is preferable to the convoluted code required to 'reuse' the original stat.

Reintroduce the 'one stat dir_walk', with singificantly more sensible behavior than the old path_info logic. If the stat() of r->filename succeeds ignore every segment that requires no symlink check (and on Win32/OS2/Netware the name matches the canonical name, assuring us that the case/aliases match the true name.) This optimization can be improved by establishing a second cache of the actual partial filenames that _are_ tested for symlinks (or canonical filename case), and then skiping such tests when the conditions match on successive passes for subrequests or redirects.

This patch optimizes away some strlen and strcat calls in ap_directory_walk. The strlen calls, in particular, had ranked as a top bottleneck in the usr-space code in recent performance profiling. Submitted by: Brian Pane <bpane@pacbell.net> Reviewed by: Justin Erenkrantz

More style-stuff. A lot of this doesn't qualify as 'readable' even so.. :-(

Some style-guide fixes (nothing functional)

Use the _setn flavor which will avoid strdup'ing these strings, and skip the cleanups we don't need. Submitted by: Brian Pane <bpane@pacbell.net>

Oh, don't you love buffer overflows? We need to allocate storage space for the terminating NULL AND the extra / we may tack on to the string at some point. How in the hell the stars were aligned for this to corrupt newv via the strcat at line 580 is unknown. Resolves segfault seen on daedalus.

recognize filename subrequests without requiring a null URI. This fixes a problem with infinite recursion of dirent subrequests. Submitted by: Bill Rowe

Fix the little bits o' breakage I introduced with the last two 'theoretical' corrections.

directory_walk trounced existing path_info declarations. I suspect _this_ was Greg Ames bug...

Streamline this function, and append the trailing slash for any directories that are resolved. Needs to be more tightly coupled to the dir_walk optimized cache.

This will probably fix recent breakage to mod_negotation and httpd.test, as well as the /manual/ returning docroot/index. Need to look for another solution. I'm suspecting path_info is possibly broken. Reverts 1.68, as suggested by Brian Havard.

Always helps to add the member

Speed it up. While this optimization wasn't obvious for the two-pass location_walk, it is significant for all subreq/redirects reusing the cached walk values.

prevent near infinite subrequest recursion with mod_negotiation enabled. This can happen if there is a partial match between a bad URI and a file with a variant extention. ap_sub_req_lookup_dirent has apparently been generating bogus subrequest URIs for ages, but they used to be ignored. Once we started calling ap_process_request_internal for all subrequests, they started causing problems. Make it explicit that rnew->uri is to be ignored for this type of subrequest.

get symlinks working again

at least *this* cast keeps gcc happy (and should keep everything else happy as well)

Revamped ap_directory_walk logic, without a path_info helper is now activated. It may be bumpy for a few days, and we have more optimizations to put in place, but it's time to get this in the developer's test code.

This is nothing but const bogosity. We have our very own manipulation, we are allowed to touch the char *'s, even if we have a const contract.

Hmmm... continue's not so good an idea here :)

Aaargh! Unwinding part of my patch before I committed the prior version, I ended up blasting these (intentional) changes as well :( This finishes up the changes for the new, replacement ap_directory_walk for testing. This code isn't active yet.

A major overhaul to the -replacement- ap_directory_walk logic. This still doesn't activate that code, I will do so probably by Monday, after more thorough testing. Introduces the ap_directory_walk::cache so we can stop wasting tons of effort in mod_autoindex and other subreq/redirect requests. This isn't thoroughly tested, I've only stepped through a half dozen common cases. If you want to play, define REPLACE_PATH_INFO_METHOD.

Correct file lookups when we are given a file within the same directory as the parent request. Also pulls a bunch of notes and code that was set aside, we don't need this with the other optimizations introduced.

ap_sub_req_output_filter: don't pass along a brigade if it becomes empty after deleting the EOS bucket. This prevents a seg fault in mod_include when the connection dies. There doesn't seem to be much point in passing empty brigades in general.

Fix for httpd-test modules/include test #17. If we are *already* a faux URI (i.e. relative file sub req) and we then make a subrequest from that faux URI to a file in the same directory, we'd try to build a URI out of the fake URI which leads to the wrong thing happening somewhere down the line. So, let's just give this special case a fake URI as well. OtherBill needs to verify this. He can back it out if he wants. I don't much care. It's one line and it seems okay...

Optimize file_walk with the same logic as location_walk. Fix both to have a bit more legibility, and tighter locality of scope for a smaller number of variables.

Split out (soon to be) common code for all _walk functions (no net change to location_walk).

Overhaul the compatibility with 1.3's subrequest and redirect processing. Eliminate URI-centric phases in ap_process_request_internal() for pure file subrequests (that don't correspond to URI space.) translate_name hook and location_walks are skipped for these requests. Moves the reset of the per_dir_config out of directory_walk into the internal request processing code, so that resources with alternate map_to_storage requirements start with clean r->server->lookup_defaults. Optimizes out the authn/authz of effectively identical subreqests and redirects, as the sub_req_lookup calls once did. Unlike 1.3, we copy r->user and r->ap_auth_type from main/prev for the request's reference. Stop copying the subrequest's r->chunked flag (Rbb assured me it looked bogus, chunking is on the parent request) and clean out other #if 0'ed cruft we don't need to refer back to anymore.

fix a gcc warning -- "/* within a comment"

Joy, joy. Relax the rules, just a wee bit, and prepare to move the fatal ending to this filename-less request a bit later in the request cycle, to give older, ported modules more time to cope without implementing the map_to_storage hook.

Currently, when the map-to-storage handler for TRACE returns DONE, the caller -- ap_process_request_internal() -- catches that and returns OK to its caller -- ap_process_request(). But ap_process_request(), seeing OK, tries to run a handler. It needs to skip that if the request was completed in ap_process_request_internal(). Reviewed by: William A. Rowe, Jr.

Fix a seg fault in mod_include. When we are generating an internal redirect, we must set r->uri to "", not a bogus string, and not NULL. [Ryan Bloom]

Optimize location_walk. We build an array of incremental matches, and on attempting a subreq/redirect or simply a second pass, and for each match in series, if the section is a match, we reuse the merge result for that section.

After some consideration - Location walk is always required (before and after any other map_to_storage operations.) Therefore, initialize any NULL r->per_dir_config at this phase (the earliest necessary point.)

I love it when a plan comes together. We hadn't prepared for this possibility that someone didn't set up the r->per_dir_config (which the subreq's didn't). Since we are first in line, we will handle it if need be.

Ahhh, a const headache. Here's a flag that is writeable.

Since we can preserve and further canonicalize the subreq_file name onto a canonical r->filename, let's do so.

Add some notes of things I noticed while proofing. We still need a resolution to rnew->chunked = r->chunked in subrequests!

resolve_symlink() is only used if REPLACE_PATH_INFO_METHOD is defined

As I understand it, this test is supposed to read like so: ---------------------------------------------------- if the base paths are the same if (strncmp(rnew->filename, fdir, fdirlen) == 0 and there's more stuff in the new filename than just the base path && rnew->filename[fdirlen] and that stuff contains no slashes && ap_strchr_c(rnew->filename + fdirlen, '/') == NULL) ---------------------------------------------------- Assuming that's a correct translation, which I believe to be the case (and which also seems to jive with the previous version of the test), then that first part darned well better check == 0, as opposed to != 0. strncmp returns 0 when they match. =-) And voila, "All tests successful, 1 test skipped." is the result from httpd-test

Invoking the handler must occur at the caller's discression, in order for the sub_req mechanism to share this code.

Move the ap_run_insert_filters to consistently occur in ap_process_request_internal. This allows the sub_req handler to alter the filters before the subreq is actually run.

Normalize all paths to run the same, common code for pre-request setup from the primary request, redirects and sub-requests. This will significantly reduce opporunities for inconsistancy (such as Ian observed, and as I repaired only a month ago.) This promotes process_request_internal to an ap_ namespace protected entity in server/request.c (from it's old home in http/http_request.c) since this fn has no http specifics. Reviewed (in concept): Cliff Woolley, Ian Holsman

The add-in/strip-off temporary trailing slash logic was really hosed in the new (not yet enabled) code path. Now it's slightly hosed.

Fix the new code (not currently enabled) for directory_walk

fix some calls to apr_pool_userdata_get() (foo** just doesn't work the way we sometimes might want :( )

Final additional comments (for the moment) about the new optimization.

Further optimization to location_walk() and clean up unused variables. We will note the end result of the merge, and if remains the same through the second pass, we have nothing whatsoever to do :)

Clean up location_walk, so that this step performs a minimum amount of redundant effort (it must be run twice, but it will no longer reparse all <Location > blocks when the request uri hadn't changed.) The location walk block is refactored, with some significant changes in variable names for legibility. Cooler still, it uses pool data instead of 'notes' for the important cache info :) Note the patch builds the <Location > per dir config from _nothing_, and then merges it into the per_dir_config. When the underlying per_dir_config changes between passes, the location_walk can simply tack back on this preconstruct onto the new per_dir_config.

Eliminate proxy: (and all other 'special') processing from the ap_directory_walk() phase. Modules that want to use special walk logic should refer to the mod_proxy map_to_location example, with it's proxy_walk and proxysection implementation. This makes either directory_walk flavor much more legible, since that phase only runs against real <Directory > blocks. On a technical note, this patch also forces the Directory to be canonical (unless it is "/" or a regex.) It also allows us to be more explicit when declaring <Directory > block errors.

Introduce the map_to_storage hook, which allows modules to bypass the directory_walk and file_walk for non-file requests. TRACE shortcut moved to http_protocol.c as APR_HOOK_MIDDLE, and the directory_walk/file_walk happen as APR_HOOK_VERY_LAST in core.c. A seperate patch to mod_proxy is required to short circuit both the TRACE and directory_walk/file_walk stuff. That patch is next.

sec, sec, who's got a sec? This gave me a headache, but I had to clear out the last patch before I rearranged this to be _readable_. Next step for everyone's sanity, provide <Proxy > directives ;)

Commit this code before another patch becomes to difficult to follow. This patch does one thing, it changes the root path "/" to reflect an element count of Zero (0). directory_walk will always accept the zero element (which sorts first, thankfully) on it's first go around. So, Unix will accept "/" when it's parsing it's first element "/". Dos/Win32 will accept "/" and "C:/" when they parse their first element, "C:/". The root sorted first, so it behaves as users expect. The syntax "//" or "//machine" will be depreciated for now, the user needs to set up the extact "//machine/share/" that they want served on Win32.

Whoops. To explain, we won't dup filename unless it really didn't match in the first place. We are about to abuse test_filename, so don't try using that copy.

Start with the presumption that canonical_filename is not likely to be set. Therefore we will canonicalize it when it doesn't match filename. The next optimization should take the path common to canonical_filename and filename, and start merging filename from there for canonicalization.

Another spot we are certain of the canonical_filename

Rather than continuing to canonicalize within directory_walk (very time consuming on all but *nix systems) we temporarily canonicalize to compare the results of the many merges, and fail on a mismatch. The apr_filepath_merge and ap_server_root_relative calls now merge the file _by canonicalizing it_. That includes resolving all /../, /./, and // misnomers. A minor effort is required to figure out who all munges the r->filename in an inappropriate manner. The final (return to optimized state) probably involves setting an r->goodname argument to r->filename, every time we properly merge through ap_server_root_relative or apr_filepath_merge().

fix some homophonic issues in comments, as well as some mispelings found near "its" or "it's" (helping our 4th grader with homework, couldn't help but grep)

Fix the new method code. We need to cast 1 to an apr_int64_t or it will be treated as a 32-bit integer, and it will wrap after being shifted 32 times. Submitted by: Cody Sherr <csherr@covalent.net> and Ryan Morgan <rmorgan@covalent.net>

Fix a segfault (was getting triggered by mod_include at least) caused by calling ap_allow_options() before setting rnew->per_dir_config. This is the "easy looking" fix but might have side effects of which I'm unaware... please double-check this change for correctness.

This was entirely broken. We cannot skip the location walk just because we are in a file subrequest (think of a file server-status sitting in the document root, this shouldn't be blindly served as a 'file'.)

Just a little cleaner.

More explanation

fix some warnings in resolve_symlink(), one of which seems to be for a genuine bug... The old logic if (!(opts & OPT_SYM_OWNER | OPT_SYM_LINKS)) wouldn't seem to work properly. I think it would act like if (!((opts & OPT_SYM_OWNER) | OPT_SYM_LINKS)) This clearly isn't intended since OPT_SYM_LINKS is a constant non-zero, such that we never really fail invalid parameters.

Replace check_symlinks in the ap_sub_req_lookup_* calls with the new resolve_symlink (also used by the new directory_walk) especially for performance and readability. Left check_symlinks in the soon-to-be-gone get_path_info flavor of directory_walk.

Add the new directory_walk logic, eliminating get_path_info and check_symlinks, in a protected define REPLACE_PATH_INFO_METHOD. This allows others to work on vetting, caching, etc, while keeping the existing logic stable till it's sufficiently optimal for beta.

Use d_is_absolute within directory_walk.

Testing reveals a nasty side effect of this aftn's patch, this is the fix.

Pull harry, difficult to read sub_req_common_validation() into its own function so that it's apparent that this is common code.

Here, finally are a few cleanups of my fat fingers.

Now, introduce ap_sub_req_lookup_dirent() for processing apr_finfo_t results from mod_negotiation and mod_autoindex.

Pull the common cruft from ap_sub_req_lookup_*() fns.

Sorry, second pass, working on legibility. This patch introduces the fill_in_sub_req_vars function to propogate the rnew values, but doesn't yet hook it in. Note that there are two discrepancies, apparently pretty bad ones, that have been moved after the 'common code' so the next patch becomes pretty simple.

Revert changes in 1.7, I had applied the patch incorrectly.

Move duplicated rnew cloning from apr_ap_sub_req_lookup_*() functions, and add an ap_sub_req_lookup_dirent() to create a subrequest from the results of an apr_dir_read() for mod_negotiation and mod_autoindex.

back out bogus "fix" for subrequest buckets using wrong pool Submitted by: Greg Stein

implement Ryan's suggested fix for buckets associated with a subrequest having private data in the wrong (i.e., subrequest) pool, leading to a segfault later in processing the main request in the patch posted on new-httpd, the temporary brigade was allocated from the connection pool; the committed code allocates the brigade from the main-request pool, as suggested by Ian Holsman

nicer to compare r->finfo.filetype with APR_NOFILE instead of 0 the sub request output filter shouldn't lose the return code from the next filter

change create_request hook to RUN_ALL/return int so handlers can throw errors

Add a hook, create_request. This hook allows modules to modify a request while it is being created. This hook is called for all request_rec's, main request, sub request, and internal redirect. When this hook is called, the the r->main, r->prev, r->next pointers have been set, so modules can determine what kind of request this is. Currently, this is only used by the core module, but protocol modules are going to need to have the ability to affect the request while it is being read.

Another chunk of code from http to core. This should continue to build on all platforms. The next job is to shuffle functions back and forth so that the server builds without mod_http.