core, modules: like r1657897 but for core and other modules than mod_proxy. More uses of ap_map_http_request_error() and AP_FILTER_ERROR so that we never return an HTTP error status from a handler if some filter generated a response already. That is, from a handler, either ap_get_brigade() (an input filter) returned AP_FILTER_ERROR and we must forward it to ap_die(), or ap_pass_brigade() (an output filter) failed with any status and we must return AP_FILTER_ERROR in any case for ap_die() to determine whether a response is needed or not.

Remove the bitfield spec from the http_ctx_t.state enum as it causes an error-prone mismatch between the enum values and the state values. Caused a busy loop in windows. Added in r1484852 and trunk-only. Submitted By: Edward Lu, Yann Ylavic Committed By: covener

Concat string at compile time when possible. Doing so, sometimes also give the opportunity to turn a 'ap_fputstrs' into a 'ap_fputs'. PR 53741

Add missing APLOGNO.

*) SECURITY: CVE-2013-5704 (cve.mitre.org) core: HTTP trailers could be used to replace HTTP headers late during request processing, potentially undoing or otherwise confusing modules that examined or modified request headers earlier. Adds "MergeTrailers" directive to restore legacy behavior. Submitted By: Edward Lu, Yann Ylavic, Joe Orton, Eric Covener Committed By: covener

Simplify code. Cases where 'loc' doesn't have any ':' or is starting with ':' are already handled by 'ap_ir_url()' Calling 'apr_isascii()' seems useless.

Use unsigned bit flags (otherwise the non-zero value to be used is -1).

Add directives to control two protocol options: HttpContentLengthHeadZero - allow Content-Length of 0 to be returned on HEAD HttpExpectStrict - allow admin to control whether we must see "100-continue" This is helpful when using Ceph's radosgw and httpd. Inspired by: Yehuda Sadeh <yehuda@inktank.com> See https://github.com/ceph/apache2/commits/precise * include/http_core.h (core_server_config): Add http_cl_head_zero and http_expect_strict fields. * modules/http/http_filters.c (ap_http_header_filter): Only clear out the C-L if http_cl_head_zero is not explictly set. * server/core.c (merge_core_server_configs): Add new fields. (set_cl_head_zero, set_expect_strict): New config helpers. (HttpContentLengthHeadZero, HttpExpectStrict): Declare new directives. * server/protocol.c (ap_read_request): Allow http_expect_strict to control if we return 417. * include/ap_mmn.h (MODULE_MAGIC_NUMBER_MAJOR, MODULE_MAGIC_NUMBER_MINOR): Bump. * CHANGES: Add a brief description.

Use a distinguishing APLOGNO for unk t-e with read-until-close behavior

80 rule

Wrap at 80 still, here at httpd project

PR 55475: Detect incomplete body in HTTP input filter and return APR_INCOMPLETE

Correct typo in comments for ap_map_http_request_error().

core: Add missing Reason-Phrase in HTTP response headers. PR 54946.

draft-ietf-httpbis-p1-messaging-23 fixes regarding interactions between TE and content-length in the same req/resp.

revert 1524161 for the time being

RFC2616 issue

Update the log message number.

Ensure that any meta buckets are stripped while parsing a chunk.

core: Use a matching counter type.

core: Make sure we allow unconstrained bodies from a proxy.

core: Remove apr_brigade_flatten(), buffering and duplicated code from the HTTP_IN filter, parse chunks in a single pass with zero copy. Reduce memory usage by 48 bytes per request.

core: Stop ap_finalize_request_protocol() and ap_get_client_block() from silently swallowing errors from the filter stack, create error buckets and return them appropriately.

core: Stop the HTTP_IN filter from attempting to write error buckets to the output filters, which is bogus in the proxy case. Create a clean mapping from APR codes to HTTP status codes, and use it where needed.

Fix spelling mistake, no code change.

Stupid CodeWarrior compiler cant take vars with struct inits.

add log tags missing in r1426877

Add an option to enforce stricter HTTP conformance This is a first stab, the checks will likely have to be revised. For now, we check * if the request line contains control characters * if the request uri has fragment or username/password * that the request method is standard or registered with RegisterHttpMethod * that the request protocol is of the form HTTP/[1-9]+.[0-9]+, or missing for 0.9 * if there is garbage in the request line after the protocol * if any request header contains control characters * if any request header has an empty name * for the host name in the URL or Host header: - if an IPv4 dotted decimal address: Reject octal or hex values, require exactly four parts - if a DNS host name: Reject non-alphanumeric characters besides '.' and '-'. As a side effect, this rejects multiple Host headers. * if any response header contains control characters * if any response header has an empty name * that the Location response header (if present) has a valid scheme and is absolute If we have a host name both from the URL and the Host header, we replace the Host header with the value from the URL to enforce RFC conformance. There is a log-only mode, but the loglevels of the logged messages need some thought/work. Currently, the checks for incoming data log for 'core' and the checks for outgoing data log for 'http'. Maybe we need a way to configure the loglevels separately from the core/http loglevels.

Avoid unnecessary %s substitution

Various code cleanup to avoid compiler, cppcheck, or clang warnings: modules/debugging/mod_firehose.c: Make some internal functions static (to do: logs_cleanup() is unused) modules/filters/mod_charset_lite.c: Remove dead assignments modules/filters/mod_include.c: likewise modules/metadata/mod_usertrack.c: likewise modules/proxy/mod_proxy_ftp.c: likewise modules/ssl/ssl_engine_pphrase.c: likewise modules/proxy/mod_proxy_balancer.c: likewise; Remove NULL check that can never happen modules/proxy/proxy_util.c: Axe NULL-check that can never happen and if it would, it would just mask another bug os/unix/unixd.c: likewise modules/http/http_filters.c: Remove sub-condition that is always true modules/lua/mod_lua.c: Add default cases to switch statements modules/generators/mod_autoindex.c: Unsigned value can never be < 0 server/util_expr_eval.c: Fix compiler warnings with VC and on OS2

Add lots of unique tags to error log messages

Downgrade some more error messages about broken client behavior to level info.

More cleanup: Expand tabs and some more indentation fixes No functional change

Cleanup effort in prep for GA push: Trim trailing whitespace... no func change

Simplify some overly complicated code using apr_brigade_putstrs and apr_pstrcat Submitted by: Christophe Jaillet <christophe jaillet wanadoo fr>, Stefan Fritsch

Fix a few warnings about use of uninitialized values.

Avoid some memory allocations by using apr_table_setn where the string arguments are const. Submitted by: Christophe JAILLET <christophe jaillet wanadoo fr> PR: 51357

Introduce ap_(get|set)_core_module_config() functions/macros and use them everywhere. We know that the core module has module_index 0. Therefore we can save some pointer operations in ap_get_module_config(cv, &core_module) and ap_set_module_config(cv, &core_module, val). As these are called rather often, this may actually have some (small) measurable effect.

If chunked encoding / content-length are corrupt, we may treat parts of one request's body as the next one's headers. To be safe, we should disable keep-alive in this case.

* Silence compiler warning

In comments: s@ XX @ XXX: @ so they can actually be found seen.

Return 400 not 503 if we have to abort due to malformed chunked encoding.

Fix processing of long chunk extensions PR 49474

Improve trace logging of sent response: - log response status - log Date and Server headers

More trace logging

- Be less verbose at levels INFO and DEBUG in mod_proxy* and mod_ssl - Add some trace logging to core and http

Use the new APLOG_USE_MODULE/AP_DECLARE_MODULE macros everywhere to take advantage of per-module loglevels

Use the more specific 408 (timed out) instead of a generic 400 during a timeout reading a chunk-length.

PR49167, unexpected 413 and double-errordoc during a timeout reading a chunk-size.

Don't keepalive when we send a non-100 response while Client is expecting 100 and may be feeding us continuation data. PR 47087

fix or complain about some invalid errno references

Make sure to not destroy bucket brigades that have been created by earlier filters. Otherwise the pool cleanups would be removed causing potential memory leaks later on.

Initialize len to 0, since for some logic paths it might be unset.

OK, might as well finish this... Allow ServerTokens Off

* Return APR_EOF if request body is shorter than the length announced by the client. PR: 33098 Submitted by: Stefan Fritsch <sf sfritsch.de>

don't use literal '100' for HTTP_CONTINUE (searchability)

Disabled DefaultType directive and removed ap_default_type() from core. We now exclude Content-Type from responses for which a media type has not been configured via mime.types, AddType, ForceType, or some other mechanism. MMN major bump to NZ time. PR: 13986

Translate locally generated "100-Continue" message to ASCII on EBCDIC systems.

Remove X-Pad work around. Submitted by: Takashi Sato <takashi lans-tv.com>

To be safe, consume the entire brigade after processing an error bucket in the HTTP output filter. Submitted by: Rudeiger Pluem Reviewed by: niq, covener

* Fix compiler warning (missing sentinel in function call).

Relax checks on HTTP Response status line from a backend. PR#44995 - Rainer Jung

* mod_proxy_http.c Ensure that the EOC bucket is inserted BEFORE an EOS bucket in bb as some resource filters like mod_deflate pass everything up to the EOS down the chain immediately and sent the remainder of the brigade later (or even never). But in this case the ap_http_header_filter does not get out of our way soon enough. http_filters.c Remove all data buckets that are in a brigade after an EOC bucket was seen, as an EOC bucket tells us that no (further) resource and protocol data should go out to the client. OTOH meta buckets are still welcome as they might trigger needed actions down in the chain (e.g. in network filters like SSL). Remark 1: It is needed to dump ALL data buckets in the brigade since an filter in between might have inserted data buckets BEFORE the EOC bucket sent by the original sender and we do NOT want this data to be sent. Remark 2: Dumping all data buckets here does not necessarily mean that no further data is send to the client as: 1. Network filters like SSL can still be triggered via meta buckets to talk with the client e.g. for a clean shutdown. 2. There could be still data that was buffered before down in the chain that gets flushed by a FLUSH or an EOS bucket. PR: 37770

The response to the TRACE method is partially garbled on an EBCDIC platform. Send the request line and trailing CRLF in ASCII. Submitted by: David Jones <oscaremma gmail.com> Reviewed and tested by: gregames

* Only send 100-continue if the status code so far indicates success. PR: 43711

Move the KeptBodySize directive, kept_body filters and the ap_parse_request_body function out of the http module and into a new module called mod_request, reducing the size of the core.

Add a function to the http filters that is able to parse an HTML form request with the type of application/x-www-form-urlencoded.

* In the case that we fail to read the response line from the backend and if we are a reverse proxy request shutdown the connection WITHOUT ANY response to trigger a retry by the client if allowed (as for idempotent requests). BUT currently we should not do this if the request is the first request on a keepalive connection as browsers like seamonkey only display an empty page in this case and do not do a retry. Related to PR 37770

Remove all references to CORE_PRIVATE.

close PR 44381

http_filters: Don't send 100-continue when 4xx is due PR 43711 Reported & tested by Ragini Bisraya Patch by Chetan Reddy Reviewed: niq

Don't add bogus duplicate Content-Language header entries. PR 11035

belt and suspenders

http_filters: make sure we get last byte of actual data in edge-case of an empty data bucket in get_remaining_chunk_line.

* Saveguard ourselves against underflows

* Address two edge cases: 1. The brigade contains only META buckets. 2. The last data bucket is of zero length. Pointed out by: niq

* HTTP error codes are int's not apr_status_t's. Noted by: Takashi Sato <serai lans-tv.com>

* Do not use local scoped apr_bucket_brigade. We use the same anyway, so we can define it for the whole function.

* Optimize and unify detection of blank / corrupt chunk size lines.

* Move duplicated code for error handling into local function (bail_out_on_error).

* Reuse bucket brigades instead of recreating them continously and thus wasting memory if we have many chunks.

* Optimize assignments by using apr_pcalloc instead of apr_palloc.

* Optimize alignment.

* Optimize solution from r609394 and remove chunk-extensions restriction that was in r609394.

* Fix cases with non blocking reads from the ap_http_filter input filter where chunk size lines or empty lines after a chunk are read incomplete. This can lead to a corruption inside the dechunking algorithm. This patch has an issue with larger chunk-extensions which need to get thrown away since we ignore them anyway. PR: 19954, 41056 Tested by: niq

What's left can exceed size_t

Case statements should go on their own line. (No functional change.) * modules/http/http_filters.c (ap_http_filter): Put case on its own line.

* If no data is available at this point of time we need to switch into the BODY_CHUNK_PART state like we do several lines later in the code in the same situation.

Typo fix

Fix patch to PR#43882 as pointed out by trawick

Safer fix to PR43882 than in r595672.

r595672 was incomplete. We need to be sure we reject multiple-encodings.

Deal with unrecognised Transfer-Encoding headers. PR#43882 (Björn Höhrmann)

core: Add the option to keep aside a request body up to a certain size that would otherwise be discarded, to be consumed by filters such as mod_include. When enabled for a directory, POST requests to shtml files can be passed through to embedded scripts as POST requests, rather being downgraded to GET requests.

Add "DefaultType None" option PR 13986 and PR 16139

Don't send spurious "100 Continue" response lines. PR 38014 [Basant Kumar Kukreja]

Return METHOD_NOT_ALLOWED, not FORBIDDEN, to a TRACE request when TraceEnable is Off. This agrees with our documentation, and with our Allow: header in response to OPTIONS.

Further refinement for PR41056 / PR 19954 (mostly-fixed in r480135.) We assume that a successful read but an empty brigade is NOT cause for EAGAIN. Testing may or may not confirm this assumption, in which case that test may be required as well.

* Do not replace a Date header set by a proxied backend server. PR: 40232

Protect against any sort of non-block read that would block. If ap_get_brigade() shows that, return EAGAIN.

Apply patch for PR 41056 (19954) to fix chunk filter. Now flushes work better.

Replace ap_get_server_version with ap_get_server_banner() and ap_get_server_description(). High-level summary: The full server version information is now included in the error log at startup as well as server status reports, irrespective of the setting of the ServerTokens directive. Third-party modules must now use ap_get_server_banner() or ap_get_server_description() in place of ap_get_server_version().

update license header text

Update the copyright year in all .c, .h and .xml files

Ensure that the proper status line is written to the client, fixing incorrect status lines caused by filters which modify r->status without resetting r->status_line, such as the built-in byterange filter. Note: For the byterange example, the handler must set r->status_line even though this is a 200 response. Some proxy-type modules blindly set r->status_line as set by the origin server and thus trigger the problem with byteranges if the origin server didn't handle the byterange.

If the proxied response lacks a Server header, then create one. PR: 38002 (Bugz) Submitted: william barker at wilshire com

As long as we have the AP_BUCKET_IS_ERROR macro, let's use it

Morph the ap_http_broken_backend_filter() proxy "specific" filter to a generic http error handling output filter.

* Move code for broken backend detection out of core filter into a new http protocol filter (ap_http_broken_backend_filter) that is only run in the proxy case.

If a connection aborts while waiting for a chunked line, flag the connection as errored out and send errors upwards.

No functional change: more indenting/formatting changes due to tabbing

No functional Change: Removing trailing whitespace. This also means that "blank" lines consisting of just spaces or tabs are now really blank lines

No functional change: simple detabbing of indented code.

* modules/http/http_filters.c (ap_send_http_trace): Silence gcc variable-may-be-used-uninitialized warnings.

Plug AllowTrace extended|on|off into proxy and http core. It still is not 'correct' until REQUEST_CHUNKED_PASS is reimplemented and passes some chunk headers, since we aren't echoing the entire request. But it gets me further on testing 1.3 -> 2.0 -> 2.1 -> 2.0 -> 1.3 proxy behaviors.

EBCDIC: Handle chunked input from client or, with proxy, origin server.

update comments for *_client_block functions. note namespace protected function names. clarify where 100 Continue is sent. remove dire warning for calling ap_should_client_block more than once since side effects appear to be long gone.

Update copyright year to 2005 and standardize on current copyright owner line.

Initial pass at refactoring some files to eliminate our 150K C source behemoths. * Makefile.in: Change order of dependencies to bring in exports.o first so that we have every symbol 'used' before the linker starts processing. * build/rules.mk.in: Add a 'program-install' target which just copies httpd. * server/Makefile.in, modules/http/config2.m4: Add in new file targets. * NWGNUmakefile, libhttpd.dsp: Blind updates for Netware and Win32. (I tried.) * server/core.c: Move core_input_filter, net_time_filter, and core_output_filter and all supporting functions to... * server/core_filters.c (copied): ...here. * modules/http/http_protocol.c: Move functions from here to there...namely: * modules/http/byterange_filter.c (copied): Relocate ap_byterange_filter() and friends. * modules/http/chunk_filter.c (copied): Relocate chunk_filter(). * modules/http/http_etag.c (copied): Relocate ap_set_etag and ap_make_etag(). * modules/http/http_filters.c (copied): Relocate ap_http_filter(), ap_http_header_filter(), ap_discard_request_body(), ap_setup_client_block(), ap_should_client_block(), and ap_get_client_block().