23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<!DOCTYPE modulesynopsis SYSTEM "/style/modulesynopsis.dtd">
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<?xml-stylesheet type="text/xsl" href="/style/manual.en.xsl"?>
8e8e6d90e74a6646ed697edb7238862f708dd6f7nd<!-- $LastChangedRevision$ -->
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin Licensed to the Apache Software Foundation (ASF) under one or more
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin contributor license agreements. See the NOTICE file distributed with
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin this work for additional information regarding copyright ownership.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin The ASF licenses this file to You under the Apache License, Version 2.0
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin (the "License"); you may not use this file except in compliance with
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin the License. You may obtain a copy of the License at
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin Unless required by applicable law or agreed to in writing, software
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin distributed under the License is distributed on an "AS IS" BASIS,
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin See the License for the specific language governing permissions and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin limitations under the License.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Multiplexes all I/O to a given file or pipe.</description>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p><code>mod_firehose</code> provides a mechanism to record data
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin being passed between the httpd server and the client at the raw
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin connection level to either a file or a pipe in such a way that the
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin data can be analysed or played back to the server at a future date.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin It can be thought of as "tcpdump for httpd".</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>Connections are recorded after the SSL has been stripped, and can
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin be used for forensic debugging.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>The <program>firehose</program> tool can be used to demultiplex
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin the recorded stream back into individual files for analysis, or
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <note><title>WARNING</title>This module IGNORES all request level
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin mechanisms to keep data private. It is the responsibility of the
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin administrator to ensure that private data is not inadvertently
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin exposed using this module.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>To enable the module, it should be compiled and loaded
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin in to your running Apache configuration, and the directives below
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin used to record the data you are interested in.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>It is possible to record both incoming and outgoing data to
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin the same filename if desired, as the direction of flow is recorded
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin within each fragment.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>It is possible to write to both normal files and fifos (pipes).
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin In the case of fifos, mod_firehose ensures that the packet size is
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin no larger than PIPE_BUF to ensure writes are atomic.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>If a pipe is being used, something must be reading from the pipe
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin before httpd is started for the pipe to be successfully opened for
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin write. If the request to open the pipe fails, mod_firehose will
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin silently stand down and not record anything, and the server will
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin keep running as normal.</p>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin <p>By default, all attempts to write will block the server. If the
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin webserver has been built against APR v2.0 or later, and an optional
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin "nonblock" parameter is specified all file writes will be non
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin blocking, and buffer overflows will cause debugging data to be lost.
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin In this case it is possible to prioritise the running of the server
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin over the recording of firehose data.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>The server typically serves multiple connections simultaneously,
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin and as a result requests and responses need to be multiplexed before
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin being written to the firehose.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>The fragment format is designed as clear text, so that a firehose
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin can be opened with and inspected by a normal text editor.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin Alternatively, the <program>firehose</program> tool can be used to
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin demultiplex the firehose back into individual requests or
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin connections.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>The size of the multiplexed fragments is governed by PIPE_BUF,
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin the maximum size of write the system is prepared to perform
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin atomically. By keeping the multiplexed fragments below PIPE_BUF in
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin size, the module guarantees that data from different fragments does
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin not interleave. The size of PIPE_BUF varies on different operating
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin systems.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>The BNF for the fragment format is as follows:</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin stream = 0*(fragment)
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin fragment = header CRLF body CRLF
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin length = <up to 16 byte hex fragment length>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin timestamp = <up to 16 byte hex timestamp microseconds since 1970>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin request = "<"
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin response = ">"
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin uuid = <formatted uuid of the connection>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin count = <hex fragment number in the connection>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin body = <the binary content of the fragment>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin SPC = <a single space>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin CRLF = <a carriage return, followed by a line feed>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>All fragments for a connection or a request will share the same
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin UUID, depending on whether connections or requests are being recorded.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin If connections are being recorded, multiple requests may appear within
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin a connection. A fragment with a zero length indicates the end of the
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin connection.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>Fragments may go missing or be dropped if the process reading the
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin fragments is too slow. If this happens, gaps will exist in the
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin connection counter numbering. A warning will be logged in the error
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin log to indicate the UUID and counter of the dropped fragment, so it
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin can be confirmed the fragment was dropped.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>It is possible that the terminating empty fragment may not appear,
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin caused by the httpd process crashing, or being terminated ungracefully.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin The terminating fragment may be dropped if the process reading the
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin fragments is not fast enough.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Capture traffic coming into the server on each connection</description>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin<syntax>FirehoseConnectionInput <var>[ block | nonblock ]</var> <var>filename</var></syntax>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<contextlist><context>server config</context></contextlist>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<compatibility>FirehoseConnectionInput is only available in Apache 2.5.0 and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrinlater.</compatibility>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>Capture traffic coming into the server on each connection. Multiple
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin requests will be captured within the same connection if keepalive is
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin present.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin FirehoseConnectionInput connection-input.firehose
ae936fc00a95426b8346f667ce60d0a40fbeee92humbedooh </highlight>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Capture traffic going out of the server on each connection</description>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin<syntax>FirehoseConnectionOutput <var>[ block | nonblock ]</var> <var>filename</var></syntax>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<contextlist><context>server config</context></contextlist>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<compatibility>FirehoseConnectionOutput is only available in Apache 2.5.0 and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrinlater.</compatibility>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>Capture traffic going out of the server on each connection.
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin Multiple requests will be captured within the same connection if
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin keepalive is present.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin FirehoseConnectionOutput connection-output.firehose
ae936fc00a95426b8346f667ce60d0a40fbeee92humbedooh </highlight>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Capture traffic coming into the server on each request</description>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin<syntax>FirehoseRequestInput <var>[ block | nonblock ]</var> <var>filename</var></syntax>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<contextlist><context>server config</context></contextlist>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<compatibility>FirehoseRequestInput is only available in Apache 2.5.0 and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrinlater.</compatibility>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>Capture traffic coming into the server on each request. Requests
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin will be captured separately, regardless of the presence of keepalive.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin FirehoseRequestInput request-input.firehose
ae936fc00a95426b8346f667ce60d0a40fbeee92humbedooh </highlight>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Capture traffic going out of the server on each request</description>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin<syntax>FirehoseRequestOutput <var>[ block | nonblock ]</var> <var>filename</var></syntax>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<contextlist><context>server config</context></contextlist>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<compatibility>FirehoseRequestOutput is only available in Apache 2.5.0 and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrinlater.</compatibility>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin <p>Capture traffic going out of the server on each request. Requests
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin will be captured separately, regardless of the presence of keepalive.</p>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin FirehoseRequestOutput request-output.firehose
ae936fc00a95426b8346f667ce60d0a40fbeee92humbedooh </highlight>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Capture traffic coming into the back of mod_proxy</description>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin<syntax>FirehoseProxyConnectionInput <var>[ block | nonblock ]</var> <var>filename</var></syntax>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<contextlist><context>server config</context></contextlist>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<compatibility>FirehoseProxyConnectionInput is only available in Apache 2.5.0 and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrinlater.</compatibility>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin FirehoseProxyConnectionInput proxy-input.firehose
ae936fc00a95426b8346f667ce60d0a40fbeee92humbedooh </highlight>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<description>Capture traffic sent out from the back of mod_proxy</description>
aef7cf144b9c027d2b38b4eaf36bc47c40ca5969minfrin<syntax>FirehoseProxyConnectionOutput <var>[ block | nonblock ]</var> <var>filename</var></syntax>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<contextlist><context>server config</context></contextlist>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin<compatibility>FirehoseProxyConnectionOutput is only available in Apache 2.5.0 and
23f1535d6a60817d2846bac0aea230ea475d7dccminfrinlater.</compatibility>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin FirehoseProxyConnectionOutput proxy-output.firehose
ae936fc00a95426b8346f667ce60d0a40fbeee92humbedooh </highlight>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</directivesynopsis>
23f1535d6a60817d2846bac0aea230ea475d7dccminfrin</modulesynopsis>