<?xml version="1.0"?>

<!DOCTYPE modulesynopsis SYSTEM "/style/modulesynopsis.dtd">

<?xml-stylesheet type="text/xsl" href="/style/manual.en.xsl"?>

<modulesynopsis metafile="mod_firehose.xml.meta">

<name>mod_firehose</name>

<description>Multiplexes all I/O to a given file or pipe.</description>

<status>Extension</status>

<sourcefile>mod_firehose.c</sourcefile>

<identifier>firehose_module</identifier>

<summary>

<p><code>mod_firehose</code> provides a mechanism to record data

being passed between the httpd server and the client at the raw

connection level to either a file or a pipe in such a way that the

data can be analysed or played back to the server at a future date.

It can be thought of as "tcpdump for httpd".</p>

<p>Connections are recorded after the SSL has been stripped, and can

be used for forensic debugging.</p>

<p>The <program>firehose</program> tool can be used to demultiplex

the recorded stream back into individual files for analysis, or

playback using a tool like <code>netcat</code>.</p>

<note><title>WARNING</title>This module IGNORES all request level

mechanisms to keep data private. It is the responsibility of the

administrator to ensure that private data is not inadvertently

exposed using this module.

</note>

</summary>

<section id="enable">

<title>Enabling a Firehose</title>

<p>To enable the module, it should be compiled and loaded

in to your running Apache configuration, and the directives below

used to record the data you are interested in.</p>

<p>It is possible to record both incoming and outgoing data to

the same filename if desired, as the direction of flow is recorded

within each fragment.</p>

<p>It is possible to write to both normal files and fifos (pipes).

In the case of fifos, mod_firehose ensures that the packet size is

no larger than PIPE_BUF to ensure writes are atomic.</p>

<p>If a pipe is being used, something must be reading from the pipe

before httpd is started for the pipe to be successfully opened for

write. If the request to open the pipe fails, mod_firehose will

silently stand down and not record anything, and the server will

keep running as normal.</p>

<p>All file writes are non blocking, and buffer overflows will cause

debugging data to be lost. The module makes the call that the

reliable running of the server takes precedence over the recording

of firehose data.</p>

</section>

<section id="format">

<title>Stream Format</title>

<p>The server typically serves multiple connections simultaneously,

and as a result requests and responses need to be multiplexed before

being written to the firehose.</p>

<p>The fragment format is designed as clear text, so that a firehose

can be opened with and inspected by a normal text editor.

Alternatively, the <program>firehose</program> tool can be used to

demultiplex the firehose back into individual requests or

connections.</p>

<p>The size of the multiplexed fragments is governed by PIPE_BUF,

the maximum size of write the system is prepared to perform

atomically. By keeping the multiplexed fragments below PIPE_BUF in

size, the module guarantees that data from different fragments does

not interleave. The size of PIPE_BUF varies on different operating

systems.</p>

<p>The BNF for the fragment format is as follows:</p>

<pre>

stream = 0*(fragment)

fragment = header CRLF body CRLF

header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count

length = &lt;up to 16 byte hex fragment length>

timestamp = &lt;up to 16 byte hex timestamp microseconds since 1970>

request = "&lt;"

response = ">"

uuid = &lt;formatted uuid of the connection>

count = &lt;hex fragment number in the connection>

body = &lt;the binary content of the fragment>

SPC = &lt;a single space>

CRLF = &lt;a carriage return, followed by a line feed>

</pre>

<p>All fragments for a connection or a request will share the same

UUID, depending on whether connections or requests are being recorded.

If connections are being recorded, multiple requests may appear within

a connection. A fragment with a zero length indicates the end of the

connection.</p>

<p>Fragments may go missing or be dropped if the process reading the

fragments is too slow. If this happens, gaps will exist in the

connection counter numbering. A warning will be logged in the error

log to indicate the UUID and counter of the dropped fragment, so it

can be confirmed the fragment was dropped.</p>

<p>It is possible that the terminating empty fragment may not appear,

caused by the httpd process crashing, or being terminated ungracefully.

The terminating fragment may be dropped if the process reading the

fragments is not fast enough.</p>

</section>

<directivesynopsis>

<name>FirehoseConnectionInput</name>

<description>Capture traffic coming into the server on each connection</description>

<syntax>FirehoseConnectionInput <var>filename</var></syntax>

<default>none</default>

<contextlist><context>server config</context></contextlist>

<compatibility>FirehoseConnectionInput is only available in Apache 2.5.0 and

later.</compatibility>

<usage>

<p>Capture traffic coming into the server on each connection. Multiple

requests will be captured within the same connection if keepalive is

present.</p>

<example><title>Example</title>

FirehoseConnectionInput connection-input.firehose

</example>

</usage>

</directivesynopsis>

<directivesynopsis>

<name>FirehoseConnectionOutput</name>

<description>Capture traffic going out of the server on each connection</description>

<syntax>FirehoseConnectionOutput <var>filename</var></syntax>

<default>none</default>

<contextlist><context>server config</context></contextlist>

<compatibility>FirehoseConnectionOutput is only available in Apache 2.5.0 and

later.</compatibility>

<usage>

<p>Capture traffic going out of the server on each connection.

Multiple requests will be captured within the same connection if

keepalive is present.</p>

<example><title>Example</title>

FirehoseConnectionOutput connection-output.firehose

</example>

</usage>

</directivesynopsis>

<directivesynopsis>

<name>FirehoseRequestInput</name>

<description>Capture traffic coming into the server on each request</description>

<syntax>FirehoseRequestInput <var>filename</var></syntax>

<default>none</default>

<contextlist><context>server config</context></contextlist>

<compatibility>FirehoseRequestInput is only available in Apache 2.5.0 and

later.</compatibility>

<usage>

<p>Capture traffic coming into the server on each request. Requests

will be captured separately, regardless of the presence of keepalive.</p>

<example><title>Example</title>

FirehoseRequestInput request-input.firehose

</example>

</usage>

</directivesynopsis>

<directivesynopsis>

<name>FirehoseRequestOutput</name>

<description>Capture traffic going out of the server on each request</description>

<syntax>FirehoseRequestOutput <var>filename</var></syntax>

<default>none</default>

<contextlist><context>server config</context></contextlist>

<compatibility>FirehoseRequestOutput is only available in Apache 2.5.0 and

later.</compatibility>

<usage>

<p>Capture traffic going out of the server on each request. Requests

will be captured separately, regardless of the presence of keepalive.</p>

<example><title>Example</title>

FirehoseRequestOutput request-output.firehose

</example>

</usage>

</directivesynopsis>

<directivesynopsis>

<name>FirehoseProxyConnectionInput</name>

<description>Capture traffic coming into the back of mod_proxy</description>

<syntax>FirehoseProxyConnectionInput <var>filename</var></syntax>

<default>none</default>

<contextlist><context>server config</context></contextlist>

<compatibility>FirehoseProxyConnectionInput is only available in Apache 2.5.0 and

later.</compatibility>

<usage>

<p>Capture traffic being received by mod_proxy.</p>

<example><title>Example</title>

FirehoseProxyConnectionInput proxy-input.firehose

</example>

</usage>

</directivesynopsis>

<directivesynopsis>

<name>FirehoseProxyConnectionOutput</name>

<description>Capture traffic sent out from the back of mod_proxy</description>

<syntax>FirehoseProxyConnectionOutput <var>filename</var></syntax>

<default>none</default>

<contextlist><context>server config</context></contextlist>

<compatibility>FirehoseProxyConnectionOutput is only available in Apache 2.5.0 and

later.</compatibility>

<usage>

<p>Capture traffic being sent out by mod_proxy.</p>

<example><title>Example</title>

FirehoseProxyConnectionOutput proxy-output.firehose

</example>

</usage>

</directivesynopsis>

</modulesynopsis>