howto.html revision f9b3be308809978f797e0c57b296147532a4313c
97a9a944b5887e91042b019776c41d5dd74557aferikabele<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
a945f35eff8b6a88009ce73de6d4c862ce58de3cslive<META NAME="description" CONTENT="Some 'how to' tips for the Apache httpd server">
a945f35eff8b6a88009ce73de6d4c862ce58de3cslive<META NAME="keywords" CONTENT="apache,redirect,robots,rotate,logfiles">
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<!-- Background white, links blue (unvisited), navy (visited), red (active) -->
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd BGCOLOR="#FFFFFF"
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd TEXT="#000000"
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd LINK="#0000FF"
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd VLINK="#000080"
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd ALINK="#FF0000"
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<!--#include virtual="header.html" -->
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<LI><A HREF="#redirect">redirect an entire server or directory to a single URL</A>
3b3b7fc78d1f5bfc2769903375050048ff41ff26nd<H2><A name="redirect">How to redirect an entire server or directory to a single URL</A></H2>
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<P>There are two chief ways to redirect all requests for an entire
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndserver to a single location: one which requires the use of
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<CODE>mod_rewrite</CODE>, and another which uses a CGI script.
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<P>First: if all you need to do is migrate a server from one name to
06ba4a61654b3763ad65f52283832ebf058fdf1csliveanother, simply use the <CODE>Redirect</CODE> directive, as supplied
06ba4a61654b3763ad65f52283832ebf058fdf1cslive<P>Since <CODE>Redirect</CODE> will forward along the complete path,
ccd7cf82b2654653ad0d29803184f64530b4f16bndhowever, it may not be appropriate - for example, when the directory
ccd7cf82b2654653ad0d29803184f64530b4f16bndstructure has changed after the move, and you simply want to direct people
ccd7cf82b2654653ad0d29803184f64530b4f16bndto the home page.
ccd7cf82b2654653ad0d29803184f64530b4f16bnd<P>The best option is to use the standard Apache module <CODE>mod_rewrite</CODE>.
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndIf that module is compiled in, the following lines:
62cf7c73d1630765688a05f89aa154b2e453dbc7ndRewriteRule /.* http://www.apache.org/ [R]
117c1f888a14e73cdd821dc6c23eb0411144a41cndThis will send an HTTP 302 Redirect back to the client, and no matter
117c1f888a14e73cdd821dc6c23eb0411144a41cndwhat they gave in the original URL, they'll be sent to
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndThe second option is to set up a <CODE>ScriptAlias</Code> pointing to
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nda <STRONG>cgi script</STRONG> which outputs a 301 or 302 status and the location
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndof the other server.</P>
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4nd<P>By using a <STRONG>cgi-script</STRONG> you can intercept various requests and
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndtreat them specially, e.g. you might want to intercept <STRONG>POST</STRONG>
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndrequests, so that the client isn't redirected to a script on the other
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndserver which expects POST information (a redirect will lose the POST
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndinformation.) You might also want to use a CGI script if you don't
0c4abc32c00611fe1d52c9661f5cc79a3f74c6d4ndwant to compile mod_rewrite into your server.
ccd7cf82b2654653ad0d29803184f64530b4f16bnd<P>Here's how to redirect all requests to a script... In the server
f804ec6e56dc58300c256b0fff0f2ca3f226c5b5ndconfiguration file,
f804ec6e56dc58300c256b0fff0f2ca3f226c5b5nd<BLOCKQUOTE><PRE>ScriptAlias / /usr/local/httpd/cgi-bin/redirect_script</PRE></BLOCKQUOTE>
06ba4a61654b3763ad65f52283832ebf058fdf1csliveand here's a simple perl script to redirect requests:
ccd7cf82b2654653ad0d29803184f64530b4f16bndprint "Status: 302 Moved Temporarily\r
ccd7cf82b2654653ad0d29803184f64530b4f16bnd<H2><A name="logreset">How to reset your log files</A></H2>
ccd7cf82b2654653ad0d29803184f64530b4f16bnd<P>Sooner or later, you'll want to reset your log files (access_log and
6771d932c350b8af84039f3c71ebf9313dd52569kesserror_log) because they are too big, or full of old information you don't
46fb146a4ec8c97d8dccb8c0c96a6cfddfc5442cnd<P><CODE>access.log</CODE> typically grows by 1Mb for each 10,000 requests.</P>
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowen<P>Most people's first attempt at replacing the logfile is to just move the
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowenlogfile or remove the logfile. This doesn't work.</P>
6771d932c350b8af84039f3c71ebf9313dd52569kess<P>Apache will continue writing to the logfile at the same offset as before the
ccd7cf82b2654653ad0d29803184f64530b4f16bndlogfile moved. This results in a new logfile being created which is just
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowenas big as the old one, but it now contains thousands (or millions) of null
ccd7cf82b2654653ad0d29803184f64530b4f16bndcharacters.</P>
6771d932c350b8af84039f3c71ebf9313dd52569kess<P>The correct procedure is to move the logfile, then signal Apache to tell it to reopen the logfiles.</P>
6771d932c350b8af84039f3c71ebf9313dd52569kess<P>Apache is signaled using the <STRONG>SIGHUP</STRONG> (-1) signal. e.g.
ccd7cf82b2654653ad0d29803184f64530b4f16bndkill -1 `cat httpd.pid`
89c751444e7e282537a4e9a529d0251eecf7915erpluem<P>Note: <CODE>httpd.pid</CODE> is a file containing the <STRONG>p</STRONG>rocess <STRONG>id</STRONG>
89c751444e7e282537a4e9a529d0251eecf7915erpluemof the Apache httpd daemon, Apache saves this in the same directory as the log
e1e8390280254f7f0580d701e583f670643d4f3fnilgun<P>Many people use this method to replace (and backup) their logfiles on a
e1e8390280254f7f0580d701e583f670643d4f3fnilgunnightly or weekly basis.</P>
79b024b81f6bb3c44dce77a7552191daf8b522d2jim<H2><A name="stoprob">How to stop or restrict robots</A></H2>
a0d937b340692a3578f1d2f2535890c520c4bf0cnd<P>Ever wondered why so many clients are interested in a file called
a0d937b340692a3578f1d2f2535890c520c4bf0cnd<CODE>robots.txt</CODE> which you don't have, and never did have?</P>
ccd7cf82b2654653ad0d29803184f64530b4f16bnd<P>These clients are called <STRONG>robots</STRONG> (also known as crawlers,
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowenspiders and other cute name) - special automated clients which
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowenwander around the web looking for interesting resources.</P>
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowen<P>Most robots are used to generate some kind of <EM>web index</EM> which
1ad1c5f9a82d056966dcca9c6108c5ace8eed446rbowenis then used by a <EM>search engine</EM> to help locate information.</P>
79b024b81f6bb3c44dce77a7552191daf8b522d2jim<P><CODE>robots.txt</CODE> provides a means to request that robots limit their
ccd7cf82b2654653ad0d29803184f64530b4f16bndactivities at the site, or more often than not, to leave the site alone.</P>
22265f1724519886e2a2b5e0ebd61477506b7379noodl<P>When the first robots were developed, they had a bad reputation for
22265f1724519886e2a2b5e0ebd61477506b7379noodlsending hundreds/thousands of requests to each site, often resulting
22265f1724519886e2a2b5e0ebd61477506b7379noodlin the site being overloaded. Things have improved dramatically since
22265f1724519886e2a2b5e0ebd61477506b7379noodlthen, thanks to <A
79b024b81f6bb3c44dce77a7552191daf8b522d2jimHREF="http://info.webcrawler.com/mak/projects/robots/guidelines.html">
ccd7cf82b2654653ad0d29803184f64530b4f16bndGuidelines for Robot Writers</A>, but even so, some robots may exhibit
79b024b81f6bb3c44dce77a7552191daf8b522d2jimunfriendly behavior which the webmaster isn't willing to tolerate, and
349eb1baf455be0d2b9009dab951e5b1ba13dd2ftrawickwill want to stop.</P>
5b111d121ddb14d690dde7c180402d0b9a030b7and<P>Another reason some webmasters want to block access to robots, is to
5b111d121ddb14d690dde7c180402d0b9a030b7andstop them indexing dynamic information. Many search engines will use the
349eb1baf455be0d2b9009dab951e5b1ba13dd2ftrawickdata collected from your pages for months to come - not much use if your
79b024b81f6bb3c44dce77a7552191daf8b522d2jimserving stock quotes, news, weather reports or anything else that will be
ccd7cf82b2654653ad0d29803184f64530b4f16bndstale by the time people find it in a search engine.</P>
79b024b81f6bb3c44dce77a7552191daf8b522d2jim<P>If you decide to exclude robots completely, or just limit the areas
ccd7cf82b2654653ad0d29803184f64530b4f16bndin which they can roam, create a <CODE>robots.txt</CODE> file; refer
e797af4d7b0cada1278d72d6c8ac77210ef78632minfrinto the <A HREF="http://info.webcrawler.com/mak/projects/robots/robots.html">robot information pages</A> provided by Martijn Koster for the syntax.</P>
e797af4d7b0cada1278d72d6c8ac77210ef78632minfrin<!--#include virtual="footer.html" -->