howto.html revision e342855032e43c841697d5ff3fd7c0c4bdee6a48
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd CONTENT="Some 'how to' tips for the Apache httpd server">
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd<META NAME="keywords" CONTENT="apache,redirect,robots,rotate,logfiles">
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd<!-- Background white, links blue (unvisited), navy (visited), red (active) -->
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd BGCOLOR="#FFFFFF"
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd TEXT="#000000"
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd LINK="#0000FF"
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd VLINK="#000080"
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd ALINK="#FF0000"
d5d794fc2f4cc9ca6d6da17cfa2cdcd8d244bacdnd<!--#include virtual="header.html" -->
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<LI><A HREF="#redirect">redirect an entire server or directory to a single
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<H2><A NAME="redirect">How to redirect an entire server or directory to a
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>There are two chief ways to redirect all requests for an entire
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndserver to a single location: one which requires the use of
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<CODE>mod_rewrite</CODE>, and another which uses a CGI script.
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>First: if all you need to do is migrate a server from one name to
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndanother, simply use the <CODE>Redirect</CODE> directive, as supplied
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Since <CODE>Redirect</CODE> will forward along the complete path,
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndhowever, it may not be appropriate - for example, when the directory
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndstructure has changed after the move, and you simply want to direct people
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndto the home page.
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>The best option is to use the standard Apache module
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndIf that module is compiled in, the following lines
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndRewriteRule /.* http://www.apache.org/ [R]
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndThis will send an HTTP 302 Redirect back to the client, and no matter
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndwhat they gave in the original URL, they'll be sent to
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndThe second option is to set up a <CODE>ScriptAlias</CODE> pointing to
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nda <STRONG>CGI script</STRONG> which outputs a 301 or 302 status and the
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndof the other server.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>By using a <STRONG>CGI script</STRONG> you can intercept various requests
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndtreat them specially, <EM>e.g.</EM>, you might want to intercept
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndrequests, so that the client isn't redirected to a script on the other
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndserver which expects POST information (a redirect will lose the POST
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndinformation.) You might also want to use a CGI script if you don't
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndwant to compile mod_rewrite into your server.
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Here's how to redirect all requests to a script... In the server
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndconfiguration file,
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<BLOCKQUOTE><PRE>ScriptAlias / /usr/local/httpd/cgi-bin/redirect_script</PRE>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd</BLOCKQUOTE>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndand here's a simple perl script to redirect requests:
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndprint "Status: 302 Moved Temporarily\r
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<H2><A NAME="logreset">How to reset your log files</A></H2>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Sooner or later, you'll want to reset your log files (access_log and
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nderror_log) because they are too big, or full of old information you don't
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P><CODE>access.log</CODE> typically grows by 1Mb for each 10,000 requests.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Most people's first attempt at replacing the logfile is to just move the
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndlogfile or remove the logfile. This doesn't work.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Apache will continue writing to the logfile at the same offset as before the
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndlogfile moved. This results in a new logfile being created which is just
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndas big as the old one, but it now contains thousands (or millions) of null
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndcharacters.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>The correct procedure is to move the logfile, then signal Apache to tell
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndit to reopen the logfiles.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Apache is signaled using the <STRONG>SIGHUP</STRONG> (-1) signal.
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndkill -1 `cat httpd.pid`
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndof the Apache httpd daemon, Apache saves this in the same directory as the log
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Many people use this method to replace (and backup) their logfiles on a
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndnightly or weekly basis.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<H2><A NAME="stoprob">How to stop or restrict robots</A></H2>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Ever wondered why so many clients are interested in a file called
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<CODE>robots.txt</CODE> which you don't have, and never did have?</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>These clients are called <STRONG>robots</STRONG> (also known as crawlers,
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndspiders and other cute name) - special automated clients which
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndwander around the web looking for interesting resources.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Most robots are used to generate some kind of <EM>web index</EM> which
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndis then used by a <EM>search engine</EM> to help locate information.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P><CODE>robots.txt</CODE> provides a means to request that robots limit their
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndactivities at the site, or more often than not, to leave the site alone.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>When the first robots were developed, they had a bad reputation for
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndsending hundreds/thousands of requests to each site, often resulting
5718d2b6ea423b4eb2d45f64bbc4c8eb61e0400fndin the site being overloaded. Things have improved dramatically since
5718d2b6ea423b4eb2d45f64bbc4c8eb61e0400fndthen, thanks to <A
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndHREF="http://info.webcrawler.com/mak/projects/robots/guidelines.html">
5718d2b6ea423b4eb2d45f64bbc4c8eb61e0400fndGuidelines for Robot Writers</A>, but even so, some robots may exhibit
5718d2b6ea423b4eb2d45f64bbc4c8eb61e0400fndunfriendly behavior which the webmaster isn't willing to tolerate, and
5718d2b6ea423b4eb2d45f64bbc4c8eb61e0400fndwill want to stop.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>Another reason some webmasters want to block access to robots, is to
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndstop them indexing dynamic information. Many search engines will use the
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nddata collected from your pages for months to come - not much use if your
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndserving stock quotes, news, weather reports or anything else that will be
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndstale by the time people find it in a search engine.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<P>If you decide to exclude robots completely, or just limit the areas
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndin which they can roam, create a <CODE>robots.txt</CODE> file; refer
4ef32fc8ca017f80aa0d8ff267722a335c9d7784ndto the <A HREF="http://info.webcrawler.com/mak/projects/robots/robots.html"
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd>robot information pages</A> provided by Martijn Koster for the syntax.</P>
4ef32fc8ca017f80aa0d8ff267722a335c9d7784nd<!--#include virtual="footer.html" -->