Your Virtual Server has three types of log files: e-mail, FTP, and web. These logs contain helpful diagnostic information as well as invaluable information about your web site visitors. Although extremely useful, your logs can cause a lot of problems if not properly maintained.

The ~/var/log/messages file contains log entries from various programs. Each entry, one per line, contains the following:

• A time stamp (recording the date and time of the log entry).
• The name of the originating program.
• The text of the log entry.

From you Virtual Server command prompt, type:

% tail -f ~/var/log/messages

The tail command prints the last ten lines of the named file. The -f option allows you to "follow" the file as it grows. Exit tail by entering <ctrl>-c.

Since the ~/usr/log/messages file has a tendency to grow large over time, you should reset it periodically.

From you Virtual Server command prompt, type:

% cat /dev/null > ~/var/log/messages

This action removes all messages recorded in the logs.

Note: Before resetting the log, prepare archival copies, if needed. You can do this, for example, by archiving your files with tar or zip and then copying them via FTP from your server to your local computer.

You may also use the vnukelog -r command. However, this command resets both the messages file and the web server log files. The vnukelog command is explained in more detail later in this chapter.

• Web Server Log Format
• Analyzing log files
• Rotating and clearing log files

• access_log (all access, agent, and referrer information is logged to ~/www/logs/access_log)
• error_log

Logged in these files is the volume of activity at each page on your web site, the type of browser used to access each page, any errors that users may have experienced downloading pages from your site, and where users were referred from when they accessed pages at your site.

Alternatively, you may configure your Virtual Server to log in the common log format by modifying your web server configuration file (~/www/conf/httpd.conf). In the common log format, all information is logged to four log files:

• access_log
• agent_log
• referrer_log
• error_log

The log format as well as other log activity is based on the directives you define in your httpd.conf configuration file. The default directive definitions should be adequate for most circumstances. However, you are free to modify the directives if you need to define log file formatting (or turn off the logging capability altogether). See Chapter 6 (Advanced Web Server Configuration) for details on log directives.

Recall that when your Virtual Server is configured, the default log preferences are set up in the combined log format:

Viewing the error_log File's Latest Entries

1. Connect to your Virtual Server via Telnet or SSH.
2. Make the www/logs directory your current working directory, by entering:

   % cd ~/www/logs/

3. From your logs directory, type:

   % tail -f error_log

Note: The tail command prints the last ten lines of the named file. The -f option allows you to "follow" the file as it grows. Exit by typing <ctrl>-c.

You can control the detail level of the error log file the LogLevel directive in your httpd.conf file.

http://www.yourcompany.com/bogus-filename.html

Assuming that the file bogus-filename.html doesn't exist, a new entry will be added to your error log file. It will look something like this:

[date and time] access to /usr/local/etc/httpd/htdocs/bogus-filename.html failed for some.remote.host, reason: File does not exist

1. Connect to your Virtual Server via Telnet or SSH.
2. Make the www/logs directory your current working directory by entering:

% cd ~/www/logs/

1. From your logs directory, type:

   % tail –f access_log

some.IP.address - user - [access date and time] "request" status bytes_sent file_sent referrer agent

Note: You can exit the tail command by entering "<ctrl>-c" at any time.

some.remote.host - user - [19/Aug/1998:13:48:56 –0600] "GET /index.php HTTP/1.0" 200 4817 "http://www.yahoo.com" "Mozilla/4.75 [en] (Windows NT 5.0; U)"

This entry suggests that on the 19^th of August 1998 at 1:48:56 in the afternoon Mountain Standard Time (or some other –0600 time zone), a remote host "some.remote.host" requested the URL "index.php" using an HTTP/1.0-compliant browser. The server found the resource requested (status code 200) and returned it to the client. The document was 4817 bytes in length. The request came from a link on Yahoo's home page (the referring site), and the user was using Netscape Navigator v4.75 ("Mozilla" is how Netscape identifies itself to web servers).

The following table explains this example in more detail.

WebTrends™ is web server log analysis software that produces graphical reports of your web site traffic. WebTrends is easy to use because it has a friendly interface. Configure WebTrends to download your Virtual Server web log files to your computer, and then create any number of professional statistical reports. The generated reports can be stored locally on your computer, or they can be automatically uploaded back to your Virtual Server.

There are a number of analysis programs available that you can install directly on your Virtual Server. Most of these programs analyze your web server log files in place and then create HTML, text, or e-mail reports of your web server traffic. We have made several of these tools available including http-analyze,  analog, and The Webalizer.

These software packages are a bit harder to use since they must be run from the command prompt, but they are simple to install and free of charge. For more details about log analysis software packages, see Enetrics Communications's web site.

Note: Some log analysis programs require a specific log format (i.e. combined or common). Make sure the log format configured on your Virtual Server is appropriate for the log analysis program you select.

Logs can grow rapidly and need to be rotated. After running the stats program of your choice, clear the logs. The command for clearing the log files is vnukelog. The vnukelog command can be used to clear the ~/usr/log/messages file as well as all Virtual Server and virtual subhost log files.

Use the -h flag to see all vnukelog options:

% vnukelog -i

Usage: vnukelog [-h] [-i] [-r]

-h   display this message
-i   enter interactive mode
-r   nuke root server logs only

Use the vnukelog command without any flags to clear the ~/usr/log/messages file and all Virtual Server and virtual subhost log files:

% vnukelog

Use the -r flag to clear just the Virtual Server log files, and leave the virtual subhost log files intact:

% vnukelog -r

Use the -i flag to enter an interactive mode that allows you to clear just the Virtual Server and virtual subhost log files you want to clear.

% vnukelog -i

Generating stats on a daily weekly or monthly schedule is important. We recommend that you use cron to automatically generate a report and rotate the logs.