Suppose you have the following problem. You have a program which stores log file

Question

Suppose you have the following problem. You have a program which stores log files in the directory: /var/log/sysProgram and the configuration files for the program reside in the directory: /etc/sysProgram/config and you wish to protect the rest of the system by 1) ensuring that the log files in /var/log/sysProgram never grow beyond 80MB AND 2) ensuring that the configuration files (about 10K) are read-only and safe from corruption by the rest of the disk. Note that both of these directories are required to be available after the system boots. Finally, you have a 100MB disk on hand with which to solve this problem.

NOTE: you will need to create the directories and insert something in them (simple text files will work), you will need to create a 100MB VirtualBox virtual disk and add it to the VM, partition it into two partitions, format them, and so on to solve the problem. You need to list commands and instructions for interactive commands and you will need to show the outputs of the various commands including the contents of fstab and the results of df –h after rebooting when finished

Explanation / Answer

Log files, though useful to troubleshoot and to track usage, tend to use up valuable disk space. Over time, they become large and unwieldy, so pinpointing an event becomes difficult. Logrotate performs the function of archiving a log file and starting a new one, thereby ‘rotating’ it.

Logrotate has been designed to ease the administration of systems that generate large numbers of log files in any format. It allows automatic rotation,

compression, removal and mailing of log files. Each log file may be handled daily, every week, every month, or when it grows too large

(rotation on the basis of a file’s size).

The application and the servers generate too many logs, making the task of troubleshooting or gaining business insights from these logs, a difficult one.

Many a time, there’s the issue of servers running on low disk space because of the very large log files on them.

Servers with huge log files create problems when the resizing of virtual machines needs to be done. Troubleshooting based on large files may take up a lot of time and valuable memory. The logrotate utility is extremely useful to solve all such problems. It helps in taking backups of log files on an hourly, daily, weekly, monthly or yearly basis with additional choice of log backup with compression. Also, file backups can be taken by setting a limit on the file size,

like 100MB, for instance. So, after the log file reaches a size of 100MB, the file will be rotated.

The synopsis is as follows:

logrotate [-dv] [-f|--force] [-s|--state file] config_file

Any number of configuration files can be given on the command line, and one file can include another config file. A simple logrotate configuration looks

like what’s shown below:

var/log/messages {

rotate 5

weekly

compress

olddir /var/log/backup/messages/

missingok

}

Here, every week, the /var/log/messages file will be compressed and backed up to the /var/log/backup/messages/ folder, and only five rotated log files will be kept around in the system.

Installing logrotate

Log rotation is a utility that comes preinstalled in Linux servers like Ubuntu, CentOS, Red Hat, etc. Check the folder at path /etc/logrotate.d.

If it is not installed, then you can install it manually by using the following commands.

For Ubuntu, type:

sudo apt-get install logrotate

For CentOS, type:

sudo yum install logrotate

Configuring logrotate

When logrotate runs, it reads its configuration files to decide where to find the log files that it needs to rotate, how often the files should be rotated

and how many archived logs to keep. There are primarily two ways to write a logrotate script and configure it to run every day, every week, every month, and so on.

1. Configuration can be done in the default global configuration file /etc/logrotate.conf; or

2. By creating separate configuration files in the directory/etc/logrotate.d/ for each service/application.

Personally, I think the latter option is a better way to write logrotate configurations, as each configuration is separate from the other. Some distributions use a variation and scripts that run logrotate daily can be found at any of the following paths:

/etc/cron.daily/logrotate

/etc/cron.daily/logrotate.cron

/etc/logrotate.d/

One logrotate configuration (filename: Tomcat) file given below will be used to compress and take daily backups of all Tomcat .log files and catalina.out files

and after rotation, the Tomcat service will get restarted. With this configuration it is clear that multiple log file backups can be taken in one go.

Multiple log files should be delimited with space.

/home/tomcat/logs/*.log /home/tomcat/logs/catalina.out {

missingok

copytruncate

daily

compress

rotate 10

olddir /var/log/backup/tomcat/

sharedscripts

postrotate

/home/tomcat/bin/catalins.sh restart > /dev/null

endscript

}

To check if the configuration is functioning properly, the command given below with the –v option can be used. Option -v means ‘verbose’ so that we can

view the progress made by the logrotate utility.

logrotate -dv /etc/logrotate.d/tomcat

Logrotate options

-d, –debug In debug mode, no changes will be made to the logs or to the logrotate state file.

-f, –force

This instructs logrotate to force the rotation, which is necessary as per logrotate: this is useful after adding new entries to a config file.

-s, –state <statefile>

Tells logrotate to use an alternate state file. This is useful if logrotate is being run by a different user for various sets of log files. The default state

file is /var/lib/logrotate.status.

-m, –mail <command> Tells logrotate which command to use when mailing logs. This command should accept two arguments: 1) the subject of the message, and

2) the recipient. The command must then read a message on standard input and mail it to the recipient. The default mail command is /bin/mail -s.

v, –verbose Turns on verbose mode.

The types of directives

Given below are some useful directives that can be included in the logrotate configuration file.

Missingok: Continues executing the next configuration in the file even if the log file is missing, instead of throwing an error.

nomissingok: Throws an error if the log file is missing.

compress: Compresses the log file in the .tar.gz format. The file can compress in another format using the compresscmd directive.

compresscmd: Specifies the command to use for log file compression.

compressext: Specifies the extension to use on the compressed log file. Only applicable if the compress option is enabled during configuration.

copy: Makes a copy of the log file but it does not make any modification in the original file. It is just like taking a snapshot of the log file.

copytruncate: Copies the original file content and then truncates it. This is useful when some processes are writing to the log file and can’t be stopped.

dateext: Adds a date extension (default YYYYMMDD), to back up the log file. Also see nodateext.

dateformat format_string: Specifies the extension for dateext. Only %Y %m %d and %s specifiers are allowed.

Ifempty: Rotates the log file even if it is empty. Also see notifempty.

olddir <directory>: Rotated log files get moved in the specified directory. Overrides noolddir.

sharedscripts: This says that postscript will run once for multiple configuration files having the same log directory. For example, the directory structure /home/tomcat/logs/*.log is the same for all log files placed in the logs folder, and in this case, postscript will run only once.

postscripts: This runs whenever a log is rotated in the configuration file specified block. The number of postscript executions for logs placed in the same directory can be overridden with sharedscripts directives.

Directives are also related to the intervals at which log files are rotated. They tell logrotate how often the log files should be rotated. The available options are:

1. Hourly (copy the file /etc/cron.daily/logrotate into the /etc/cron.hourly/ directory)

2. Daily

3. Weekly

4. Monthly

5. Yearly

Log files may also be rotated on the basis of file size. We can instruct logrotate to rotate files when the size of the file is greater than, let’s say, 100KB, 100MB, 10GB, etc.

Some directives tell logrotate what number of rotated files to keep before deleting the old ones. In the following example, it will keep four rotated log files.

rotate 4

You can also use directives to remove rotated logs that are older than X number of days. The age is only checked if the log file is to be rotated.

The files are mailed, instead of being deleted, to the configured address if maillast and mail are configured.

One can get the full list of commands used in logrotate configuration files by checking the man page:

man logrotate

Logrotate is one of the best utilities available in the Linux OS. It is ideal to take backups of applications, servers or any logs.

Related Questions

Navigate

• Browse (All)
• Browse S
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.