[rsyslog] how to best rotate files

[david at lang.hm]

On Mon, 15 Sep 2008, Rainer Gerhards wrote:

> This, too, is quite old, to-be-replaced on demand functionality ...
> without any demand for replacement up to now ;)
>
> Do you know this guide:
> http://www.rsyslog.com/doc-log_rotation_fix_size.html
>
> The shell script could spawn off an async process, but I have never
> tried this...
>
> Also <inline>:
> On Sun, 2008-09-14 at 23:13 -0700, david at lang.hm wrote:
>> looking through the rsyslog.conf man page I see ways to tell rsyslogd to
>> run a command when a file is over a given size, but I see no examples or
>> details of what that command can/should do.
>>
>> what I want to do is to move the file out of the way, kick rsyslogd so
>> that it starts writing to a new file, then compress the old file (which
>> can take a significant amount of time)
>>
>> traditionally I have rotated syslogs by doing a mv of the logfile followed
>> by a kill -HUP of the syslog process (usually out of crontab)
>>
>> In my testing of rsyslog I have been doing this, but once in a while
>> rsyslog doesn't release the file it's writing and start a new one, instead
>> it keeps writing to the old file.
>
> This very much smells like a bug. Can you elaborate on what you did?

rsyslogd 3.18.3 on one machine with the config file

$ModLoad immark
$MarkMessagePeriod 1200
$ModLoad imuxsock
$ModLoad imudp
$ModLoad imklog
$UDPServerRun 514
$AllowedSender UDP, 127.0.0.1, 192.0.0.0/8
$umask 0000
$FileGroup root
$FileOwner root
$template TraditionalFormat,"%timegenerated% %HOSTNAME% 
%syslogtag%%msg:::drop-last-lf%\n"
*.*     /var/log/messages;TraditionalFormat      # log to a file in the 
traditional format

tcpreplay on a second machine spewing log entries at it at a rate of 
~30,000 logs/sec

crontab running the following script every 5 min

#!/bin/sh
#

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

umask 022
year=`date +%Y`
month=`date +%m`
day=`date +%d`
fdate=`date +%Y%m%d.%H%M`
logroot=/var/log
logroll=$logroot/oldlogs

cd $logroot
mkdir -p $logroll/$year/$month/$day >/dev/null 2>/dev/null
mv messages $logroll/$year/$month/$day/messages.$fdate
pkill -HUP rsyslogd
2>/dev/null &
gzip -9 $logroll/$year/$month/$day/messages.$fdate


I get a couple of rotations sucessfully (with ~22,000 logs/sec in the 
files), but then rsyslog seems to hang.
lsof show it with the file open that was moved (and later deleted by 
gzip).
I need to do a kill -9 of rsyslogd to get it to recover.

David Lang

>>
>> so this means that my command can't spawn a background task that does a mv
>> of the file followed by a kill -HUP of rsyslogd.
>>
>> what should I be doing?
>>
>> David Lang
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>