[rsyslog] Development of failsafe disk based queue

[David Ecker]

Hi Rainer,

the assert error seemed to be fixed in HEAD.

Mounting the ext3 filesystem with (noatime,sync,dirsync,rw) seemed to
work a lot better.

/etc/rsyslog.conf
----------------
$WorkDirectory /rsyslog/
$ActionQueueFileName buffer
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType Disk
$ActionQueueMaxFileSize 100m
$ActionQueueSize 10000000
$ActionQueueCheckpointInterval 1
$ActionResumeRetryCount -1
*.* @@10.8.0.1:514
-----------------

I found out that the queue was limited to 1000 elements if you do not
define ActionQueueSize inside the config.

One difference was that version 3.18.3 actually created one file per msg
if I did set ActionQueueCheckpointInterval  to 1. Right now only one
file in addition to the .qi file is created containing all messages. I
haven't waited to see what will happen if I reach 100MB on the data file.

If I shutdown rsyslog normally some status information seems to be
written to the .qi file (508 bytes -> 1024 bytes). After restarting the
qi file actually shrinks to 508 bytes again.

If I kill rsyslogd (SIGKILL) the qi file is not updated with this
information (as expected). But it looks like the queue is still working
correctly after restarting rsyslogd at least it doesn't invalidate the
queue or loosing messages.

Turning the system off immediatly seems to be working most of the times.
After 11 tries the last one failed. It actually looked like that I
turned the system off during a write.

bye
David Ecker

Rainer Gerhards schrieb:
> Please use the version from git. I didn't realize that the bug affects
> normal operations, but obviously it does. This is fixed and I'll see
> that I release 3.21.6 ASAP, but I am not sure if I manage to do this
> today.
>
> Gitweb available at http://git.adiscon.com
>
> Rainer
>