[rsyslog] Development of failsafe disk based queue

[Rainer Gerhards]

David,

On Wed, 2008-10-01 at 07:10 -0700, david at lang.hm wrote:
> On Wed, 1 Oct 2008, Rainer Gerhards wrote:
> 
> > David (Ecker ;)),
> >
> > I'd appreciate if you let me know your results after you have tested
> > with $..QueueCheckpointInterval 1
> 
> also, try mounting the filesystem -o sync
> 
> this should make everything on it a sync write without Rainer having to 
> change his code (won't work for other situations, but will definantly work 
> for a test)

I after all this discussion I will take a few minutes to see if I can
capture results in code. There are two things and I guess you have an
opinion on that.

First of all, the doc I have on fsync() is a bit cryptic. It says:

----
fsync()  transfers  ("flushes")  all  modified in-core data of (i.e.,
modified buffer cache pages for) the file referred to by the file
descriptor fd to  the disk  device (or other permanent storage device)
where that file resides.  The call blocks until the device reports that
the transfer has completed.  It also flushes  metadata information
associated with the file (see stat(2)).
----

so it looks like fsync() is sufficient to flush both the data as well as
the size information (so that data is not only present but can actually
be read ;). But in the next paragraph, the man page says:

----
Calling  fsync()  does  not necessarily ensure that the entry in the
directory containing the file has also reached disk.  For that an
explicit fsync() on  a file descriptor for the directory is also needed.
----

which sound much like the opposite. I know there are many flavours of
*nix and even though this is a POSIX call, things may be different in
different OS...

Do you have an opinion of we can assume a call to fsync() on the
descriptor in questions solves the issue? This would probably avoid the
need to open with always synced writes and thus could offer a somewhat
better performance.

Also, the thought about O_NOATIME looks interesting. For internal queue
files, I think the last access time is of very limited interest, so it
would probably be useful to always specify this option (at least by
default).

Feedback appreciated.
Rainer