[rsyslog] Development of failsafe disk based queue

[Rainer Gerhards]

On Wed, 2008-10-01 at 06:24 -0700, david at lang.hm wrote:

> one possible thing is tht if the write has not completed then the system 
> sending you the logs has not received confirmation that you have the log 
> yet, so they are the ones responsible for it.

I concur. But as I understood the scenario here, the log messages are
emited from a process *inside* the failing machine. So that process
fails, too, and we do not have any interim that has a copy of the data.
So if it is not persisted to disk, it is lost. Anyhow, this requirement
has been relaxed in later posting.

Also note that I was just thinking about the physical layer, considering
a single physical write - far away from rsyslog.

> it's only after you acknowledge the message (via relp or equivalent) that 
> you are required to not loose the log message.

I concur and this is how RELP handles this. Well, actually I think there
currently is a very slight (but still existing) window of exposure. I
think RELP acks when the message is submited to the queue engine. That
does not necessarily mean the message is already present on disk. Also,
I think some mild duplication of messages may happen with RELP in a
power fail scenario. It is not doing a two-phase commit, thought it
tries very hard to get a perfect understanding of what is written and
what not. I could check this with spec/code, but I think this is not
justified at this point in time ;)

> 
> with ext3 and data=journaled you can do this. the writes to the journal 
> are done by the filesystem in such a way that they can be considered 
> atomic (either they happen and are reliable, or they can be treated as if 
> they never happened), once that write (or rather the sync on that write) 
> has completed the data is safe. the filesystem will later modify the 
> actual sector on disk, but it jumps through hoops to make that safe from 
> power outages.
> 

sure, definitely. And I assume a sync'ed write will return only after
all this has happened. So journaled ext3 should be able to solve the
problem described here.

Rainer