[rsyslog] omfile redesign / $OutChannel will go away!

[Rainer Gerhards]

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, June 30, 2009 12:36 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] omfile redesign / $OutChannel will go away!
> 
> On Tue, 30 Jun 2009, Sergei Zhirikov wrote:
> 
> > Rainer Gerhards wrote:
> >> Hi all,
> >>
> >> The most prominent use case is to size-limit files. $OutChannel provides
> the
> >> ability to execute a command when a certain file size is reached. The
> command
> >> may then rotate the file. After command execution, omfile checks the
file
> >> size again. If it still is above the configured limit, this file will be
> >> disabled (no longer be written to until restart).
> >>
> >> If you use this for size-limiting, please let me know how you do it.
Please
> >> also let me know if you would like to see things handled differently.
> >>
> >> Looking forward to your feedback.
> >>
> >> Thanks,
> >> Rainer
> >>
> >
> > Hi Rainer,
> >
> > I do use $OutChannel for size-limiting and it would be a disappointment
> > to me if that feature disappeared. As a matter of fact, it was the main
> > reason why I chose to use rsyslog in the first place.
> >
> > I find the "classic" way of using cron+logrotate+SIGHUP quite ugly and
> > in some cases unacceptable. I have a small home gateway with read-only
> > main file system, so all the log files are written into a tmpfs mount.
> > It happened a couple of times that some misconfigured or malfunctioning
> > software produced a lot of logging and filled in most of the available
> > RAM before the next scheduled run of logrotate. In this situation the
> > size limiting feature of rsyslog is exactly what I need in order to be
> > able to avoid such this kind of problems.
> 
> this is a very legitimate use. I suspect the biggest issue is how to give
> you this capability without slowing down the people who don't use it.
> 
> I wonder if rainerscript could be extended to where it could be used to do
> this?
> 
> something like:
> 
> for each message logged, add the length of the message to a counter, and
> have an if statement that calls the external program to roll the log and
> reset the counter.

I probably should have phrased the initial question less agressive, but than
I'd probably not received any feedback. From the feedback I received so far,
there seem to be two use cases:

a) circular logs
b) stop logging when file hits max size (was that really what this post
described?)

In any case, these two seem to be the only things that are done with
outchannel. So when $outchannel goes away (and it will at some time), the
"regular" file writer will have options to support these use cases - probably
more elegant than before. For example, for case a), circular logs, I do not
need to involve an external process, so I can remove that feature without
harm, as long as I replace it with something suitable (and think that omfile
now uses the stream writer, which *already does* circular "logging" for disk
queues).

So functionality in actual need will not go away, as long as people tell what
they need.

If I look at the b) case, I suspect that this is not even the intent of the
original poster. I think he would be much better served with a circular log
that records as many log records as fit into a finite, operator-specified log
file size. Or am I wrong here? (If nobody tells me I am wrong, I'll probably
remove the b) use case in the long term!).

Rainer