[rsyslog] first time using rsyslog - some questions

[Rainer Gerhards]

Hi,

I finished 1.17.0 (too much went in it for 1.16.1 ;)), so I finally have
time for an intelligent answer ;) See inline below...

On Tue, 2007-07-17 at 08:28 -0400, seth vidal wrote:
> On Tue, 2007-07-17 at 08:15 +0200, Rainer Gerhards wrote:
> 
> > Great to have someone experienced with syslog-ng on the list. I really
> > like to make rsyslog the best syslogd ever and it is most interesting to
> > learn what syslog-ng users find the most important features. In my point
> > of view, we have some things syslog-ng can not do, but others where
> > syslog-ng is ahead. I'd like to change that, but in order of importance
> > to the user ;)
> 
> Great! Featurewise, right now, rsyslog has a lot of the pieces in the
> right place to be a great syslog server. My primary uses of syslog-ng
> was:
> 1. large scale central file server (500ish systems logging everything
> centrally)
> 2. ssl-encrypting/tcp syslog server for communicating over wans.
> 3. invoking sec for specialized event behavior
> 
That sounds great. I would be really interested in your experience with
1.17.0 in this environment (now is the time to try to break it,
feature-wise its more or less ready for fedora 8 ;)).

> 
> 
> > It's on the todo list, most probably to be implemented this week. All
> > interested please see:
> > 
> > https://sourceforge.net/tracker/index.php?func=detail&aid=1755143&group_
> > id=123448&atid=696552
> > 
> 
> Great to hear! Thanks.

I had a good time today, its in.

> 
> One of the use cases for me is:
> 
> /var/log/hosts/%HOSTNAME%/%YEAR%/%MONTH%/%DAY%/messages
> /var/log/hosts/%HOSTNAME%/%YEAR%/%MONTH%/%DAY%/secure.log
> /var/log/hosts/%HOSTNAME%/%YEAR%/%MONTH%/%DAY%/kern.log
> etc
> 
This is a sample of what you can do now:

$umask 0000 # let default umask not get into our way
$CreateDirs on
$DynaCacheFileSize 50 # depending on number of hosts
$FileOwner whoever
$FileGroup loggroup
$FileCreateMode 0600
$DirOwner whoever
$DirGroup loggroup
$DirCreateMode 0600
$FailOnChownFailure off # its more important for us that logs be written
$template file-msgs,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY
%/messages
$RepeatedMessageReduction on # or off?
*.* -?file-msgs

It is important to keep in mind that all parameters keep in effect until
they are either overwritten (by a new directive) or
$ResetConfigVariables is specified.

> "
$
> so that for log cleanup I can simply do:
> tmpwatch -f 720 /var/log/hosts/
> 
> and that removes all logs older than 30 days.
> 
> No need to run logrotate, no need to hup the server. It also makes logs
> rsyncable b/c the filenames/path change as they progress. This lets me
> rsync my logs to another box, process them for results/reports and not
> disrupt or bog down my primary syslog server.
> 
> So if automatically-creating-dirs is made possible then I can do the
> above on my central syslog server.

Please check out - it's brand-new code, so probably some things go wrong
at some place (of course, not in lab ;)).

[snip]

> > I've already some ideas for a new log file format. I won't elaborate
> > right now, because I'd like to iron out 1.16.1 this morning and that's a
> > priority. However, the log file will still be quite different from
> > syslog-ng, following a more simple concept (at least I think so). I
> > searched the web a lot and found numerous questions on the syslog-ng log
> > file format. So I think for the next version of rsyslogs format, it
> > should be very easy to grasp yet powerful. And, yes, I agree the current
> > way of doing things is not optimal. OK, already written more than I
> > wanted: my plan is to get feature-complete, then restructure the code to
> > be finally modular (I inherited a non-modular design form syslogd) and
> > then define output and input plugins. In the course of that, you'll see
> > config file changes. I am thinking along these lines:
> > 
[snip]

> What would you think of a config file format that's a bit closer to a
> language? So you could do more case-like statements for what to get
> logged where? for example:
> 
> my_emerg_output = /var/log/mail/emergency
> my_generic_output = |/usr/bin/keep_count_of_messages
> my_normal_logs = /var/log/mail/normal
> 
> if facility == 'mail':
>   if priority == 'emerg':
>      output == my_emerg_output
>   elif priority < 'emerg' and priority > 'debug':
>      filter_out(host='localhost')
>      output == my_generic_output, my_normal_logs
>   else:
>       output == /dev/null
> 
> So this would mean:
>  if the facility is mail and priority emerg then send it to my mail
> emergency file
>  if the facility is mail and the priority is between emerg and debug
> then remove all logs where the sending host is 'localhost' and then send
> it to my generic output AND to my normal logs
>  otherwise, dump the output to /dev/null.
> 
> 
> Would something like that be reasonable? Alternatively, can you tell me
> a better way to do something like the above with what you have in mind
> for the config file? I'm really not wed to any one format - I just like
> it when config file examples are easy to put together and parse.
> 

I have to admit I have not yet thought about the filtering side. The
modules are input - filter - output. Whatever it is, it won't happen for
2.0.0, which shall be the result of the current development branch in
the not so distant future (let's say 3 to 5 weeks from now). I'll
consider your recommendation when I think about a new way to handle
filters. But no promise ;)

> 
> For additional feature sets:
> Something that syslog-ng cannot do but I've always wanted a syslog
> daemon to do is store-and-forward remote logging and/or failover remote
> logging.
> 
> ie:
> If you're using tcp-based logging from client to server you can
> determine that the server is no longer accepting your messages from the
> client. In that event, have the client start packing away all of the log
> entries it would normally send to the server into a tmpfile some place.
> When the server comes back push all of the logs over to it from earliest
> to latest. Yes, the logs will be out of order but that's the least of
> the worries, at least you have the data remotely. Setting a limit on the
> size of the tmpfile makes sense to me and when that limit is hit start
> discarding the EARLIER log entries for the later ones.

This *is* on the upcoming feature list. But to implement it, a number of
core changes must be applied first, including a change of the threading
model. I already have a rough idea on most of these things, so I
wouldn't outrule that the whole thing materializes this year (but again,
no promise ;)).
> 
> As an option related to this. Allow a remote log server configuration
> option to include multiple hostnames/destinations in decreasing
> priority.
> So as long as the first one is up, you log to it, when it stops being up
> you log to the next one in the list and so on and so on.

That sounds fairly easy. Let's see if I can fit that in without touching
too much code.

I've added a tracker:

http://sourceforge.net/tracker/index.php?func=detail&aid=1755530&group_id=123448&atid=696555

Thanks for the comments,
Rainer