[rsyslog] first time using rsyslog - some questions

seth vidal skvidal at linux.duke.edu
Tue Jul 17 14:28:24 CEST 2007


On Tue, 2007-07-17 at 08:15 +0200, Rainer Gerhards wrote:

> Great to have someone experienced with syslog-ng on the list. I really
> like to make rsyslog the best syslogd ever and it is most interesting to
> learn what syslog-ng users find the most important features. In my point
> of view, we have some things syslog-ng can not do, but others where
> syslog-ng is ahead. I'd like to change that, but in order of importance
> to the user ;)

Great! Featurewise, right now, rsyslog has a lot of the pieces in the
right place to be a great syslog server. My primary uses of syslog-ng
was:
1. large scale central file server (500ish systems logging everything
centrally)
2. ssl-encrypting/tcp syslog server for communicating over wans.
3. invoking sec for specialized event behavior



> It's on the todo list, most probably to be implemented this week. All
> interested please see:
> 
> https://sourceforge.net/tracker/index.php?func=detail&aid=1755143&group_
> id=123448&atid=696552
> 

Great to hear! Thanks.

One of the use cases for me is:

/var/log/hosts/%HOSTNAME%/%YEAR%/%MONTH%/%DAY%/messages
/var/log/hosts/%HOSTNAME%/%YEAR%/%MONTH%/%DAY%/secure.log
/var/log/hosts/%HOSTNAME%/%YEAR%/%MONTH%/%DAY%/kern.log
etc

so that for log cleanup I can simply do:
tmpwatch -f 720 /var/log/hosts/

and that removes all logs older than 30 days.

No need to run logrotate, no need to hup the server. It also makes logs
rsyncable b/c the filenames/path change as they progress. This lets me
rsync my logs to another box, process them for results/reports and not
disrupt or bog down my primary syslog server.

So if automatically-creating-dirs is made possible then I can do the
above on my central syslog server.
 
> > 2. related to 1 is there a way to set specific file ownerships not
> just
> > modes on the logs? I'd like to set the logs to be owned by
> > root:log_readers and to be 640, along with all directories that it
> > autocreates. That way no one else can look at the logs and I don't
> have
> > to do anything hurky like 'sg' before running rsyslog.
> 
> Not yet on the feature list, but definitely interesting. I've filed a
> new feature request:
> 
> https://sourceforge.net/tracker/index.php?func=detail&aid=1755276&group_
> id=123448&atid=696555

great! monitoring that rfe now, thanks.


> I've already some ideas for a new log file format. I won't elaborate
> right now, because I'd like to iron out 1.16.1 this morning and that's a
> priority. However, the log file will still be quite different from
> syslog-ng, following a more simple concept (at least I think so). I
> searched the web a lot and found numerous questions on the syslog-ng log
> file format. So I think for the next version of rsyslogs format, it
> should be very easy to grasp yet powerful. And, yes, I agree the current
> way of doing things is not optimal. OK, already written more than I
> wanted: my plan is to get feature-complete, then restructure the code to
> be finally modular (I inherited a non-modular design form syslogd) and
> then define output and input plugins. In the course of that, you'll see
> config file changes. I am thinking along these lines:
> 
> (for a listener)
> 
> $protocol tcp
> $port 514
> $RepeatedMsgReduction on # already in 1.16.1
> $EscapeControlCharactersOnReceive off # already in 1.16.1
> $listen 127.0.0.1
> 
> Similar things for the output side...
> 
> Looking forward to more comments.

What would you think of a config file format that's a bit closer to a
language? So you could do more case-like statements for what to get
logged where? for example:

my_emerg_output = /var/log/mail/emergency
my_generic_output = |/usr/bin/keep_count_of_messages
my_normal_logs = /var/log/mail/normal

if facility == 'mail':
  if priority == 'emerg':
     output == my_emerg_output
  elif priority < 'emerg' and priority > 'debug':
     filter_out(host='localhost')
     output == my_generic_output, my_normal_logs
  else:
      output == /dev/null

So this would mean:
 if the facility is mail and priority emerg then send it to my mail
emergency file
 if the facility is mail and the priority is between emerg and debug
then remove all logs where the sending host is 'localhost' and then send
it to my generic output AND to my normal logs
 otherwise, dump the output to /dev/null.


Would something like that be reasonable? Alternatively, can you tell me
a better way to do something like the above with what you have in mind
for the config file? I'm really not wed to any one format - I just like
it when config file examples are easy to put together and parse.


For additional feature sets:
Something that syslog-ng cannot do but I've always wanted a syslog
daemon to do is store-and-forward remote logging and/or failover remote
logging.

ie:
If you're using tcp-based logging from client to server you can
determine that the server is no longer accepting your messages from the
client. In that event, have the client start packing away all of the log
entries it would normally send to the server into a tmpfile some place.
When the server comes back push all of the logs over to it from earliest
to latest. Yes, the logs will be out of order but that's the least of
the worries, at least you have the data remotely. Setting a limit on the
size of the tmpfile makes sense to me and when that limit is hit start
discarding the EARLIER log entries for the later ones.

As an option related to this. Allow a remote log server configuration
option to include multiple hostnames/destinations in decreasing
priority.
So as long as the first one is up, you log to it, when it stops being up
you log to the next one in the list and so on and so on.

Thanks,
-sv





More information about the rsyslog mailing list