From rgerhards at hq.adiscon.com Tue Jul 1 15:36:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 1 Jul 2008 15:36:11 +0200 Subject: [rsyslog] rsyslog 3.19.8 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092F2@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.8, a new release of the development branch. Most importantly, it adds a bugfix for TLS mode. A TLS session was lost if the OS returned a retry request via the API. This happened frequently on some platforms. The situation is now properly handled. Feature-wise error number have been added to most rsyslog messages, making it easier to troubleshoot problems. There are also some other minor improvements and bug fixes. This is a recommended update for all devel branch users. Change Log: http://www.rsyslog.com/Article247.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-116.phtml I hope this release is useful. Feedback is appreciated. Rainer Gerhards From mycleanjunk at gmail.com Wed Jul 2 05:16:41 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Tue, 1 Jul 2008 20:16:41 -0700 Subject: [rsyslog] configuration file question and error message question Message-ID: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> Hi, I am using rsyslog 3.16.2. When I specify the script of command to issue when a log reaches its max size, how do I specify the command with parameters? This is what I did: $outchannel ch_common,/var/log/messages/1048675,\ mv -f /var/log/messages /var/log/messages.0 Do I use a quotation mark or a single quote or something else? Your help says to specify a script but does that line except a single word for the script, meaning it can not take parameters? To clarify, if I have a script that is named rotate and I want to use it for multiple logs, I would like to say "rotate /var/log/message". Is this possible? I am getting this error message: rsyslogd: omfwd.c:318: doTryResume: Assertion `0' failed. What does this mean? Thanks for your help!! I really like how rsyslogd has a lot of configurable properties. Scott From rgerhards at hq.adiscon.com Wed Jul 2 08:28:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 2 Jul 2008 08:28:59 +0200 Subject: [rsyslog] configuration file question and error message question In-Reply-To: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> References: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092F7@grfint2.intern.adiscon.com> Hi, > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Wednesday, July 02, 2008 5:17 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] configuration file question and error message > question > > Hi, > > I am using rsyslog 3.16.2. > > When I specify the script of command to issue when a log reaches its > max size, how do I specify the command with parameters? The command can have EXACTLY one parameter. > This is what I > did: > > $outchannel ch_common,/var/log/messages/1048675,\ > mv -f /var/log/messages /var/log/messages.0 > > Do I use a quotation mark or a single quote or something else? Your > help says to specify a script but does that line except a single word > for the script, meaning it can not take parameters? To clarify, if I > have a script that is named rotate and I want to use it for multiple > logs, I would like to say "rotate /var/log/message". Is this possible? Yes, but you can not specify more than one parameter. The outchannel code is scheduled to be superseded by something new that works along the lines of the regular log file (just with an additional parameter). However, until this is done, we need to live with the current restriction. > > > I am getting this error message: > rsyslogd: omfwd.c:318: doTryResume: Assertion `0' failed. > > What does this mean? Something goes really wrong ;) I have checked that assertion and there seems to be some problem with the retry log. Can you reproduce it? If so, can you provide a full debug log (run interactively with -dn option)? > > Thanks for your help!! I really like how rsyslogd has a lot of > configurable properties. :) Rainer > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Wed Jul 2 10:15:07 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 01:15:07 -0700 Subject: [rsyslog] rsyslog threads questions Message-ID: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Hi, I have 3.16.2 which was recently released. I see that under certain conditions rsyslogd spawns a lot of threads: 5949 root 11216 S rsyslogd 5950 root 11216 S rsyslogd 5951 root 11216 S rsyslogd 5952 root 11216 S rsyslogd 5953 root 11216 S rsyslogd 5954 root 11216 S rsyslogd 5985 root Z [rsyslogd] 6445 root Z [rsyslogd] I had to kill the rsyslogd and restart it. The first invocation had a pid of 219 before it had to be killed. The second invocation of pid which you see above starts with 5949. The difference is the amount of zombie threads that were invoked by rsyslogd before I had to kill the first invocation of it. The question is under what conditions does rsyslogd spawn a new thread/process and why was it a zombie? I am running rsyslogd in an embedded environment and not a regular laptop/desktop. In addition, I am using busybox and I believe the syslog buffer size is set to something very low or perhaps none at all. Would this be a factor? Furthermore, I ran rsyslogd with -c3 and also without -c3 and both cases happen. Are these issues already known and fixed in a later version? Sorry, if I am asking the same questions or have the same issues as previous people but without the ability to search (or at least, I don't know how to) the archive, I don't know if my problem/questions has already been seen and/or resolved. Thank you very much for your support. Scott From rgerhards at hq.adiscon.com Wed Jul 2 10:27:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 02 Jul 2008 10:27:18 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Message-ID: <1214987239.7184.13.camel@rgf9dev.intern.adiscon.com> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > Hi, > > I have 3.16.2 which was recently released. I see that under certain > conditions rsyslogd spawns a lot of threads: > 5949 root 11216 S rsyslogd > 5950 root 11216 S rsyslogd > 5951 root 11216 S rsyslogd > 5952 root 11216 S rsyslogd > 5953 root 11216 S rsyslogd > 5954 root 11216 S rsyslogd > 5985 root Z [rsyslogd] > 6445 root Z [rsyslogd] > > I had to kill the rsyslogd and restart it. The first invocation had a > pid of 219 before it had to be killed. The second invocation of pid > which you see above starts with 5949. The difference is the amount of > zombie threads that were invoked by rsyslogd before I had to kill the > first invocation of it. I have no explanation yet for the zombies. They should not happen and so far I have never seen them. We may need to go through a debug log (which will become very large) to find out what's going on. > The question is under what conditions does rsyslogd spawn a new > thread/process and why was it a zombie? Unfortunately, there is no quick answer. A quick one may be: when it needs them, based on queue watermark settings and based on you configuration. But to really understand it, you need to read this doc: http://www.rsyslog.com/doc-queues.html The doc also describes all the knobs that you can use to control thread creation. There are many ;) > I am running rsyslogd in an > embedded environment and not a regular laptop/desktop. Interesting use case... > In addition, I > am using busybox and I believe the syslog buffer size is set to what do yo mean by "syslog buffer size"? The length of a receive buffer? It is 2K, thus single messages up to 2K are supported. It can be changed by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) support only up to 1K. > something very low or perhaps none at all. Would this be a factor? > Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > cases happen. The compatibility modes do not affect queue operation. > Are these issues already known and fixed in a later version? Sorry, if > I am asking the same questions or have the same issues as previous > people but without the ability to search (or at least, I don't know > how to) the archive, I don't know if my problem/questions has already > been seen and/or resolved. If we need to find out about the zombies, we need to move on to the current devel version. So I would give that a try in any case. 3.16.2 will (most probably) be replaced by 3.18.0 (based on the current beta) next week. So I won't touch it any longer. Looking forward to your feedback, Rainer > > Thank you very much for your support. > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 2 11:07:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 2 Jul 2008 11:07:28 +0200 Subject: [rsyslog] rsyslog error message repository Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092FA@grfint2.intern.adiscon.com> Hi all, starting with 3.19.8, rsyslog finally offers specific error codes as part of the syslog tag. For example, rsyslogd-2040 means that a file could not be found. I have added these tags both to facilitate log parsing as well as easy troubleshooting. But a tag is only as good as the information that it helps to find. Consequently, I have started to describe error cases inside the knowledge base's event repository: http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog.html So far, there is only a limited set of messages available (to phrase it politely ;)), but I plan to increase it over time. Note that there is an interactive feature where questions to the message can directly be posted to the forum. I hope this is useful. If you run into an error message that is not-yet described, let us know and we'll add an entry. In the long term, the new knowledge base part should be able to solve most problems. Feedback is appreciated, Rainer From mycleanjunk at gmail.com Wed Jul 2 21:04:23 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 12:04:23 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Message-ID: <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> Hi Rainer, Thanks for your reply. Looking at the default settings (from the online help's configuration page), they are what I wanted. The main messages queue is set to fix sized array with 1 worker thread created at maximum and action queues are direct mode which according to the queue document page, means that there will not be a worker thread created. Is my understanding correct? If yes, how do I quickly check without using the -d option if the defaults are set correctly? Or what do I look for in the debug messages that gets printed out to ensure this? You also mentioned that version 3.18.0 is probably going to be released as the stable version next week. I see on the webpage there is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? Also, how come I did not get your reply in my email inbox? My account settings look correct. Thanks, Scott Phuong As for the syslog buffer size, that applies to syslogd and does not apply to rsyslog. My configuration files do not change the Action queue or Worker queue parameters at all. Looking at On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > Hi, > > I have 3.16.2 which was recently released. I see that under certain > conditions rsyslogd spawns a lot of threads: > 5949 root 11216 S rsyslogd > 5950 root 11216 S rsyslogd > 5951 root 11216 S rsyslogd > 5952 root 11216 S rsyslogd > 5953 root 11216 S rsyslogd > 5954 root 11216 S rsyslogd > 5985 root Z [rsyslogd] > 6445 root Z [rsyslogd] > > I had to kill the rsyslogd and restart it. The first invocation had a > pid of 219 before it had to be killed. The second invocation of pid > which you see above starts with 5949. The difference is the amount of > zombie threads that were invoked by rsyslogd before I had to kill the > first invocation of it. I have no explanation yet for the zombies. They should not happen and so far I have never seen them. We may need to go through a debug log (which will become very large) to find out what's going on. > The question is under what conditions does rsyslogd spawn a new > thread/process and why was it a zombie? Unfortunately, there is no quick answer. A quick one may be: when it needs them, based on queue watermark settings and based on you configuration. But to really understand it, you need to read this doc: http://www.rsyslog.com/doc-queues.html The doc also describes all the knobs that you can use to control thread creation. There are many ;) > I am running rsyslogd in an > embedded environment and not a regular laptop/desktop. Interesting use case... > In addition, I > am using busybox and I believe the syslog buffer size is set to what do yo mean by "syslog buffer size"? The length of a receive buffer? It is 2K, thus single messages up to 2K are supported. It can be changed by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) support only up to 1K. > something very low or perhaps none at all. Would this be a factor? > Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > cases happen. The compatibility modes do not affect queue operation. > Are these issues already known and fixed in a later version? Sorry, if > I am asking the same questions or have the same issues as previous > people but without the ability to search (or at least, I don't know > how to) the archive, I don't know if my problem/questions has already > been seen and/or resolved. If we need to find out about the zombies, we need to move on to the current devel version. So I would give that a try in any case. 3.16.2 will (most probably) be replaced by 3.18.0 (based on the current beta) next week. So I won't touch it any longer. Looking forward to your feedback, Rainer > > Thank you very much for your support. > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Thu Jul 3 08:48:02 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 23:48:02 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Message-ID: <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> Sorry for sending a lot of emails. I believe I have root-cause the issue in 3.17.5. It appears that when the rsyslog.conf file does something like this: emerg.* *;mytemplate The code executes wallmsg which does forks. This is how I got a lot of processes were being created in my scenario. Although one would hope that a system doesn't misbehave this awful, nonetheless it does happen. First, it seems that the emergency messages should go to all those that are logged in via whatever mechanism (i.e. telnet, console, stty, ssh, etc...), this message should appear on all those "screens". I see that it does not even if I just logged 1 message at this serverity. I believe this is how syslog described setting up a wall message and what it is. Is this a bug? Lastly, when the child of the fork exits, it does not appear to be removed from Linux and still continues to eat up memory and is reported as a zombie. It appears that when the workerthread has been "destroyed/deconstructed" does things clear up. I am not sure if this is a rsyslogd problem or a linux or gcc problem. Any ideas? Thanks, Scott On Wed, Jul 2, 2008 at 5:57 PM, Scott Phuong wrote: > Hi, > > I've attached four files. Two of which are debug dumps, one is the > conf file and the last one is a test case scenario that constantly > fails on my end. I hope this gives a little more information. > Furthermore, the dumps are from 3.17.5 which is the "closest" version > to 3.18.0 that I was able to find. > > Both failed scenarios occur when lots of messages were being flooded > to rsyslogd at a very fast rate (look at logtest.c) The > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads > that it took up so much memory that executing any command as simple as > 'ls -l' would not work from the command line. I think the number of > threads grew as much as the number of messages. In the latter > scenario, after killing logtest.c, it didn't look like the those > zombies threads went away until I did a CTRL+C to the rsyslogd which > was running in the foreground since I use the "-dn" option. > > This is on an embedded system that runs significantly slower than a > desktop or laptop so maybe it would be harder to reproduce on a > regular computer. I looked at all the parameters that I believe could > affect this and believe for the most part the defaults are more than > adequate. The main message queue never looked like it hit the high > water mark but it did hit the lower one. So, I don't think messages > were being dropped (not sure) or an overflow condition occurred. > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > compiled using GCC and the standard GNU C libraries version 3.4.5. > Rsyslog source code is cross-compiled using the following configure > line: > > ./configure --disable-zlib --disable-largefile > --enable-share=yes > --prefix=/ > --host=arm-unknown-linux-gnu > ac_cv_func_malloc_0_nonnull=yes > ac_cv_func_realloc_0_nonnull=yes > ac_cv_func_lstat_dereferences_slashed_symlink=yes > ac_cv_func_stat_empty_string_bug=no > enable_debug=no > enable_rtinst=no > > Lastly, the logtest was executed with just the "-s" parameter. It is a > simple C file that I came up with. > > I took a look at the debug messages and it does not appear that new > threads are created via calls to wtpStartWrkr in wtp.c. > > Any help I can bring to solve this issue, please let me know. I hope I > am not doing anything wrong here. > > Thanks, > > Scott >> >> On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong wrote: >>> Hi Rainer, >>> >>> Thanks for your reply. Looking at the default settings (from the >>> online help's configuration page), they are what I wanted. The main >>> messages queue is set to fix sized array with 1 worker thread created >>> at maximum and action queues are direct mode which according to the >>> queue document page, means that there will not be a worker thread >>> created. Is my understanding correct? If yes, how do I quickly check >>> without using the -d option if the defaults are set correctly? Or what >>> do I look for in the debug messages that gets printed out to ensure >>> this? >>> >>> You also mentioned that version 3.18.0 is probably going to be >>> released as the stable version next week. I see on the webpage there >>> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >>> >>> Also, how come I did not get your reply in my email inbox? My account >>> settings look correct. >>> >>> Thanks, >>> >>> Scott Phuong >>> >>> As for the syslog buffer size, that applies to syslogd and does not >>> apply to rsyslog. >>> >>> >>> >>> My configuration files do not change the Action queue or Worker queue >>> parameters at all. Looking at >>> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >>>> Hi, >>>> >>>> I have 3.16.2 which was recently released. I see that under certain >>>> conditions rsyslogd spawns a lot of threads: >>>> 5949 root 11216 S rsyslogd >>>> 5950 root 11216 S rsyslogd >>>> 5951 root 11216 S rsyslogd >>>> 5952 root 11216 S rsyslogd >>>> 5953 root 11216 S rsyslogd >>>> 5954 root 11216 S rsyslogd >>>> 5985 root Z [rsyslogd] >>>> 6445 root Z [rsyslogd] >>>> >>>> I had to kill the rsyslogd and restart it. The first invocation had a >>>> pid of 219 before it had to be killed. The second invocation of pid >>>> which you see above starts with 5949. The difference is the amount of >>>> zombie threads that were invoked by rsyslogd before I had to kill the >>>> first invocation of it. >>> >>> I have no explanation yet for the zombies. They should not happen and so >>> far I have never seen them. We may need to go through a debug log (which >>> will become very large) to find out what's going on. >>> >>>> The question is under what conditions does rsyslogd spawn a new >>>> thread/process and why was it a zombie? >>> >>> Unfortunately, there is no quick answer. A quick one may be: when it >>> needs them, based on queue watermark settings and based on you >>> configuration. But to really understand it, you need to read this doc: >>> >>> http://www.rsyslog.com/doc-queues.html >>> >>> The doc also describes all the knobs that you can use to control thread >>> creation. There are many ;) >>> >>>> I am running rsyslogd in an >>>> embedded environment and not a regular laptop/desktop. >>> >>> Interesting use case... >>> >>>> In addition, I >>>> am using busybox and I believe the syslog buffer size is set to >>> >>> what do yo mean by "syslog buffer size"? The length of a receive buffer? >>> It is 2K, thus single messages up to 2K are supported. It can be changed >>> by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) >>> support only up to 1K. >>> >>>> something very low or perhaps none at all. Would this be a factor? >>>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >>>> cases happen. >>> >>> The compatibility modes do not affect queue operation. >>> >>>> Are these issues already known and fixed in a later version? Sorry, if >>>> I am asking the same questions or have the same issues as previous >>>> people but without the ability to search (or at least, I don't know >>>> how to) the archive, I don't know if my problem/questions has already >>>> been seen and/or resolved. >>> >>> If we need to find out about the zombies, we need to move on to the >>> current devel version. So I would give that a try in any case. 3.16.2 >>> will (most probably) be replaced by 3.18.0 (based on the current beta) >>> next week. So I won't touch it any longer. >>> >>> Looking forward to your feedback, >>> Rainer >>> >>>> >>>> Thank you very much for your support. >>>> >>>> Scott >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > From rgerhards at hq.adiscon.com Thu Jul 3 08:59:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 3 Jul 2008 08:59:26 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com><17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309307@grfint2.intern.adiscon.com> Hi Scott, will reply in more detail later today, but one thing quickly. This is very good information. The wall code is ooooooooooold, stems back to sysklogd and has only been slightly modified. I have to admit it is not much on my testing radar. So I assume you found the bug. I think I'll need to re-design it. With the queue engine, forking off should not really be necessary. I need to see if that goes into the v3-stable soon or is kept at the devel tree. I am a bit hesitant to move it to stable, as it involves a good chance for additional bugs... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Thursday, July 03, 2008 8:48 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog threads questions > > Sorry for sending a lot of emails. I believe I have root-cause the > issue in 3.17.5. It appears that when the rsyslog.conf file does > something like this: > emerg.* *;mytemplate > > The code executes wallmsg which does forks. This is how I got a lot of > processes were being created in my scenario. Although one would hope > that a system doesn't misbehave this awful, nonetheless it does > happen. First, it seems that the emergency messages should go to all > those that are logged in via whatever mechanism (i.e. telnet, console, > stty, ssh, etc...), this message should appear on all those "screens". > I see that it does not even if I just logged 1 message at this > serverity. I believe this is how syslog described setting up a wall > message and what it is. Is this a bug? > > Lastly, when the child of the fork exits, it does not appear to be > removed from Linux and still continues to eat up memory and is > reported as a zombie. It appears that when the workerthread has been > "destroyed/deconstructed" does things clear up. I am not sure if this > is a rsyslogd problem or a linux or gcc problem. Any ideas? > > Thanks, > > Scott > > > > On Wed, Jul 2, 2008 at 5:57 PM, Scott Phuong > wrote: > > Hi, > > > > I've attached four files. Two of which are debug dumps, one is the > > conf file and the last one is a test case scenario that constantly > > fails on my end. I hope this gives a little more information. > > Furthermore, the dumps are from 3.17.5 which is the "closest" version > > to 3.18.0 that I was able to find. > > > > Both failed scenarios occur when lots of messages were being flooded > > to rsyslogd at a very fast rate (look at logtest.c) The > > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" > threads > > that it took up so much memory that executing any command as simple > as > > 'ls -l' would not work from the command line. I think the number of > > threads grew as much as the number of messages. In the latter > > scenario, after killing logtest.c, it didn't look like the those > > zombies threads went away until I did a CTRL+C to the rsyslogd which > > was running in the foreground since I use the "-dn" option. > > > > This is on an embedded system that runs significantly slower than a > > desktop or laptop so maybe it would be harder to reproduce on a > > regular computer. I looked at all the parameters that I believe could > > affect this and believe for the most part the defaults are more than > > adequate. The main message queue never looked like it hit the high > > water mark but it did hit the lower one. So, I don't think messages > > were being dropped (not sure) or an overflow condition occurred. > > > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > > compiled using GCC and the standard GNU C libraries version 3.4.5. > > Rsyslog source code is cross-compiled using the following configure > > line: > > > > ./configure --disable-zlib --disable-largefile > > --enable-share=yes > > --prefix=/ > > --host=arm-unknown-linux-gnu > > ac_cv_func_malloc_0_nonnull=yes > > ac_cv_func_realloc_0_nonnull=yes > > > ac_cv_func_lstat_dereferences_slashed_symlink=yes > > ac_cv_func_stat_empty_string_bug=no > > enable_debug=no > > enable_rtinst=no > > > > Lastly, the logtest was executed with just the "-s" parameter. It is > a > > simple C file that I came up with. > > > > I took a look at the debug messages and it does not appear that new > > threads are created via calls to wtpStartWrkr in wtp.c. > > > > Any help I can bring to solve this issue, please let me know. I hope > I > > am not doing anything wrong here. > > > > Thanks, > > > > Scott > >> > >> On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > wrote: > >>> Hi Rainer, > >>> > >>> Thanks for your reply. Looking at the default settings (from the > >>> online help's configuration page), they are what I wanted. The main > >>> messages queue is set to fix sized array with 1 worker thread > created > >>> at maximum and action queues are direct mode which according to the > >>> queue document page, means that there will not be a worker thread > >>> created. Is my understanding correct? If yes, how do I quickly > check > >>> without using the -d option if the defaults are set correctly? Or > what > >>> do I look for in the debug messages that gets printed out to ensure > >>> this? > >>> > >>> You also mentioned that version 3.18.0 is probably going to be > >>> released as the stable version next week. I see on the webpage > there > >>> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? > >>> > >>> Also, how come I did not get your reply in my email inbox? My > account > >>> settings look correct. > >>> > >>> Thanks, > >>> > >>> Scott Phuong > >>> > >>> As for the syslog buffer size, that applies to syslogd and does not > >>> apply to rsyslog. > >>> > >>> > >>> > >>> My configuration files do not change the Action queue or Worker > queue > >>> parameters at all. Looking at > >>> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >>>> Hi, > >>>> > >>>> I have 3.16.2 which was recently released. I see that under > certain > >>>> conditions rsyslogd spawns a lot of threads: > >>>> 5949 root 11216 S rsyslogd > >>>> 5950 root 11216 S rsyslogd > >>>> 5951 root 11216 S rsyslogd > >>>> 5952 root 11216 S rsyslogd > >>>> 5953 root 11216 S rsyslogd > >>>> 5954 root 11216 S rsyslogd > >>>> 5985 root Z [rsyslogd] > >>>> 6445 root Z [rsyslogd] > >>>> > >>>> I had to kill the rsyslogd and restart it. The first invocation > had a > >>>> pid of 219 before it had to be killed. The second invocation of > pid > >>>> which you see above starts with 5949. The difference is the amount > of > >>>> zombie threads that were invoked by rsyslogd before I had to kill > the > >>>> first invocation of it. > >>> > >>> I have no explanation yet for the zombies. They should not happen > and so > >>> far I have never seen them. We may need to go through a debug log > (which > >>> will become very large) to find out what's going on. > >>> > >>>> The question is under what conditions does rsyslogd spawn a new > >>>> thread/process and why was it a zombie? > >>> > >>> Unfortunately, there is no quick answer. A quick one may be: when > it > >>> needs them, based on queue watermark settings and based on you > >>> configuration. But to really understand it, you need to read this > doc: > >>> > >>> http://www.rsyslog.com/doc-queues.html > >>> > >>> The doc also describes all the knobs that you can use to control > thread > >>> creation. There are many ;) > >>> > >>>> I am running rsyslogd in an > >>>> embedded environment and not a regular laptop/desktop. > >>> > >>> Interesting use case... > >>> > >>>> In addition, I > >>>> am using busybox and I believe the syslog buffer size is set to > >>> > >>> what do yo mean by "syslog buffer size"? The length of a receive > buffer? > >>> It is 2K, thus single messages up to 2K are supported. It can be > changed > >>> by modifying the MAXLINE define. Note that stock syslogd (and > RFC3164) > >>> support only up to 1K. > >>> > >>>> something very low or perhaps none at all. Would this be a factor? > >>>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > >>>> cases happen. > >>> > >>> The compatibility modes do not affect queue operation. > >>> > >>>> Are these issues already known and fixed in a later version? > Sorry, if > >>>> I am asking the same questions or have the same issues as previous > >>>> people but without the ability to search (or at least, I don't > know > >>>> how to) the archive, I don't know if my problem/questions has > already > >>>> been seen and/or resolved. > >>> > >>> If we need to find out about the zombies, we need to move on to the > >>> current devel version. So I would give that a try in any case. > 3.16.2 > >>> will (most probably) be replaced by 3.18.0 (based on the current > beta) > >>> next week. So I won't touch it any longer. > >>> > >>> Looking forward to your feedback, > >>> Rainer > >>> > >>>> > >>>> Thank you very much for your support. > >>>> > >>>> Scott > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Thu Jul 3 02:57:39 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 17:57:39 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> Message-ID: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Hi, I've attached four files. Two of which are debug dumps, one is the conf file and the last one is a test case scenario that constantly fails on my end. I hope this gives a little more information. Furthermore, the dumps are from 3.17.5 which is the "closest" version to 3.18.0 that I was able to find. Both failed scenarios occur when lots of messages were being flooded to rsyslogd at a very fast rate (look at logtest.c) The my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads that it took up so much memory that executing any command as simple as 'ls -l' would not work from the command line. I think the number of threads grew as much as the number of messages. In the latter scenario, after killing logtest.c, it didn't look like the those zombies threads went away until I did a CTRL+C to the rsyslogd which was running in the foreground since I use the "-dn" option. This is on an embedded system that runs significantly slower than a desktop or laptop so maybe it would be harder to reproduce on a regular computer. I looked at all the parameters that I believe could affect this and believe for the most part the defaults are more than adequate. The main message queue never looked like it hit the high water mark but it did hit the lower one. So, I don't think messages were being dropped (not sure) or an overflow condition occurred. The processor is ARM-based and it is using Linux kernel 2.6.16.12 and compiled using GCC and the standard GNU C libraries version 3.4.5. Rsyslog source code is cross-compiled using the following configure line: ./configure --disable-zlib --disable-largefile --enable-share=yes --prefix=/ --host=arm-unknown-linux-gnu ac_cv_func_malloc_0_nonnull=yes ac_cv_func_realloc_0_nonnull=yes ac_cv_func_lstat_dereferences_slashed_symlink=yes ac_cv_func_stat_empty_string_bug=no enable_debug=no enable_rtinst=no Lastly, the logtest was executed with just the "-s" parameter. It is a simple C file that I came up with. I took a look at the debug messages and it does not appear that new threads are created via calls to wtpStartWrkr in wtp.c. Any help I can bring to solve this issue, please let me know. I hope I am not doing anything wrong here. Thanks, Scott > > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong wrote: >> Hi Rainer, >> >> Thanks for your reply. Looking at the default settings (from the >> online help's configuration page), they are what I wanted. The main >> messages queue is set to fix sized array with 1 worker thread created >> at maximum and action queues are direct mode which according to the >> queue document page, means that there will not be a worker thread >> created. Is my understanding correct? If yes, how do I quickly check >> without using the -d option if the defaults are set correctly? Or what >> do I look for in the debug messages that gets printed out to ensure >> this? >> >> You also mentioned that version 3.18.0 is probably going to be >> released as the stable version next week. I see on the webpage there >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >> >> Also, how come I did not get your reply in my email inbox? My account >> settings look correct. >> >> Thanks, >> >> Scott Phuong >> >> As for the syslog buffer size, that applies to syslogd and does not >> apply to rsyslog. >> >> >> >> My configuration files do not change the Action queue or Worker queue >> parameters at all. Looking at >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >>> Hi, >>> >>> I have 3.16.2 which was recently released. I see that under certain >>> conditions rsyslogd spawns a lot of threads: >>> 5949 root 11216 S rsyslogd >>> 5950 root 11216 S rsyslogd >>> 5951 root 11216 S rsyslogd >>> 5952 root 11216 S rsyslogd >>> 5953 root 11216 S rsyslogd >>> 5954 root 11216 S rsyslogd >>> 5985 root Z [rsyslogd] >>> 6445 root Z [rsyslogd] >>> >>> I had to kill the rsyslogd and restart it. The first invocation had a >>> pid of 219 before it had to be killed. The second invocation of pid >>> which you see above starts with 5949. The difference is the amount of >>> zombie threads that were invoked by rsyslogd before I had to kill the >>> first invocation of it. >> >> I have no explanation yet for the zombies. They should not happen and so >> far I have never seen them. We may need to go through a debug log (which >> will become very large) to find out what's going on. >> >>> The question is under what conditions does rsyslogd spawn a new >>> thread/process and why was it a zombie? >> >> Unfortunately, there is no quick answer. A quick one may be: when it >> needs them, based on queue watermark settings and based on you >> configuration. But to really understand it, you need to read this doc: >> >> http://www.rsyslog.com/doc-queues.html >> >> The doc also describes all the knobs that you can use to control thread >> creation. There are many ;) >> >>> I am running rsyslogd in an >>> embedded environment and not a regular laptop/desktop. >> >> Interesting use case... >> >>> In addition, I >>> am using busybox and I believe the syslog buffer size is set to >> >> what do yo mean by "syslog buffer size"? The length of a receive buffer? >> It is 2K, thus single messages up to 2K are supported. It can be changed >> by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) >> support only up to 1K. >> >>> something very low or perhaps none at all. Would this be a factor? >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >>> cases happen. >> >> The compatibility modes do not affect queue operation. >> >>> Are these issues already known and fixed in a later version? Sorry, if >>> I am asking the same questions or have the same issues as previous >>> people but without the ability to search (or at least, I don't know >>> how to) the archive, I don't know if my problem/questions has already >>> been seen and/or resolved. >> >> If we need to find out about the zombies, we need to move on to the >> current devel version. So I would give that a try in any case. 3.16.2 >> will (most probably) be replaced by 3.18.0 (based on the current beta) >> next week. So I won't touch it any longer. >> >> Looking forward to your feedback, >> Rainer >> >>> >>> Thank you very much for your support. >>> >>> Scott >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > From niallel at gmail.com Thu Jul 3 23:59:53 2008 From: niallel at gmail.com (niall el-assaad) Date: Thu, 3 Jul 2008 22:59:53 +0100 Subject: [rsyslog] Problem matching localhost Message-ID: Hi, I'm running 2.0.0-11 (version included with redhat 5.2) I want to filter all the messages from external syslog devices to one file and all messages from the localhost to another file. However even with the -x option turned on when a local service (such as crond) sends a message to the log the hostname is set to the domain name of the server. So I can't use the following to match: :HOSTNAME, isequal, "localhost" /var/log/messages :HOSTNAME, !isequal, "localhost" /var/log/externalsyslog I could replace "localhost" with "dnsname" to get it to work, but I would like a generic method that will work on all the syslog servers I have. Is there some switch that will cause rsyslog to report the local services as sending from localhost or 127.0.0.1 rather than the hostname of the localhost. thanks, niall From rgerhards at hq.adiscon.com Fri Jul 4 08:12:39 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 08:12:39 +0200 Subject: [rsyslog] Problem matching localhost In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30932A@grfint2.intern.adiscon.com> I replied to your forum post: http://kb.monitorware.com/post13018.html#p13018 I suggest we keep discussing it in the forum to reduce double work. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of niall el-assaad > Sent: Friday, July 04, 2008 12:00 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Problem matching localhost > > Hi, > > I'm running 2.0.0-11 (version included with redhat 5.2) > > I want to filter all the messages from external syslog devices to one > file > and all messages from the localhost to another file. > > However even with the -x option turned on when a local service (such as > crond) sends a message to the log the hostname is set to the domain > name of > the server. > > So I can't use the following to match: > :HOSTNAME, isequal, "localhost" /var/log/messages > :HOSTNAME, !isequal, "localhost" /var/log/externalsyslog > > I could replace "localhost" with "dnsname" to get it to work, but I > would > like a generic method that will work on all the syslog servers I have. > Is there some switch that will cause rsyslog to report the local > services as > sending from localhost or 127.0.0.1 rather than the hostname of the > localhost. > > thanks, > > niall > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 4 10:31:08 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 10:31:08 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> Scott, I have rewritten the omusrmsg plugin. This is currently part of the development tree, but will become part of the (new) beta when we do a version number shift next week. I'd appreciate if you could give it a try. It is available here: http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz I am hesitant to put this in the upcoming new v3-stable. There were lots of changes and one rule for a stable is that there should be no non-fix type of changes for at least 4 weeks before it turns into a stable. It of course is debatable if the change I made is a fix or not. In some sense it is, but on the other hand it is a rewrite that changed a lot. So given the fact that nobody on a "regular" machine saw an issue the past year, I would not like to bring the new code directly into the stable. If that causes problems for you, you may want to try simply using the omusrmsg.c code together with the v3-stable. I haven't tried it, but it should work well. All changes were just to that file, and there are no dependencies to anything external. OK... but now let's see first if it fixes the issue ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Thursday, July 03, 2008 2:58 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog threads questions > > Hi, > > I've attached four files. Two of which are debug dumps, one is the > conf file and the last one is a test case scenario that constantly > fails on my end. I hope this gives a little more information. > Furthermore, the dumps are from 3.17.5 which is the "closest" version > to 3.18.0 that I was able to find. > > Both failed scenarios occur when lots of messages were being flooded > to rsyslogd at a very fast rate (look at logtest.c) The > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads > that it took up so much memory that executing any command as simple as > 'ls -l' would not work from the command line. I think the number of > threads grew as much as the number of messages. In the latter > scenario, after killing logtest.c, it didn't look like the those > zombies threads went away until I did a CTRL+C to the rsyslogd which > was running in the foreground since I use the "-dn" option. > > This is on an embedded system that runs significantly slower than a > desktop or laptop so maybe it would be harder to reproduce on a > regular computer. I looked at all the parameters that I believe could > affect this and believe for the most part the defaults are more than > adequate. The main message queue never looked like it hit the high > water mark but it did hit the lower one. So, I don't think messages > were being dropped (not sure) or an overflow condition occurred. > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > compiled using GCC and the standard GNU C libraries version 3.4.5. > Rsyslog source code is cross-compiled using the following configure > line: > > ./configure --disable-zlib --disable-largefile > --enable-share=yes > --prefix=/ > --host=arm-unknown-linux-gnu > ac_cv_func_malloc_0_nonnull=yes > ac_cv_func_realloc_0_nonnull=yes > > ac_cv_func_lstat_dereferences_slashed_symlink=yes > ac_cv_func_stat_empty_string_bug=no > enable_debug=no > enable_rtinst=no > > Lastly, the logtest was executed with just the "-s" parameter. It is a > simple C file that I came up with. > > I took a look at the debug messages and it does not appear that new > threads are created via calls to wtpStartWrkr in wtp.c. > > Any help I can bring to solve this issue, please let me know. I hope I > am not doing anything wrong here. > > Thanks, > > Scott > > > > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > wrote: > >> Hi Rainer, > >> > >> Thanks for your reply. Looking at the default settings (from the > >> online help's configuration page), they are what I wanted. The main > >> messages queue is set to fix sized array with 1 worker thread > created > >> at maximum and action queues are direct mode which according to the > >> queue document page, means that there will not be a worker thread > >> created. Is my understanding correct? If yes, how do I quickly > check > >> without using the -d option if the defaults are set correctly? Or > what > >> do I look for in the debug messages that gets printed out to ensure > >> this? > >> > >> You also mentioned that version 3.18.0 is probably going to be > >> released as the stable version next week. I see on the webpage there > >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? > >> > >> Also, how come I did not get your reply in my email inbox? My > account > >> settings look correct. > >> > >> Thanks, > >> > >> Scott Phuong > >> > >> As for the syslog buffer size, that applies to syslogd and does not > >> apply to rsyslog. > >> > >> > >> > >> My configuration files do not change the Action queue or Worker > queue > >> parameters at all. Looking at > >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >>> Hi, > >>> > >>> I have 3.16.2 which was recently released. I see that under certain > >>> conditions rsyslogd spawns a lot of threads: > >>> 5949 root 11216 S rsyslogd > >>> 5950 root 11216 S rsyslogd > >>> 5951 root 11216 S rsyslogd > >>> 5952 root 11216 S rsyslogd > >>> 5953 root 11216 S rsyslogd > >>> 5954 root 11216 S rsyslogd > >>> 5985 root Z [rsyslogd] > >>> 6445 root Z [rsyslogd] > >>> > >>> I had to kill the rsyslogd and restart it. The first invocation had > a > >>> pid of 219 before it had to be killed. The second invocation of pid > >>> which you see above starts with 5949. The difference is the amount > of > >>> zombie threads that were invoked by rsyslogd before I had to kill > the > >>> first invocation of it. > >> > >> I have no explanation yet for the zombies. They should not happen > and so > >> far I have never seen them. We may need to go through a debug log > (which > >> will become very large) to find out what's going on. > >> > >>> The question is under what conditions does rsyslogd spawn a new > >>> thread/process and why was it a zombie? > >> > >> Unfortunately, there is no quick answer. A quick one may be: when it > >> needs them, based on queue watermark settings and based on you > >> configuration. But to really understand it, you need to read this > doc: > >> > >> http://www.rsyslog.com/doc-queues.html > >> > >> The doc also describes all the knobs that you can use to control > thread > >> creation. There are many ;) > >> > >>> I am running rsyslogd in an > >>> embedded environment and not a regular laptop/desktop. > >> > >> Interesting use case... > >> > >>> In addition, I > >>> am using busybox and I believe the syslog buffer size is set to > >> > >> what do yo mean by "syslog buffer size"? The length of a receive > buffer? > >> It is 2K, thus single messages up to 2K are supported. It can be > changed > >> by modifying the MAXLINE define. Note that stock syslogd (and > RFC3164) > >> support only up to 1K. > >> > >>> something very low or perhaps none at all. Would this be a factor? > >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > >>> cases happen. > >> > >> The compatibility modes do not affect queue operation. > >> > >>> Are these issues already known and fixed in a later version? Sorry, > if > >>> I am asking the same questions or have the same issues as previous > >>> people but without the ability to search (or at least, I don't know > >>> how to) the archive, I don't know if my problem/questions has > already > >>> been seen and/or resolved. > >> > >> If we need to find out about the zombies, we need to move on to the > >> current devel version. So I would give that a try in any case. > 3.16.2 > >> will (most probably) be replaced by 3.18.0 (based on the current > beta) > >> next week. So I won't touch it any longer. > >> > >> Looking forward to your feedback, > >> Rainer > >> > >>> > >>> Thank you very much for your support. > >>> > >>> Scott > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > From rgerhards at hq.adiscon.com Fri Jul 4 11:24:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 11:24:29 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> Hi list, I am posting a question here in the hope that some of the subscribers may be able to lend me a helping hand. Recently, I have begun to add a testbench to rsyslog. The idea is that over time the project should have canned tests which are easy to run on each version (as part of make distcheck at latest), increasing the overall code quality. "All" (so far two ;)) tests are located in the tests subdirectory. One test fails if compiled in release mode, that is rscript-parse.c. I have tracked down the failure cause and it is different struct member alignment in the ctok_token_t structure. The alignment is different in the rsyslog runtime and this test tool (I have compared the offsets computed for pToken->tok and they are different). So far, so good. But I do not find the cause of this misalignment. I checked the make output, and as it looks both the runtime as well as the tool are compiled with the same compiler options. Also, in debug mode it works OK, but in release mode (--disable-debug) it fails. I am sure the problem is something quite simple, but I have run out of ideas of what it may be (or, more probable, I simply overlook something). Any help would be deeply appreciated. Thanks, Rainer From mycleanjunk at gmail.com Fri Jul 4 12:28:58 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:28:58 -0700 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> Message-ID: <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> Here is my analysis: In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a #define and is defined to obj_t objData which is the same for both NDEBUG defined and not defined. But obj_t is a typedef of struct obj_s. In the structure definition for obj_s in obj-types.h are the following lines: #ifndef NDEBUG unsigned int iObjCooCKiE; #endif NDEBUG is defined in config.h which was included in the runtime for rsyslog but not for test-parse.c I didn't compile the code to test my theory. I just did a quick check. Hope that solved your problem. Scott I think this may be why the you are getting this structure misalignment? On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards wrote: > Hi list, > > I am posting a question here in the hope that some of the subscribers > may be able to lend me a helping hand. > > Recently, I have begun to add a testbench to rsyslog. The idea is that > over time the project should have canned tests which are easy to run on > each version (as part of make distcheck at latest), increasing the > overall code quality. > > "All" (so far two ;)) tests are located in the tests subdirectory. One > test fails if compiled in release mode, that is rscript-parse.c. I have > tracked down the failure cause and it is different struct member > alignment in the ctok_token_t structure. The alignment is different in > the rsyslog runtime and this test tool (I have compared the offsets > computed for pToken->tok and they are different). So far, so good. But I > do not find the cause of this misalignment. I checked the make output, > and as it looks both the runtime as well as the tool are compiled with > the same compiler options. Also, in debug mode it works OK, but in > release mode (--disable-debug) it fails. > > I am sure the problem is something quite simple, but I have run out of > ideas of what it may be (or, more probable, I simply overlook > something). > > Any help would be deeply appreciated. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mycleanjunk at gmail.com Fri Jul 4 12:31:45 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:31:45 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> Message-ID: <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> Hi Rainer, Thanks!! I'll give this a try next week and let you know the results. If it works, I think it is okay not to include in the upcoming v3 stable since it is due to release very soon. But if it does work, it would be great to have it in the following stable release. Scott On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards wrote: > Scott, > > I have rewritten the omusrmsg plugin. This is currently part of the > development tree, but will become part of the (new) beta when we do a > version number shift next week. I'd appreciate if you could give it a > try. It is available here: > > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz > > I am hesitant to put this in the upcoming new v3-stable. There were lots > of changes and one rule for a stable is that there should be no non-fix > type of changes for at least 4 weeks before it turns into a stable. It > of course is debatable if the change I made is a fix or not. In some > sense it is, but on the other hand it is a rewrite that changed a lot. > So given the fact that nobody on a "regular" machine saw an issue the > past year, I would not like to bring the new code directly into the > stable. If that causes problems for you, you may want to try simply > using the omusrmsg.c code together with the v3-stable. I haven't tried > it, but it should work well. All changes were just to that file, and > there are no dependencies to anything external. > > OK... but now let's see first if it fixes the issue ;) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Thursday, July 03, 2008 2:58 AM >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] rsyslog threads questions >> >> Hi, >> >> I've attached four files. Two of which are debug dumps, one is the >> conf file and the last one is a test case scenario that constantly >> fails on my end. I hope this gives a little more information. >> Furthermore, the dumps are from 3.17.5 which is the "closest" version >> to 3.18.0 that I was able to find. >> >> Both failed scenarios occur when lots of messages were being flooded >> to rsyslogd at a very fast rate (look at logtest.c) The >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads >> that it took up so much memory that executing any command as simple as >> 'ls -l' would not work from the command line. I think the number of >> threads grew as much as the number of messages. In the latter >> scenario, after killing logtest.c, it didn't look like the those >> zombies threads went away until I did a CTRL+C to the rsyslogd which >> was running in the foreground since I use the "-dn" option. >> >> This is on an embedded system that runs significantly slower than a >> desktop or laptop so maybe it would be harder to reproduce on a >> regular computer. I looked at all the parameters that I believe could >> affect this and believe for the most part the defaults are more than >> adequate. The main message queue never looked like it hit the high >> water mark but it did hit the lower one. So, I don't think messages >> were being dropped (not sure) or an overflow condition occurred. >> >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 and >> compiled using GCC and the standard GNU C libraries version 3.4.5. >> Rsyslog source code is cross-compiled using the following configure >> line: >> >> ./configure --disable-zlib --disable-largefile >> --enable-share=yes >> --prefix=/ >> --host=arm-unknown-linux-gnu >> ac_cv_func_malloc_0_nonnull=yes >> ac_cv_func_realloc_0_nonnull=yes >> >> ac_cv_func_lstat_dereferences_slashed_symlink=yes >> ac_cv_func_stat_empty_string_bug=no >> enable_debug=no >> enable_rtinst=no >> >> Lastly, the logtest was executed with just the "-s" parameter. It is a >> simple C file that I came up with. >> >> I took a look at the debug messages and it does not appear that new >> threads are created via calls to wtpStartWrkr in wtp.c. >> >> Any help I can bring to solve this issue, please let me know. I hope I >> am not doing anything wrong here. >> >> Thanks, >> >> Scott >> > >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > >> wrote: >> >> Hi Rainer, >> >> >> >> Thanks for your reply. Looking at the default settings (from the >> >> online help's configuration page), they are what I wanted. The main >> >> messages queue is set to fix sized array with 1 worker thread >> created >> >> at maximum and action queues are direct mode which according to the >> >> queue document page, means that there will not be a worker thread >> >> created. Is my understanding correct? If yes, how do I quickly >> check >> >> without using the -d option if the defaults are set correctly? Or >> what >> >> do I look for in the debug messages that gets printed out to ensure >> >> this? >> >> >> >> You also mentioned that version 3.18.0 is probably going to be >> >> released as the stable version next week. I see on the webpage > there >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >> >> >> >> Also, how come I did not get your reply in my email inbox? My >> account >> >> settings look correct. >> >> >> >> Thanks, >> >> >> >> Scott Phuong >> >> >> >> As for the syslog buffer size, that applies to syslogd and does not >> >> apply to rsyslog. >> >> >> >> >> >> >> >> My configuration files do not change the Action queue or Worker >> queue >> >> parameters at all. Looking at >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >> >>> Hi, >> >>> >> >>> I have 3.16.2 which was recently released. I see that under > certain >> >>> conditions rsyslogd spawns a lot of threads: >> >>> 5949 root 11216 S rsyslogd >> >>> 5950 root 11216 S rsyslogd >> >>> 5951 root 11216 S rsyslogd >> >>> 5952 root 11216 S rsyslogd >> >>> 5953 root 11216 S rsyslogd >> >>> 5954 root 11216 S rsyslogd >> >>> 5985 root Z [rsyslogd] >> >>> 6445 root Z [rsyslogd] >> >>> >> >>> I had to kill the rsyslogd and restart it. The first invocation > had >> a >> >>> pid of 219 before it had to be killed. The second invocation of > pid >> >>> which you see above starts with 5949. The difference is the amount >> of >> >>> zombie threads that were invoked by rsyslogd before I had to kill >> the >> >>> first invocation of it. >> >> >> >> I have no explanation yet for the zombies. They should not happen >> and so >> >> far I have never seen them. We may need to go through a debug log >> (which >> >> will become very large) to find out what's going on. >> >> >> >>> The question is under what conditions does rsyslogd spawn a new >> >>> thread/process and why was it a zombie? >> >> >> >> Unfortunately, there is no quick answer. A quick one may be: when > it >> >> needs them, based on queue watermark settings and based on you >> >> configuration. But to really understand it, you need to read this >> doc: >> >> >> >> http://www.rsyslog.com/doc-queues.html >> >> >> >> The doc also describes all the knobs that you can use to control >> thread >> >> creation. There are many ;) >> >> >> >>> I am running rsyslogd in an >> >>> embedded environment and not a regular laptop/desktop. >> >> >> >> Interesting use case... >> >> >> >>> In addition, I >> >>> am using busybox and I believe the syslog buffer size is set to >> >> >> >> what do yo mean by "syslog buffer size"? The length of a receive >> buffer? >> >> It is 2K, thus single messages up to 2K are supported. It can be >> changed >> >> by modifying the MAXLINE define. Note that stock syslogd (and >> RFC3164) >> >> support only up to 1K. >> >> >> >>> something very low or perhaps none at all. Would this be a factor? >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >> >>> cases happen. >> >> >> >> The compatibility modes do not affect queue operation. >> >> >> >>> Are these issues already known and fixed in a later version? > Sorry, >> if >> >>> I am asking the same questions or have the same issues as previous >> >>> people but without the ability to search (or at least, I don't > know >> >>> how to) the archive, I don't know if my problem/questions has >> already >> >>> been seen and/or resolved. >> >> >> >> If we need to find out about the zombies, we need to move on to the >> >> current devel version. So I would give that a try in any case. >> 3.16.2 >> >> will (most probably) be replaced by 3.18.0 (based on the current >> beta) >> >> next week. So I won't touch it any longer. >> >> >> >> Looking forward to your feedback, >> >> Rainer >> >> >> >>> >> >>> Thank you very much for your support. >> >>> >> >>> Scott >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Fri Jul 4 12:32:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:32:03 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> Hi Scott, thanks for the help, I'll further investigate. BUT: both the runtime AND the tool are compiled in non-debug mode. So in this case, neither of them should have the object cookie, thus the alignment should be the same. But, as I said, I am grateful for any clue and will check if there is some issue along these lines :) In any case, it seems to be inconsistent and should be fixed :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:29 PM > To: rsyslog-users > Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > > Here is my analysis: > In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a > #define and is defined to obj_t objData which is the same for both > NDEBUG defined and not defined. But obj_t is a typedef of struct > obj_s. In the structure definition for obj_s in obj-types.h are the > following lines: > #ifndef NDEBUG > unsigned int iObjCooCKiE; > #endif > > NDEBUG is defined in config.h which was included in the runtime for > rsyslog but not for test-parse.c > > I didn't compile the code to test my theory. I just did a quick check. > > Hope that solved your problem. > > Scott > > > I think this may be why the you are getting this structure > misalignment? > > > On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards > wrote: > > Hi list, > > > > I am posting a question here in the hope that some of the subscribers > > may be able to lend me a helping hand. > > > > Recently, I have begun to add a testbench to rsyslog. The idea is > that > > over time the project should have canned tests which are easy to run > on > > each version (as part of make distcheck at latest), increasing the > > overall code quality. > > > > "All" (so far two ;)) tests are located in the tests subdirectory. > One > > test fails if compiled in release mode, that is rscript-parse.c. I > have > > tracked down the failure cause and it is different struct member > > alignment in the ctok_token_t structure. The alignment is different > in > > the rsyslog runtime and this test tool (I have compared the offsets > > computed for pToken->tok and they are different). So far, so good. > But I > > do not find the cause of this misalignment. I checked the make > output, > > and as it looks both the runtime as well as the tool are compiled > with > > the same compiler options. Also, in debug mode it works OK, but in > > release mode (--disable-debug) it fails. > > > > I am sure the problem is something quite simple, but I have run out > of > > ideas of what it may be (or, more probable, I simply overlook > > something). > > > > Any help would be deeply appreciated. > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 4 12:35:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:35:23 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com><17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> Hi Scott, just for your understanding: once a release is stable, it will only receive fixes. So the fix we are looking it needs to become part of the devel, be migrated to beta and then be migrated to stable. In general, one migration step takes between 4 and 12 weeks, depending on what was done. So far, the average seems on the average, that is two month. So two cycles mean roughly four month. HOWEVER, I intend to put it into the next beta, which will bring us to just one cycle. So it would be available in roughly two month. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog threads questions > > Hi Rainer, > > Thanks!! I'll give this a try next week and let you know the results. > If it works, I think it is okay not to include in the upcoming v3 > stable since it is due to release very soon. But if it does work, it > would be great to have it in the following stable release. > > Scott > > On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards > wrote: > > Scott, > > > > I have rewritten the omusrmsg plugin. This is currently part of the > > development tree, but will become part of the (new) beta when we do a > > version number shift next week. I'd appreciate if you could give it a > > try. It is available here: > > > > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz > > > > I am hesitant to put this in the upcoming new v3-stable. There were > lots > > of changes and one rule for a stable is that there should be no non- > fix > > type of changes for at least 4 weeks before it turns into a stable. > It > > of course is debatable if the change I made is a fix or not. In some > > sense it is, but on the other hand it is a rewrite that changed a > lot. > > So given the fact that nobody on a "regular" machine saw an issue the > > past year, I would not like to bring the new code directly into the > > stable. If that causes problems for you, you may want to try simply > > using the omusrmsg.c code together with the v3-stable. I haven't > tried > > it, but it should work well. All changes were just to that file, and > > there are no dependencies to anything external. > > > > OK... but now let's see first if it fixes the issue ;) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong > >> Sent: Thursday, July 03, 2008 2:58 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: Re: [rsyslog] rsyslog threads questions > >> > >> Hi, > >> > >> I've attached four files. Two of which are debug dumps, one is the > >> conf file and the last one is a test case scenario that constantly > >> fails on my end. I hope this gives a little more information. > >> Furthermore, the dumps are from 3.17.5 which is the "closest" > version > >> to 3.18.0 that I was able to find. > >> > >> Both failed scenarios occur when lots of messages were being flooded > >> to rsyslogd at a very fast rate (look at logtest.c) The > >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" > threads > >> that it took up so much memory that executing any command as simple > as > >> 'ls -l' would not work from the command line. I think the number of > >> threads grew as much as the number of messages. In the latter > >> scenario, after killing logtest.c, it didn't look like the those > >> zombies threads went away until I did a CTRL+C to the rsyslogd which > >> was running in the foreground since I use the "-dn" option. > >> > >> This is on an embedded system that runs significantly slower than a > >> desktop or laptop so maybe it would be harder to reproduce on a > >> regular computer. I looked at all the parameters that I believe > could > >> affect this and believe for the most part the defaults are more than > >> adequate. The main message queue never looked like it hit the high > >> water mark but it did hit the lower one. So, I don't think messages > >> were being dropped (not sure) or an overflow condition occurred. > >> > >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 > and > >> compiled using GCC and the standard GNU C libraries version 3.4.5. > >> Rsyslog source code is cross-compiled using the following configure > >> line: > >> > >> ./configure --disable-zlib --disable-largefile > >> --enable-share=yes > >> --prefix=/ > >> --host=arm-unknown-linux-gnu > >> ac_cv_func_malloc_0_nonnull=yes > >> ac_cv_func_realloc_0_nonnull=yes > >> > >> ac_cv_func_lstat_dereferences_slashed_symlink=yes > >> ac_cv_func_stat_empty_string_bug=no > >> enable_debug=no > >> enable_rtinst=no > >> > >> Lastly, the logtest was executed with just the "-s" parameter. It is > a > >> simple C file that I came up with. > >> > >> I took a look at the debug messages and it does not appear that new > >> threads are created via calls to wtpStartWrkr in wtp.c. > >> > >> Any help I can bring to solve this issue, please let me know. I hope > I > >> am not doing anything wrong here. > >> > >> Thanks, > >> > >> Scott > >> > > >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > > > >> wrote: > >> >> Hi Rainer, > >> >> > >> >> Thanks for your reply. Looking at the default settings (from the > >> >> online help's configuration page), they are what I wanted. The > main > >> >> messages queue is set to fix sized array with 1 worker thread > >> created > >> >> at maximum and action queues are direct mode which according to > the > >> >> queue document page, means that there will not be a worker thread > >> >> created. Is my understanding correct? If yes, how do I quickly > >> check > >> >> without using the -d option if the defaults are set correctly? Or > >> what > >> >> do I look for in the debug messages that gets printed out to > ensure > >> >> this? > >> >> > >> >> You also mentioned that version 3.18.0 is probably going to be > >> >> released as the stable version next week. I see on the webpage > > there > >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to > 3.18.0? > >> >> > >> >> Also, how come I did not get your reply in my email inbox? My > >> account > >> >> settings look correct. > >> >> > >> >> Thanks, > >> >> > >> >> Scott Phuong > >> >> > >> >> As for the syslog buffer size, that applies to syslogd and does > not > >> >> apply to rsyslog. > >> >> > >> >> > >> >> > >> >> My configuration files do not change the Action queue or Worker > >> queue > >> >> parameters at all. Looking at > >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >> >>> Hi, > >> >>> > >> >>> I have 3.16.2 which was recently released. I see that under > > certain > >> >>> conditions rsyslogd spawns a lot of threads: > >> >>> 5949 root 11216 S rsyslogd > >> >>> 5950 root 11216 S rsyslogd > >> >>> 5951 root 11216 S rsyslogd > >> >>> 5952 root 11216 S rsyslogd > >> >>> 5953 root 11216 S rsyslogd > >> >>> 5954 root 11216 S rsyslogd > >> >>> 5985 root Z [rsyslogd] > >> >>> 6445 root Z [rsyslogd] > >> >>> > >> >>> I had to kill the rsyslogd and restart it. The first invocation > > had > >> a > >> >>> pid of 219 before it had to be killed. The second invocation of > > pid > >> >>> which you see above starts with 5949. The difference is the > amount > >> of > >> >>> zombie threads that were invoked by rsyslogd before I had to > kill > >> the > >> >>> first invocation of it. > >> >> > >> >> I have no explanation yet for the zombies. They should not happen > >> and so > >> >> far I have never seen them. We may need to go through a debug log > >> (which > >> >> will become very large) to find out what's going on. > >> >> > >> >>> The question is under what conditions does rsyslogd spawn a new > >> >>> thread/process and why was it a zombie? > >> >> > >> >> Unfortunately, there is no quick answer. A quick one may be: when > > it > >> >> needs them, based on queue watermark settings and based on you > >> >> configuration. But to really understand it, you need to read this > >> doc: > >> >> > >> >> http://www.rsyslog.com/doc-queues.html > >> >> > >> >> The doc also describes all the knobs that you can use to control > >> thread > >> >> creation. There are many ;) > >> >> > >> >>> I am running rsyslogd in an > >> >>> embedded environment and not a regular laptop/desktop. > >> >> > >> >> Interesting use case... > >> >> > >> >>> In addition, I > >> >>> am using busybox and I believe the syslog buffer size is set to > >> >> > >> >> what do yo mean by "syslog buffer size"? The length of a receive > >> buffer? > >> >> It is 2K, thus single messages up to 2K are supported. It can be > >> changed > >> >> by modifying the MAXLINE define. Note that stock syslogd (and > >> RFC3164) > >> >> support only up to 1K. > >> >> > >> >>> something very low or perhaps none at all. Would this be a > factor? > >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and > both > >> >>> cases happen. > >> >> > >> >> The compatibility modes do not affect queue operation. > >> >> > >> >>> Are these issues already known and fixed in a later version? > > Sorry, > >> if > >> >>> I am asking the same questions or have the same issues as > previous > >> >>> people but without the ability to search (or at least, I don't > > know > >> >>> how to) the archive, I don't know if my problem/questions has > >> already > >> >>> been seen and/or resolved. > >> >> > >> >> If we need to find out about the zombies, we need to move on to > the > >> >> current devel version. So I would give that a try in any case. > >> 3.16.2 > >> >> will (most probably) be replaced by 3.18.0 (based on the current > >> beta) > >> >> next week. So I won't touch it any longer. > >> >> > >> >> Looking forward to your feedback, > >> >> Rainer > >> >> > >> >>> > >> >>> Thank you very much for your support. > >> >>> > >> >>> Scott > >> >>> _______________________________________________ > >> >>> rsyslog mailing list > >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > >> > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Fri Jul 4 12:38:12 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:38:12 -0700 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> Message-ID: <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> Hi Rainer, But in release mode, NDEBUG is intentionally defined in the config.h. This is what I recalled from looking at 3.16.1 and 3.17.5. So from the perspective of rsyslog, NDEBUG is defined and the opposite for the test-parse.c (because config.h is not included). Scott On Fri, Jul 4, 2008 at 3:32 AM, Rainer Gerhards wrote: > Hi Scott, > > thanks for the help, I'll further investigate. BUT: both the runtime AND > the tool are compiled in non-debug mode. So in this case, neither of > them should have the object cookie, thus the alignment should be the > same. But, as I said, I am grateful for any clue and will check if there > is some issue along these lines :) In any case, it seems to be > inconsistent and should be fixed :) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Friday, July 04, 2008 12:29 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] struct alignment problem - who can help? ;) >> >> Here is my analysis: >> In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a >> #define and is defined to obj_t objData which is the same for both >> NDEBUG defined and not defined. But obj_t is a typedef of struct >> obj_s. In the structure definition for obj_s in obj-types.h are the >> following lines: >> #ifndef NDEBUG >> unsigned int iObjCooCKiE; >> #endif >> >> NDEBUG is defined in config.h which was included in the runtime for >> rsyslog but not for test-parse.c >> >> I didn't compile the code to test my theory. I just did a quick check. >> >> Hope that solved your problem. >> >> Scott >> >> >> I think this may be why the you are getting this structure >> misalignment? >> >> >> On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards >> wrote: >> > Hi list, >> > >> > I am posting a question here in the hope that some of the > subscribers >> > may be able to lend me a helping hand. >> > >> > Recently, I have begun to add a testbench to rsyslog. The idea is >> that >> > over time the project should have canned tests which are easy to run >> on >> > each version (as part of make distcheck at latest), increasing the >> > overall code quality. >> > >> > "All" (so far two ;)) tests are located in the tests subdirectory. >> One >> > test fails if compiled in release mode, that is rscript-parse.c. I >> have >> > tracked down the failure cause and it is different struct member >> > alignment in the ctok_token_t structure. The alignment is different >> in >> > the rsyslog runtime and this test tool (I have compared the offsets >> > computed for pToken->tok and they are different). So far, so good. >> But I >> > do not find the cause of this misalignment. I checked the make >> output, >> > and as it looks both the runtime as well as the tool are compiled >> with >> > the same compiler options. Also, in debug mode it works OK, but in >> > release mode (--disable-debug) it fails. >> > >> > I am sure the problem is something quite simple, but I have run out >> of >> > ideas of what it may be (or, more probable, I simply overlook >> > something). >> > >> > Any help would be deeply appreciated. >> > >> > Thanks, >> > Rainer >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mycleanjunk at gmail.com Fri Jul 4 12:39:32 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:39:32 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> Message-ID: <17945960807040339y44e5f75cgde6b4593e5c99584@mail.gmail.com> Hi Rainer, Thanks for the information. I didn't know that. Scott On Fri, Jul 4, 2008 at 3:35 AM, Rainer Gerhards wrote: > Hi Scott, > > just for your understanding: once a release is stable, it will only > receive fixes. So the fix we are looking it needs to become part of the > devel, be migrated to beta and then be migrated to stable. In general, > one migration step takes between 4 and 12 weeks, depending on what was > done. So far, the average seems on the average, that is two month. So > two cycles mean roughly four month. HOWEVER, I intend to put it into the > next beta, which will bring us to just one cycle. So it would be > available in roughly two month. > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Friday, July 04, 2008 12:32 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog threads questions >> >> Hi Rainer, >> >> Thanks!! I'll give this a try next week and let you know the results. >> If it works, I think it is okay not to include in the upcoming v3 >> stable since it is due to release very soon. But if it does work, it >> would be great to have it in the following stable release. >> >> Scott >> >> On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards >> wrote: >> > Scott, >> > >> > I have rewritten the omusrmsg plugin. This is currently part of the >> > development tree, but will become part of the (new) beta when we do > a >> > version number shift next week. I'd appreciate if you could give it > a >> > try. It is available here: >> > >> > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz >> > >> > I am hesitant to put this in the upcoming new v3-stable. There were >> lots >> > of changes and one rule for a stable is that there should be no non- >> fix >> > type of changes for at least 4 weeks before it turns into a stable. >> It >> > of course is debatable if the change I made is a fix or not. In some >> > sense it is, but on the other hand it is a rewrite that changed a >> lot. >> > So given the fact that nobody on a "regular" machine saw an issue > the >> > past year, I would not like to bring the new code directly into the >> > stable. If that causes problems for you, you may want to try simply >> > using the omusrmsg.c code together with the v3-stable. I haven't >> tried >> > it, but it should work well. All changes were just to that file, and >> > there are no dependencies to anything external. >> > >> > OK... but now let's see first if it fixes the issue ;) >> > >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> >> Sent: Thursday, July 03, 2008 2:58 AM >> >> To: rsyslog at lists.adiscon.com >> >> Subject: Re: [rsyslog] rsyslog threads questions >> >> >> >> Hi, >> >> >> >> I've attached four files. Two of which are debug dumps, one is the >> >> conf file and the last one is a test case scenario that constantly >> >> fails on my end. I hope this gives a little more information. >> >> Furthermore, the dumps are from 3.17.5 which is the "closest" >> version >> >> to 3.18.0 that I was able to find. >> >> >> >> Both failed scenarios occur when lots of messages were being > flooded >> >> to rsyslogd at a very fast rate (look at logtest.c) The >> >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while >> >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" >> threads >> >> that it took up so much memory that executing any command as simple >> as >> >> 'ls -l' would not work from the command line. I think the number of >> >> threads grew as much as the number of messages. In the latter >> >> scenario, after killing logtest.c, it didn't look like the those >> >> zombies threads went away until I did a CTRL+C to the rsyslogd > which >> >> was running in the foreground since I use the "-dn" option. >> >> >> >> This is on an embedded system that runs significantly slower than a >> >> desktop or laptop so maybe it would be harder to reproduce on a >> >> regular computer. I looked at all the parameters that I believe >> could >> >> affect this and believe for the most part the defaults are more > than >> >> adequate. The main message queue never looked like it hit the high >> >> water mark but it did hit the lower one. So, I don't think messages >> >> were being dropped (not sure) or an overflow condition occurred. >> >> >> >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 >> and >> >> compiled using GCC and the standard GNU C libraries version 3.4.5. >> >> Rsyslog source code is cross-compiled using the following configure >> >> line: >> >> >> >> ./configure --disable-zlib --disable-largefile >> >> --enable-share=yes >> >> --prefix=/ >> >> --host=arm-unknown-linux-gnu >> >> ac_cv_func_malloc_0_nonnull=yes >> >> ac_cv_func_realloc_0_nonnull=yes >> >> >> >> ac_cv_func_lstat_dereferences_slashed_symlink=yes >> >> ac_cv_func_stat_empty_string_bug=no >> >> enable_debug=no >> >> enable_rtinst=no >> >> >> >> Lastly, the logtest was executed with just the "-s" parameter. It > is >> a >> >> simple C file that I came up with. >> >> >> >> I took a look at the debug messages and it does not appear that new >> >> threads are created via calls to wtpStartWrkr in wtp.c. >> >> >> >> Any help I can bring to solve this issue, please let me know. I > hope >> I >> >> am not doing anything wrong here. >> >> >> >> Thanks, >> >> >> >> Scott >> >> > >> >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong >> > >> >> wrote: >> >> >> Hi Rainer, >> >> >> >> >> >> Thanks for your reply. Looking at the default settings (from > the >> >> >> online help's configuration page), they are what I wanted. The >> main >> >> >> messages queue is set to fix sized array with 1 worker thread >> >> created >> >> >> at maximum and action queues are direct mode which according to >> the >> >> >> queue document page, means that there will not be a worker > thread >> >> >> created. Is my understanding correct? If yes, how do I quickly >> >> check >> >> >> without using the -d option if the defaults are set correctly? > Or >> >> what >> >> >> do I look for in the debug messages that gets printed out to >> ensure >> >> >> this? >> >> >> >> >> >> You also mentioned that version 3.18.0 is probably going to be >> >> >> released as the stable version next week. I see on the webpage >> > there >> >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to >> 3.18.0? >> >> >> >> >> >> Also, how come I did not get your reply in my email inbox? My >> >> account >> >> >> settings look correct. >> >> >> >> >> >> Thanks, >> >> >> >> >> >> Scott Phuong >> >> >> >> >> >> As for the syslog buffer size, that applies to syslogd and does >> not >> >> >> apply to rsyslog. >> >> >> >> >> >> >> >> >> >> >> >> My configuration files do not change the Action queue or Worker >> >> queue >> >> >> parameters at all. Looking at >> >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >> >> >>> Hi, >> >> >>> >> >> >>> I have 3.16.2 which was recently released. I see that under >> > certain >> >> >>> conditions rsyslogd spawns a lot of threads: >> >> >>> 5949 root 11216 S rsyslogd >> >> >>> 5950 root 11216 S rsyslogd >> >> >>> 5951 root 11216 S rsyslogd >> >> >>> 5952 root 11216 S rsyslogd >> >> >>> 5953 root 11216 S rsyslogd >> >> >>> 5954 root 11216 S rsyslogd >> >> >>> 5985 root Z [rsyslogd] >> >> >>> 6445 root Z [rsyslogd] >> >> >>> >> >> >>> I had to kill the rsyslogd and restart it. The first invocation >> > had >> >> a >> >> >>> pid of 219 before it had to be killed. The second invocation of >> > pid >> >> >>> which you see above starts with 5949. The difference is the >> amount >> >> of >> >> >>> zombie threads that were invoked by rsyslogd before I had to >> kill >> >> the >> >> >>> first invocation of it. >> >> >> >> >> >> I have no explanation yet for the zombies. They should not > happen >> >> and so >> >> >> far I have never seen them. We may need to go through a debug > log >> >> (which >> >> >> will become very large) to find out what's going on. >> >> >> >> >> >>> The question is under what conditions does rsyslogd spawn a new >> >> >>> thread/process and why was it a zombie? >> >> >> >> >> >> Unfortunately, there is no quick answer. A quick one may be: > when >> > it >> >> >> needs them, based on queue watermark settings and based on you >> >> >> configuration. But to really understand it, you need to read > this >> >> doc: >> >> >> >> >> >> http://www.rsyslog.com/doc-queues.html >> >> >> >> >> >> The doc also describes all the knobs that you can use to control >> >> thread >> >> >> creation. There are many ;) >> >> >> >> >> >>> I am running rsyslogd in an >> >> >>> embedded environment and not a regular laptop/desktop. >> >> >> >> >> >> Interesting use case... >> >> >> >> >> >>> In addition, I >> >> >>> am using busybox and I believe the syslog buffer size is set to >> >> >> >> >> >> what do yo mean by "syslog buffer size"? The length of a receive >> >> buffer? >> >> >> It is 2K, thus single messages up to 2K are supported. It can be >> >> changed >> >> >> by modifying the MAXLINE define. Note that stock syslogd (and >> >> RFC3164) >> >> >> support only up to 1K. >> >> >> >> >> >>> something very low or perhaps none at all. Would this be a >> factor? >> >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and >> both >> >> >>> cases happen. >> >> >> >> >> >> The compatibility modes do not affect queue operation. >> >> >> >> >> >>> Are these issues already known and fixed in a later version? >> > Sorry, >> >> if >> >> >>> I am asking the same questions or have the same issues as >> previous >> >> >>> people but without the ability to search (or at least, I don't >> > know >> >> >>> how to) the archive, I don't know if my problem/questions has >> >> already >> >> >>> been seen and/or resolved. >> >> >> >> >> >> If we need to find out about the zombies, we need to move on to >> the >> >> >> current devel version. So I would give that a try in any case. >> >> 3.16.2 >> >> >> will (most probably) be replaced by 3.18.0 (based on the current >> >> beta) >> >> >> next week. So I won't touch it any longer. >> >> >> >> >> >> Looking forward to your feedback, >> >> >> Rainer >> >> >> >> >> >>> >> >> >>> Thank you very much for your support. >> >> >>> >> >> >>> Scott >> >> >>> _______________________________________________ >> >> >>> rsyslog mailing list >> >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> >> >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Fri Jul 4 12:44:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:44:24 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com><17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933D@grfint2.intern.adiscon.com> ROFL... you got it! I have forgotten to #include "config.h"! I knew that it must have been something simple. Thanks a lot, you saved me from pulling out the rest of my hair ;) Rainer PS: and, yes, it looks like I need to reevaluate mode-specific structure members now that we aim at a real runtime. Doesn't look so smart any longer... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > > Hi Rainer, > > But in release mode, NDEBUG is intentionally defined in the config.h. > This is what I recalled from looking at 3.16.1 and 3.17.5. So from the > perspective of rsyslog, NDEBUG is defined and the opposite for the > test-parse.c (because config.h is not included). > > Scott > > On Fri, Jul 4, 2008 at 3:32 AM, Rainer Gerhards > wrote: > > Hi Scott, > > > > thanks for the help, I'll further investigate. BUT: both the runtime > AND > > the tool are compiled in non-debug mode. So in this case, neither of > > them should have the object cookie, thus the alignment should be the > > same. But, as I said, I am grateful for any clue and will check if > there > > is some issue along these lines :) In any case, it seems to be > > inconsistent and should be fixed :) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong > >> Sent: Friday, July 04, 2008 12:29 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > >> > >> Here is my analysis: > >> In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a > >> #define and is defined to obj_t objData which is the same for both > >> NDEBUG defined and not defined. But obj_t is a typedef of struct > >> obj_s. In the structure definition for obj_s in obj-types.h are the > >> following lines: > >> #ifndef NDEBUG > >> unsigned int iObjCooCKiE; > >> #endif > >> > >> NDEBUG is defined in config.h which was included in the runtime for > >> rsyslog but not for test-parse.c > >> > >> I didn't compile the code to test my theory. I just did a quick > check. > >> > >> Hope that solved your problem. > >> > >> Scott > >> > >> > >> I think this may be why the you are getting this structure > >> misalignment? > >> > >> > >> On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards > >> wrote: > >> > Hi list, > >> > > >> > I am posting a question here in the hope that some of the > > subscribers > >> > may be able to lend me a helping hand. > >> > > >> > Recently, I have begun to add a testbench to rsyslog. The idea is > >> that > >> > over time the project should have canned tests which are easy to > run > >> on > >> > each version (as part of make distcheck at latest), increasing the > >> > overall code quality. > >> > > >> > "All" (so far two ;)) tests are located in the tests subdirectory. > >> One > >> > test fails if compiled in release mode, that is rscript-parse.c. I > >> have > >> > tracked down the failure cause and it is different struct member > >> > alignment in the ctok_token_t structure. The alignment is > different > >> in > >> > the rsyslog runtime and this test tool (I have compared the > offsets > >> > computed for pToken->tok and they are different). So far, so good. > >> But I > >> > do not find the cause of this misalignment. I checked the make > >> output, > >> > and as it looks both the runtime as well as the tool are compiled > >> with > >> > the same compiler options. Also, in debug mode it works OK, but in > >> > release mode (--disable-debug) it fails. > >> > > >> > I am sure the problem is something quite simple, but I have run > out > >> of > >> > ideas of what it may be (or, more probable, I simply overlook > >> > something). > >> > > >> > Any help would be deeply appreciated. > >> > > >> > Thanks, > >> > Rainer > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From friedl at hq.adiscon.com Mon Jul 7 16:02:21 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 7 Jul 2008 16:02:21 +0200 Subject: [rsyslog] rsyslog 3.19.9 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED69@grfint2.intern.adiscon.com> Hi all, Rsyslog 3.19.9, a member of the development branch, has been released. Documentation on how to setup TLS syslog has been greatly improved. Also, omusrmsg has been rewritten to utilize rsyslog's threading instead of forking of children to send the messages. This is expected to also solve some issues with zombie processes. Finally, an issue with TLS anonymous mode, which still required client certificates, and a problem with the RainerScript parser, which did not detect every syntax error, was solved. This is a recommended update for all development branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-117.phtml Changelog: http://www.rsyslog.com/Article250.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rfujita at redhat.com Wed Jul 9 10:34:27 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 17:34:27 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. Message-ID: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> Hi List, $subject And I guess that documents are managed with Wiki. Do you have a plan to do i10n/i18n with rsyslog? Best Rio. ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 10:46:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 9 Jul 2008 10:46:34 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> Hi Rio-san, many thanks for your help! Much appreciated. As far as the documentation goes, there are actually two places. One is the wiki ( http://wiki.rsyslog.com ) , but then there is also a set (called "the core set") of html documentation. The reasoning is as follows: The wiki is a great resource and easy to edit. However, it does not provide the versioning we need. For example, there are changes in the way things can be configured between v2 and v3. In the wiki, version-specifc pages are a bit hard to find. Also, the wiki is only available online. To solve this, we have the core set: html files which document rsyslog's config directives as well as generic use cases. This is contained in git and under version control. An exactly matching version of the documentation is distributed with each source tarball. I think it would be very useful to have a Japanese version of (at least some) documents of the core set. If you are interested in that, I would add your translations to git and the distribution set. > Do you have a plan to do i10n/i18n with rsyslog? Could you elaborate a little bit on this? Unfortunately, I do neither speak Japanse nor any other language with a non-western script. I asked around several times and some folks from Japan told me that rsyslog works well with Japanese characters. But I am always interested in enhancing this support - I just need some guidance what is needed. Looking forward to your relpy, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Wednesday, July 09, 2008 10:34 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] I'd like to translate documents into Japanese. > > Hi List, > > $subject > And I guess that documents are managed with Wiki. > Do you have a plan to do i10n/i18n with rsyslog? > > Best Rio. > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Wed Jul 9 11:33:29 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 18:33:29 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> Message-ID: <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> Hi Rainer-san, Thank you for your quick response! Too quick, I was surprised. I understand the status of the project and what I should do. #The reason why I asked how the contributors collaborate for rsyslog was that I cloned the repository with git and I couldn't find any docbook or .po files. As soon as possible, I'll start to translate docs into Japanese. BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. But I can't find the source file of this version in the git tree and Wiki. Why? Best Rio. On 2008/07/09, at 17:46, Rainer Gerhards wrote: > Hi Rio-san, > > many thanks for your help! Much appreciated. > > As far as the documentation goes, there are actually two places. One > is > the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > (called "the core set") of html documentation. The reasoning is as > follows: The wiki is a great resource and easy to edit. However, it > does > not provide the versioning we need. For example, there are changes in > the way things can be configured between v2 and v3. In the wiki, > version-specifc pages are a bit hard to find. Also, the wiki is only > available online. > > To solve this, we have the core set: html files which document > rsyslog's > config directives as well as generic use cases. This is contained in > git > and under version control. An exactly matching version of the > documentation is distributed with each source tarball. > > I think it would be very useful to have a Japanese version of (at > least > some) documents of the core set. If you are interested in that, I > would > add your translations to git and the distribution set. > >> Do you have a plan to do i10n/i18n with rsyslog? > > Could you elaborate a little bit on this? Unfortunately, I do neither > speak Japanse nor any other language with a non-western script. I > asked > around several times and some folks from Japan told me that rsyslog > works well with Japanese characters. But I am always interested in > enhancing this support - I just need some guidance what is needed. > > Looking forward to your relpy, > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Wednesday, July 09, 2008 10:34 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] I'd like to translate documents into Japanese. >> >> Hi List, >> >> $subject >> And I guess that documents are managed with Wiki. >> Do you have a plan to do i10n/i18n with rsyslog? >> >> Best Rio. >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 11:43:17 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 11:43:17 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> Message-ID: <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: > Hi Rainer-san, > > Thank you for your quick response! > Too quick, I was surprised. I am trying my best - doesn't always work :) > > I understand the status of the project and what I should do. > > #The reason why I asked how the contributors collaborate for rsyslog > was that I cloned the repository with git and I couldn't find any > docbook or .po files. The doc set is in the ./doc subfolder, with .html files (obviously not what you looked for :)). I think it would be a good time now to create a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the files and I would integrate them as soon as I have them availalbe. > > As soon as possible, I'll start to translate docs into Japanese. > > BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. > But I can't find the source file of this version in the git tree and > Wiki. > Why? I restructured the directory structure a bit in 3.19. It is now in the ./tools directory, because it belongs to the rsyslogd tool. Here is a quick link: http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD All the best, Rainer > > Best Rio. > > On 2008/07/09, at 17:46, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > many thanks for your help! Much appreciated. > > > > As far as the documentation goes, there are actually two places. One > > is > > the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > > (called "the core set") of html documentation. The reasoning is as > > follows: The wiki is a great resource and easy to edit. However, it > > does > > not provide the versioning we need. For example, there are changes in > > the way things can be configured between v2 and v3. In the wiki, > > version-specifc pages are a bit hard to find. Also, the wiki is only > > available online. > > > > To solve this, we have the core set: html files which document > > rsyslog's > > config directives as well as generic use cases. This is contained in > > git > > and under version control. An exactly matching version of the > > documentation is distributed with each source tarball. > > > > I think it would be very useful to have a Japanese version of (at > > least > > some) documents of the core set. If you are interested in that, I > > would > > add your translations to git and the distribution set. > > > >> Do you have a plan to do i10n/i18n with rsyslog? > > > > Could you elaborate a little bit on this? Unfortunately, I do neither > > speak Japanse nor any other language with a non-western script. I > > asked > > around several times and some folks from Japan told me that rsyslog > > works well with Japanese characters. But I am always interested in > > enhancing this support - I just need some guidance what is needed. > > > > Looking forward to your relpy, > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Wednesday, July 09, 2008 10:34 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] I'd like to translate documents into Japanese. > >> > >> Hi List, > >> > >> $subject > >> And I guess that documents are managed with Wiki. > >> Do you have a plan to do i10n/i18n with rsyslog? > >> > >> Best Rio. > >> > >> > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Wed Jul 9 11:59:48 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 18:59:48 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Message-ID: Hi Rainer-san, On 2008/07/09, at 18:43, Rainer Gerhards wrote: > I am trying my best - doesn't always work :) I'm working for Red Hat's Tokyo office. Do you live in Germany? If so, we can co-work in the daytime! > The doc set is in the ./doc subfolder, with .html files (obviously not > what you looked for :)). I think it would be a good time now to create > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > files and I would integrate them as soon as I have them availalbe. I agree with you totally. Because Fedora 9 has included rsyslog, lots Japanese and other non English/German speakers have started using it. Sooner or later, we'll need to create ./doc/other_lang directories :) > I restructured the directory structure a bit in 3.19. It is now in > the ./tools directory, because it belongs to the rsyslogd tool. > > Here is a quick link: > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD Oh, I find the file rsyslogd.8 in tools directory. I'll start translating it first. Best Rio. On 2008/07/09, at 18:43, Rainer Gerhards wrote: > Hi Rio-san, > > On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: >> Hi Rainer-san, >> >> Thank you for your quick response! >> Too quick, I was surprised. > > I am trying my best - doesn't always work :) > >> >> I understand the status of the project and what I should do. >> >> #The reason why I asked how the contributors collaborate for rsyslog >> was that I cloned the repository with git and I couldn't find any >> docbook or .po files. > > The doc set is in the ./doc subfolder, with .html files (obviously not > what you looked for :)). I think it would be a good time now to create > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > files and I would integrate them as soon as I have them availalbe. >> >> As soon as possible, I'll start to translate docs into Japanese. >> >> BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. >> But I can't find the source file of this version in the git tree and >> Wiki. >> Why? > > I restructured the directory structure a bit in 3.19. It is now in > the ./tools directory, because it belongs to the rsyslogd tool. > > Here is a quick link: > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > All the best, > Rainer >> >> Best Rio. >> >> On 2008/07/09, at 17:46, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> many thanks for your help! Much appreciated. >>> >>> As far as the documentation goes, there are actually two places. One >>> is >>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a set >>> (called "the core set") of html documentation. The reasoning is as >>> follows: The wiki is a great resource and easy to edit. However, it >>> does >>> not provide the versioning we need. For example, there are changes >>> in >>> the way things can be configured between v2 and v3. In the wiki, >>> version-specifc pages are a bit hard to find. Also, the wiki is only >>> available online. >>> >>> To solve this, we have the core set: html files which document >>> rsyslog's >>> config directives as well as generic use cases. This is contained in >>> git >>> and under version control. An exactly matching version of the >>> documentation is distributed with each source tarball. >>> >>> I think it would be very useful to have a Japanese version of (at >>> least >>> some) documents of the core set. If you are interested in that, I >>> would >>> add your translations to git and the distribution set. >>> >>>> Do you have a plan to do i10n/i18n with rsyslog? >>> >>> Could you elaborate a little bit on this? Unfortunately, I do >>> neither >>> speak Japanse nor any other language with a non-western script. I >>> asked >>> around several times and some folks from Japan told me that rsyslog >>> works well with Japanese characters. But I am always interested in >>> enhancing this support - I just need some guidance what is needed. >>> >>> Looking forward to your relpy, >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Wednesday, July 09, 2008 10:34 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] I'd like to translate documents into Japanese. >>>> >>>> Hi List, >>>> >>>> $subject >>>> And I guess that documents are managed with Wiki. >>>> Do you have a plan to do i10n/i18n with rsyslog? >>>> >>>> Best Rio. >>>> >>>> >>> ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> ######################################################################## >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> ######################################################################## >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 12:06:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 12:06:11 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Message-ID: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Wed, 2008-07-09 at 18:59 +0900, Ryo Fujita wrote: > Hi Rainer-san, > > On 2008/07/09, at 18:43, Rainer Gerhards wrote: > > > I am trying my best - doesn't always work :) > > > I'm working for Red Hat's Tokyo office. > Do you live in Germany? > If so, we can co-work in the daytime! indeed, I do. So we can at least try. > > > The doc set is in the ./doc subfolder, with .html files (obviously not > > what you looked for :)). I think it would be a good time now to create > > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > > files and I would integrate them as soon as I have them availalbe. > > I agree with you totally. > Because Fedora 9 has included rsyslog, > lots Japanese and other non English/German speakers have started using > it. > Sooner or later, we'll need to create ./doc/other_lang directories :) I'll begin this when I start the next devel branch. I scheduled it for this week, but I got a bug report for the beta, which I would like to fix first. But in any case... soon! > > > I restructured the directory structure a bit in 3.19. It is now in > > the ./tools directory, because it belongs to the rsyslogd tool. > > > > Here is a quick link: > > > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > Oh, I find the file rsyslogd.8 in tools directory. > I'll start translating it first. Excellent. If you go through ./doc, you'll also notice that there is room for improvement. I think the structure is not very good today. Suggestions are welcome :) Rainer > > Best Rio. > > On 2008/07/09, at 18:43, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: > >> Hi Rainer-san, > >> > >> Thank you for your quick response! > >> Too quick, I was surprised. > > > > I am trying my best - doesn't always work :) > > > >> > >> I understand the status of the project and what I should do. > >> > >> #The reason why I asked how the contributors collaborate for rsyslog > >> was that I cloned the repository with git and I couldn't find any > >> docbook or .po files. > > > > The doc set is in the ./doc subfolder, with .html files (obviously not > > what you looked for :)). I think it would be a good time now to create > > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > > files and I would integrate them as soon as I have them availalbe. > >> > >> As soon as possible, I'll start to translate docs into Japanese. > >> > >> BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. > >> But I can't find the source file of this version in the git tree and > >> Wiki. > >> Why? > > > > I restructured the directory structure a bit in 3.19. It is now in > > the ./tools directory, because it belongs to the rsyslogd tool. > > > > Here is a quick link: > > > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > > > > All the best, > > Rainer > >> > >> Best Rio. > >> > >> On 2008/07/09, at 17:46, Rainer Gerhards wrote: > >> > >>> Hi Rio-san, > >>> > >>> many thanks for your help! Much appreciated. > >>> > >>> As far as the documentation goes, there are actually two places. One > >>> is > >>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > >>> (called "the core set") of html documentation. The reasoning is as > >>> follows: The wiki is a great resource and easy to edit. However, it > >>> does > >>> not provide the versioning we need. For example, there are changes > >>> in > >>> the way things can be configured between v2 and v3. In the wiki, > >>> version-specifc pages are a bit hard to find. Also, the wiki is only > >>> available online. > >>> > >>> To solve this, we have the core set: html files which document > >>> rsyslog's > >>> config directives as well as generic use cases. This is contained in > >>> git > >>> and under version control. An exactly matching version of the > >>> documentation is distributed with each source tarball. > >>> > >>> I think it would be very useful to have a Japanese version of (at > >>> least > >>> some) documents of the core set. If you are interested in that, I > >>> would > >>> add your translations to git and the distribution set. > >>> > >>>> Do you have a plan to do i10n/i18n with rsyslog? > >>> > >>> Could you elaborate a little bit on this? Unfortunately, I do > >>> neither > >>> speak Japanse nor any other language with a non-western script. I > >>> asked > >>> around several times and some folks from Japan told me that rsyslog > >>> works well with Japanese characters. But I am always interested in > >>> enhancing this support - I just need some guidance what is needed. > >>> > >>> Looking forward to your relpy, > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>>> Sent: Wednesday, July 09, 2008 10:34 AM > >>>> To: rsyslog at lists.adiscon.com > >>>> Subject: [rsyslog] I'd like to translate documents into Japanese. > >>>> > >>>> Hi List, > >>>> > >>>> $subject > >>>> And I guess that documents are managed with Wiki. > >>>> Do you have a plan to do i10n/i18n with rsyslog? > >>>> > >>>> Best Rio. > >>>> > >>>> > >>> ####################################################################### > >>>> # > >>>> Ryo Fujita > >>>> Senior Solution Architect, RHCE > >>>> Red Hat K.K. > >>>> TEL +81-3-5798-8500 > >>>> FAX +81-3-5798-8599 > >>>> Ebisu Neonato 8F > >>>> 4-1-18 Ebisu, Shibuya-ku, > >>>> Tokyo Japan 1500013 > >>>> > >>> ####################################################################### > >>>> # > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> ######################################################################## > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> ######################################################################## > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 9 12:32:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 12:32:38 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Message-ID: <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> Hi Rio-san and others, one question comes up on my mind: > > Oh, I find the file rsyslogd.8 in tools directory. > > I'll start translating it first. The man file is inside the tools directory, but not at all language specific. Do you think it makes sense to create ./tools/mans// (with lan being the ISO 2-char language code) Or is there any other suggestion? Feedback appreciated, Rainer From rfujita at redhat.com Wed Jul 9 17:10:46 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 00:10:46 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Message-ID: <4FD7940E-8E4F-41BB-AC31-578082706F55@redhat.com> Hi Rainer-san, > I'll begin this when I start the next devel branch. I scheduled it for > this week, but I got a bug report for the beta, which I would like to > fix first. But in any case... soon! It's a good timing. I need some time to investigate the best way to include translation stuff into the tree. > Excellent. If you go through ./doc, you'll also notice that there is > room for improvement. I think the structure is not very good today. > Suggestions are welcome :) My colleagues are the best to ask how to restructure the tree in order to translate docs. And one of my colleagues, Noriko Mizumoto-san is the editor of fedora- trans ja project. http://docs.fedoraproject.org/translation-quick-start-guide/ja_JP/index.html I'll ask her tomorrow. Best Rio. On 2008/07/09, at 19:06, Rainer Gerhards wrote: > Hi Rio-san, > > On Wed, 2008-07-09 at 18:59 +0900, Ryo Fujita wrote: >> Hi Rainer-san, >> >> On 2008/07/09, at 18:43, Rainer Gerhards wrote: >> >>> I am trying my best - doesn't always work :) >> >> >> I'm working for Red Hat's Tokyo office. >> Do you live in Germany? >> If so, we can co-work in the daytime! > > indeed, I do. So we can at least try. > >> >>> The doc set is in the ./doc subfolder, with .html files (obviously >>> not >>> what you looked for :)). I think it would be a good time now to >>> create >>> a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over >>> the >>> files and I would integrate them as soon as I have them availalbe. >> >> I agree with you totally. >> Because Fedora 9 has included rsyslog, >> lots Japanese and other non English/German speakers have started >> using >> it. >> Sooner or later, we'll need to create ./doc/other_lang directories :) > > I'll begin this when I start the next devel branch. I scheduled it for > this week, but I got a bug report for the beta, which I would like to > fix first. But in any case... soon! > >> >>> I restructured the directory structure a bit in 3.19. It is now in >>> the ./tools directory, because it belongs to the rsyslogd tool. >>> >>> Here is a quick link: >>> >>> http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD >> >> >> Oh, I find the file rsyslogd.8 in tools directory. >> I'll start translating it first. > > Excellent. If you go through ./doc, you'll also notice that there is > room for improvement. I think the structure is not very good today. > Suggestions are welcome :) > > Rainer >> >> Best Rio. >> >> On 2008/07/09, at 18:43, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: >>>> Hi Rainer-san, >>>> >>>> Thank you for your quick response! >>>> Too quick, I was surprised. >>> >>> I am trying my best - doesn't always work :) >>> >>>> >>>> I understand the status of the project and what I should do. >>>> >>>> #The reason why I asked how the contributors collaborate for >>>> rsyslog >>>> was that I cloned the repository with git and I couldn't find any >>>> docbook or .po files. >>> >>> The doc set is in the ./doc subfolder, with .html files (obviously >>> not >>> what you looked for :)). I think it would be a good time now to >>> create >>> a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over >>> the >>> files and I would integrate them as soon as I have them availalbe. >>>> >>>> As soon as possible, I'll start to translate docs into Japanese. >>>> >>>> BTW, Fedora 9 has the man page, rsyslogd(8) with the version >>>> 3.14.0. >>>> But I can't find the source file of this version in the git tree >>>> and >>>> Wiki. >>>> Why? >>> >>> I restructured the directory structure a bit in 3.19. It is now in >>> the ./tools directory, because it belongs to the rsyslogd tool. >>> >>> Here is a quick link: >>> >>> http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD >>> >>> >>> All the best, >>> Rainer >>>> >>>> Best Rio. >>>> >>>> On 2008/07/09, at 17:46, Rainer Gerhards wrote: >>>> >>>>> Hi Rio-san, >>>>> >>>>> many thanks for your help! Much appreciated. >>>>> >>>>> As far as the documentation goes, there are actually two places. >>>>> One >>>>> is >>>>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a >>>>> set >>>>> (called "the core set") of html documentation. The reasoning is as >>>>> follows: The wiki is a great resource and easy to edit. However, >>>>> it >>>>> does >>>>> not provide the versioning we need. For example, there are changes >>>>> in >>>>> the way things can be configured between v2 and v3. In the wiki, >>>>> version-specifc pages are a bit hard to find. Also, the wiki is >>>>> only >>>>> available online. >>>>> >>>>> To solve this, we have the core set: html files which document >>>>> rsyslog's >>>>> config directives as well as generic use cases. This is >>>>> contained in >>>>> git >>>>> and under version control. An exactly matching version of the >>>>> documentation is distributed with each source tarball. >>>>> >>>>> I think it would be very useful to have a Japanese version of (at >>>>> least >>>>> some) documents of the core set. If you are interested in that, I >>>>> would >>>>> add your translations to git and the distribution set. >>>>> >>>>>> Do you have a plan to do i10n/i18n with rsyslog? >>>>> >>>>> Could you elaborate a little bit on this? Unfortunately, I do >>>>> neither >>>>> speak Japanse nor any other language with a non-western script. I >>>>> asked >>>>> around several times and some folks from Japan told me that >>>>> rsyslog >>>>> works well with Japanese characters. But I am always interested in >>>>> enhancing this support - I just need some guidance what is needed. >>>>> >>>>> Looking forward to your relpy, >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>>> Sent: Wednesday, July 09, 2008 10:34 AM >>>>>> To: rsyslog at lists.adiscon.com >>>>>> Subject: [rsyslog] I'd like to translate documents into Japanese. >>>>>> >>>>>> Hi List, >>>>>> >>>>>> $subject >>>>>> And I guess that documents are managed with Wiki. >>>>>> Do you have a plan to do i10n/i18n with rsyslog? >>>>>> >>>>>> Best Rio. >>>>>> >>>>>> >>>>> ####################################################################### >>>>>> # >>>>>> Ryo Fujita >>>>>> Senior Solution Architect, RHCE >>>>>> Red Hat K.K. >>>>>> TEL +81-3-5798-8500 >>>>>> FAX +81-3-5798-8599 >>>>>> Ebisu Neonato 8F >>>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>>> Tokyo Japan 1500013 >>>>>> >>>>> ####################################################################### >>>>>> # >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> ######################################################################## >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> ######################################################################## >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> ######################################################################## >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> ######################################################################## >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rfujita at redhat.com Wed Jul 9 17:15:01 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 00:15:01 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> Message-ID: <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> Hi List, +1 But I'll ask it to my colleagues who contribute to translation matter tomorrow. I'm sure that they know the smartest way to do it. Best Rio. On 2008/07/09, at 19:32, Rainer Gerhards wrote: > Hi Rio-san and others, > > one question comes up on my mind: > >>> Oh, I find the file rsyslogd.8 in tools directory. >>> I'll start translating it first. > > The man file is inside the tools directory, but not at all language > specific. Do you think it makes sense to create > > ./tools/mans// (with lan being the ISO 2-char language code) > > Or is there any other suggestion? > > Feedback appreciated, > Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 17:16:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 9 Jul 2008 17:16:01 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com><5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com><1215596597.7184.29.camel@rgf9dev.intern.adiscon.com><1215597971.7184.33.camel@rgf9dev.intern.adiscon.com><1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED92@grfint2.intern.adiscon.com> Thanks, I am standing by for more votes and the insight! :) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Wednesday, July 09, 2008 5:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] I'd like to translate documents into Japanese. > > Hi List, > > +1 > > But I'll ask it to my colleagues who contribute to translation matter > tomorrow. > I'm sure that they know the smartest way to do it. > > Best Rio. > > On 2008/07/09, at 19:32, Rainer Gerhards wrote: > > > Hi Rio-san and others, > > > > one question comes up on my mind: > > > >>> Oh, I find the file rsyslogd.8 in tools directory. > >>> I'll start translating it first. > > > > The man file is inside the tools directory, but not at all language > > specific. Do you think it makes sense to create > > > > ./tools/mans// (with lan being the ISO 2-char language code) > > > > Or is there any other suggestion? > > > > Feedback appreciated, > > Rainer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Thu Jul 10 08:51:21 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 15:51:21 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure Message-ID: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> Hi List, I consulted with my colleagues about the smartest way to reconstruct the documents of rsyslog. 1.Work Flow matter The attached image is just an idea for rational flowchart to make translation easier. #If this ML forbids me to attach any files, you can see it on my site. http://rio.tc/2008/07/10-144007.php I'm worried that this flow will give the contributers much trouble. - HTML(from Wiki) to DocBook conversion 'tidy' will help us a little, but exporter of media Wiki is poor to export 'well-formed HTML'. We need to edit and check them manually. Of course, I willingly do this. But it may take a number of days. - Writers need to edit DocBook XML after the reconstruction. As you know, HTML format is not good for a source of multi-output. DocBook XML is a perfect one except for edit :) - Automake related matter There can be two streams to generate man and html from DocBook XML. One : When you edit DocBook, you commit DocBook, HTML and man files to git tree. Of course this method requires us to prepare an environment where you can use xsltproc/docbook2man. Two : By adding some sequences into Makefile, HTML and man files can be generated automatically. All persons who want to build rsyslog from tar ball need to prepare the environment. And we need to put some check routines for dependencies into Makefile. 2.git tree reconstruct only with 'doc' directory `-- doc |-- Makefile.am |-- conf | |-- en | | `-- rsyslog-example.conf | |-- OTHER_LANGUAGES(iso code) | `-- jp | `-- rsyslog-example.conf # annotations are translated. |-- html | |-- en | | |-- bugs.html | | |-- OTHER_HTMLS | | `-- version_naming.html | |-- OTHER_LANGUAGES(iso code) | `-- jp | |-- bugs.html | |-- OTHER_HTMLS | `-- version_naming.html |-- images | |-- gssapi.png | |-- OTHER_IMAGES | `-- tls_cert_ca.jpg |-- man | |-- en | | |-- man5 | | | `-- rsyslog.conf.5 | | `-- man8 | | `-- rsyslogd.8 | |-- OTHER_LANGUAGES(iso code) | `-- jp | |-- man5 | | `-- rsyslog.conf.5 | `-- man8 | `-- rsyslogd.8 `-- src |-- dias | |-- classes.dia | |-- OTHER_DIA_FILES | `-- tls_cert_ca.dia `-- docbook |-- en | |-- bugs.xml | |-- rsyslog.conf.5.xml | |-- rsyslogd.8.xml | |-- OTHER_XMLS | `-- version_naming.xml `-- ja |-- bugs.html |-- rsyslog.conf.5.xml |-- rsyslogd.8.xml |-- OTHER_XMLS `-- version_naming.html # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ directory. Your suggestions will be highly appreciated!! Best Rio. ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Thu Jul 10 09:03:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 10 Jul 2008 09:03:18 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> Hi Rio-san, I need to leave soon, but a quick feedback: the proposal looks *very* interesting. I need to digest it a bit. I have no experience with DocBook, so I probably need to check that out, first. That may take a short while (should you happen to know a good "getting started guide", I'd grateful if you send me a link ;)). From first thought, I would prefer to generate the html et al outputs from a single source (where only that source is in git). Feedback from other list members is also appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Thursday, July 10, 2008 8:51 AM > To: rsyslog-users > Subject: [rsyslog] Suggestion for Documentation Reconstructure > > Hi List, > > I consulted with my colleagues about the smartest way to reconstruct > the documents of rsyslog. > > 1.Work Flow matter > The attached image is just an idea for rational flowchart to make > translation easier. > #If this ML forbids me to attach any files, you can see it on my site. > http://rio.tc/2008/07/10-144007.php > > I'm worried that this flow will give the contributers much trouble. > > - HTML(from Wiki) to DocBook conversion > 'tidy' will help us a little, but exporter of media Wiki is poor to > export 'well-formed HTML'. > We need to edit and check them manually. > Of course, I willingly do this. But it may take a number of days. > > - Writers need to edit DocBook XML after the reconstruction. > As you know, HTML format is not good for a source of multi-output. > DocBook XML is a perfect one except for edit :) > > - Automake related matter > There can be two streams to generate man and html from DocBook XML. > One : When you edit DocBook, you commit DocBook, HTML and man files > to git tree. > Of course this method requires us to prepare an > environment where you can use xsltproc/docbook2man. > Two : By adding some sequences into Makefile, HTML and man files > can be generated automatically. > All persons who want to build rsyslog from tar ball need > to prepare the environment. > And we need to put some check routines for dependencies > into Makefile. > > 2.git tree reconstruct only with 'doc' directory > > `-- doc > |-- Makefile.am > |-- conf > | |-- en > | | `-- rsyslog-example.conf > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | `-- rsyslog-example.conf # annotations are translated. > |-- html > | |-- en > | | |-- bugs.html > | | |-- OTHER_HTMLS > | | `-- version_naming.html > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | |-- bugs.html > | |-- OTHER_HTMLS > | `-- version_naming.html > |-- images > | |-- gssapi.png > | |-- OTHER_IMAGES > | `-- tls_cert_ca.jpg > |-- man > | |-- en > | | |-- man5 > | | | `-- rsyslog.conf.5 > | | `-- man8 > | | `-- rsyslogd.8 > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | |-- man5 > | | `-- rsyslog.conf.5 > | `-- man8 > | `-- rsyslogd.8 > `-- src > |-- dias > | |-- classes.dia > | |-- OTHER_DIA_FILES > | `-- tls_cert_ca.dia > `-- docbook > |-- en > | |-- bugs.xml > | |-- rsyslog.conf.5.xml > | |-- rsyslogd.8.xml > | |-- OTHER_XMLS > | `-- version_naming.xml > `-- ja > |-- bugs.html > |-- rsyslog.conf.5.xml > |-- rsyslogd.8.xml > |-- OTHER_XMLS > `-- version_naming.html > > # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ > directory. > > Your suggestions will be highly appreciated!! > > Best Rio. > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # From rfujita at redhat.com Thu Jul 10 09:39:37 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 16:39:37 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> Message-ID: <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Hi Rainer-san, DocBook.org is here. http://www.docbook.org/ And IBM has a good guide for beginners. http://www.ibm.com/developerworks/xml/library/xml-matters3.html Recently, Red Hat uses DocBook format to generate web pages ,PDFs and so on. For example, https://www.redhat.com/docs/manuals/enterprise/ A document having both html and PDF is generated from DocBook. gnome-doc-utils and kdesdk-utils include very convenient tools to handle DocBook and related formats. Best Rio. On 2008/07/10, at 16:03, Rainer Gerhards wrote: > Hi Rio-san, > > I need to leave soon, but a quick feedback: the proposal looks *very* > interesting. I need to digest it a bit. I have no experience with > DocBook, so I probably need to check that out, first. That may take a > short while (should you happen to know a good "getting started guide", > I'd grateful if you send me a link ;)). From first thought, I would > prefer to generate the html et al outputs from a single source (where > only that source is in git). > > Feedback from other list members is also appreciated. > > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Thursday, July 10, 2008 8:51 AM >> To: rsyslog-users >> Subject: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi List, >> >> I consulted with my colleagues about the smartest way to reconstruct >> the documents of rsyslog. >> >> 1.Work Flow matter >> The attached image is just an idea for rational flowchart to make >> translation easier. >> #If this ML forbids me to attach any files, you can see it on my >> site. >> http://rio.tc/2008/07/10-144007.php >> >> I'm worried that this flow will give the contributers much trouble. >> >> - HTML(from Wiki) to DocBook conversion >> 'tidy' will help us a little, but exporter of media Wiki is poor to >> export 'well-formed HTML'. >> We need to edit and check them manually. >> Of course, I willingly do this. But it may take a number of days. >> >> - Writers need to edit DocBook XML after the reconstruction. >> As you know, HTML format is not good for a source of multi-output. >> DocBook XML is a perfect one except for edit :) >> >> - Automake related matter >> There can be two streams to generate man and html from DocBook XML. >> One : When you edit DocBook, you commit DocBook, HTML and man files >> to git tree. >> Of course this method requires us to prepare an >> environment where you can use xsltproc/docbook2man. >> Two : By adding some sequences into Makefile, HTML and man files >> can be generated automatically. >> All persons who want to build rsyslog from tar ball need >> to prepare the environment. >> And we need to put some check routines for dependencies >> into Makefile. >> >> 2.git tree reconstruct only with 'doc' directory >> >> `-- doc >> |-- Makefile.am >> |-- conf >> | |-- en >> | | `-- rsyslog-example.conf >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | `-- rsyslog-example.conf # annotations are translated. >> |-- html >> | |-- en >> | | |-- bugs.html >> | | |-- OTHER_HTMLS >> | | `-- version_naming.html >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | |-- bugs.html >> | |-- OTHER_HTMLS >> | `-- version_naming.html >> |-- images >> | |-- gssapi.png >> | |-- OTHER_IMAGES >> | `-- tls_cert_ca.jpg >> |-- man >> | |-- en >> | | |-- man5 >> | | | `-- rsyslog.conf.5 >> | | `-- man8 >> | | `-- rsyslogd.8 >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | |-- man5 >> | | `-- rsyslog.conf.5 >> | `-- man8 >> | `-- rsyslogd.8 >> `-- src >> |-- dias >> | |-- classes.dia >> | |-- OTHER_DIA_FILES >> | `-- tls_cert_ca.dia >> `-- docbook >> |-- en >> | |-- bugs.xml >> | |-- rsyslog.conf.5.xml >> | |-- rsyslogd.8.xml >> | |-- OTHER_XMLS >> | `-- version_naming.xml >> `-- ja >> |-- bugs.html >> |-- rsyslog.conf.5.xml >> |-- rsyslogd.8.xml >> |-- OTHER_XMLS >> `-- version_naming.html >> >> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ >> directory. >> >> Your suggestions will be highly appreciated!! >> >> Best Rio. >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 11 16:35:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:35:59 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. It includes all features and bug-fixes from the beta, among others the ability to natively send email. There is one minor bugfix for 3.17.5 included, where RainerScript did not always properly detect syntax errors. This is a recommended update for all v3-stable branch users. Change Log: http://www.rsyslog.com/Article254.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml Please note that I also created a 3.16.3 branch, immediately outdated, for those that prefer to keep the previous version for a couple of days. That version contains an important bugfix which prevents a big memory leak when running the queue in one of the disk modes. This patch is of course also included in 3.18.0. Find 3.16.3 here: http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml Just to make sure I am not misunderstood: 3.18.0 is the current v3-stable. Other v3-stables are not officially supported (except, of course, via a commercial support contract). 3.16.3 will not receive any patches in the future. So I suggest moving over to 3.18.0. It has been tested for over two month and looks quite ready for prime time. I would also like to express my sincere thanks to all of you who provided bug reports and patches. This is immensely helpful. I hope the release is useful. As always, feedback is appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Fri Jul 11 16:42:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:42:04 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> HI all, Michael Biebl just told me that some html files are missing in the 3.18.0 tarball. Please hold download, I am updating it right now... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, July 11, 2008 4:36 PM > To: rsyslog-users > Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > Hi all, > > we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. > It includes all features and bug-fixes from the beta, among others the > ability to natively send email. There is one minor bugfix for 3.17.5 > included, where RainerScript did not always properly detect syntax > errors. This is a recommended update for all v3-stable branch users. > > Change Log: > > http://www.rsyslog.com/Article254.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > Please note that I also created a 3.16.3 branch, immediately outdated, > for those that prefer to keep the previous version for a couple of > days. > That version contains an important bugfix which prevents a big memory > leak when running the queue in one of the disk modes. This patch is of > course also included in 3.18.0. Find 3.16.3 here: > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > Just to make sure I am not misunderstood: 3.18.0 is the current > v3-stable. Other v3-stables are not officially supported (except, of > course, via a commercial support contract). 3.16.3 will not receive any > patches in the future. So I suggest moving over to 3.18.0. It has been > tested for over two month and looks quite ready for prime time. > > I would also like to express my sincere thanks to all of you who > provided bug reports and patches. This is immensely helpful. > > I hope the release is useful. As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 11 16:57:27 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:57:27 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDBC@grfint2.intern.adiscon.com> ...and now this issue is fixed. The new tarball is online. If in doubt which one you loaded, please check the md5sum. The issue was that some documentation files were not properly included inside the tarball. The automatic checks did not detect that. The problem seems to have lurked for quite a while. Many thanks to Michael Biebl for pointing out the issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, July 11, 2008 4:42 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > HI all, > > Michael Biebl just told me that some html files are missing in the > 3.18.0 tarball. Please hold download, I am updating it right now... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Friday, July 11, 2008 4:36 PM > > To: rsyslog-users > > Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > > > Hi all, > > > > we have just released rsyslog 3.18.0, a new v3-stable version. > Rsyslog > > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta > branch. > > It includes all features and bug-fixes from the beta, among others > the > > ability to natively send email. There is one minor bugfix for 3.17.5 > > included, where RainerScript did not always properly detect syntax > > errors. This is a recommended update for all v3-stable branch users. > > > > Change Log: > > > > http://www.rsyslog.com/Article254.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > > > Please note that I also created a 3.16.3 branch, immediately > outdated, > > for those that prefer to keep the previous version for a couple of > > days. > > That version contains an important bugfix which prevents a big memory > > leak when running the queue in one of the disk modes. This patch is > of > > course also included in 3.18.0. Find 3.16.3 here: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > > > Just to make sure I am not misunderstood: 3.18.0 is the current > > v3-stable. Other v3-stables are not officially supported (except, of > > course, via a commercial support contract). 3.16.3 will not receive > any > > patches in the future. So I suggest moving over to 3.18.0. It has > been > > tested for over two month and looks quite ready for prime time. > > > > I would also like to express my sincere thanks to all of you who > > provided bug reports and patches. This is immensely helpful. > > > > I hope the release is useful. As always, feedback is appreciated. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Jul 14 12:06:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 14 Jul 2008 12:06:00 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Hi all, Michael Biebl noticed a small inconsistency in kernel log format: 3.16.x (and before): Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. 3.18.0: Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg started. Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. As you can see, there is a space after kernel: in the pre-3.18.0 messages. This also brings us down to a general inconsistency: Unfortunately, RFC 3164 defines the after TAG not as a delimiter but as part of the CONTENT of the message. This leads to some inconsistency in practice. Depending on who generated the message, there may be a space after the TAG or not. If it is, that space must be submitted as part of the message itself. In versions up to 3.16.x, kernel log messages were generated by old code, which did not know about tag and simply generated a non-structured message. With 3.17.x and above, the klog code is rewritten and correctly fills in all properties. This leads to the "missing" space, because the space is neither part of the tag nor part of the message. I have now three options: 1. leave as is (without a space) In that case, some log parsers may run into troubles 2. add a at the end of the TAG That will probably lead to even more confusion, as matches against TAG would need to include that space. 3. add a in front of each kernel message This comes close to the previous mode, but is "a bit" clumpsy and hackish. It will also make all database tables etc have messages starting with . Similar as 2, MSG matching rules are affected (but I consider this less severe, as matches inside the MSG are usually driven by searches and not absolute positions). I think that real solutions are numbers 1 and 3, with 3 probably having the best arguments. However, I have to admit I do not like this option very much at all. A fourth option would be to add two new property replacer argument "add at end if none is already there" and "drop first if there is one". I could then modify the default templates to use the first one with Tag and the later with MSG (instead of the regular ones). That would be a good fix, probably my favorite, BUT it would also cause format change, now in other messages. Plus, it is a little bit to code and I prefer to do as few code changes to a stable as possible. This issue has unfortunately been lurking ever since the beginning of rsyslog (as it is rooted in rfc 3164) and I should probably have it addressed earlier. But as you see: it is ugly... ;) So question now: what do the list members think? Feedback is highly appreciated. Thanks, Rainer From pascal at impressionet.ch Mon Jul 14 13:31:57 2008 From: pascal at impressionet.ch (Pascal Mainini) Date: Mon, 14 Jul 2008 13:31:57 +0200 Subject: [rsyslog] MARK frequency, too Message-ID: <487B392D.3090408@impressionet.ch> Hi all I'm refering to this mail: http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html I'm having the same issue as described in the Mail above, $MarkMessagePeriod gets completly ignored. I tried putting it at various places in the config-file, with tabs or whitespaces etc., without any result. I'm using the package from debian etch (on debian etch, of course). Find below my configfile (with hostnames anonymized). /etc/rsyslog.d is empty, so no further config should get included... Also, options for rsyslogd are "-c3 -s ". Also, I was wondering: I have a selector *.* which directs anything into a logfile - but not the mark-messages. Why is it that way? And are there maybe other messages which are ignored by a *.* selector? Help would be very much appreciated! Thanks a lot in advance, kind regards, Pascal ---- # /etc/rsyslog.conf Configuration file for rsyslog v3. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) $ModLoad immark # provides --MARK-- message capability $MarkMessagePeriod 300 # provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use default timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # # Set the default permissions for all log files. # $FileOwner root $FileGroup adm $FileCreateMode 0640 # # Include all config files in /etc/rsyslog.d/ # $IncludeConfig /etc/rsyslog.d/*.conf ############### #### RULES #### ############### +hostA *.* -/var/log/remote/hostA.log & -/var/log/full -hostA +hostB *.* -/var/log/remote/hostB.log & -/var/log/full -hostB +hostC *.* -/var/log/remote/hostC.log & -/var/log/full -hostC ####################### #### LOCAL LOGGING #### ####################### +localhostname *.* -/var/log/full # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #*.* -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # #daemon.*;mail.*;\ # news.err;\ # *.=debug;*.=info;\ # *.=notice;*.=warn |/dev/xconsole # From rgerhards at hq.adiscon.com Mon Jul 14 13:43:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 14 Jul 2008 13:43:42 +0200 Subject: [rsyslog] MARK frequency, too In-Reply-To: <487B392D.3090408@impressionet.ch> References: <487B392D.3090408@impressionet.ch> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> Hi, looks like I overlooked hte other post you mention. I'll have a look into the issue, but need to finish some other things first. I'd appreciate if you could file a bug report, so that it will not be forgotten. You can do that at http://bugzilla.adiscon.com Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Pascal Mainini > Sent: Monday, July 14, 2008 1:32 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] MARK frequency, too > > Hi all > > I'm refering to this mail: > > http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html > > I'm having the same issue as described in the Mail above, > $MarkMessagePeriod gets completly ignored. I tried putting it at > various places in the config-file, with tabs or whitespaces etc., > without any result. I'm using the package from debian etch (on > debian etch, of course). > > Find below my configfile (with hostnames anonymized). > /etc/rsyslog.d is empty, so no further config should get included... > Also, options for rsyslogd are "-c3 -s ". > > Also, I was wondering: I have a selector *.* which directs anything > into a logfile - but not the mark-messages. Why is it that way? > And are there maybe other messages which are ignored by a *.* selector? > > Help would be very much appreciated! > > Thanks a lot in advance, kind regards, > > Pascal > > ---- > > # /etc/rsyslog.conf Configuration file for rsyslog v3. > # > # For more information see > # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html > > > ################# > #### MODULES #### > ################# > > $ModLoad imuxsock # provides support for local system logging > $ModLoad imklog # provides kernel logging support (previously done by > rklogd) > > $ModLoad immark # provides --MARK-- message capability > $MarkMessagePeriod 300 > > # provides UDP syslog reception > $ModLoad imudp > $UDPServerRun 514 > > # provides TCP syslog reception > #$ModLoad imtcp > #$InputTCPServerRun 514 > > > ########################### > #### GLOBAL DIRECTIVES #### > ########################### > > # > # Use default timestamp format. > # To enable high precision timestamps, comment out the following line. > # > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # > # Set the default permissions for all log files. > # > $FileOwner root > $FileGroup adm > $FileCreateMode 0640 > > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/*.conf > > > ############### > #### RULES #### > ############### > > +hostA > *.* -/var/log/remote/hostA.log > & -/var/log/full > -hostA > > +hostB > *.* -/var/log/remote/hostB.log > & -/var/log/full > -hostB > > +hostC > *.* -/var/log/remote/hostC.log > & -/var/log/full > -hostC > > ####################### > #### LOCAL LOGGING #### > ####################### > > +localhostname > > *.* -/var/log/full > # > # First some standard log files. Log by facility. > # > auth,authpriv.* /var/log/auth.log > *.*;auth,authpriv.none -/var/log/syslog > #*.* -/var/log/syslog > #cron.* /var/log/cron.log > daemon.* -/var/log/daemon.log > kern.* -/var/log/kern.log > lpr.* -/var/log/lpr.log > mail.* -/var/log/mail.log > user.* -/var/log/user.log > > # > # Logging for the mail system. Split it up so that > # it is easy to write scripts to parse these files. > # > mail.info -/var/log/mail.info > mail.warn -/var/log/mail.warn > mail.err /var/log/mail.err > > # > # Logging for INN news system. > # > news.crit /var/log/news/news.crit > news.err /var/log/news/news.err > news.notice -/var/log/news/news.notice > > # > # Some "catch-all" log files. > # > *.=debug;\ > auth,authpriv.none;\ > news.none;mail.none -/var/log/debug > *.=info;*.=notice;*.=warn;\ > auth,authpriv.none;\ > cron,daemon.none;\ > mail,news.none -/var/log/messages > > # > # Emergencies are sent to everybody logged in. > # > *.emerg * > > # > # I like to have messages displayed on the console, but only on a > virtual > # console I usually leave idle. > # > #daemon,mail.*;\ > # news.=crit;news.=err;news.=notice;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn /dev/tty8 > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use > it, > # you must invoke `xconsole' with the `-file' option: > # > # $ xconsole -file /dev/xconsole [...] > # > # NOTE: adjust the list below, or you'll go crazy if you have a > reasonably > # busy site.. > # > #daemon.*;mail.*;\ > # news.err;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn |/dev/xconsole > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From aoz.syn at gmail.com Mon Jul 14 16:17:31 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 14 Jul 2008 08:17:31 -0600 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: <4255c2570807140717h1a118d86n6dccc31777a98111@mail.gmail.com> > So question now: what do the list members think? I think I may be partially re-stating #4 by saying this, but I think the most pragmatic approach would be to implement strict RFC compliance, then add non-default options to break that in one way or another for backwards compatibility. Make it explicit (in the option name and/or documentation) that by doing so they're breaking RFC 3164; you could even request a courtesy contact to you or the list so you could roughly track its use and thereby when you can drop it (if ever). Of course, that makes more code and more complexity; I'd also support leaving people with inflexible/broken parsers out in the cold, but I'm pretty heartless like that. From mbiebl at gmail.com Tue Jul 15 14:54:48 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Tue, 15 Jul 2008 14:54:48 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: 2008/7/14 Rainer Gerhards : > Hi all, > > Michael Biebl noticed a small inconsistency in kernel log format: > > 3.16.x (and before): > Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): > wlan0: link becomes ready > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > 3.18.0: > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > started. > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. another example: Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down See the <6> after the kernel: tag. It's also new. Is this some kind of log level, set by the kernel? I have never seen, that other syslogger (like syslog-ng, sysklogd set this). > > As you can see, there is a space after kernel: in the pre-3.18.0 > messages. This also brings us down to a general inconsistency: > > Unfortunately, RFC 3164 defines the after TAG not as a delimiter > but as part of the CONTENT of the message. This leads to some > inconsistency in practice. Depending on who generated the message, there > may be a space after the TAG or not. If it is, that space must be > submitted as part of the message itself. > > In versions up to 3.16.x, kernel log messages were generated by old > code, which did not know about tag and simply generated a non-structured > message. With 3.17.x and above, the klog code is rewritten and correctly > fills in all properties. This leads to the "missing" space, because the > space is neither part of the tag nor part of the message. > > I have now three options: > > 1. leave as is (without a space) > In that case, some log parsers may run into troubles This was a rather vague concern of mine. I don't actually know, if other software is affected by this change. > 2. add a at the end of the TAG > That will probably lead to even more confusion, as matches against TAG > would need to include that space. > > 3. add a in front of each kernel message > This comes close to the previous mode, but is "a bit" clumpsy and > hackish. It will also make all database tables etc have messages > starting with . Similar as 2, MSG matching rules are affected (but I > consider this less severe, as matches inside the MSG are usually driven > by searches and not absolute positions). Just curious: For other log messages, that are not generated by the kernel, you have a space after the programname: tag, e.g. Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 Is the space after dhclient: part of the message or part of the :programname tag? Imho, if there is no clear definition in the RFC, how the message should be written, it's best to stay backwards compatible. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Tue Jul 15 15:03:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 15:03:42 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > 2008/7/14 Rainer Gerhards : > > Hi all, > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > 3.16.x (and before): > > Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): > > wlan0: link becomes ready > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > 3.18.0: > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > > started. > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > another example: > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > See the <6> after the kernel: tag. It's also new. > Is this some kind of log level, set by the kernel? This does not look correct. Under FreeBSD, it is typical to have a PRI inside kernel messages. But for linux kernels I tested with, I did not see it and (I think) so I did not support it. But if you see it on Debian, it seems to happen. A cure is to use the same logic that is used for BSD. > I have never seen, that other syslogger (like syslog-ng, sysklogd set this). > > > > > As you can see, there is a space after kernel: in the pre-3.18.0 > > messages. This also brings us down to a general inconsistency: > > > > Unfortunately, RFC 3164 defines the after TAG not as a delimiter > > but as part of the CONTENT of the message. This leads to some > > inconsistency in practice. Depending on who generated the message, there > > may be a space after the TAG or not. If it is, that space must be > > submitted as part of the message itself. > > > > In versions up to 3.16.x, kernel log messages were generated by old > > code, which did not know about tag and simply generated a non-structured > > message. With 3.17.x and above, the klog code is rewritten and correctly > > fills in all properties. This leads to the "missing" space, because the > > space is neither part of the tag nor part of the message. > > > > I have now three options: > > > > 1. leave as is (without a space) > > In that case, some log parsers may run into troubles > > This was a rather vague concern of mine. I don't actually know, if > other software is affected by this change. I share this concern, but also have no additional information. I wanted to point out the options. I have to admit I am still undecided. But that nobody violently objected so far makes this look like it is not so important... > > > 2. add a at the end of the TAG > > That will probably lead to even more confusion, as matches against TAG > > would need to include that space. > > > > 3. add a in front of each kernel message > > This comes close to the previous mode, but is "a bit" clumpsy and > > hackish. It will also make all database tables etc have messages > > starting with . Similar as 2, MSG matching rules are affected (but I > > consider this less severe, as matches inside the MSG are usually driven > > by searches and not absolute positions). > > Just curious: For other log messages, that are not generated by the > kernel, you have a space after the programname: tag, e.g. > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > Is the space after dhclient: part of the message or part of the > :programname tag? In our private communication, I thought it is part of TAG. But that is wrong. It actually is part of MSG - at least for legacy/RFC3164 syslog. For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is well-defined and part of neither. There, it is a delimiter (as one would expect). I you use rsyslog only with syslog-protocol messages (and the syslog-protocol templates), this is more or less no issue. But who does today...? ;) The real problem is that legacy syslog has many different interpretations and we break one or the other in either way... > > Imho, if there is no clear definition in the RFC, how the message > should be written, it's best to stay backwards compatible. Is that a vote for option 3, stick a space in front of each kernel message? ;) > > > Cheers, > Michael > > From rgerhards at hq.adiscon.com Tue Jul 15 16:10:20 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:10:20 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2and 3.18.0 In-Reply-To: <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> Michael, I have reviewed some of the code in question. It looks like a bug. It's actually not a klog driver issue, the PRI parsing is done at an upper klog layer and thus should work for linux as well. I would like to generate a version with some additional instrumentation. Could you run it and report the results back (maybe via private mail to keep the list free of noise). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 15, 2008 3:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] slight kernel log format inconsistency between > 3.16.2and 3.18.0 > > On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > > 2008/7/14 Rainer Gerhards : > > > Hi all, > > > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > > > 3.16.x (and before): > > > Jul 11 17:33:28 pluto kernel: [14177.432349] > ADDRCONF(NETDEV_CHANGE): > > > wlan0: link becomes ready > > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > > > > 3.18.0: > > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > > > started. > > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > > > another example: > > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > > See the <6> after the kernel: tag. It's also new. > > Is this some kind of log level, set by the kernel? > > This does not look correct. Under FreeBSD, it is typical to have a PRI > inside kernel messages. But for linux kernels I tested with, I did not > see it and (I think) so I did not support it. But if you see it on > Debian, it seems to happen. A cure is to use the same logic that is > used > for BSD. > > > I have never seen, that other syslogger (like syslog-ng, sysklogd set > this). > > > > > > > > As you can see, there is a space after kernel: in the pre- > 3.18.0 > > > messages. This also brings us down to a general inconsistency: > > > > > > Unfortunately, RFC 3164 defines the after TAG not as a > delimiter > > > but as part of the CONTENT of the message. This leads to some > > > inconsistency in practice. Depending on who generated the message, > there > > > may be a space after the TAG or not. If it is, that space must be > > > submitted as part of the message itself. > > > > > > In versions up to 3.16.x, kernel log messages were generated by old > > > code, which did not know about tag and simply generated a non- > structured > > > message. With 3.17.x and above, the klog code is rewritten and > correctly > > > fills in all properties. This leads to the "missing" space, because > the > > > space is neither part of the tag nor part of the message. > > > > > > I have now three options: > > > > > > 1. leave as is (without a space) > > > In that case, some log parsers may run into troubles > > > > This was a rather vague concern of mine. I don't actually know, if > > other software is affected by this change. > > I share this concern, but also have no additional information. I wanted > to point out the options. I have to admit I am still undecided. But > that > nobody violently objected so far makes this look like it is not so > important... > > > > > > 2. add a at the end of the TAG > > > That will probably lead to even more confusion, as matches against > TAG > > > would need to include that space. > > > > > > 3. add a in front of each kernel message > > > This comes close to the previous mode, but is "a bit" clumpsy and > > > hackish. It will also make all database tables etc have messages > > > starting with . Similar as 2, MSG matching rules are affected > (but I > > > consider this less severe, as matches inside the MSG are usually > driven > > > by searches and not absolute positions). > > > > Just curious: For other log messages, that are not generated by the > > kernel, you have a space after the programname: tag, e.g. > > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > > > Is the space after dhclient: part of the message or part of the > > :programname tag? > > In our private communication, I thought it is part of TAG. But that is > wrong. It actually is part of MSG - at least for legacy/RFC3164 syslog. > > For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is > well-defined and part of neither. There, it is a delimiter (as one > would > expect). I you use rsyslog only with syslog-protocol messages (and the > syslog-protocol templates), this is more or less no issue. But who does > today...? ;) > > The real problem is that legacy syslog has many different > interpretations and we break one or the other in either way... > > > > > Imho, if there is no clear definition in the RFC, how the message > > should be written, it's best to stay backwards compatible. > > Is that a vote for option 3, stick a space in front of each kernel > message? ;) > > > > > > > Cheers, > > Michael > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 15 16:22:07 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:22:07 +0200 Subject: [rsyslog] MARK frequency, too In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> References: <487B392D.3090408@impressionet.ch> <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDC@grfint2.intern.adiscon.com> FYI: I have found and fixed this bug: http://bugzilla.adiscon.com/process_bug.cgi Thanks for reporting :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, July 14, 2008 1:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] MARK frequency, too > > Hi, > > looks like I overlooked hte other post you mention. I'll have a look > into the issue, but need to finish some other things first. I'd > appreciate if you could file a bug report, so that it will not be > forgotten. You can do that at > > http://bugzilla.adiscon.com > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Pascal Mainini > > Sent: Monday, July 14, 2008 1:32 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] MARK frequency, too > > > > Hi all > > > > I'm refering to this mail: > > > > http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html > > > > I'm having the same issue as described in the Mail above, > > $MarkMessagePeriod gets completly ignored. I tried putting it at > > various places in the config-file, with tabs or whitespaces etc., > > without any result. I'm using the package from debian etch (on > > debian etch, of course). > > > > Find below my configfile (with hostnames anonymized). > > /etc/rsyslog.d is empty, so no further config should get included... > > Also, options for rsyslogd are "-c3 -s ". > > > > Also, I was wondering: I have a selector *.* which directs anything > > into a logfile - but not the mark-messages. Why is it that way? > > And are there maybe other messages which are ignored by a *.* > selector? > > > > Help would be very much appreciated! > > > > Thanks a lot in advance, kind regards, > > > > Pascal > > > > ---- > > > > # /etc/rsyslog.conf Configuration file for rsyslog v3. > > # > > # For more information see > > # > /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html > > > > > > ################# > > #### MODULES #### > > ################# > > > > $ModLoad imuxsock # provides support for local system logging > > $ModLoad imklog # provides kernel logging support (previously done > by > > rklogd) > > > > $ModLoad immark # provides --MARK-- message capability > > $MarkMessagePeriod 300 > > > > # provides UDP syslog reception > > $ModLoad imudp > > $UDPServerRun 514 > > > > # provides TCP syslog reception > > #$ModLoad imtcp > > #$InputTCPServerRun 514 > > > > > > ########################### > > #### GLOBAL DIRECTIVES #### > > ########################### > > > > # > > # Use default timestamp format. > > # To enable high precision timestamps, comment out the following > line. > > # > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > > > # > > # Set the default permissions for all log files. > > # > > $FileOwner root > > $FileGroup adm > > $FileCreateMode 0640 > > > > # > > # Include all config files in /etc/rsyslog.d/ > > # > > $IncludeConfig /etc/rsyslog.d/*.conf > > > > > > ############### > > #### RULES #### > > ############### > > > > +hostA > > *.* -/var/log/remote/hostA.log > > & -/var/log/full > > -hostA > > > > +hostB > > *.* -/var/log/remote/hostB.log > > & -/var/log/full > > -hostB > > > > +hostC > > *.* -/var/log/remote/hostC.log > > & -/var/log/full > > -hostC > > > > ####################### > > #### LOCAL LOGGING #### > > ####################### > > > > +localhostname > > > > *.* -/var/log/full > > # > > # First some standard log files. Log by facility. > > # > > auth,authpriv.* /var/log/auth.log > > *.*;auth,authpriv.none -/var/log/syslog > > #*.* -/var/log/syslog > > #cron.* /var/log/cron.log > > daemon.* -/var/log/daemon.log > > kern.* -/var/log/kern.log > > lpr.* -/var/log/lpr.log > > mail.* -/var/log/mail.log > > user.* -/var/log/user.log > > > > # > > # Logging for the mail system. Split it up so that > > # it is easy to write scripts to parse these files. > > # > > mail.info -/var/log/mail.info > > mail.warn -/var/log/mail.warn > > mail.err /var/log/mail.err > > > > # > > # Logging for INN news system. > > # > > news.crit /var/log/news/news.crit > > news.err /var/log/news/news.err > > news.notice -/var/log/news/news.notice > > > > # > > # Some "catch-all" log files. > > # > > *.=debug;\ > > auth,authpriv.none;\ > > news.none;mail.none -/var/log/debug > > *.=info;*.=notice;*.=warn;\ > > auth,authpriv.none;\ > > cron,daemon.none;\ > > mail,news.none -/var/log/messages > > > > # > > # Emergencies are sent to everybody logged in. > > # > > *.emerg * > > > > # > > # I like to have messages displayed on the console, but only on a > > virtual > > # console I usually leave idle. > > # > > #daemon,mail.*;\ > > # news.=crit;news.=err;news.=notice;\ > > # *.=debug;*.=info;\ > > # *.=notice;*.=warn /dev/tty8 > > > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use > > it, > > # you must invoke `xconsole' with the `-file' option: > > # > > # $ xconsole -file /dev/xconsole [...] > > # > > # NOTE: adjust the list below, or you'll go crazy if you have a > > reasonably > > # busy site.. > > # > > #daemon.*;mail.*;\ > > # news.err;\ > > # *.=debug;*.=info;\ > > # *.=notice;*.=warn |/dev/xconsole > > # > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 15 16:38:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:38:46 +0200 Subject: [rsyslog] rsyslog 3.19.10 (beta) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDF@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.10. This starts a new iteration of the beta branch, bringing all the great features of the previous development version to it. It contains all features from 3.19.9 and also a number of bug fixes. The most important one is for a bad memory leak that occurred if queues were running in one of the disk-queuing modes. Also, a number of cross-platform problems on FreeBSD were sorted out. This is a suggested update for all beta branch users. Just in case you wonder: at this time there is NO development branch version available. One will be announced in the not so distant future. ChangeLog: http://www.rsyslog.com/Article256.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-120.phtml As always, feedback is appreciated. I hope this release is useful. Rainer Gerhards From hks.private at gmail.com Tue Jul 15 20:14:39 2008 From: hks.private at gmail.com ((private) HKS) Date: Tue, 15 Jul 2008 14:14:39 -0400 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: FYI, the links at the rsyslog status page are a bit funky: http://www.rsyslog.com/doc-status.html Beta 3.19.10's download link takes you to a page describing 3.17.3 beta with a download link for 3.19.10. v3 stable 3.18.0's download link takes you to a page describing 3.19.10 beta with a download link for the same. v2 stable 2.0.5's download link takes you to a page mostly describing 2.0.5 but with the rsyslog version misreported (and perhaps md5 checksum?). -HKS On Fri, Jul 11, 2008 at 10:35 AM, Rainer Gerhards wrote: > Hi all, > > we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. > It includes all features and bug-fixes from the beta, among others the > ability to natively send email. There is one minor bugfix for 3.17.5 > included, where RainerScript did not always properly detect syntax > errors. This is a recommended update for all v3-stable branch users. > > Change Log: > > http://www.rsyslog.com/Article254.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > Please note that I also created a 3.16.3 branch, immediately outdated, > for those that prefer to keep the previous version for a couple of days. > That version contains an important bugfix which prevents a big memory > leak when running the queue in one of the disk modes. This patch is of > course also included in 3.18.0. Find 3.16.3 here: > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > Just to make sure I am not misunderstood: 3.18.0 is the current > v3-stable. Other v3-stables are not officially supported (except, of > course, via a commercial support contract). 3.16.3 will not receive any > patches in the future. So I suggest moving over to 3.18.0. It has been > tested for over two month and looks quite ready for prime time. > > I would also like to express my sincere thanks to all of you who > provided bug reports and patches. This is immensely helpful. > > I hope the release is useful. As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Wed Jul 16 08:30:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 16 Jul 2008 08:30:56 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDE3@grfint2.intern.adiscon.com> Hi HKS, Thanks, looks like I had a bad time with the links... Corrected now. I have also checked 2.0.5, the version was incorrect (copy&paste error), but the md5sum is correct. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Tuesday, July 15, 2008 8:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > FYI, the links at the rsyslog status page are a bit funky: > http://www.rsyslog.com/doc-status.html > > Beta 3.19.10's download link takes you to a page describing 3.17.3 > beta with a download link for 3.19.10. > > v3 stable 3.18.0's download link takes you to a page describing > 3.19.10 beta with a download link for the same. > > v2 stable 2.0.5's download link takes you to a page mostly describing > 2.0.5 but with the rsyslog version misreported (and perhaps md5 > checksum?). > > -HKS > > On Fri, Jul 11, 2008 at 10:35 AM, Rainer Gerhards > wrote: > > Hi all, > > > > we have just released rsyslog 3.18.0, a new v3-stable version. > Rsyslog > > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta > branch. > > It includes all features and bug-fixes from the beta, among others > the > > ability to natively send email. There is one minor bugfix for 3.17.5 > > included, where RainerScript did not always properly detect syntax > > errors. This is a recommended update for all v3-stable branch users. > > > > Change Log: > > > > http://www.rsyslog.com/Article254.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > > > Please note that I also created a 3.16.3 branch, immediately > outdated, > > for those that prefer to keep the previous version for a couple of > days. > > That version contains an important bugfix which prevents a big memory > > leak when running the queue in one of the disk modes. This patch is > of > > course also included in 3.18.0. Find 3.16.3 here: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > > > Just to make sure I am not misunderstood: 3.18.0 is the current > > v3-stable. Other v3-stables are not officially supported (except, of > > course, via a commercial support contract). 3.16.3 will not receive > any > > patches in the future. So I suggest moving over to 3.18.0. It has > been > > tested for over two month and looks quite ready for prime time. > > > > I would also like to express my sincere thanks to all of you who > > provided bug reports and patches. This is immensely helpful. > > > > I hope the release is useful. As always, feedback is appreciated. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 17 12:43:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 12:43:54 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Hi all, I just got an idea for a feature inspired by the ramlog project. I have blogged about it: http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h tml If you have some minutes to spare, please review and provide feedback. Thanks, Rainer From mic at npgx.com.au Thu Jul 17 13:09:27 2008 From: mic at npgx.com.au (Michael Mansour) Date: Thu, 17 Jul 2008 22:09:27 +1100 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <20080717110306.M99550@npgx.com.au> Hi Rainer, > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. I've read and honestly have no opinion on it :) The last time I tried delayed writes with rsyslog the machines I set them up on basically died (load averages above 30, so they were complete unresponsive). I log rsyslog to a MySQL DB (for maillogs only) and it took me a while to realise it was the rsyslog changes I made that killed the mail servers. They even hung on boot ie. would not complete a boot process. When I got into single user mode and didn't start up any services, I was able to change the rsyslog.conf file back to my older one, and everything was fine again. This was some versions back (but still the 3.x stable series) but because these were production servers, I didn't bother analysing what went wrong or why it happened, I was just glad I had my mail servers back again and working. I might give this a second attempt at some stage this year, because I like the feature for queuing entries if the DB is down, but until then, I really don't have much to say about the delayed writes stuff :) Michael. > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From rgerhards at hq.adiscon.com Thu Jul 17 13:45:49 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 13:45:49 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <20080717110306.M99550@npgx.com.au> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <20080717110306.M99550@npgx.com.au> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE12@grfint2.intern.adiscon.com> Hi Michael, I think you were hit by this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=86 interestingly, nobody who experienced it bothered to report, until recently. Thus it was for quite a while inside the core engine ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Mansour > Sent: Thursday, July 17, 2008 1:09 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback request: in-memory buffering messages > > Hi Rainer, > > > Hi all, > > > > I just got an idea for a feature inspired by the ramlog project. I > have > > blogged about it: > > > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram- > buffer.h > > tml > > > > If you have some minutes to spare, please review and provide > feedback. > > I've read and honestly have no opinion on it :) > > The last time I tried delayed writes with rsyslog the machines I set > them up > on basically died (load averages above 30, so they were complete > unresponsive). > > I log rsyslog to a MySQL DB (for maillogs only) and it took me a while > to > realise it was the rsyslog changes I made that killed the mail servers. > They > even hung on boot ie. would not complete a boot process. When I got > into > single user mode and didn't start up any services, I was able to change > the > rsyslog.conf file back to my older one, and everything was fine again. > > This was some versions back (but still the 3.x stable series) but > because > these were production servers, I didn't bother analysing what went > wrong or > why it happened, I was just glad I had my mail servers back again and > working. > > I might give this a second attempt at some stage this year, because I > like the > feature for queuing entries if the DB is down, but until then, I really > don't > have much to say about the delayed writes stuff :) > > Michael. > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > ------- End of Original Message ------- > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Thu Jul 17 16:13:10 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 17 Jul 2008 23:13:10 +0900 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <5646AC09-7597-4549-8193-BD6C187DEA7E@redhat.com> Hi Rainer-san, +1 How to use logging facilities varies according to circumstances. For example, RHEL is used for from edge to mission critical. Our customers want robust logging functions, like writing to MySQL or transporting via TCP/RELP. But some users want 'greener' machines. It sounds interesting to combine the feature to write log messages to RAM and the powertop project. http://www.lesswatts.org/projects/powertop/ Best Rio. On 2008/07/17, at 19:43, Rainer Gerhards wrote: > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I > have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From hks.private at gmail.com Thu Jul 17 16:19:11 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 17 Jul 2008 10:19:11 -0400 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: This is certainly something I would use. I've got a number of mission critical devices running off of flash drives (OpenBSD firewalls), and the less they have to write to disk the better. This would also be handy on a high-use database machine, where excessive disk writes could have a significant impact on service delivery. I doubt it would see very widely used, but it would make rsyslog appealing to another niche. -HKS On Thu, Jul 17, 2008 at 6:43 AM, Rainer Gerhards wrote: > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From aoz.syn at gmail.com Thu Jul 17 16:20:07 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 08:20:07 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml Prior art exists and I think it's well worth the effort, particularly for busy servers on reliable power. If I read your post right, you're considering something rather similar to the syslog-ng options 'flush_lines' and 'flush_timeout'? From rgerhards at hq.adiscon.com Thu Jul 17 16:22:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 16:22:43 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> > Prior art exists and I think it's well worth the effort, particularly > for busy servers on reliable power. If I read your post right, you're > considering something rather similar to the syslog-ng options > 'flush_lines' and 'flush_timeout'? I actually (really ;)) do not know syslog-ng and keep myself somewhat away from it to prevent accidental steal of "whatever" from there. But now that you raise the point: could you quickly describe how it works. >From your mail, it sounds like what I am thinking about... Rainer From mbiebl at gmail.com Thu Jul 17 16:32:08 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Thu, 17 Jul 2008 16:32:08 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Message-ID: Hi, I just wanted to say, that I like the idea of using docbook/xml to generate the (html) documentation (and possibly the man pages). Imo this would make the documentation more consistent (consistent font sizes, headers, footers etc.), more usable (You'd get something like a TOC and possibly better cross-linking) and perhaps easier keeping up-to-date. Cheers, Michael 2008/7/10 Ryo Fujita : > Hi Rainer-san, > > DocBook.org is here. > http://www.docbook.org/ > > And IBM has a good guide for beginners. > http://www.ibm.com/developerworks/xml/library/xml-matters3.html > > Recently, Red Hat uses DocBook format to generate web pages ,PDFs and > so on. > For example, > https://www.redhat.com/docs/manuals/enterprise/ > A document having both html and PDF is generated from DocBook. > > gnome-doc-utils and kdesdk-utils include very convenient tools to > handle DocBook and related formats. > > Best Rio. > > On 2008/07/10, at 16:03, Rainer Gerhards wrote: > >> Hi Rio-san, >> >> I need to leave soon, but a quick feedback: the proposal looks *very* >> interesting. I need to digest it a bit. I have no experience with >> DocBook, so I probably need to check that out, first. That may take a >> short while (should you happen to know a good "getting started guide", >> I'd grateful if you send me a link ;)). From first thought, I would >> prefer to generate the html et al outputs from a single source (where >> only that source is in git). >> >> Feedback from other list members is also appreciated. >> >> Rainer >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>> Sent: Thursday, July 10, 2008 8:51 AM >>> To: rsyslog-users >>> Subject: [rsyslog] Suggestion for Documentation Reconstructure >>> >>> Hi List, >>> >>> I consulted with my colleagues about the smartest way to reconstruct >>> the documents of rsyslog. >>> >>> 1.Work Flow matter >>> The attached image is just an idea for rational flowchart to make >>> translation easier. >>> #If this ML forbids me to attach any files, you can see it on my >>> site. >>> http://rio.tc/2008/07/10-144007.php >>> >>> I'm worried that this flow will give the contributers much trouble. >>> >>> - HTML(from Wiki) to DocBook conversion >>> 'tidy' will help us a little, but exporter of media Wiki is poor to >>> export 'well-formed HTML'. >>> We need to edit and check them manually. >>> Of course, I willingly do this. But it may take a number of days. >>> >>> - Writers need to edit DocBook XML after the reconstruction. >>> As you know, HTML format is not good for a source of multi-output. >>> DocBook XML is a perfect one except for edit :) >>> >>> - Automake related matter >>> There can be two streams to generate man and html from DocBook XML. >>> One : When you edit DocBook, you commit DocBook, HTML and man files >>> to git tree. >>> Of course this method requires us to prepare an >>> environment where you can use xsltproc/docbook2man. >>> Two : By adding some sequences into Makefile, HTML and man files >>> can be generated automatically. >>> All persons who want to build rsyslog from tar ball need >>> to prepare the environment. >>> And we need to put some check routines for dependencies >>> into Makefile. >>> >>> 2.git tree reconstruct only with 'doc' directory >>> >>> `-- doc >>> |-- Makefile.am >>> |-- conf >>> | |-- en >>> | | `-- rsyslog-example.conf >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | `-- rsyslog-example.conf # annotations are translated. >>> |-- html >>> | |-- en >>> | | |-- bugs.html >>> | | |-- OTHER_HTMLS >>> | | `-- version_naming.html >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | |-- bugs.html >>> | |-- OTHER_HTMLS >>> | `-- version_naming.html >>> |-- images >>> | |-- gssapi.png >>> | |-- OTHER_IMAGES >>> | `-- tls_cert_ca.jpg >>> |-- man >>> | |-- en >>> | | |-- man5 >>> | | | `-- rsyslog.conf.5 >>> | | `-- man8 >>> | | `-- rsyslogd.8 >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | |-- man5 >>> | | `-- rsyslog.conf.5 >>> | `-- man8 >>> | `-- rsyslogd.8 >>> `-- src >>> |-- dias >>> | |-- classes.dia >>> | |-- OTHER_DIA_FILES >>> | `-- tls_cert_ca.dia >>> `-- docbook >>> |-- en >>> | |-- bugs.xml >>> | |-- rsyslog.conf.5.xml >>> | |-- rsyslogd.8.xml >>> | |-- OTHER_XMLS >>> | `-- version_naming.xml >>> `-- ja >>> |-- bugs.html >>> |-- rsyslog.conf.5.xml >>> |-- rsyslogd.8.xml >>> |-- OTHER_XMLS >>> `-- version_naming.html >>> >>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ >>> directory. >>> >>> Your suggestions will be highly appreciated!! >>> >>> Best Rio. >>> >>> >> ####################################################################### >>> # >>> Ryo Fujita >>> Senior Solution Architect, RHCE >>> Red Hat K.K. >>> TEL +81-3-5798-8500 >>> FAX +81-3-5798-8599 >>> Ebisu Neonato 8F >>> 4-1-18 Ebisu, Shibuya-ku, >>> Tokyo Japan 1500013 >>> >> ####################################################################### >>> # >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From aoz.syn at gmail.com Thu Jul 17 17:08:22 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 09:08:22 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> Message-ID: <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> > I actually (really ;)) do not know syslog-ng and keep myself somewhat > away from it to prevent accidental steal of "whatever" from there. But > now that you raise the point: could you quickly describe how it works. > From your mail, it sounds like what I am thinking about... I 100% agree with and applaud that separation. As you've stated before, they make a good product and don't warrant any interference. Not touching the docs myself (since I know too much of it by heart anyway), flush_lines allows the admin to configure a particular number of log entries to buffer before forcing a flush to disk, whereas flush_timeout configures the maximum interval between disk flushes. Unlike ramlog, it seems to do this with internal allocations and doesn't rely on a ramdisk. I usually set it rather conservatively (20 and 600 respectively), but I can definitely see being more aggressive on a dedicated log collector with critical power or a laptop in ultra-low power mode. This has been a feature of the public version of syslog-ng for as long as I can remember (or four years, whichever is sooner ;). Combined with disk queues I can see a very nice tiered approach to handling extremely high volumes of log data in a rather reliable manner. From rgerhards at hq.adiscon.com Thu Jul 17 18:29:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 18:29:30 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> Message-ID: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> On Thu, 2008-07-17 at 09:08 -0600, RB wrote: > > I actually (really ;)) do not know syslog-ng and keep myself somewhat > > away from it to prevent accidental steal of "whatever" from there. But > > now that you raise the point: could you quickly describe how it works. > > From your mail, it sounds like what I am thinking about... > > I 100% agree with and applaud that separation. As you've stated > before, they make a good product and don't warrant any interference. Definitely. > > Not touching the docs myself (since I know too much of it by heart > anyway), flush_lines allows the admin to configure a particular number > of log entries to buffer before forcing a flush to disk, whereas > flush_timeout configures the maximum interval between disk flushes. > Unlike ramlog, it seems to do this with internal allocations and > doesn't rely on a ramdisk. I usually set it rather conservatively (20 > and 600 respectively), but I can definitely see being more aggressive > on a dedicated log collector with critical power or a laptop in > ultra-low power mode. This begins to make more and more sense. Just one thing to make sure we are talking along the same lines. The ramdisk approach provides a way to save the IO while still being able to look at the log lines as fast as they come in. If, in rsyslog, I create a memory buffer, I can persist lines to disk only after it is "time to write them to disk" (whatever that triggers). So you will not be able to observe the messages while they stick inside rsyslog's queue. I guess for many cases this is not really relevant. And of course, it is something that must be configurable on an action basis, so different files may use different settings. Thinking more about a potential algorithm, I tend to think it would probably useful to write log files in chunks of n bytes instead of n lines. I am thinking along the lines of matching up the output buffer with the disk sector size (or any multiple of it) so that partial writes of the same sector are avoided. Of course, that involves some stating of the file to obtain the size of the first buffer to be written (filesize mod sector [allocation unit] size). I also requires me to find out the allocation unit size for a given file system. It obviously involves writing incomplete lines. It requires additional code. So the best solution may actually be to permit partial writes (as initially thought) but recommend large buffers. The overall effect could justify the performance impact from doing non-optimal writes. But I am in too much detail. The actual questions are: a) is it OK that log data is visible only after the (write) delay? b) does it sound useful to buffer based on allocation unit sizes? Thanks, Rainer > > This has been a feature of the public version of syslog-ng for as long > as I can remember (or four years, whichever is sooner ;). Combined > with disk queues I can see a very nice tiered approach to handling > extremely high volumes of log data in a rather reliable manner. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 17 18:33:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 18:33:38 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Message-ID: <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> I still haven't found the time to have an in-depth look at docbook, but I strongly tend to agree to Michael's conclusion. It looks like a problem solver for some ugly problems I am facing. I will probably re-evaluate the release goal for 3.21.x. It may happen that I push back the script engine evolution to after summer to do some cleanup, performance improvement, doc work and maybe a rewrite of the file output handler. I find it a bit hard to think what is more important, but I think we can stay with the non-scriptable config for two more month. Maybe I can add custom functions as a smaller improvement sometime while doing the other things. In any case, I am happy to have Rio-san over here and as such someone to ask for solutions to the doc problem. Thanks a lot! Rainer On Thu, 2008-07-17 at 16:32 +0200, Michael Biebl wrote: > Hi, > > I just wanted to say, that I like the idea of using docbook/xml to > generate the (html) documentation (and possibly the man pages). > Imo this would make the documentation more consistent (consistent font > sizes, headers, footers etc.), more usable (You'd get something like a > TOC and possibly better cross-linking) and perhaps easier keeping > up-to-date. > > Cheers, > Michael > > > 2008/7/10 Ryo Fujita : > > Hi Rainer-san, > > > > DocBook.org is here. > > http://www.docbook.org/ > > > > And IBM has a good guide for beginners. > > http://www.ibm.com/developerworks/xml/library/xml-matters3.html > > > > Recently, Red Hat uses DocBook format to generate web pages ,PDFs and > > so on. > > For example, > > https://www.redhat.com/docs/manuals/enterprise/ > > A document having both html and PDF is generated from DocBook. > > > > gnome-doc-utils and kdesdk-utils include very convenient tools to > > handle DocBook and related formats. > > > > Best Rio. > > > > On 2008/07/10, at 16:03, Rainer Gerhards wrote: > > > >> Hi Rio-san, > >> > >> I need to leave soon, but a quick feedback: the proposal looks *very* > >> interesting. I need to digest it a bit. I have no experience with > >> DocBook, so I probably need to check that out, first. That may take a > >> short while (should you happen to know a good "getting started guide", > >> I'd grateful if you send me a link ;)). From first thought, I would > >> prefer to generate the html et al outputs from a single source (where > >> only that source is in git). > >> > >> Feedback from other list members is also appreciated. > >> > >> Rainer > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>> Sent: Thursday, July 10, 2008 8:51 AM > >>> To: rsyslog-users > >>> Subject: [rsyslog] Suggestion for Documentation Reconstructure > >>> > >>> Hi List, > >>> > >>> I consulted with my colleagues about the smartest way to reconstruct > >>> the documents of rsyslog. > >>> > >>> 1.Work Flow matter > >>> The attached image is just an idea for rational flowchart to make > >>> translation easier. > >>> #If this ML forbids me to attach any files, you can see it on my > >>> site. > >>> http://rio.tc/2008/07/10-144007.php > >>> > >>> I'm worried that this flow will give the contributers much trouble. > >>> > >>> - HTML(from Wiki) to DocBook conversion > >>> 'tidy' will help us a little, but exporter of media Wiki is poor to > >>> export 'well-formed HTML'. > >>> We need to edit and check them manually. > >>> Of course, I willingly do this. But it may take a number of days. > >>> > >>> - Writers need to edit DocBook XML after the reconstruction. > >>> As you know, HTML format is not good for a source of multi-output. > >>> DocBook XML is a perfect one except for edit :) > >>> > >>> - Automake related matter > >>> There can be two streams to generate man and html from DocBook XML. > >>> One : When you edit DocBook, you commit DocBook, HTML and man files > >>> to git tree. > >>> Of course this method requires us to prepare an > >>> environment where you can use xsltproc/docbook2man. > >>> Two : By adding some sequences into Makefile, HTML and man files > >>> can be generated automatically. > >>> All persons who want to build rsyslog from tar ball need > >>> to prepare the environment. > >>> And we need to put some check routines for dependencies > >>> into Makefile. > >>> > >>> 2.git tree reconstruct only with 'doc' directory > >>> > >>> `-- doc > >>> |-- Makefile.am > >>> |-- conf > >>> | |-- en > >>> | | `-- rsyslog-example.conf > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | `-- rsyslog-example.conf # annotations are translated. > >>> |-- html > >>> | |-- en > >>> | | |-- bugs.html > >>> | | |-- OTHER_HTMLS > >>> | | `-- version_naming.html > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | |-- bugs.html > >>> | |-- OTHER_HTMLS > >>> | `-- version_naming.html > >>> |-- images > >>> | |-- gssapi.png > >>> | |-- OTHER_IMAGES > >>> | `-- tls_cert_ca.jpg > >>> |-- man > >>> | |-- en > >>> | | |-- man5 > >>> | | | `-- rsyslog.conf.5 > >>> | | `-- man8 > >>> | | `-- rsyslogd.8 > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | |-- man5 > >>> | | `-- rsyslog.conf.5 > >>> | `-- man8 > >>> | `-- rsyslogd.8 > >>> `-- src > >>> |-- dias > >>> | |-- classes.dia > >>> | |-- OTHER_DIA_FILES > >>> | `-- tls_cert_ca.dia > >>> `-- docbook > >>> |-- en > >>> | |-- bugs.xml > >>> | |-- rsyslog.conf.5.xml > >>> | |-- rsyslogd.8.xml > >>> | |-- OTHER_XMLS > >>> | `-- version_naming.xml > >>> `-- ja > >>> |-- bugs.html > >>> |-- rsyslog.conf.5.xml > >>> |-- rsyslogd.8.xml > >>> |-- OTHER_XMLS > >>> `-- version_naming.html > >>> > >>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ > >>> directory. > >>> > >>> Your suggestions will be highly appreciated!! > >>> > >>> Best Rio. > >>> > >>> > >> ####################################################################### > >>> # > >>> Ryo Fujita > >>> Senior Solution Architect, RHCE > >>> Red Hat K.K. > >>> TEL +81-3-5798-8500 > >>> FAX +81-3-5798-8599 > >>> Ebisu Neonato 8F > >>> 4-1-18 Ebisu, Shibuya-ku, > >>> Tokyo Japan 1500013 > >>> > >> ####################################################################### > >>> # > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > ######################################################################## > > Ryo Fujita > > Senior Solution Architect, RHCE > > Red Hat K.K. > > TEL +81-3-5798-8500 > > FAX +81-3-5798-8599 > > Ebisu Neonato 8F > > 4-1-18 Ebisu, Shibuya-ku, > > Tokyo Japan 1500013 > > ######################################################################## > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > From martinmie at PartyGaming.com Thu Jul 17 18:41:19 2008 From: martinmie at PartyGaming.com (Martin Mielke) Date: Thu, 17 Jul 2008 18:41:19 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com><4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com><4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: <0B1B9163138571439EAF804646F3F6AA050C3078@GIBSVWIN004X.partygaming.local> Hi all, [ big snip ] > > The actual questions are: > > a) is it OK that log data is visible only after the (write) delay? Hmmm.. no. This would eliminate the "magic" of watching system events in real-time as in the commonly used 'tail -n0 -f /var/log/messages' which is often used to troubleshoot problems... Unless you provide some tools to implement such functionality by watching the log events in memory... huh... My 2 cents... Martin > b) does it sound useful to buffer based on allocation unit sizes? > > Thanks, > Rainer > > > > This has been a feature of the public version of syslog-ng for as long > > as I can remember (or four years, whichever is sooner ;). Combined > > with disk queues I can see a very nice tiered approach to handling > > extremely high volumes of log data in a rather reliable manner. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Thu Jul 17 20:16:03 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 17 Jul 2008 14:16:03 -0400 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: > The actual questions are: > > a) is it OK that log data is visible only after the (write) delay? > b) does it sound useful to buffer based on allocation unit sizes? a) It depends on the situation. If you're using this to conserve power or flash-disk writes, then it's going to be a pain. If you're using this to tune performance in a high log-volume environment, then this is an acceptable tradeoff. Perhaps a secondary script that would send a signal to rsyslogd and print the lines to STDOUT would be an acceptable compromise? b) It's more granular than many (most?) users will need, including myself, but I'm sure some people will find this very useful. -HKS From aoz.syn at gmail.com Thu Jul 17 21:11:24 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 13:11:24 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: <4255c2570807171211q76742249o6ae3917feec70a62@mail.gmail.com> > a) is it OK that log data is visible only after the (write) delay? For my purposes it is; I know some of the other responses have been that it's not, but perhaps setting up a signal (maybe USR2) for triggering the flush mechanism would help those situations. Of course, people wanting "real-time" data would probably either leave off the configurations (knowing they're doing so at the expense of performance) or change it at runtime and HUP. > b) does it sound useful to buffer based on allocation unit sizes? I had actually considered suggesting this as a third option (differentiating from other implementations), but abandoned it since I was unsure of the value and felt the implementation may be overly complex. If you implement byte/allocation-aligned flushes, my preference would be to have it in addition to but mutually (configuration) exclusive with record-aligned ones. Partial-sector writes are less of a concern to me in most cases than getting the whole message to disk, but I can again see the usefulness of allocation alignment in a very high-volume environment or one where disk performance is constrained. Regardless, it's a fascinating development and one that will come in very handy. From rfujita at redhat.com Fri Jul 18 04:19:53 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 11:19:53 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> Message-ID: <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Hi, As I found an excellent template in the SRPM of docbook2X package, I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. First of all, let me finish the work related to man pages. I hope Rainer-san and other contributors devote yourself to make rsyslog better :) Best Rio. On 2008/07/18, at 1:33, Rainer Gerhards wrote: > I still haven't found the time to have an in-depth look at docbook, > but > I strongly tend to agree to Michael's conclusion. It looks like a > problem solver for some ugly problems I am facing. > > I will probably re-evaluate the release goal for 3.21.x. It may happen > that I push back the script engine evolution to after summer to do > some > cleanup, performance improvement, doc work and maybe a rewrite of the > file output handler. I find it a bit hard to think what is more > important, but I think we can stay with the non-scriptable config for > two more month. Maybe I can add custom functions as a smaller > improvement sometime while doing the other things. > > In any case, I am happy to have Rio-san over here and as such > someone to > ask for solutions to the doc problem. Thanks a lot! > > Rainer > > On Thu, 2008-07-17 at 16:32 +0200, Michael Biebl wrote: >> Hi, >> >> I just wanted to say, that I like the idea of using docbook/xml to >> generate the (html) documentation (and possibly the man pages). >> Imo this would make the documentation more consistent (consistent >> font >> sizes, headers, footers etc.), more usable (You'd get something >> like a >> TOC and possibly better cross-linking) and perhaps easier keeping >> up-to-date. >> >> Cheers, >> Michael >> >> >> 2008/7/10 Ryo Fujita : >>> Hi Rainer-san, >>> >>> DocBook.org is here. >>> http://www.docbook.org/ >>> >>> And IBM has a good guide for beginners. >>> http://www.ibm.com/developerworks/xml/library/xml-matters3.html >>> >>> Recently, Red Hat uses DocBook format to generate web pages ,PDFs >>> and >>> so on. >>> For example, >>> https://www.redhat.com/docs/manuals/enterprise/ >>> A document having both html and PDF is generated from DocBook. >>> >>> gnome-doc-utils and kdesdk-utils include very convenient tools to >>> handle DocBook and related formats. >>> >>> Best Rio. >>> >>> On 2008/07/10, at 16:03, Rainer Gerhards wrote: >>> >>>> Hi Rio-san, >>>> >>>> I need to leave soon, but a quick feedback: the proposal looks >>>> *very* >>>> interesting. I need to digest it a bit. I have no experience with >>>> DocBook, so I probably need to check that out, first. That may >>>> take a >>>> short while (should you happen to know a good "getting started >>>> guide", >>>> I'd grateful if you send me a link ;)). From first thought, I would >>>> prefer to generate the html et al outputs from a single source >>>> (where >>>> only that source is in git). >>>> >>>> Feedback from other list members is also appreciated. >>>> >>>> Rainer >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>> Sent: Thursday, July 10, 2008 8:51 AM >>>>> To: rsyslog-users >>>>> Subject: [rsyslog] Suggestion for Documentation Reconstructure >>>>> >>>>> Hi List, >>>>> >>>>> I consulted with my colleagues about the smartest way to >>>>> reconstruct >>>>> the documents of rsyslog. >>>>> >>>>> 1.Work Flow matter >>>>> The attached image is just an idea for rational flowchart to make >>>>> translation easier. >>>>> #If this ML forbids me to attach any files, you can see it on my >>>>> site. >>>>> http://rio.tc/2008/07/10-144007.php >>>>> >>>>> I'm worried that this flow will give the contributers much >>>>> trouble. >>>>> >>>>> - HTML(from Wiki) to DocBook conversion >>>>> 'tidy' will help us a little, but exporter of media Wiki is >>>>> poor to >>>>> export 'well-formed HTML'. >>>>> We need to edit and check them manually. >>>>> Of course, I willingly do this. But it may take a number of days. >>>>> >>>>> - Writers need to edit DocBook XML after the reconstruction. >>>>> As you know, HTML format is not good for a source of multi- >>>>> output. >>>>> DocBook XML is a perfect one except for edit :) >>>>> >>>>> - Automake related matter >>>>> There can be two streams to generate man and html from DocBook >>>>> XML. >>>>> One : When you edit DocBook, you commit DocBook, HTML and man >>>>> files >>>>> to git tree. >>>>> Of course this method requires us to prepare an >>>>> environment where you can use xsltproc/docbook2man. >>>>> Two : By adding some sequences into Makefile, HTML and man files >>>>> can be generated automatically. >>>>> All persons who want to build rsyslog from tar ball >>>>> need >>>>> to prepare the environment. >>>>> And we need to put some check routines for >>>>> dependencies >>>>> into Makefile. >>>>> >>>>> 2.git tree reconstruct only with 'doc' directory >>>>> >>>>> `-- doc >>>>> |-- Makefile.am >>>>> |-- conf >>>>> | |-- en >>>>> | | `-- rsyslog-example.conf >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | `-- rsyslog-example.conf # annotations are translated. >>>>> |-- html >>>>> | |-- en >>>>> | | |-- bugs.html >>>>> | | |-- OTHER_HTMLS >>>>> | | `-- version_naming.html >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | |-- bugs.html >>>>> | |-- OTHER_HTMLS >>>>> | `-- version_naming.html >>>>> |-- images >>>>> | |-- gssapi.png >>>>> | |-- OTHER_IMAGES >>>>> | `-- tls_cert_ca.jpg >>>>> |-- man >>>>> | |-- en >>>>> | | |-- man5 >>>>> | | | `-- rsyslog.conf.5 >>>>> | | `-- man8 >>>>> | | `-- rsyslogd.8 >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | |-- man5 >>>>> | | `-- rsyslog.conf.5 >>>>> | `-- man8 >>>>> | `-- rsyslogd.8 >>>>> `-- src >>>>> |-- dias >>>>> | |-- classes.dia >>>>> | |-- OTHER_DIA_FILES >>>>> | `-- tls_cert_ca.dia >>>>> `-- docbook >>>>> |-- en >>>>> | |-- bugs.xml >>>>> | |-- rsyslog.conf.5.xml >>>>> | |-- rsyslogd.8.xml >>>>> | |-- OTHER_XMLS >>>>> | `-- version_naming.xml >>>>> `-- ja >>>>> |-- bugs.html >>>>> |-- rsyslog.conf.5.xml >>>>> |-- rsyslogd.8.xml >>>>> |-- OTHER_XMLS >>>>> `-- version_naming.html >>>>> >>>>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./ >>>>> tools/ >>>>> directory. >>>>> >>>>> Your suggestions will be highly appreciated!! >>>>> >>>>> Best Rio. >>>>> >>>>> >>>> ####################################################################### >>>>> # >>>>> Ryo Fujita >>>>> Senior Solution Architect, RHCE >>>>> Red Hat K.K. >>>>> TEL +81-3-5798-8500 >>>>> FAX +81-3-5798-8599 >>>>> Ebisu Neonato 8F >>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>> Tokyo Japan 1500013 >>>>> >>>> ####################################################################### >>>>> # >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> ######################################################################## >>> Ryo Fujita >>> Senior Solution Architect, RHCE >>> Red Hat K.K. >>> TEL +81-3-5798-8500 >>> FAX +81-3-5798-8599 >>> Ebisu Neonato 8F >>> 4-1-18 Ebisu, Shibuya-ku, >>> Tokyo Japan 1500013 >>> ######################################################################## >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From mbiebl at gmail.com Fri Jul 18 04:38:31 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 18 Jul 2008 04:38:31 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Message-ID: 2008/7/18 Ryo Fujita : > Hi, > > As I found an excellent template in the SRPM of docbook2X package, > I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. Do you have a public git repo somewhere and a separate branch, where you make these changes? It would be interesting to track the progress. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rfujita at redhat.com Fri Jul 18 05:17:36 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 12:17:36 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Message-ID: <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> Hi Mike-san, I have no git repo now. So, I sent the files I converted to Rainer-san directly. But I recently feel a necessity to build a public repo. If I have a time to do it this weekend, I'll try it :) Best Rio. On 2008/07/18, at 11:38, Michael Biebl wrote: > 2008/7/18 Ryo Fujita : >> Hi, >> >> As I found an excellent template in the SRPM of docbook2X package, >> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > > Do you have a public git repo somewhere and a separate branch, where > you make these changes? > > It would be interesting to track the progress. > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 08:14:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 08:14:26 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com> <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Hi Rio-san, I would really appreciate if you could build a git repro. I think that would be a very valuable resource and help streamline the integration process :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 5:18 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi Mike-san, > > I have no git repo now. > So, I sent the files I converted to Rainer-san directly. > But I recently feel a necessity to build a public repo. > If I have a time to do it this weekend, I'll try it :) > > Best Rio. > > On 2008/07/18, at 11:38, Michael Biebl wrote: > > > 2008/7/18 Ryo Fujita : > >> Hi, > >> > >> As I found an excellent template in the SRPM of docbook2X package, > >> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > > > > Do you have a public git repo somewhere and a separate branch, where > > you make these changes? > > > > It would be interesting to track the progress. > > > > Cheers, > > Michael > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 08:43:33 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 15:43:33 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com> <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Message-ID: Hi all, Please check the following git repo. http://rio.tc/git/rsyslog Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? # It's my first time to build my own git repo :) On 2008/07/18, at 15:14, Rainer Gerhards wrote: > Hi Rio-san, > > I would really appreciate if you could build a git repro. I think that > would be a very valuable resource and help streamline the integration > process :) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 5:18 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi Mike-san, >> >> I have no git repo now. >> So, I sent the files I converted to Rainer-san directly. >> But I recently feel a necessity to build a public repo. >> If I have a time to do it this weekend, I'll try it :) >> >> Best Rio. >> >> On 2008/07/18, at 11:38, Michael Biebl wrote: >> >>> 2008/7/18 Ryo Fujita : >>>> Hi, >>>> >>>> As I found an excellent template in the SRPM of docbook2X package, >>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>> >>> Do you have a public git repo somewhere and a separate branch, where >>> you make these changes? >>> >>> It would be interesting to track the progress. >>> >>> Cheers, >>> Michael >>> >>> -- >>> Why is it that all of the instruments seeking intelligent life in > the >>> universe are pointed away from Earth? >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 09:24:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 09:24:19 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> Hi Rio-san, I get a "file not found" error. Unfortunately, I am far from being a git-wizard, so I can not provide much advise :( Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 8:44 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi all, > > Please check the following git repo. > http://rio.tc/git/rsyslog > Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? > > # It's my first time to build my own git repo :) > > On 2008/07/18, at 15:14, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > I would really appreciate if you could build a git repro. I think > that > > would be a very valuable resource and help streamline the integration > > process :) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Friday, July 18, 2008 5:18 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >> > >> Hi Mike-san, > >> > >> I have no git repo now. > >> So, I sent the files I converted to Rainer-san directly. > >> But I recently feel a necessity to build a public repo. > >> If I have a time to do it this weekend, I'll try it :) > >> > >> Best Rio. > >> > >> On 2008/07/18, at 11:38, Michael Biebl wrote: > >> > >>> 2008/7/18 Ryo Fujita : > >>>> Hi, > >>>> > >>>> As I found an excellent template in the SRPM of docbook2X package, > >>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > >>> > >>> Do you have a public git repo somewhere and a separate branch, > where > >>> you make these changes? > >>> > >>> It would be interesting to track the progress. > >>> > >>> Cheers, > >>> Michael > >>> > >>> -- > >>> Why is it that all of the instruments seeking intelligent life in > > the > >>> universe are pointed away from Earth? > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 09:32:59 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 16:32:59 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> Message-ID: <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> Hi Rainer-san, Did you access with git? Access with web browser may fail or get 404 not found. I succeeded as follows. $ git clone http://rio.tc/git/rsyslog Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ Getting alternates list for http://rio.tc/git/rsyslog Getting pack list for http://rio.tc/git/rsyslog Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 $ cd rsyslog $ git branch -r origin/HEAD origin/beta origin/master origin/queuememleak origin/rfujita origin/rscript origin/tests origin/v1-stable origin/v2-stable origin/v3-stable origin/v3stable-klogbug origin/wall $ git-pull http://rio.tc/git/rsyslog rfujita Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http Updating 37bac26..9d841de Fast forward man/Makefile.am | 101 ++++ man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ man/en/man8/rsyslogd.8 | 303 +++++++++++ man/entities/xinclude.dtd | 31 + man/ja/man8/rsyslogd.8 | 319 +++++++++++ man/xml-source/en/rsyslog.conf.5.xml | 994 +++++++++++++++++++++++++ ++++++++ man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ man/xml-source/ja/rsyslog.conf.5.xml | 997 +++++++++++++++++++++++++ +++++++++ man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ man/xslt/docbook.xsl | 127 +++++ man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ man/xslt/man.xsl | 70 +++ man/xslt/settings.xsl | 11 + 13 files changed, 5464 insertions(+), 0 deletions(-) create mode 100644 man/Makefile.am create mode 100644 man/en/man5/rsyslog.conf.5 create mode 100644 man/en/man8/rsyslogd.8 create mode 100644 man/entities/xinclude.dtd create mode 100644 man/ja/man8/rsyslogd.8 create mode 100644 man/xml-source/en/rsyslog.conf.5.xml create mode 100644 man/xml-source/en/rsyslogd.8.xml create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml create mode 100644 man/xml-source/ja/rsyslogd.8.xml create mode 100644 man/xslt/docbook.xsl create mode 100644 man/xslt/expand-sambadoc.xsl create mode 100644 man/xslt/man.xsl create mode 100644 man/xslt/settings.xsl On 2008/07/18, at 16:24, Rainer Gerhards wrote: > Hi Rio-san, > > I get a "file not found" error. Unfortunately, I am far from being a > git-wizard, so I can not provide much advise :( > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 8:44 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi all, >> >> Please check the following git repo. >> http://rio.tc/git/rsyslog >> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? >> >> # It's my first time to build my own git repo :) >> >> On 2008/07/18, at 15:14, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> I would really appreciate if you could build a git repro. I think >> that >>> would be a very valuable resource and help streamline the > integration >>> process :) >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Friday, July 18, 2008 5:18 AM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >>>> >>>> Hi Mike-san, >>>> >>>> I have no git repo now. >>>> So, I sent the files I converted to Rainer-san directly. >>>> But I recently feel a necessity to build a public repo. >>>> If I have a time to do it this weekend, I'll try it :) >>>> >>>> Best Rio. >>>> >>>> On 2008/07/18, at 11:38, Michael Biebl wrote: >>>> >>>>> 2008/7/18 Ryo Fujita : >>>>>> Hi, >>>>>> >>>>>> As I found an excellent template in the SRPM of docbook2X > package, >>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>>>> >>>>> Do you have a public git repo somewhere and a separate branch, >> where >>>>> you make these changes? >>>>> >>>>> It would be interesting to track the progress. >>>>> >>>>> Cheers, >>>>> Michael >>>>> >>>>> -- >>>>> Why is it that all of the instruments seeking intelligent life in >>> the >>>>> universe are pointed away from Earth? >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> >> > ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> >> > ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 09:46:33 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 09:46:33 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> Hi Rio-san, doh ;) Yes, I was after the web interface. I now pulled the repo with git and everything works fine :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 9:33 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi Rainer-san, > > Did you access with git? > Access with web browser may fail or get 404 not found. > > I succeeded as follows. > > $ git clone http://rio.tc/git/rsyslog > Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ > Getting alternates list for http://rio.tc/git/rsyslog > Getting pack list for http://rio.tc/git/rsyslog > Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b > Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b > which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 > > > $ cd rsyslog > $ git branch -r > origin/HEAD > origin/beta > origin/master > origin/queuememleak > origin/rfujita > origin/rscript > origin/tests > origin/v1-stable > origin/v2-stable > origin/v3-stable > origin/v3stable-klogbug > origin/wall > > $ git-pull http://rio.tc/git/rsyslog rfujita > Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http > Updating 37bac26..9d841de > > Fast forward > man/Makefile.am | 101 ++++ > man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ > man/en/man8/rsyslogd.8 | 303 +++++++++++ > man/entities/xinclude.dtd | 31 + > man/ja/man8/rsyslogd.8 | 319 +++++++++++ > man/xml-source/en/rsyslog.conf.5.xml | 994 +++++++++++++++++++++++++ > ++++++++ > man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ > man/xml-source/ja/rsyslog.conf.5.xml | 997 +++++++++++++++++++++++++ > +++++++++ > man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ > man/xslt/docbook.xsl | 127 +++++ > man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ > man/xslt/man.xsl | 70 +++ > man/xslt/settings.xsl | 11 + > 13 files changed, 5464 insertions(+), 0 deletions(-) > create mode 100644 man/Makefile.am > create mode 100644 man/en/man5/rsyslog.conf.5 > create mode 100644 man/en/man8/rsyslogd.8 > create mode 100644 man/entities/xinclude.dtd > create mode 100644 man/ja/man8/rsyslogd.8 > create mode 100644 man/xml-source/en/rsyslog.conf.5.xml > create mode 100644 man/xml-source/en/rsyslogd.8.xml > create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml > create mode 100644 man/xml-source/ja/rsyslogd.8.xml > create mode 100644 man/xslt/docbook.xsl > create mode 100644 man/xslt/expand-sambadoc.xsl > create mode 100644 man/xslt/man.xsl > create mode 100644 man/xslt/settings.xsl > > On 2008/07/18, at 16:24, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > I get a "file not found" error. Unfortunately, I am far from being a > > git-wizard, so I can not provide much advise :( > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Friday, July 18, 2008 8:44 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >> > >> Hi all, > >> > >> Please check the following git repo. > >> http://rio.tc/git/rsyslog > >> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? > >> > >> # It's my first time to build my own git repo :) > >> > >> On 2008/07/18, at 15:14, Rainer Gerhards wrote: > >> > >>> Hi Rio-san, > >>> > >>> I would really appreciate if you could build a git repro. I think > >> that > >>> would be a very valuable resource and help streamline the > > integration > >>> process :) > >>> > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>>> Sent: Friday, July 18, 2008 5:18 AM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >>>> > >>>> Hi Mike-san, > >>>> > >>>> I have no git repo now. > >>>> So, I sent the files I converted to Rainer-san directly. > >>>> But I recently feel a necessity to build a public repo. > >>>> If I have a time to do it this weekend, I'll try it :) > >>>> > >>>> Best Rio. > >>>> > >>>> On 2008/07/18, at 11:38, Michael Biebl wrote: > >>>> > >>>>> 2008/7/18 Ryo Fujita : > >>>>>> Hi, > >>>>>> > >>>>>> As I found an excellent template in the SRPM of docbook2X > > package, > >>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > >>>>> > >>>>> Do you have a public git repo somewhere and a separate branch, > >> where > >>>>> you make these changes? > >>>>> > >>>>> It would be interesting to track the progress. > >>>>> > >>>>> Cheers, > >>>>> Michael > >>>>> > >>>>> -- > >>>>> Why is it that all of the instruments seeking intelligent life in > >>> the > >>>>> universe are pointed away from Earth? > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>> > >> > > > ####################################################################### > >>>> # > >>>> Ryo Fujita > >>>> Senior Solution Architect, RHCE > >>>> Red Hat K.K. > >>>> TEL +81-3-5798-8500 > >>>> FAX +81-3-5798-8599 > >>>> Ebisu Neonato 8F > >>>> 4-1-18 Ebisu, Shibuya-ku, > >>>> Tokyo Japan 1500013 > >>>> > >>> > >> > > > ####################################################################### > >>>> # > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 09:50:28 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 16:50:28 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> Message-ID: Fine! I don't know how to publish git directory for web browser. If I find the way to do, all you can access with browser :) Best Rio. On 2008/07/18, at 16:46, Rainer Gerhards wrote: > Hi Rio-san, > > doh ;) Yes, I was after the web interface. I now pulled the repo with > git and everything works fine :) > > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 9:33 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi Rainer-san, >> >> Did you access with git? >> Access with web browser may fail or get 404 not found. >> >> I succeeded as follows. >> >> $ git clone http://rio.tc/git/rsyslog >> Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ >> Getting alternates list for http://rio.tc/git/rsyslog >> Getting pack list for http://rio.tc/git/rsyslog >> Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b >> Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b >> which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 >> >> >> $ cd rsyslog >> $ git branch -r >> origin/HEAD >> origin/beta >> origin/master >> origin/queuememleak >> origin/rfujita >> origin/rscript >> origin/tests >> origin/v1-stable >> origin/v2-stable >> origin/v3-stable >> origin/v3stable-klogbug >> origin/wall >> >> $ git-pull http://rio.tc/git/rsyslog rfujita >> Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http >> Updating 37bac26..9d841de >> >> Fast forward >> man/Makefile.am | 101 ++++ >> man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ >> man/en/man8/rsyslogd.8 | 303 +++++++++++ >> man/entities/xinclude.dtd | 31 + >> man/ja/man8/rsyslogd.8 | 319 +++++++++++ >> man/xml-source/en/rsyslog.conf.5.xml | 994 > +++++++++++++++++++++++++ >> ++++++++ >> man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ >> man/xml-source/ja/rsyslog.conf.5.xml | 997 > +++++++++++++++++++++++++ >> +++++++++ >> man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ >> man/xslt/docbook.xsl | 127 +++++ >> man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ >> man/xslt/man.xsl | 70 +++ >> man/xslt/settings.xsl | 11 + >> 13 files changed, 5464 insertions(+), 0 deletions(-) >> create mode 100644 man/Makefile.am >> create mode 100644 man/en/man5/rsyslog.conf.5 >> create mode 100644 man/en/man8/rsyslogd.8 >> create mode 100644 man/entities/xinclude.dtd >> create mode 100644 man/ja/man8/rsyslogd.8 >> create mode 100644 man/xml-source/en/rsyslog.conf.5.xml >> create mode 100644 man/xml-source/en/rsyslogd.8.xml >> create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml >> create mode 100644 man/xml-source/ja/rsyslogd.8.xml >> create mode 100644 man/xslt/docbook.xsl >> create mode 100644 man/xslt/expand-sambadoc.xsl >> create mode 100644 man/xslt/man.xsl >> create mode 100644 man/xslt/settings.xsl >> >> On 2008/07/18, at 16:24, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> I get a "file not found" error. Unfortunately, I am far from being a >>> git-wizard, so I can not provide much advise :( >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Friday, July 18, 2008 8:44 AM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >>>> >>>> Hi all, >>>> >>>> Please check the following git repo. >>>> http://rio.tc/git/rsyslog >>>> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? >>>> >>>> # It's my first time to build my own git repo :) >>>> >>>> On 2008/07/18, at 15:14, Rainer Gerhards wrote: >>>> >>>>> Hi Rio-san, >>>>> >>>>> I would really appreciate if you could build a git repro. I think >>>> that >>>>> would be a very valuable resource and help streamline the >>> integration >>>>> process :) >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>>> Sent: Friday, July 18, 2008 5:18 AM >>>>>> To: rsyslog-users >>>>>> Subject: Re: [rsyslog] Suggestion for Documentation > Reconstructure >>>>>> >>>>>> Hi Mike-san, >>>>>> >>>>>> I have no git repo now. >>>>>> So, I sent the files I converted to Rainer-san directly. >>>>>> But I recently feel a necessity to build a public repo. >>>>>> If I have a time to do it this weekend, I'll try it :) >>>>>> >>>>>> Best Rio. >>>>>> >>>>>> On 2008/07/18, at 11:38, Michael Biebl wrote: >>>>>> >>>>>>> 2008/7/18 Ryo Fujita : >>>>>>>> Hi, >>>>>>>> >>>>>>>> As I found an excellent template in the SRPM of docbook2X >>> package, >>>>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>>>>>> >>>>>>> Do you have a public git repo somewhere and a separate branch, >>>> where >>>>>>> you make these changes? >>>>>>> >>>>>>> It would be interesting to track the progress. >>>>>>> >>>>>>> Cheers, >>>>>>> Michael >>>>>>> >>>>>>> -- >>>>>>> Why is it that all of the instruments seeking intelligent life > in >>>>> the >>>>>>> universe are pointed away from Earth? >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> >>>>>> >>>>> >>>> >>> >> > ####################################################################### >>>>>> # >>>>>> Ryo Fujita >>>>>> Senior Solution Architect, RHCE >>>>>> Red Hat K.K. >>>>>> TEL +81-3-5798-8500 >>>>>> FAX +81-3-5798-8599 >>>>>> Ebisu Neonato 8F >>>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>>> Tokyo Japan 1500013 >>>>>> >>>>> >>>> >>> >> > ####################################################################### >>>>>> # >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> >> > ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> >> > ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 14:34:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 14:34:41 +0200 Subject: [rsyslog] slight kernel log format inconsistencybetween 3.16.2and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com><1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE2F@grfint2.intern.adiscon.com> FYI: I have now taken action against the format problem. I think the changelog entries tells it all: - added a new property replacer option "sp-if-no-1st-sp" to cover a problem with RFC 3164 based interpretation of tag separation. While it is a generic approach, it fixes a format problem introduced in 3.18.0, where kernel messages no longer had a space after the tag. This is done by a modification of the default templates. Please note that this may affect some messages where there intentionally is no space between the tag and the first character of the message content. If so, this needs to be worked around via a specific template. However, we consider this scenario to be quite remote and, even if it exists, it is not expected that it will actually cause problems with log parsers (instead, we assume the new default template behavior may fix previous problems with log parsers due to the missing space). It is not a perfect solution, but I hope a pragmatic and good-enough one. Other than format issues, I had also some performance concerns. What I have now implemented affects performance very little. Most importantly, it enables us to stay away from copying large strings in memory just to prepend a space. If some has a concern, please voice it. This patch will be part of 3.21.0 (as it looks to be released today) and, if no hard objection is received, 3.18.1. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 15, 2008 4:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] slight kernel log format inconsistencybetween > 3.16.2and 3.18.0 > > Michael, > > I have reviewed some of the code in question. It looks like a bug. It's > actually not a klog driver issue, the PRI parsing is done at an upper > klog layer and thus should work for linux as well. I would like to > generate a version with some additional instrumentation. Could you run > it and report the results back (maybe via private mail to keep the list > free of noise). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Tuesday, July 15, 2008 3:04 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] slight kernel log format inconsistency between > > 3.16.2and 3.18.0 > > > > On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > > > 2008/7/14 Rainer Gerhards : > > > > Hi all, > > > > > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > > > > > 3.16.x (and before): > > > > Jul 11 17:33:28 pluto kernel: [14177.432349] > > ADDRCONF(NETDEV_CHANGE): > > > > wlan0: link becomes ready > > > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > > > > > > > 3.18.0: > > > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = > /proc/kmsg > > > > started. > > > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > > > > > another example: > > > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > > > See the <6> after the kernel: tag. It's also new. > > > Is this some kind of log level, set by the kernel? > > > > This does not look correct. Under FreeBSD, it is typical to have a > PRI > > inside kernel messages. But for linux kernels I tested with, I did > not > > see it and (I think) so I did not support it. But if you see it on > > Debian, it seems to happen. A cure is to use the same logic that is > > used > > for BSD. > > > > > I have never seen, that other syslogger (like syslog-ng, sysklogd > set > > this). > > > > > > > > > > > As you can see, there is a space after kernel: in the pre- > > 3.18.0 > > > > messages. This also brings us down to a general inconsistency: > > > > > > > > Unfortunately, RFC 3164 defines the after TAG not as a > > delimiter > > > > but as part of the CONTENT of the message. This leads to some > > > > inconsistency in practice. Depending on who generated the > message, > > there > > > > may be a space after the TAG or not. If it is, that space must be > > > > submitted as part of the message itself. > > > > > > > > In versions up to 3.16.x, kernel log messages were generated by > old > > > > code, which did not know about tag and simply generated a non- > > structured > > > > message. With 3.17.x and above, the klog code is rewritten and > > correctly > > > > fills in all properties. This leads to the "missing" space, > because > > the > > > > space is neither part of the tag nor part of the message. > > > > > > > > I have now three options: > > > > > > > > 1. leave as is (without a space) > > > > In that case, some log parsers may run into troubles > > > > > > This was a rather vague concern of mine. I don't actually know, if > > > other software is affected by this change. > > > > I share this concern, but also have no additional information. I > wanted > > to point out the options. I have to admit I am still undecided. But > > that > > nobody violently objected so far makes this look like it is not so > > important... > > > > > > > > > 2. add a at the end of the TAG > > > > That will probably lead to even more confusion, as matches > against > > TAG > > > > would need to include that space. > > > > > > > > 3. add a in front of each kernel message > > > > This comes close to the previous mode, but is "a bit" clumpsy and > > > > hackish. It will also make all database tables etc have messages > > > > starting with . Similar as 2, MSG matching rules are affected > > (but I > > > > consider this less severe, as matches inside the MSG are usually > > driven > > > > by searches and not absolute positions). > > > > > > Just curious: For other log messages, that are not generated by the > > > kernel, you have a space after the programname: tag, e.g. > > > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > > > > > Is the space after dhclient: part of the message or part of the > > > :programname tag? > > > > In our private communication, I thought it is part of TAG. But that > is > > wrong. It actually is part of MSG - at least for legacy/RFC3164 > syslog. > > > > For the new IETF syslog RFC series (syslog-protocol, -tls et al) it > is > > well-defined and part of neither. There, it is a delimiter (as one > > would > > expect). I you use rsyslog only with syslog-protocol messages (and > the > > syslog-protocol templates), this is more or less no issue. But who > does > > today...? ;) > > > > The real problem is that legacy syslog has many different > > interpretations and we break one or the other in either way... > > > > > > > > Imho, if there is no clear definition in the RFC, how the message > > > should be written, it's best to stay backwards compatible. > > > > Is that a vote for option 3, stick a space in front of each kernel > > message? ;) > > > > > > > > > > > Cheers, > > > Michael > > > > > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 18 14:42:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 14:42:12 +0200 Subject: [rsyslog] preview of 3.18.1 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE30@grfint2.intern.adiscon.com> Hi all, as you are probably aware, 3.18.1 contains a "fix" for a format issue with kernel messages. This fix has a somewhat broader scope. In an ideal world, I would have preferred to run it through the beta phase and not apply it directly to stable. But this is counter-productive, as the format issue may affect users of v3-stable *right now*. So I am doing the second best thing: I have prepared a preview tarball and hope that some of you will implement it and tell me if it works OK. I do not expect any problems, but, hey, there is always a difference between lab and practice. I have pushed back 3.18.1 in the hopes to get some feedback. I need to release 3.18.1 soon, preferably by next Monday or Tuesday. So I would appreciate any feedback you can offer (including "no problem experienced" feedback). The tarball is available at: http://download.rsyslog.com/rsyslog/rsyslog-3.18.1-Test1.tar.gz This most probably is the official 3.18.1, except for some changes to cover the new release number (except, of course, feedback requires changes). Thanks, Rainer From friedl at hq.adiscon.com Fri Jul 18 17:25:23 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 18 Jul 2008 17:25:23 +0200 Subject: [rsyslog] rsyslog 3.21.0 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE36@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.21.0, the first release of the new 3.21.x devel branch. It includes all fixes and additions of the current v3-stable and beta plus an improved testbench. Please note that this includes some vital bugfixes from earlier releases. 3.21.0 lays the foundation for the next round of rsyslog enhancements. It is a recommended update for all devel branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-121.phtml Changelog: http://www.rsyslog.com/Article258.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Mon Jul 21 18:16:24 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 21 Jul 2008 18:16:24 +0200 Subject: [rsyslog] rsyslog 3.18.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE53@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.18.1. It is a v3-stable branch version. It offers a number of bug fixes and solves portability issues on FreeBSD. Most importantly, a format difference in kernel logs between 3.16.x and 3.18.0 has been corrected. This is a recommended update for all v3-stable branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-122.phtml Changelog: http://www.rsyslog.com/Article260.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From martinmie at PartyGaming.com Tue Jul 22 10:38:32 2008 From: martinmie at PartyGaming.com (Martin Mielke) Date: Tue, 22 Jul 2008 10:38:32 +0200 Subject: [rsyslog] Whitepapers on perfomance Message-ID: <0B1B9163138571439EAF804646F3F6AA051AD910@GIBSVWIN004X.partygaming.local> Hi list, Can someone point me to a official document covering rsyslog's performance under heavy load? TIA, Martin From rgerhards at hq.adiscon.com Tue Jul 22 12:16:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 22 Jul 2008 12:16:28 +0200 Subject: [rsyslog] rsyslog scheduled as new default syslogd on Debian Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> Hi folks, I thought I share the good news. I have just blogged about the details: http://blog.gerhards.net/2008/07/rsyslog-will-become-debians-default.htm l Special thanks go to Michael Biebl, who is the driving force behind this development. Rainer From rfujita at redhat.com Wed Jul 23 19:10:56 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 24 Jul 2008 02:10:56 +0900 Subject: [rsyslog] rsyslog scheduled as new default syslogd on Debian In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> Message-ID: Hi Rainer-san, Congrats! And the news cheer me up, too! Best Rio. On 2008/07/22, at 19:16, Rainer Gerhards wrote: > Hi folks, > > I thought I share the good news. I have just blogged about the > details: > > http://blog.gerhards.net/2008/07/rsyslog-will-become-debians-default.htm > l > > Special thanks go to Michael Biebl, who is the driving force behind > this > development. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From mmcgrath at redhat.com Wed Jul 23 21:21:46 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 14:21:46 -0500 (CDT) Subject: [rsyslog] rsyslog dropping logs Message-ID: I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central logging server. When running tcpdump I'm seeing all the udp packets coming in but many of them are not getting logged. And we're talking like 10% or so getting logged (maybe less) and the rest are just lost. I've attached my config file. (side note, if I'm doing something stupid in the config please correct me) -Mike -------------- next part -------------- #rsyslog v3 config file #### GLOBAL DIRECTIVES #### # Use default timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # File syncing capability is disabled by default. This feature is usually not required, # not useful and an extreme performance hit #$ActionFileEnableSync on # An "In-Memory Queue" is created for remote logging. # $WorkDirectory /var/spool/rsyslog # where to place spool files # $ActionQueueFileName queue # unique name prefix for spool files # $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) # $ActionQueueSaveOnShutdown on # save messages to disk on shutdown # $ActionQueueType LinkedList # run asynchronously # $ActionResumeRetryCount -1 # infinety retries if host is down #### MODULES #### $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # provides kernel logging support (previously done by rklogd) #$ModLoad immark.so # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp.so $UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp.so #$InputTCPServerRun 514 $FileGroup sysadmin $FileCreateMode 0640 $DirGroup sysadmin $DirCreateMode 0750 #### RULES #### # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;local1.none;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log $template RawMessage,"%msg%\n" $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" if $app-name contains 'access.log' then -?HttpAccessTemplate;RawMessage $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" if $app-name contains 'error.log' then -?HttpErrorTemplate;RawMessage $template DynaFile,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/messages.log" *.*;local1.none -?DynaFile From Samuel.Kielek at marriott.com Wed Jul 23 21:44:25 2008 From: Samuel.Kielek at marriott.com (Kielek, Samuel) Date: Wed, 23 Jul 2008 15:44:25 -0400 Subject: [rsyslog] rsyslog dropping logs In-Reply-To: References: Message-ID: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> Mike, You're using some v3 features that will not work with v2 such as the conditional filtering. I'll email you off list with a copy of my config (also using RHEL 5.2 here). -Sam -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Mike McGrath Sent: Wednesday, July 23, 2008 3:22 PM To: rsyslog at lists.adiscon.com Subject: [rsyslog] rsyslog dropping logs I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central logging server. When running tcpdump I'm seeing all the udp packets coming in but many of them are not getting logged. And we're talking like 10% or so getting logged (maybe less) and the rest are just lost. I've attached my config file. (side note, if I'm doing something stupid in the config please correct me) -Mike From mmcgrath at redhat.com Wed Jul 23 23:04:24 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 16:04:24 -0500 (CDT) Subject: [rsyslog] Leading whitespace Message-ID: Right now I'm doing live logging of my http servers to my rsyslog host. I've got this code on the rsyslog server: $template RawMessage,"%msg%\n" $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" On the http servers I've got: CustomLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-access.log" combined ErrorLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-error.log" It works great except for one thing. The logs have an additional whitespace in front of each line. for example: 111.111.111.11 - - [23/Jul/2008:21:01:32 +0000] "GET /static/images/fedora-logo.png HTTP/1.1" 200 5354 "http://fedoraproject.org/static/css/fedora.css" "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.15) Gecko/20080702Fedora/2.0.0.15-1.fc8 Firefox/2.0.0.15" Any ideas? -Mike From mmcgrath at redhat.com Wed Jul 23 23:06:24 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 16:06:24 -0500 (CDT) Subject: [rsyslog] rsyslog dropping logs In-Reply-To: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> References: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> Message-ID: On Wed, 23 Jul 2008, Kielek, Samuel wrote: > Mike, > > You're using some v3 features that will not work with v2 such as the > conditional filtering. I'll email you off list with a copy of my config > (also using RHEL 5.2 here). > Thanks, I upgraded to a v3 version with my old config, same issue. Used the config provided off list and it works great. even in v3. -Mike From rgerhards at hq.adiscon.com Thu Jul 24 12:27:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 24 Jul 2008 12:27:05 +0200 Subject: [rsyslog] Leading whitespace In-Reply-To: References: Message-ID: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> This is rooted in RFC 3164. A description can be found here: http://lists.adiscon.net/pipermail/rsyslog/2008-July/000893.html If you want to consistently get rid of that SP, you can remove it via the property replacer. I think this works: ?$template RawMessage,"%msg:2%\n" if not, then this ;) ??$template RawMessage,"%msg:2:2048%\n" Let us know if that cured the problem. Rainer On Wed, 2008-07-23 at 16:04 -0500, Mike McGrath wrote: > Right now I'm doing live logging of my http servers to my rsyslog host. > I've got this code on the rsyslog server: > > $template RawMessage,"%msg%\n" > $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" > $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" > > > On the http servers I've got: > > CustomLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-access.log" combined > ErrorLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-error.log" > > > It works great except for one thing. The logs have an additional > whitespace in front of each line. for example: > > 111.111.111.11 - - [23/Jul/2008:21:01:32 +0000] "GET /static/images/fedora-logo.png HTTP/1.1" 200 5354 "http://fedoraproject.org/static/css/fedora.css" "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.15) Gecko/20080702Fedora/2.0.0.15-1.fc8 Firefox/2.0.0.15" > > Any ideas? > > -Mike > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 24 12:40:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 24 Jul 2008 12:40:24 +0200 Subject: [rsyslog] rsyslog dropping logs In-Reply-To: References: Message-ID: <1216896024.7184.189.camel@rgf9dev.intern.adiscon.com> (I am not commenting on v2 vs. v3 as this is already done) First of all, we need to keep in mind that UDP is inherently lossy. Even when a frame is seen received by the local stack, it does not mean that it will eventually be forwarded to the application. If message bursts come in very quickly and the OS scheduler does not schedule the app fast enough to receive this messages (or the app is too slow in itself! ;)) new frames may overwrite frames inside the stack's receive buffers. So it is always a good idea to avoid UDP if that's possible. HOWEVER, I, too, find it somewhat unusual that around 90% of all incoming frames are discarded before the rsyslog receiver could process them. One explanation I have is that you have bursts (or volume in general) that outperforms the configured actions. Having seen the config file, and seeing it does not include any database writer, it is hard to imagine this should happen, assuming reasonable hardware sizing is used. A cause could be excessive synchronous writes. Many rules do not put a dash in front of the file name and without it (in v2), every write is immediately synced. This is very costly. But still, I have never seen that this alone outperforms a system. To dig deeper into what is happening, a debug log would be most useful, together with the information which frames have been seen in tcpdump but NOT in one of the log files. You can enable debug mode via -dn command line switch and is recommended to run rsyslog interactively while doing so. Then, you can simply capture its output via stdout redirection. Please note that debug mode generates considerable output, and requires considerable additional processing time. In any case, though, it should show us where the bottleneck is. Please note that I need a consistent excerpt from the debug log that shows how things began and how it worked during the fault conditions. Usually, this means I need everything ;) Debug logs may also reveal sensitive information, even passwords, so you should be careful in what you do. I am used to log files around the size of 1GB. With reasonable compression, the transfer is usually not a problem (but I suggest you place them on a server for me to download). Download links and/or smaller logs you can email me privately at rgerhards at gmail.com (please NOT at my primary, adiscon, email address). I hope this helps and I am looking forward for the additional information. Rainer On Wed, 2008-07-23 at 14:21 -0500, Mike McGrath wrote: > I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central > logging server. When running tcpdump I'm seeing all the udp packets > coming in but many of them are not getting logged. And we're talking > like 10% or so getting logged (maybe less) and the rest are just lost. > I've attached my config file. > > (side note, if I'm doing something stupid in the config please correct me) > > -Mike > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From mmcgrath at redhat.com Thu Jul 24 15:41:59 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 24 Jul 2008 08:41:59 -0500 (CDT) Subject: [rsyslog] Leading whitespace In-Reply-To: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> References: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> Message-ID: On Thu, 24 Jul 2008, Rainer Gerhards wrote: > This is rooted in RFC 3164. A description can be found here: > > http://lists.adiscon.net/pipermail/rsyslog/2008-July/000893.html > > If you want to consistently get rid of that SP, you can remove it via > the property replacer. I think this works: > > ?$template RawMessage,"%msg:2%\n" > > if not, then this ;) > > ??$template RawMessage,"%msg:2:2048%\n" > > Let us know if that cured the problem. > %msg:2:2048% seems to work perfectly, thank you. -Mike From Kevin.Goutos at budget.state.ny.us Thu Jul 24 21:50:04 2008 From: Kevin.Goutos at budget.state.ny.us (Goutos, Kevin (DOB)) Date: Thu, 24 Jul 2008 15:50:04 -0400 Subject: [rsyslog] Help with Ommail Configuration Message-ID: Hello all, First off, I am not very Linux savvy. I don't have a lot of experience other then a basic course. This is probably way past my knowledge, but I really need to get it done. Appreciate any help you guys have to offer. I am working on a Red Hat Enterprise 4 box and I am running the latest edition of rsyslog. I currently have rsyslog configured to receive messages remotely via UDP. I am trying to set it up so that it will send out E-mail messages to the system Admin's based on the severity level of the log files I am receiving. I would like it so that any device that sends a log with a critical, alert, or emergency level facility will send out an e-mail to a specific address. Here is my rsyslog.conf file. I used the sample code from Rainer Gerhards configuration page. I tried sending a test syslog with 'hard disk fatal failure' in it, but it is not sending out mail. Also tried without the templates below thinking it would just send me an email for every syslog that I received, but it doesn't appear to be sending mail. Any thoughts on what I am doing wrong. I'm sure there is a lot I need to do, so please let me know. Thanks! $template mailSubject,"disk problem on %hostname%" $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' I edited out the 3 lines below in the code for security reasons.. $ActionMailSMTPServer $ActionMailFrom $ActionMailTo Here is my code from rsyslog.conf below. # if you experience problems, check # http://www.rsyslog.com/troubleshoot for assistance # rsyslog v3: load input modules # If you do not load inputs, nothing happens! # You may need to set the module load path if modules are not found. $ModLoad immark.so # provides --MARK-- message capability $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # kernel logging (formerly provided by rklogd) $ModLoad ommail $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none -/var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* -/var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit -/var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log #Catch all incoming syslog messages *.* /var/log/catchall;TraditionalFormatWithPRI # Remote Logging (we use TCP for reliable delivery) # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. $WorkDirectory /rsyslog/spool # where to place spool files $ActionQueueFileName uniqName # unique name prefix for spool files $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueType LinkedList # run asynchronously $ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional *.* @10.57.106.140:514 $ModLoad ommail $ActionMailSMTPServer $ActionMailFrom $ActionMailTo $template mailSubject,"disk problem on %hostname%" $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" $ActionMailSubject mailSubject # make sure we receive a mail only once in six # hours (21,600 seconds ;)) $ActionExecOnlyOnceEveryInterval 21600 # the if ... then ... mailBody mus be on one line! if $msg contains 'hard disk fatal failure' then :ommail:;mailBody # ######### Receiving Messages from Remote Hosts ########## # TCP Syslog Server: # provides TCP syslog reception and GSS-API (if compiled to support it) $ModLoad imtcp.so # load module $InputTCPServerRun 514 # start up TCP listener at port 514 # UDP Syslog Server: $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port -------------------------------------------------------- This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. From hks.private at gmail.com Thu Jul 24 23:01:42 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 24 Jul 2008 17:01:42 -0400 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: A few things to try: - You load ommail.so twice, once at the top and once right above your $ActionMail... lines. I don't think this will break it, but it's unnecessary - delete the second one. - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going to attempt sending a message once every 6 hours. For testing purposes, this will be obnoxious. Until you're ready for production, change it to: $ActionExecOnlyOnceEveryInterval 30 - Send everything to make sure you're not missing it based on something in the property-replacer/templates/whatever. Replace "if $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: *.* :ommail:;mailBody Try again. Try logging a few messages from the localhost first (with RHEL you can just run "logger test" to log a user.notice message with content "test") and see if you get them. If you don't, check the mail logs on your mail server to see if it ever received the message. If not, it's time to break out tcpdump and see if the packets are ever being generated. Hope that helps. -HKS On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) wrote: > Hello all, > > First off, I am not very Linux savvy. I don't have a lot of experience > other then a basic course. This is probably way past my knowledge, but I > really need to get it done. Appreciate any help you guys have to offer. > > I am working on a Red Hat Enterprise 4 box and I am running the latest > edition of rsyslog. I currently have rsyslog configured to receive > messages remotely via UDP. I am trying to set it up so that it will send > out E-mail messages to the system Admin's based on the severity level of > the log files I am receiving. I would like it so that any device that > sends a log with a critical, alert, or emergency level facility will > send out an e-mail to a specific address. > > Here is my rsyslog.conf file. I used the sample code from Rainer > Gerhards configuration page. I tried sending a test syslog with 'hard > disk fatal failure' in it, but it is not sending out mail. Also tried > without the templates below thinking it would just send me an email for > every syslog that I received, but it doesn't appear to be sending mail. > Any thoughts on what I am doing wrong. I'm sure there is a lot I need to > do, so please let me know. Thanks! > > $template mailSubject,"disk problem on %hostname%" > $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > > I edited out the 3 lines below in the code for security reasons.. > $ActionMailSMTPServer > $ActionMailFrom > $ActionMailTo > > > > Here is my code from rsyslog.conf below. > > > > # if you experience problems, check > # http://www.rsyslog.com/troubleshoot for assistance > > # rsyslog v3: load input modules > # If you do not load inputs, nothing happens! > # You may need to set the module load path if modules are not found. > > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging (e.g. > via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > $ModLoad ommail > > $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > > # Log all kernel messages to the console. > # Logging much else clutters up the screen. > #kern.* /dev/console > > # Log anything (except mail) of level info or higher. > # Don't log private authentication messages! > *.info;mail.none;authpriv.none;cron.none > -/var/log/messages > > # The authpriv file has restricted access. > authpriv.* /var/log/secure > > # Log all the mail messages in one place. > mail.* > -/var/log/maillog > > # Log cron stuff > cron.* -/var/log/cron > > # Everybody gets emergency messages > *.emerg * > > # Save news errors of level crit and higher in a special file. > uucp,news.crit > -/var/log/spooler > > # Save boot messages also to boot.log > local7.* > /var/log/boot.log > > #Catch all incoming syslog messages > *.* > /var/log/catchall;TraditionalFormatWithPRI > > # Remote Logging (we use TCP for reliable delivery) > # An on-disk queue is created for this action. If the remote host is > # down, messages are spooled to disk and sent when it is up again. > $WorkDirectory /rsyslog/spool # where to place spool files > $ActionQueueFileName uniqName # unique name prefix for spool files > $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > possible) > $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > $ActionQueueType LinkedList # run asynchronously > $ActionResumeRetryCount -1 # infinite retries if host is down > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > *.* @10.57.106.140:514 > > $ModLoad ommail > $ActionMailSMTPServer > $ActionMailFrom > $ActionMailTo > $template mailSubject,"disk problem on %hostname%" > $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > $ActionMailSubject mailSubject > # make sure we receive a mail only once in six > # hours (21,600 seconds ;)) > $ActionExecOnlyOnceEveryInterval 21600 > # the if ... then ... mailBody mus be on one line! > if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > > > # ######### Receiving Messages from Remote Hosts ########## > # TCP Syslog Server: > # provides TCP syslog reception and GSS-API (if compiled to support it) > $ModLoad imtcp.so # load module > $InputTCPServerRun 514 # start up TCP listener at port 514 > > # UDP Syslog Server: > $ModLoad imudp.so # provides UDP syslog reception > $UDPServerRun 514 # start a UDP syslog server at standard port > -------------------------------------------------------- > This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From hks.private at gmail.com Thu Jul 24 23:35:39 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 24 Jul 2008 17:35:39 -0400 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: Oh - one other possibility. rsyslogd has to have mail enabled at compile time to work with it at runtime. check your catchall logfile - if it has messages like this, you didn't compile it in: rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object file: No such file or directory To fix this, you'll need to reinstall. This time, when you run ./configure be sure to include --enable-mail. make install clean and you should be good to go... -HKS On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS wrote: > A few things to try: > > - You load ommail.so twice, once at the top and once right above your > $ActionMail... lines. I don't think this will break it, but it's > unnecessary - delete the second one. > > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > to attempt sending a message once every 6 hours. For testing purposes, > this will be obnoxious. Until you're ready for production, change it > to: > $ActionExecOnlyOnceEveryInterval 30 > > - Send everything to make sure you're not missing it based on > something in the property-replacer/templates/whatever. Replace "if > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: > *.* :ommail:;mailBody > > Try again. Try logging a few messages from the localhost first (with > RHEL you can just run "logger test" to log a user.notice message with > content "test") and see if you get them. > > If you don't, check the mail logs on your mail server to see if it > ever received the message. If not, it's time to break out tcpdump and > see if the packets are ever being generated. > > Hope that helps. > > -HKS > > > > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > wrote: >> Hello all, >> >> First off, I am not very Linux savvy. I don't have a lot of experience >> other then a basic course. This is probably way past my knowledge, but I >> really need to get it done. Appreciate any help you guys have to offer. >> >> I am working on a Red Hat Enterprise 4 box and I am running the latest >> edition of rsyslog. I currently have rsyslog configured to receive >> messages remotely via UDP. I am trying to set it up so that it will send >> out E-mail messages to the system Admin's based on the severity level of >> the log files I am receiving. I would like it so that any device that >> sends a log with a critical, alert, or emergency level facility will >> send out an e-mail to a specific address. >> >> Here is my rsyslog.conf file. I used the sample code from Rainer >> Gerhards configuration page. I tried sending a test syslog with 'hard >> disk fatal failure' in it, but it is not sending out mail. Also tried >> without the templates below thinking it would just send me an email for >> every syslog that I received, but it doesn't appear to be sending mail. >> Any thoughts on what I am doing wrong. I'm sure there is a lot I need to >> do, so please let me know. Thanks! >> >> $template mailSubject,"disk problem on %hostname%" >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >> >> I edited out the 3 lines below in the code for security reasons.. >> $ActionMailSMTPServer >> $ActionMailFrom >> $ActionMailTo >> >> >> >> Here is my code from rsyslog.conf below. >> >> >> >> # if you experience problems, check >> # http://www.rsyslog.com/troubleshoot for assistance >> >> # rsyslog v3: load input modules >> # If you do not load inputs, nothing happens! >> # You may need to set the module load path if modules are not found. >> >> $ModLoad immark.so # provides --MARK-- message capability >> $ModLoad imuxsock.so # provides support for local system logging (e.g. >> via logger command) >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >> $ModLoad ommail >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >> >> # Log all kernel messages to the console. >> # Logging much else clutters up the screen. >> #kern.* /dev/console >> >> # Log anything (except mail) of level info or higher. >> # Don't log private authentication messages! >> *.info;mail.none;authpriv.none;cron.none >> -/var/log/messages >> >> # The authpriv file has restricted access. >> authpriv.* /var/log/secure >> >> # Log all the mail messages in one place. >> mail.* >> -/var/log/maillog >> >> # Log cron stuff >> cron.* -/var/log/cron >> >> # Everybody gets emergency messages >> *.emerg * >> >> # Save news errors of level crit and higher in a special file. >> uucp,news.crit >> -/var/log/spooler >> >> # Save boot messages also to boot.log >> local7.* >> /var/log/boot.log >> >> #Catch all incoming syslog messages >> *.* >> /var/log/catchall;TraditionalFormatWithPRI >> >> # Remote Logging (we use TCP for reliable delivery) >> # An on-disk queue is created for this action. If the remote host is >> # down, messages are spooled to disk and sent when it is up again. >> $WorkDirectory /rsyslog/spool # where to place spool files >> $ActionQueueFileName uniqName # unique name prefix for spool files >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >> possible) >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> $ActionQueueType LinkedList # run asynchronously >> $ActionResumeRetryCount -1 # infinite retries if host is down >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> *.* @10.57.106.140:514 >> >> $ModLoad ommail >> $ActionMailSMTPServer >> $ActionMailFrom >> $ActionMailTo >> $template mailSubject,"disk problem on %hostname%" >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >> $ActionMailSubject mailSubject >> # make sure we receive a mail only once in six >> # hours (21,600 seconds ;)) >> $ActionExecOnlyOnceEveryInterval 21600 >> # the if ... then ... mailBody mus be on one line! >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >> >> >> # ######### Receiving Messages from Remote Hosts ########## >> # TCP Syslog Server: >> # provides TCP syslog reception and GSS-API (if compiled to support it) >> $ModLoad imtcp.so # load module >> $InputTCPServerRun 514 # start up TCP listener at port 514 >> >> # UDP Syslog Server: >> $ModLoad imudp.so # provides UDP syslog reception >> $UDPServerRun 514 # start a UDP syslog server at standard port >> -------------------------------------------------------- >> This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > From rgerhards at hq.adiscon.com Fri Jul 25 11:41:50 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 11:41:50 +0200 Subject: [rsyslog] the philosophy behind rsyslog, phplogcon et al... Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEA0@grfint2.intern.adiscon.com> Hi all, This page is for those of you interested in the philosophy behind rsyslog and its sister projects: http://blog.gerhards.net/2008/07/what-is-event-and-what-event-log.html Be warned, it is a bit technical, but I think quite useful to look behind the curtain. Feedback is welcome. Thanks, Rainer From rgerhards at hq.adiscon.com Fri Jul 25 12:31:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 12:31:51 +0200 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEA4@grfint2.intern.adiscon.com> Thanks, HKS, for the good answers. There is one thing I would like to bring to your attention: I often see that people do not check for rsyslog error messages themselves. That complicates setup. I do not blame anyone, but would like to make you aware of the problem. I have just blogged about a potential (partial) solution and will try to implement it: http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht ml Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Thursday, July 24, 2008 11:36 PM > To: rsyslog-users > Subject: Re: [rsyslog] Help with Ommail Configuration > > Oh - one other possibility. rsyslogd has to have mail enabled at > compile time to work with it at runtime. check your catchall logfile - > if it has messages like this, you didn't compile it in: > > rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > file: No such file or directory > > To fix this, you'll need to reinstall. This time, when you run > ./configure be sure to include --enable-mail. make install clean and > you should be good to go... > > -HKS > > On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS > wrote: > > A few things to try: > > > > - You load ommail.so twice, once at the top and once right above > your > > $ActionMail... lines. I don't think this will break it, but it's > > unnecessary - delete the second one. > > > > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > > to attempt sending a message once every 6 hours. For testing > purposes, > > this will be obnoxious. Until you're ready for production, change it > > to: > > $ActionExecOnlyOnceEveryInterval 30 > > > > - Send everything to make sure you're not missing it based on > > something in the property-replacer/templates/whatever. Replace "if > > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: > > *.* :ommail:;mailBody > > > > Try again. Try logging a few messages from the localhost first (with > > RHEL you can just run "logger test" to log a user.notice message with > > content "test") and see if you get them. > > > > If you don't, check the mail logs on your mail server to see if it > > ever received the message. If not, it's time to break out tcpdump and > > see if the packets are ever being generated. > > > > Hope that helps. > > > > -HKS > > > > > > > > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > > wrote: > >> Hello all, > >> > >> First off, I am not very Linux savvy. I don't have a lot of > experience > >> other then a basic course. This is probably way past my knowledge, > but I > >> really need to get it done. Appreciate any help you guys have to > offer. > >> > >> I am working on a Red Hat Enterprise 4 box and I am running the > latest > >> edition of rsyslog. I currently have rsyslog configured to receive > >> messages remotely via UDP. I am trying to set it up so that it will > send > >> out E-mail messages to the system Admin's based on the severity > level of > >> the log files I am receiving. I would like it so that any device > that > >> sends a log with a critical, alert, or emergency level facility will > >> send out an e-mail to a specific address. > >> > >> Here is my rsyslog.conf file. I used the sample code from Rainer > >> Gerhards configuration page. I tried sending a test syslog with > 'hard > >> disk fatal failure' in it, but it is not sending out mail. Also > tried > >> without the templates below thinking it would just send me an email > for > >> every syslog that I received, but it doesn't appear to be sending > mail. > >> Any thoughts on what I am doing wrong. I'm sure there is a lot I > need to > >> do, so please let me know. Thanks! > >> > >> $template mailSubject,"disk problem on %hostname%" > >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >> > >> I edited out the 3 lines below in the code for security reasons.. > >> $ActionMailSMTPServer > >> $ActionMailFrom > >> $ActionMailTo > >> > >> > >> > >> Here is my code from rsyslog.conf below. > >> > >> > >> > >> # if you experience problems, check > >> # http://www.rsyslog.com/troubleshoot for assistance > >> > >> # rsyslog v3: load input modules > >> # If you do not load inputs, nothing happens! > >> # You may need to set the module load path if modules are not found. > >> > >> $ModLoad immark.so # provides --MARK-- message capability > >> $ModLoad imuxsock.so # provides support for local system logging > (e.g. > >> via logger command) > >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >> $ModLoad ommail > >> > >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >> > >> # Log all kernel messages to the console. > >> # Logging much else clutters up the screen. > >> #kern.* /dev/console > >> > >> # Log anything (except mail) of level info or higher. > >> # Don't log private authentication messages! > >> *.info;mail.none;authpriv.none;cron.none > >> -/var/log/messages > >> > >> # The authpriv file has restricted access. > >> authpriv.* > /var/log/secure > >> > >> # Log all the mail messages in one place. > >> mail.* > >> -/var/log/maillog > >> > >> # Log cron stuff > >> cron.* - > /var/log/cron > >> > >> # Everybody gets emergency messages > >> *.emerg * > >> > >> # Save news errors of level crit and higher in a special file. > >> uucp,news.crit > >> -/var/log/spooler > >> > >> # Save boot messages also to boot.log > >> local7.* > >> /var/log/boot.log > >> > >> #Catch all incoming syslog messages > >> *.* > >> /var/log/catchall;TraditionalFormatWithPRI > >> > >> # Remote Logging (we use TCP for reliable delivery) > >> # An on-disk queue is created for this action. If the remote host is > >> # down, messages are spooled to disk and sent when it is up again. > >> $WorkDirectory /rsyslog/spool # where to place spool files > >> $ActionQueueFileName uniqName # unique name prefix for spool files > >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >> possible) > >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >> $ActionQueueType LinkedList # run asynchronously > >> $ActionResumeRetryCount -1 # infinite retries if host is down > >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> *.* @10.57.106.140:514 > >> > >> $ModLoad ommail > >> $ActionMailSMTPServer > >> $ActionMailFrom > >> $ActionMailTo > >> $template mailSubject,"disk problem on %hostname%" > >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >> $ActionMailSubject mailSubject > >> # make sure we receive a mail only once in six > >> # hours (21,600 seconds ;)) > >> $ActionExecOnlyOnceEveryInterval 21600 > >> # the if ... then ... mailBody mus be on one line! > >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >> > >> > >> # ######### Receiving Messages from Remote Hosts ########## > >> # TCP Syslog Server: > >> # provides TCP syslog reception and GSS-API (if compiled to support > it) > >> $ModLoad imtcp.so # load module > >> $InputTCPServerRun 514 # start up TCP listener at port 514 > >> > >> # UDP Syslog Server: > >> $ModLoad imudp.so # provides UDP syslog reception > >> $UDPServerRun 514 # start a UDP syslog server at standard port > >> -------------------------------------------------------- > >> This e-mail, including any attachments, may be confidential, > privileged or otherwise legally protected. If you have received this e- > mail in error, or from someone who was not authorized to send it to > you, do not disseminate, copy or otherwise use this e-mail or its > attachments. Please notify the sender immediately if you have received > this e-mail by mistake, and delete it from your system. > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Fri Jul 25 16:33:26 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 25 Jul 2008 10:33:26 -0400 Subject: [rsyslog] rsyslog error messages Message-ID: Hi Rainer, Interesting post. The tendency to ignore errors/diagnostics is not limited to rsyslog, it's something that plagues pretty much every piece of software ever created anywhere. Some users are ignorant of basic diagnostic tools like syslog, --help flags, tcpdump, and man pages. Others just need a bit of prodding before they slap themselves and realize that they forgot some basic troubleshooting. Some are lazy slobs. However, rsyslog lends itself to error reports that lack troubleshooting for one huge reason: errors are not reliably reported in the default configuration. It's happened a few times that I start rsyslogd at the command line, and yet there's nothing in the process table, and nothing dumped to my logs. I've been able to track down the problem with -dn switches, but that involves weeding through a lot of irrelevant information. There are three modifications I'd love to see that would help resolve this: 1 - Configuration errors should be fatal 2 - Configuration and other runtime errors should be printed to STDERR 3 - By default, rsyslogd should log all of its own errors (fatal or not) to /var/log/rsyslog.log. This should also be user-configurable My reasoning: 1 - If rsyslog doesn't understand your config file, obviously it won't be doing what you intend it to. There's little benefit in running with a horked configuration, but if you don't test thoroughly, the cost could be devastating. Some users won't know they're not logging things until they need them - then, if they keep their jobs, they probably won't keep rsyslog ("This software doesn't even log ssh attempts!"). 2 - This will avoid a lot of silly questions without any supporting information. At least when you get a complaint like "Rsyslogd won't start!" it will generally include a "It just says 'configuration file invalid: broken syntax at line 135.'" For most users, this kind of error reporting will be enough to lead them to a solution. If nothing else, it removes the need to explain how to find the errors. 3 - For those more subtle problems, and things you might miss upon bootup. This also gives you a simple, easily accessible debugging source with minimal security implications and additional code and no required user configuration. I'm not sure how the code is written, but I'd prefer to see this happen even if the configuration file can't be read. Perhaps an $RsyslogErrors directive that, by default, references a different module that just dumps directly to a file? Users can configure it to put rsyslog errors through the actual logging engine and then manage it with typical syslog selector/action lines. The HTTP server is an interesting idea, but not something I would make use of. The potential security problems are enormous, and it would introduce too many additional factors to the debugging process: is the firewall right? Was there already a process listening on that port? Is the user's configuration file correct? These are the kinds of things you already have to deal with on rsyslog itself - why force the debugging process to go through them again? Hope that helps, and sorry for the long email. -HKS On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards wrote: > Thanks, HKS, for the good answers. There is one thing I would like to > bring to your attention: I often see that people do not check for > rsyslog error messages themselves. That complicates setup. I do not > blame anyone, but would like to make you aware of the problem. > > I have just blogged about a potential (partial) solution and will try to > implement it: > > http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > ml > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of (private) HKS >> Sent: Thursday, July 24, 2008 11:36 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Help with Ommail Configuration >> >> Oh - one other possibility. rsyslogd has to have mail enabled at >> compile time to work with it at runtime. check your catchall logfile - >> if it has messages like this, you didn't compile it in: >> >> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', >> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object >> file: No such file or directory >> >> To fix this, you'll need to reinstall. This time, when you run >> ./configure be sure to include --enable-mail. make install clean and >> you should be good to go... >> >> -HKS >> >> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >> wrote: >> > A few things to try: >> > >> > - You load ommail.so twice, once at the top and once right above >> your >> > $ActionMail... lines. I don't think this will break it, but it's >> > unnecessary - delete the second one. >> > >> > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going >> > to attempt sending a message once every 6 hours. For testing >> purposes, >> > this will be obnoxious. Until you're ready for production, change it >> > to: >> > $ActionExecOnlyOnceEveryInterval 30 >> > >> > - Send everything to make sure you're not missing it based on >> > something in the property-replacer/templates/whatever. Replace "if >> > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > with: >> > *.* :ommail:;mailBody >> > >> > Try again. Try logging a few messages from the localhost first (with >> > RHEL you can just run "logger test" to log a user.notice message > with >> > content "test") and see if you get them. >> > >> > If you don't, check the mail logs on your mail server to see if it >> > ever received the message. If not, it's time to break out tcpdump > and >> > see if the packets are ever being generated. >> > >> > Hope that helps. >> > >> > -HKS >> > >> > >> > >> > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) >> > wrote: >> >> Hello all, >> >> >> >> First off, I am not very Linux savvy. I don't have a lot of >> experience >> >> other then a basic course. This is probably way past my knowledge, >> but I >> >> really need to get it done. Appreciate any help you guys have to >> offer. >> >> >> >> I am working on a Red Hat Enterprise 4 box and I am running the >> latest >> >> edition of rsyslog. I currently have rsyslog configured to receive >> >> messages remotely via UDP. I am trying to set it up so that it will >> send >> >> out E-mail messages to the system Admin's based on the severity >> level of >> >> the log files I am receiving. I would like it so that any device >> that >> >> sends a log with a critical, alert, or emergency level facility > will >> >> send out an e-mail to a specific address. >> >> >> >> Here is my rsyslog.conf file. I used the sample code from Rainer >> >> Gerhards configuration page. I tried sending a test syslog with >> 'hard >> >> disk fatal failure' in it, but it is not sending out mail. Also >> tried >> >> without the templates below thinking it would just send me an email >> for >> >> every syslog that I received, but it doesn't appear to be sending >> mail. >> >> Any thoughts on what I am doing wrong. I'm sure there is a lot I >> need to >> >> do, so please let me know. Thanks! >> >> >> >> $template mailSubject,"disk problem on %hostname%" >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >> >> >> >> I edited out the 3 lines below in the code for security reasons.. >> >> $ActionMailSMTPServer >> >> $ActionMailFrom >> >> $ActionMailTo >> >> >> >> >> >> >> >> Here is my code from rsyslog.conf below. >> >> >> >> >> >> >> >> # if you experience problems, check >> >> # http://www.rsyslog.com/troubleshoot for assistance >> >> >> >> # rsyslog v3: load input modules >> >> # If you do not load inputs, nothing happens! >> >> # You may need to set the module load path if modules are not > found. >> >> >> >> $ModLoad immark.so # provides --MARK-- message capability >> >> $ModLoad imuxsock.so # provides support for local system logging >> (e.g. >> >> via logger command) >> >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >> >> $ModLoad ommail >> >> >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >> >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >> >> >> >> # Log all kernel messages to the console. >> >> # Logging much else clutters up the screen. >> >> #kern.* > /dev/console >> >> >> >> # Log anything (except mail) of level info or higher. >> >> # Don't log private authentication messages! >> >> *.info;mail.none;authpriv.none;cron.none >> >> -/var/log/messages >> >> >> >> # The authpriv file has restricted access. >> >> authpriv.* >> /var/log/secure >> >> >> >> # Log all the mail messages in one place. >> >> mail.* >> >> -/var/log/maillog >> >> >> >> # Log cron stuff >> >> cron.* - >> /var/log/cron >> >> >> >> # Everybody gets emergency messages >> >> *.emerg * >> >> >> >> # Save news errors of level crit and higher in a special file. >> >> uucp,news.crit >> >> -/var/log/spooler >> >> >> >> # Save boot messages also to boot.log >> >> local7.* >> >> /var/log/boot.log >> >> >> >> #Catch all incoming syslog messages >> >> *.* >> >> /var/log/catchall;TraditionalFormatWithPRI >> >> >> >> # Remote Logging (we use TCP for reliable delivery) >> >> # An on-disk queue is created for this action. If the remote host > is >> >> # down, messages are spooled to disk and sent when it is up again. >> >> $WorkDirectory /rsyslog/spool # where to place spool files >> >> $ActionQueueFileName uniqName # unique name prefix for spool files >> >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >> >> possible) >> >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> >> $ActionQueueType LinkedList # run asynchronously >> >> $ActionResumeRetryCount -1 # infinite retries if host is down >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> >> *.* @10.57.106.140:514 >> >> >> >> $ModLoad ommail >> >> $ActionMailSMTPServer >> >> $ActionMailFrom >> >> $ActionMailTo >> >> $template mailSubject,"disk problem on %hostname%" >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >> >> $ActionMailSubject mailSubject >> >> # make sure we receive a mail only once in six >> >> # hours (21,600 seconds ;)) >> >> $ActionExecOnlyOnceEveryInterval 21600 >> >> # the if ... then ... mailBody mus be on one line! >> >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >> >> >> >> >> >> # ######### Receiving Messages from Remote Hosts ########## >> >> # TCP Syslog Server: >> >> # provides TCP syslog reception and GSS-API (if compiled to support >> it) >> >> $ModLoad imtcp.so # load module >> >> $InputTCPServerRun 514 # start up TCP listener at port 514 >> >> >> >> # UDP Syslog Server: >> >> $ModLoad imudp.so # provides UDP syslog reception >> >> $UDPServerRun 514 # start a UDP syslog server at standard port >> >> -------------------------------------------------------- >> >> This e-mail, including any attachments, may be confidential, >> privileged or otherwise legally protected. If you have received this > e- >> mail in error, or from someone who was not authorized to send it to >> you, do not disseminate, copy or otherwise use this e-mail or its >> attachments. Please notify the sender immediately if you have received >> this e-mail by mistake, and delete it from your system. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rfujita at redhat.com Fri Jul 25 16:56:04 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 25 Jul 2008 23:56:04 +0900 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: Hi HKS-san, +1 > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.' In addition, '... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.' Any ideas? Best Rio. On 2008/07/25, at 23:33, (private) HKS wrote: > Hi Rainer, > > Interesting post. The tendency to ignore errors/diagnostics is not > limited to rsyslog, it's something that plagues pretty much every > piece of software ever created anywhere. Some users are ignorant of > basic diagnostic tools like syslog, --help flags, tcpdump, and man > pages. Others just need a bit of prodding before they slap themselves > and realize that they forgot some basic troubleshooting. Some are lazy > slobs. > > However, rsyslog lends itself to error reports that lack > troubleshooting for one huge reason: errors are not reliably reported > in the default configuration. It's happened a few times that I start > rsyslogd at the command line, and yet there's nothing in the process > table, and nothing dumped to my logs. I've been able to track down the > problem with -dn switches, but that involves weeding through a lot of > irrelevant information. > > There are three modifications I'd love to see that would help > resolve this: > > 1 - Configuration errors should be fatal > 2 - Configuration and other runtime errors should be printed to STDERR > 3 - By default, rsyslogd should log all of its own errors (fatal or > not) to /var/log/rsyslog.log. This should also be user-configurable > > My reasoning: > > 1 - If rsyslog doesn't understand your config file, obviously it won't > be doing what you intend it to. There's little benefit in running with > a horked configuration, but if you don't test thoroughly, the cost > could be devastating. Some users won't know they're not logging things > until they need them - then, if they keep their jobs, they probably > won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.'" For most users, this kind of > error reporting will be enough to lead them to a solution. If nothing > else, it removes the need to explain how to find the errors. > > 3 - For those more subtle problems, and things you might miss upon > bootup. This also gives you a simple, easily accessible debugging > source with minimal security implications and additional code and no > required user configuration. I'm not sure how the code is written, but > I'd prefer to see this happen even if the configuration file can't be > read. Perhaps an $RsyslogErrors directive that, by default, references > a different module that just dumps directly to a file? Users can > configure it to put rsyslog errors through the actual logging engine > and then manage it with typical syslog selector/action lines. > > The HTTP server is an interesting idea, but not something I would make > use of. The potential security problems are enormous, and it would > introduce too many additional factors to the debugging process: is the > firewall right? Was there already a process listening on that port? Is > the user's configuration file correct? These are the kinds of things > you already have to deal with on rsyslog itself - why force the > debugging process to go through them again? > > Hope that helps, and sorry for the long email. > -HKS > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > wrote: >> Thanks, HKS, for the good answers. There is one thing I would like to >> bring to your attention: I often see that people do not check for >> rsyslog error messages themselves. That complicates setup. I do not >> blame anyone, but would like to make you aware of the problem. >> >> I have just blogged about a potential (partial) solution and will >> try to >> implement it: >> >> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht >> ml >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of (private) HKS >>> Sent: Thursday, July 24, 2008 11:36 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] Help with Ommail Configuration >>> >>> Oh - one other possibility. rsyslogd has to have mail enabled at >>> compile time to work with it at runtime. check your catchall >>> logfile - >>> if it has messages like this, you didn't compile it in: >>> >>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', >>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object >>> file: No such file or directory >>> >>> To fix this, you'll need to reinstall. This time, when you run >>> ./configure be sure to include --enable-mail. make install clean and >>> you should be good to go... >>> >>> -HKS >>> >>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >> > >>> wrote: >>>> A few things to try: >>>> >>>> - You load ommail.so twice, once at the top and once right above >>> your >>>> $ActionMail... lines. I don't think this will break it, but it's >>>> unnecessary - delete the second one. >>>> >>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going >>>> to attempt sending a message once every 6 hours. For testing >>> purposes, >>>> this will be obnoxious. Until you're ready for production, change >>>> it >>>> to: >>>> $ActionExecOnlyOnceEveryInterval 30 >>>> >>>> - Send everything to make sure you're not missing it based on >>>> something in the property-replacer/templates/whatever. Replace "if >>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody" >> with: >>>> *.* :ommail:;mailBody >>>> >>>> Try again. Try logging a few messages from the localhost first >>>> (with >>>> RHEL you can just run "logger test" to log a user.notice message >> with >>>> content "test") and see if you get them. >>>> >>>> If you don't, check the mail logs on your mail server to see if it >>>> ever received the message. If not, it's time to break out tcpdump >> and >>>> see if the packets are ever being generated. >>>> >>>> Hope that helps. >>>> >>>> -HKS >>>> >>>> >>>> >>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) >>>> wrote: >>>>> Hello all, >>>>> >>>>> First off, I am not very Linux savvy. I don't have a lot of >>> experience >>>>> other then a basic course. This is probably way past my knowledge, >>> but I >>>>> really need to get it done. Appreciate any help you guys have to >>> offer. >>>>> >>>>> I am working on a Red Hat Enterprise 4 box and I am running the >>> latest >>>>> edition of rsyslog. I currently have rsyslog configured to receive >>>>> messages remotely via UDP. I am trying to set it up so that it >>>>> will >>> send >>>>> out E-mail messages to the system Admin's based on the severity >>> level of >>>>> the log files I am receiving. I would like it so that any device >>> that >>>>> sends a log with a critical, alert, or emergency level facility >> will >>>>> send out an e-mail to a specific address. >>>>> >>>>> Here is my rsyslog.conf file. I used the sample code from Rainer >>>>> Gerhards configuration page. I tried sending a test syslog with >>> 'hard >>>>> disk fatal failure' in it, but it is not sending out mail. Also >>> tried >>>>> without the templates below thinking it would just send me an >>>>> email >>> for >>>>> every syslog that I received, but it doesn't appear to be sending >>> mail. >>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I >>> need to >>>>> do, so please let me know. Thanks! >>>>> >>>>> $template mailSubject,"disk problem on %hostname%" >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >>>>> >>>>> I edited out the 3 lines below in the code for security reasons.. >>>>> $ActionMailSMTPServer >>>>> $ActionMailFrom >>>>> $ActionMailTo >>>>> >>>>> >>>>> >>>>> Here is my code from rsyslog.conf below. >>>>> >>>>> >>>>> >>>>> # if you experience problems, check >>>>> # http://www.rsyslog.com/troubleshoot for assistance >>>>> >>>>> # rsyslog v3: load input modules >>>>> # If you do not load inputs, nothing happens! >>>>> # You may need to set the module load path if modules are not >> found. >>>>> >>>>> $ModLoad immark.so # provides --MARK-- message capability >>>>> $ModLoad imuxsock.so # provides support for local system logging >>> (e.g. >>>>> via logger command) >>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >>>>> $ModLoad ommail >>>>> >>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >>>>> >>>>> # Log all kernel messages to the console. >>>>> # Logging much else clutters up the screen. >>>>> #kern.* >> /dev/console >>>>> >>>>> # Log anything (except mail) of level info or higher. >>>>> # Don't log private authentication messages! >>>>> *.info;mail.none;authpriv.none;cron.none >>>>> -/var/log/messages >>>>> >>>>> # The authpriv file has restricted access. >>>>> authpriv.* >>> /var/log/secure >>>>> >>>>> # Log all the mail messages in one place. >>>>> mail.* >>>>> -/var/log/maillog >>>>> >>>>> # Log cron stuff >>>>> cron.* - >>> /var/log/cron >>>>> >>>>> # Everybody gets emergency messages >>>>> *.emerg * >>>>> >>>>> # Save news errors of level crit and higher in a special file. >>>>> uucp,news.crit >>>>> -/var/log/spooler >>>>> >>>>> # Save boot messages also to boot.log >>>>> local7.* >>>>> /var/log/boot.log >>>>> >>>>> #Catch all incoming syslog messages >>>>> *.* >>>>> /var/log/catchall;TraditionalFormatWithPRI >>>>> >>>>> # Remote Logging (we use TCP for reliable delivery) >>>>> # An on-disk queue is created for this action. If the remote host >> is >>>>> # down, messages are spooled to disk and sent when it is up again. >>>>> $WorkDirectory /rsyslog/spool # where to place spool files >>>>> $ActionQueueFileName uniqName # unique name prefix for spool files >>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >>>>> possible) >>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >>>>> $ActionQueueType LinkedList # run asynchronously >>>>> $ActionResumeRetryCount -1 # infinite retries if host is down >>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port >>>>> optional >>>>> *.* @10.57.106.140:514 >>>>> >>>>> $ModLoad ommail >>>>> $ActionMailSMTPServer >>>>> $ActionMailFrom >>>>> $ActionMailTo >>>>> $template mailSubject,"disk problem on %hostname%" >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >>>>> $ActionMailSubject mailSubject >>>>> # make sure we receive a mail only once in six >>>>> # hours (21,600 seconds ;)) >>>>> $ActionExecOnlyOnceEveryInterval 21600 >>>>> # the if ... then ... mailBody mus be on one line! >>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >>>>> >>>>> >>>>> # ######### Receiving Messages from Remote Hosts ########## >>>>> # TCP Syslog Server: >>>>> # provides TCP syslog reception and GSS-API (if compiled to >>>>> support >>> it) >>>>> $ModLoad imtcp.so # load module >>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 >>>>> >>>>> # UDP Syslog Server: >>>>> $ModLoad imudp.so # provides UDP syslog reception >>>>> $UDPServerRun 514 # start a UDP syslog server at standard port >>>>> -------------------------------------------------------- >>>>> This e-mail, including any attachments, may be confidential, >>> privileged or otherwise legally protected. If you have received this >> e- >>> mail in error, or from someone who was not authorized to send it to >>> you, do not disseminate, copy or otherwise use this e-mail or its >>> attachments. Please notify the sender immediately if you have >>> received >>> this e-mail by mistake, and delete it from your system. >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 25 17:16:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 17:16:12 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> On Fri, 2008-07-25 at 10:33 -0400, (private) HKS wrote: > Hi Rainer, > > Interesting post. The tendency to ignore errors/diagnostics is not > limited to rsyslog, it's something that plagues pretty much every > piece of software ever created anywhere. I totally agree, but I think the situation is even worse for a syslogd. Because it *is* the usual error reporting system. I think we are a bit in the same situation (if you allow that analogy) like HIV which is attacking the human immune system. We have a problem in the troubleshooting system itself, and it may even prevent us from conveying troubleshooting information :( > Some users are ignorant of > basic diagnostic tools like syslog, --help flags, tcpdump, and man > pages. Others just need a bit of prodding before they slap themselves > and realize that they forgot some basic troubleshooting. Some are lazy > slobs. > > However, rsyslog lends itself to error reports that lack > troubleshooting for one huge reason: errors are not reliably reported > in the default configuration. > It's happened a few times that I start > rsyslogd at the command line, and yet there's nothing in the process > table, and nothing dumped to my logs. Could you elaborate a bit about above sentence - I have to admit I do not fully get you... > I've been able to track down the > problem with -dn switches, but that involves weeding through a lot of > irrelevant information. > > There are three modifications I'd love to see that would help resolve this: > > 1 - Configuration errors should be fatal > 2 - Configuration and other runtime errors should be printed to STDERR Does that really help? If so, that would be extremely easy to add. But does somebody see them? > 3 - By default, rsyslogd should log all of its own errors (fatal or > not) to /var/log/rsyslog.log. This should also be user-configurable This default comes into the way of some distros: they use different pathes. I really don't like the idea of embedding any information about the file system into the binary. Just think about embedded systems... > > My reasoning: > > 1 - If rsyslog doesn't understand your config file, obviously it won't > be doing what you intend it to. There's little benefit in running with > a horked configuration, but if you don't test thoroughly, the cost > could be devastating. Some users won't know they're not logging things > until they need them - then, if they keep their jobs, they probably > won't keep rsyslog ("This software doesn't even log ssh attempts!"). That's quite controversal and and does not match rsyslog's root philosophy. Let me explain: The syslogd is an extremely important piece of software. If it is not functioning, a) vital data can be lost b) the system as whole may hang (due to the fact that the log socket gets filled up and so other processes block on it) As such, rsyslogd is written in a sense that it tries to continue to work under almost all circumstances. It even tries to survive a system-wide low memory condition, though I am not 100% convinced it will manage to do that in all cases. At least it is coded careful to survive as much as possible. Now let's assume that a configuration error happens that prevents rsyslogd from executing an otherwise perfect configuration. Should we actually stop it from running? And live with the results? Or should we do whatever we can do and do that as good as possible? What, if it is not a configuration issue but a hard disk failure or an admin error that make part of the config unavailable (e.g. not able to load module, not able to write file)? Assuming we can carry out other actions - should we really abort? This may even come at the price that we can NOT warn users (via wall) of a serious problem. And what with warnings? Things that look wrong, but may be OK. Reason to abort? Or continue run? Tough decision. So I still think the right thing to do is to run as long as there is limited sense in it, because everything else would actually be worse. Please note that rsyslogd even comes with a hardcoded emergency configuration which is used in case the real configuration file can not be obtained. What I could add, however, is an option to make it stop on any config error (not warning). However, I fear that, if turned on, the results can be fatal in some cases... > > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.'" For most users, this kind of > error reporting will be enough to lead them to a solution. If nothing > else, it removes the need to explain how to find the errors. > > 3 - For those more subtle problems, and things you might miss upon > bootup. This also gives you a simple, easily accessible debugging > source with minimal security implications and additional code and no > required user configuration. I'm not sure how the code is written, but > I'd prefer to see this happen even if the configuration file can't be > read. Perhaps an $RsyslogErrors directive that, by default, references > a different module that just dumps directly to a file? Users can > configure it to put rsyslog errors through the actual logging engine > and then manage it with typical syslog selector/action lines. All internally-generated messages are run through a specific interface. For the "diag plugin", I will define a hook, where a diag server can register to receive these messages. Maybe a good compromise would be to add this $RsyslogErrors directive and make it point to a file. So, still, we do not necessarily have it, but package maintainers would hopefully include it and point it to a specific file (what still means I do not necessarily know where to look). A problem is source tarball install, in which case the directive may not be set. But with a proper sample rsyslog.conf... > The HTTP server is an interesting idea, but not something I would make > use of. The potential security problems are enormous, and it would > introduce too many additional factors to the debugging process: is the > firewall right? Was there already a process listening on that port? Is > the user's configuration file correct? These are the kinds of things > you already have to deal with on rsyslog itself - why force the > debugging process to go through them again? I see (and share) much of your argument. This is why I stayed away for such a long time. But if you think a bit more about it, it has *very interesting* capabilities. Eg. I may also use it to gather some real-time view of internal state, emit commands like HUP and do a number of other nice things. Again, security is problematic. In any case, I'll give it a try. I have started to write a small plugin which will dump the error messages within 10 minutes after startup. It will not be a real http server but rather a listener that response to anyone who connects to it (so that nc can be used as a client ;)). This is very rough, but it has two side-effects: a) it provides something to actually experiment with (and to extend if we decide to keep that route) b) it helps create the plumbing inside the internal interface that could also be used by any other method, so the potential waste of time is limited So while I object some suggestions, I am very interested in a discussion. The rsyslog "keep running" philosophy *is* a discussion topic, just expect me to argue strongly in favor of it (pls think about the HIV analogy to see why I think we have a unique situation in case of a syslogd). Thanks for the good post, and I guess I just wrote another long mail ;) Rainer > > Hope that helps, and sorry for the long email. > -HKS > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > wrote: > > Thanks, HKS, for the good answers. There is one thing I would like to > > bring to your attention: I often see that people do not check for > > rsyslog error messages themselves. That complicates setup. I do not > > blame anyone, but would like to make you aware of the problem. > > > > I have just blogged about a potential (partial) solution and will try to > > implement it: > > > > http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > > ml > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of (private) HKS > >> Sent: Thursday, July 24, 2008 11:36 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Help with Ommail Configuration > >> > >> Oh - one other possibility. rsyslogd has to have mail enabled at > >> compile time to work with it at runtime. check your catchall logfile - > >> if it has messages like this, you didn't compile it in: > >> > >> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > >> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > >> file: No such file or directory > >> > >> To fix this, you'll need to reinstall. This time, when you run > >> ./configure be sure to include --enable-mail. make install clean and > >> you should be good to go... > >> > >> -HKS > >> > >> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS > >> wrote: > >> > A few things to try: > >> > > >> > - You load ommail.so twice, once at the top and once right above > >> your > >> > $ActionMail... lines. I don't think this will break it, but it's > >> > unnecessary - delete the second one. > >> > > >> > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > >> > to attempt sending a message once every 6 hours. For testing > >> purposes, > >> > this will be obnoxious. Until you're ready for production, change it > >> > to: > >> > $ActionExecOnlyOnceEveryInterval 30 > >> > > >> > - Send everything to make sure you're not missing it based on > >> > something in the property-replacer/templates/whatever. Replace "if > >> > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > > with: > >> > *.* :ommail:;mailBody > >> > > >> > Try again. Try logging a few messages from the localhost first (with > >> > RHEL you can just run "logger test" to log a user.notice message > > with > >> > content "test") and see if you get them. > >> > > >> > If you don't, check the mail logs on your mail server to see if it > >> > ever received the message. If not, it's time to break out tcpdump > > and > >> > see if the packets are ever being generated. > >> > > >> > Hope that helps. > >> > > >> > -HKS > >> > > >> > > >> > > >> > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > >> > wrote: > >> >> Hello all, > >> >> > >> >> First off, I am not very Linux savvy. I don't have a lot of > >> experience > >> >> other then a basic course. This is probably way past my knowledge, > >> but I > >> >> really need to get it done. Appreciate any help you guys have to > >> offer. > >> >> > >> >> I am working on a Red Hat Enterprise 4 box and I am running the > >> latest > >> >> edition of rsyslog. I currently have rsyslog configured to receive > >> >> messages remotely via UDP. I am trying to set it up so that it will > >> send > >> >> out E-mail messages to the system Admin's based on the severity > >> level of > >> >> the log files I am receiving. I would like it so that any device > >> that > >> >> sends a log with a critical, alert, or emergency level facility > > will > >> >> send out an e-mail to a specific address. > >> >> > >> >> Here is my rsyslog.conf file. I used the sample code from Rainer > >> >> Gerhards configuration page. I tried sending a test syslog with > >> 'hard > >> >> disk fatal failure' in it, but it is not sending out mail. Also > >> tried > >> >> without the templates below thinking it would just send me an email > >> for > >> >> every syslog that I received, but it doesn't appear to be sending > >> mail. > >> >> Any thoughts on what I am doing wrong. I'm sure there is a lot I > >> need to > >> >> do, so please let me know. Thanks! > >> >> > >> >> $template mailSubject,"disk problem on %hostname%" > >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >> >> > >> >> I edited out the 3 lines below in the code for security reasons.. > >> >> $ActionMailSMTPServer > >> >> $ActionMailFrom > >> >> $ActionMailTo > >> >> > >> >> > >> >> > >> >> Here is my code from rsyslog.conf below. > >> >> > >> >> > >> >> > >> >> # if you experience problems, check > >> >> # http://www.rsyslog.com/troubleshoot for assistance > >> >> > >> >> # rsyslog v3: load input modules > >> >> # If you do not load inputs, nothing happens! > >> >> # You may need to set the module load path if modules are not > > found. > >> >> > >> >> $ModLoad immark.so # provides --MARK-- message capability > >> >> $ModLoad imuxsock.so # provides support for local system logging > >> (e.g. > >> >> via logger command) > >> >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >> >> $ModLoad ommail > >> >> > >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >> >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >> >> > >> >> # Log all kernel messages to the console. > >> >> # Logging much else clutters up the screen. > >> >> #kern.* > > /dev/console > >> >> > >> >> # Log anything (except mail) of level info or higher. > >> >> # Don't log private authentication messages! > >> >> *.info;mail.none;authpriv.none;cron.none > >> >> -/var/log/messages > >> >> > >> >> # The authpriv file has restricted access. > >> >> authpriv.* > >> /var/log/secure > >> >> > >> >> # Log all the mail messages in one place. > >> >> mail.* > >> >> -/var/log/maillog > >> >> > >> >> # Log cron stuff > >> >> cron.* - > >> /var/log/cron > >> >> > >> >> # Everybody gets emergency messages > >> >> *.emerg * > >> >> > >> >> # Save news errors of level crit and higher in a special file. > >> >> uucp,news.crit > >> >> -/var/log/spooler > >> >> > >> >> # Save boot messages also to boot.log > >> >> local7.* > >> >> /var/log/boot.log > >> >> > >> >> #Catch all incoming syslog messages > >> >> *.* > >> >> /var/log/catchall;TraditionalFormatWithPRI > >> >> > >> >> # Remote Logging (we use TCP for reliable delivery) > >> >> # An on-disk queue is created for this action. If the remote host > > is > >> >> # down, messages are spooled to disk and sent when it is up again. > >> >> $WorkDirectory /rsyslog/spool # where to place spool files > >> >> $ActionQueueFileName uniqName # unique name prefix for spool files > >> >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >> >> possible) > >> >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >> >> $ActionQueueType LinkedList # run asynchronously > >> >> $ActionResumeRetryCount -1 # infinite retries if host is down > >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> >> *.* @10.57.106.140:514 > >> >> > >> >> $ModLoad ommail > >> >> $ActionMailSMTPServer > >> >> $ActionMailFrom > >> >> $ActionMailTo > >> >> $template mailSubject,"disk problem on %hostname%" > >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >> >> $ActionMailSubject mailSubject > >> >> # make sure we receive a mail only once in six > >> >> # hours (21,600 seconds ;)) > >> >> $ActionExecOnlyOnceEveryInterval 21600 > >> >> # the if ... then ... mailBody mus be on one line! > >> >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >> >> > >> >> > >> >> # ######### Receiving Messages from Remote Hosts ########## > >> >> # TCP Syslog Server: > >> >> # provides TCP syslog reception and GSS-API (if compiled to support > >> it) > >> >> $ModLoad imtcp.so # load module > >> >> $InputTCPServerRun 514 # start up TCP listener at port 514 > >> >> > >> >> # UDP Syslog Server: > >> >> $ModLoad imudp.so # provides UDP syslog reception > >> >> $UDPServerRun 514 # start a UDP syslog server at standard port > >> >> -------------------------------------------------------- > >> >> This e-mail, including any attachments, may be confidential, > >> privileged or otherwise legally protected. If you have received this > > e- > >> mail in error, or from someone who was not authorized to send it to > >> you, do not disseminate, copy or otherwise use this e-mail or its > >> attachments. Please notify the sender immediately if you have received > >> this e-mail by mistake, and delete it from your system. > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 25 17:25:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 17:25:31 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: <1216999531.7184.219.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Fri, 2008-07-25 at 23:56 +0900, Ryo Fujita wrote: > Hi HKS-san, > > +1 > > > 2 - This will avoid a lot of silly questions without any supporting > > information. At least when you get a complaint like "Rsyslogd won't > > start!" it will generally include a "It just says 'configuration file > > invalid: broken syntax at line 135.' > > In addition, > '... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.' Same thought over here :) The recent build even emits live links to troubleshooting information. Log lines look like this: 2008-07-25T17:22:18.523900+02:00 rgf9dev rsyslogd-3003: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] Note the part in the bracket at the end of the message. That repository is far from being complete, but I hope more and more information will be added. I hope this is a useful addition. Together with the error number, we can hopefully most often pinpoint the source of the problem (though the 3003 above is a good example of where it may not be possible to tell the exact cause, but for sure give a good hint). BTW: this is also one thing that makes me think about the scripting engine. It shall provide better error reporting capabilties, but that's harder than it initially sounds ;) Rainer > > Any ideas? > > Best Rio. > > On 2008/07/25, at 23:33, (private) HKS wrote: > > > Hi Rainer, > > > > Interesting post. The tendency to ignore errors/diagnostics is not > > limited to rsyslog, it's something that plagues pretty much every > > piece of software ever created anywhere. Some users are ignorant of > > basic diagnostic tools like syslog, --help flags, tcpdump, and man > > pages. Others just need a bit of prodding before they slap themselves > > and realize that they forgot some basic troubleshooting. Some are lazy > > slobs. > > > > However, rsyslog lends itself to error reports that lack > > troubleshooting for one huge reason: errors are not reliably reported > > in the default configuration. It's happened a few times that I start > > rsyslogd at the command line, and yet there's nothing in the process > > table, and nothing dumped to my logs. I've been able to track down the > > problem with -dn switches, but that involves weeding through a lot of > > irrelevant information. > > > > There are three modifications I'd love to see that would help > > resolve this: > > > > 1 - Configuration errors should be fatal > > 2 - Configuration and other runtime errors should be printed to STDERR > > 3 - By default, rsyslogd should log all of its own errors (fatal or > > not) to /var/log/rsyslog.log. This should also be user-configurable > > > > My reasoning: > > > > 1 - If rsyslog doesn't understand your config file, obviously it won't > > be doing what you intend it to. There's little benefit in running with > > a horked configuration, but if you don't test thoroughly, the cost > > could be devastating. Some users won't know they're not logging things > > until they need them - then, if they keep their jobs, they probably > > won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > > > 2 - This will avoid a lot of silly questions without any supporting > > information. At least when you get a complaint like "Rsyslogd won't > > start!" it will generally include a "It just says 'configuration file > > invalid: broken syntax at line 135.'" For most users, this kind of > > error reporting will be enough to lead them to a solution. If nothing > > else, it removes the need to explain how to find the errors. > > > > 3 - For those more subtle problems, and things you might miss upon > > bootup. This also gives you a simple, easily accessible debugging > > source with minimal security implications and additional code and no > > required user configuration. I'm not sure how the code is written, but > > I'd prefer to see this happen even if the configuration file can't be > > read. Perhaps an $RsyslogErrors directive that, by default, references > > a different module that just dumps directly to a file? Users can > > configure it to put rsyslog errors through the actual logging engine > > and then manage it with typical syslog selector/action lines. > > > > The HTTP server is an interesting idea, but not something I would make > > use of. The potential security problems are enormous, and it would > > introduce too many additional factors to the debugging process: is the > > firewall right? Was there already a process listening on that port? Is > > the user's configuration file correct? These are the kinds of things > > you already have to deal with on rsyslog itself - why force the > > debugging process to go through them again? > > > > Hope that helps, and sorry for the long email. > > -HKS > > > > > > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > > wrote: > >> Thanks, HKS, for the good answers. There is one thing I would like to > >> bring to your attention: I often see that people do not check for > >> rsyslog error messages themselves. That complicates setup. I do not > >> blame anyone, but would like to make you aware of the problem. > >> > >> I have just blogged about a potential (partial) solution and will > >> try to > >> implement it: > >> > >> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > >> ml > >> > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of (private) HKS > >>> Sent: Thursday, July 24, 2008 11:36 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] Help with Ommail Configuration > >>> > >>> Oh - one other possibility. rsyslogd has to have mail enabled at > >>> compile time to work with it at runtime. check your catchall > >>> logfile - > >>> if it has messages like this, you didn't compile it in: > >>> > >>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > >>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > >>> file: No such file or directory > >>> > >>> To fix this, you'll need to reinstall. This time, when you run > >>> ./configure be sure to include --enable-mail. make install clean and > >>> you should be good to go... > >>> > >>> -HKS > >>> > >>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >>> > > >>> wrote: > >>>> A few things to try: > >>>> > >>>> - You load ommail.so twice, once at the top and once right above > >>> your > >>>> $ActionMail... lines. I don't think this will break it, but it's > >>>> unnecessary - delete the second one. > >>>> > >>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > >>>> to attempt sending a message once every 6 hours. For testing > >>> purposes, > >>>> this will be obnoxious. Until you're ready for production, change > >>>> it > >>>> to: > >>>> $ActionExecOnlyOnceEveryInterval 30 > >>>> > >>>> - Send everything to make sure you're not missing it based on > >>>> something in the property-replacer/templates/whatever. Replace "if > >>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > >> with: > >>>> *.* :ommail:;mailBody > >>>> > >>>> Try again. Try logging a few messages from the localhost first > >>>> (with > >>>> RHEL you can just run "logger test" to log a user.notice message > >> with > >>>> content "test") and see if you get them. > >>>> > >>>> If you don't, check the mail logs on your mail server to see if it > >>>> ever received the message. If not, it's time to break out tcpdump > >> and > >>>> see if the packets are ever being generated. > >>>> > >>>> Hope that helps. > >>>> > >>>> -HKS > >>>> > >>>> > >>>> > >>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > >>>> wrote: > >>>>> Hello all, > >>>>> > >>>>> First off, I am not very Linux savvy. I don't have a lot of > >>> experience > >>>>> other then a basic course. This is probably way past my knowledge, > >>> but I > >>>>> really need to get it done. Appreciate any help you guys have to > >>> offer. > >>>>> > >>>>> I am working on a Red Hat Enterprise 4 box and I am running the > >>> latest > >>>>> edition of rsyslog. I currently have rsyslog configured to receive > >>>>> messages remotely via UDP. I am trying to set it up so that it > >>>>> will > >>> send > >>>>> out E-mail messages to the system Admin's based on the severity > >>> level of > >>>>> the log files I am receiving. I would like it so that any device > >>> that > >>>>> sends a log with a critical, alert, or emergency level facility > >> will > >>>>> send out an e-mail to a specific address. > >>>>> > >>>>> Here is my rsyslog.conf file. I used the sample code from Rainer > >>>>> Gerhards configuration page. I tried sending a test syslog with > >>> 'hard > >>>>> disk fatal failure' in it, but it is not sending out mail. Also > >>> tried > >>>>> without the templates below thinking it would just send me an > >>>>> email > >>> for > >>>>> every syslog that I received, but it doesn't appear to be sending > >>> mail. > >>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I > >>> need to > >>>>> do, so please let me know. Thanks! > >>>>> > >>>>> $template mailSubject,"disk problem on %hostname%" > >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >>>>> > >>>>> I edited out the 3 lines below in the code for security reasons.. > >>>>> $ActionMailSMTPServer > >>>>> $ActionMailFrom > >>>>> $ActionMailTo > >>>>> > >>>>> > >>>>> > >>>>> Here is my code from rsyslog.conf below. > >>>>> > >>>>> > >>>>> > >>>>> # if you experience problems, check > >>>>> # http://www.rsyslog.com/troubleshoot for assistance > >>>>> > >>>>> # rsyslog v3: load input modules > >>>>> # If you do not load inputs, nothing happens! > >>>>> # You may need to set the module load path if modules are not > >> found. > >>>>> > >>>>> $ModLoad immark.so # provides --MARK-- message capability > >>>>> $ModLoad imuxsock.so # provides support for local system logging > >>> (e.g. > >>>>> via logger command) > >>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >>>>> $ModLoad ommail > >>>>> > >>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >>>>> > >>>>> # Log all kernel messages to the console. > >>>>> # Logging much else clutters up the screen. > >>>>> #kern.* > >> /dev/console > >>>>> > >>>>> # Log anything (except mail) of level info or higher. > >>>>> # Don't log private authentication messages! > >>>>> *.info;mail.none;authpriv.none;cron.none > >>>>> -/var/log/messages > >>>>> > >>>>> # The authpriv file has restricted access. > >>>>> authpriv.* > >>> /var/log/secure > >>>>> > >>>>> # Log all the mail messages in one place. > >>>>> mail.* > >>>>> -/var/log/maillog > >>>>> > >>>>> # Log cron stuff > >>>>> cron.* - > >>> /var/log/cron > >>>>> > >>>>> # Everybody gets emergency messages > >>>>> *.emerg * > >>>>> > >>>>> # Save news errors of level crit and higher in a special file. > >>>>> uucp,news.crit > >>>>> -/var/log/spooler > >>>>> > >>>>> # Save boot messages also to boot.log > >>>>> local7.* > >>>>> /var/log/boot.log > >>>>> > >>>>> #Catch all incoming syslog messages > >>>>> *.* > >>>>> /var/log/catchall;TraditionalFormatWithPRI > >>>>> > >>>>> # Remote Logging (we use TCP for reliable delivery) > >>>>> # An on-disk queue is created for this action. If the remote host > >> is > >>>>> # down, messages are spooled to disk and sent when it is up again. > >>>>> $WorkDirectory /rsyslog/spool # where to place spool files > >>>>> $ActionQueueFileName uniqName # unique name prefix for spool files > >>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >>>>> possible) > >>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >>>>> $ActionQueueType LinkedList # run asynchronously > >>>>> $ActionResumeRetryCount -1 # infinite retries if host is down > >>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port > >>>>> optional > >>>>> *.* @10.57.106.140:514 > >>>>> > >>>>> $ModLoad ommail > >>>>> $ActionMailSMTPServer > >>>>> $ActionMailFrom > >>>>> $ActionMailTo > >>>>> $template mailSubject,"disk problem on %hostname%" > >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >>>>> $ActionMailSubject mailSubject > >>>>> # make sure we receive a mail only once in six > >>>>> # hours (21,600 seconds ;)) > >>>>> $ActionExecOnlyOnceEveryInterval 21600 > >>>>> # the if ... then ... mailBody mus be on one line! > >>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >>>>> > >>>>> > >>>>> # ######### Receiving Messages from Remote Hosts ########## > >>>>> # TCP Syslog Server: > >>>>> # provides TCP syslog reception and GSS-API (if compiled to > >>>>> support > >>> it) > >>>>> $ModLoad imtcp.so # load module > >>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 > >>>>> > >>>>> # UDP Syslog Server: > >>>>> $ModLoad imudp.so # provides UDP syslog reception > >>>>> $UDPServerRun 514 # start a UDP syslog server at standard port > >>>>> -------------------------------------------------------- > >>>>> This e-mail, including any attachments, may be confidential, > >>> privileged or otherwise legally protected. If you have received this > >> e- > >>> mail in error, or from someone who was not authorized to send it to > >>> you, do not disseminate, copy or otherwise use this e-mail or its > >>> attachments. Please notify the sender immediately if you have > >>> received > >>> this e-mail by mistake, and delete it from your system. > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Fri Jul 25 17:44:18 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 25 Jul 2008 11:44:18 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> Message-ID: Responses inline. > I totally agree, but I think the situation is even worse for a syslogd. > Because it *is* the usual error reporting system. I think we are a bit > in the same situation (if you allow that analogy) like HIV which is > attacking the human immune system. We have a problem in the > troubleshooting system itself, and it may even prevent us from conveying > troubleshooting information :( That's a good way of putting it. This is one of the reasons I think it's so important to print errors to STDERR - if the logging mechanisms are broken for some reason, then at least they're going somewhere. And I sincerely hope that users have the sense to start it from the command line the first time rather than sticking it into their system startup scripts and rebooting. >> It's happened a few times that I start >> rsyslogd at the command line, and yet there's nothing in the process >> table, and nothing dumped to my logs. > > Could you elaborate a bit about above sentence - I have to admit I do > not fully get you... I've messed up my configs bad enough a couple times that when I start rsyslogd, nothing apparently happens. No errors to STDERR, no errors in my logs, and the process isn't running. Rewriting the config or running with -dn is the solution. >> 2 - Configuration and other runtime errors should be printed to STDERR > > Does that really help? If so, that would be extremely easy to add. But > does somebody see them? Absolutely. The first time I configure any daemon, I do it in a test environment and run it by hand until I have the configuration acting exactly as I want. Most users take the "run-by-hand" precaution even if they don't have the luxury of a test setup. These will also pop up when a user tries to restart the daemon after making a config change. I'm not sure if they'll be visible when a HUP signal is sent, but that would make me awfully happy. >> 3 - By default, rsyslogd should log all of its own errors (fatal or >> not) to /var/log/rsyslog.log. This should also be user-configurable > > This default comes into the way of some distros: they use different > pathes. I really don't like the idea of embedding any information about > the file system into the binary. Just think about embedded systems... Good point. >> 1 - If rsyslog doesn't understand your config file, obviously it won't >> be doing what you intend it to. There's little benefit in running with >> a horked configuration, but if you don't test thoroughly, the cost >> could be devastating. Some users won't know they're not logging things >> until they need them - then, if they keep their jobs, they probably >> won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > That's quite controversal and and does not match rsyslog's root > philosophy. Let me explain: > > The syslogd is an extremely important piece of software. If it is not > functioning, > > a) vital data can be lost > b) the system as whole may hang (due to the fact that the log > socket gets filled up and so other processes block on it) > > As such, rsyslogd is written in a sense that it tries to continue to > work under almost all circumstances. It even tries to survive a > system-wide low memory condition, though I am not 100% convinced it will > manage to do that in all cases. At least it is coded careful to survive > as much as possible. > > Now let's assume that a configuration error happens that prevents > rsyslogd from executing an otherwise perfect configuration. Should we > actually stop it from running? And live with the results? Or should we > do whatever we can do and do that as good as possible? What, if it is > not a configuration issue but a hard disk failure or an admin error that > make part of the config unavailable (e.g. not able to load module, not > able to write file)? Assuming we can carry out other actions - should we > really abort? This may even come at the price that we can NOT warn users > (via wall) of a serious problem. And what with warnings? Things that > look wrong, but may be OK. Reason to abort? Or continue run? Tough > decision. So I still think the right thing to do is to run as long as > there is limited sense in it, because everything else would actually be > worse. Please note that rsyslogd even comes with a hardcoded emergency > configuration which is used in case the real configuration file can not > be obtained. > > What I could add, however, is an option to make it stop on any config > error (not warning). However, I fear that, if turned on, the results can > be fatal in some cases... You make a lot of good points here. Does the hardcoded emergency configuration that rsyslogd uses actually log? I only ask because it's easy (if you fail to read the documentation) to write a config file that doesn't do anything. Would it make sense for rsyslog to determine whether the passed configuration actually asks it to do anything, and complaining if not? Printing errors to STDERR would help with this. Another possibility might be adding a flag (-n is typical for a lot of systems, but that's already in use) that will tell rsyslog to parse the configuration, spit out any problems, and quit. That way, administrators can modify a config file on a live system and verify it's syntax without hosing the actual rsyslogd process. >> 3 - For those more subtle problems, and things you might miss upon >> bootup. This also gives you a simple, easily accessible debugging >> source with minimal security implications and additional code and no >> required user configuration. I'm not sure how the code is written, but >> I'd prefer to see this happen even if the configuration file can't be >> read. Perhaps an $RsyslogErrors directive that, by default, references >> a different module that just dumps directly to a file? Users can >> configure it to put rsyslog errors through the actual logging engine >> and then manage it with typical syslog selector/action lines. > > All internally-generated messages are run through a specific interface. > For the "diag plugin", I will define a hook, where a diag server can > register to receive these messages. Maybe a good compromise would be to > add this $RsyslogErrors directive and make it point to a file. So, > still, we do not necessarily have it, but package maintainers would > hopefully include it and point it to a specific file (what still means I > do not necessarily know where to look). A problem is source tarball > install, in which case the directive may not be set. But with a proper > sample rsyslog.conf... I think a locally installed sample rsyslog.conf is a solid workaround for the "different distros, different locations" problem. > >> The HTTP server is an interesting idea, but not something I would make >> use of. The potential security problems are enormous, and it would >> introduce too many additional factors to the debugging process: is the >> firewall right? Was there already a process listening on that port? Is >> the user's configuration file correct? These are the kinds of things >> you already have to deal with on rsyslog itself - why force the >> debugging process to go through them again? > > I see (and share) much of your argument. This is why I stayed away for > such a long time. But if you think a bit more about it, it has *very > interesting* capabilities. Eg. I may also use it to gather some > real-time view of internal state, emit commands like HUP and do a number > of other nice things. Again, security is problematic. > > In any case, I'll give it a try. I have started to write a small plugin > which will dump the error messages within 10 minutes after startup. It > will not be a real http server but rather a listener that response to > anyone who connects to it (so that nc can be used as a client ;)). This > is very rough, but it has two side-effects: > > a) it provides something to actually experiment with (and to extend if > we decide to keep that route) > b) it helps create the plumbing inside the internal interface that could > also be used by any other method, so the potential waste of time is > limited It sounds interesting, and I agree that there's a lot of potential there. Among them, the possibility of adding an XML-RPC API to allow for greater interface flexibility...but frankly, that scares the hell out of me. ;) I'll wait to see what you come up with. -HKS From rgerhards at hq.adiscon.com Fri Jul 25 18:07:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 18:07:51 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> Message-ID: <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> inline... On Fri, 2008-07-25 at 11:44 -0400, (private) HKS wrote: > Responses inline. > > > I totally agree, but I think the situation is even worse for a syslogd. > > Because it *is* the usual error reporting system. I think we are a bit > > in the same situation (if you allow that analogy) like HIV which is > > attacking the human immune system. We have a problem in the > > troubleshooting system itself, and it may even prevent us from conveying > > troubleshooting information :( > > That's a good way of putting it. This is one of the reasons I think > it's so important to print errors to STDERR - if the logging > mechanisms are broken for some reason, then at least they're going > somewhere. And I sincerely hope that users have the sense to start it > from the command line the first time rather than sticking it into > their system startup scripts and rebooting. If I look at some of the support calls I see, I doubt that ;) The problem is that there are a lot of novices who have never ever done any real system administration. But as the rsyslog homepage says: "Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user." Look at the end of the sentence. In any case, I see the utility of what you say, if not in the novice case, then with more experienced users. > >> It's happened a few times that I start > >> rsyslogd at the command line, and yet there's nothing in the process > >> table, and nothing dumped to my logs. > > > > Could you elaborate a bit about above sentence - I have to admit I do > > not fully get you... > > I've messed up my configs bad enough a couple times that when I start > rsyslogd, nothing apparently happens. No errors to STDERR, no errors > in my logs, and the process isn't running. Rewriting the config or > running with -dn is the solution. Ah, ok, now I understand. Thanks. > > > > >> 2 - Configuration and other runtime errors should be printed to STDERR > > > > Does that really help? If so, that would be extremely easy to add. But > > does somebody see them? > > Absolutely. The first time I configure any daemon, I do it in a test > environment and run it by hand until I have the configuration acting > exactly as I want. I understand. That makes a lot of sense. As I said, it should be easy to implement. I'll once again shuffle priorities a bit and make that the top one. Should not take too long to implement. I'll probably have a switch to disable it, but being enabled by default. > Most users take the "run-by-hand" precaution even > if they don't have the luxury of a test setup. These will also pop up > when a user tries to restart the daemon after making a config change. > I'm not sure if they'll be visible when a HUP signal is sent, but that > would make me awfully happy. So expect to be happy some time next week ;) > > > >> 3 - By default, rsyslogd should log all of its own errors (fatal or > >> not) to /var/log/rsyslog.log. This should also be user-configurable > > > > This default comes into the way of some distros: they use different > > pathes. I really don't like the idea of embedding any information about > > the file system into the binary. Just think about embedded systems... > > Good point. > > > >> 1 - If rsyslog doesn't understand your config file, obviously it won't > >> be doing what you intend it to. There's little benefit in running with > >> a horked configuration, but if you don't test thoroughly, the cost > >> could be devastating. Some users won't know they're not logging things > >> until they need them - then, if they keep their jobs, they probably > >> won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > > > That's quite controversal and and does not match rsyslog's root > > philosophy. Let me explain: > > > > The syslogd is an extremely important piece of software. If it is not > > functioning, > > > > a) vital data can be lost > > b) the system as whole may hang (due to the fact that the log > > socket gets filled up and so other processes block on it) > > > > As such, rsyslogd is written in a sense that it tries to continue to > > work under almost all circumstances. It even tries to survive a > > system-wide low memory condition, though I am not 100% convinced it will > > manage to do that in all cases. At least it is coded careful to survive > > as much as possible. > > > > Now let's assume that a configuration error happens that prevents > > rsyslogd from executing an otherwise perfect configuration. Should we > > actually stop it from running? And live with the results? Or should we > > do whatever we can do and do that as good as possible? What, if it is > > not a configuration issue but a hard disk failure or an admin error that > > make part of the config unavailable (e.g. not able to load module, not > > able to write file)? Assuming we can carry out other actions - should we > > really abort? This may even come at the price that we can NOT warn users > > (via wall) of a serious problem. And what with warnings? Things that > > look wrong, but may be OK. Reason to abort? Or continue run? Tough > > decision. So I still think the right thing to do is to run as long as > > there is limited sense in it, because everything else would actually be > > worse. Please note that rsyslogd even comes with a hardcoded emergency > > configuration which is used in case the real configuration file can not > > be obtained. > > > > What I could add, however, is an option to make it stop on any config > > error (not warning). However, I fear that, if turned on, the results can > > be fatal in some cases... > > You make a lot of good points here. Does the hardcoded emergency > configuration that rsyslogd uses actually log? Yes, but in a very limited sense. It puts *.err onto the console, does a wall for *.panic and writes everything else to the controlling tty. I have to admit that this code isn't awfully well tested, usually only when something is done at the config code (and sometimes not even then...). > I only ask because it's > easy (if you fail to read the documentation) to write a config file > that doesn't do anything. Would it make sense for rsyslog to determine > whether the passed configuration actually asks it to do anything, and > complaining if not? That's an interesting idea. I am not sure if this is already checked. I can at least check if there is no action at all, which means there is no useful work to be done. The question remains, as usual, where to complain to. But that could be another situation where the hardcoded config could be activated. I have also thought about an implicit write user, with the user being root by default (and being overwritable via command line and config file). > > Printing errors to STDERR would help with this. Another possibility > might be adding a flag (-n is typical for a lot of systems, but that's > already in use) that will tell rsyslog to parse the configuration, > spit out any problems, and quit. That way, administrators can modify a > config file on a live system and verify it's syntax without hosing the > actual rsyslogd process. That's also a good idea. Rsyslog is quite modular (not yet as much as I would like it to be, but well enough). So I could even generate a dedicated config check tool. That would not touch any running configuration at all. But maybe the switch is easier ;) I need to think on how this affects the testbed I have begun to work on. Looks like this could also drive some automated tests. > > > > >> 3 - For those more subtle problems, and things you might miss upon > >> bootup. This also gives you a simple, easily accessible debugging > >> source with minimal security implications and additional code and no > >> required user configuration. I'm not sure how the code is written, but > >> I'd prefer to see this happen even if the configuration file can't be > >> read. Perhaps an $RsyslogErrors directive that, by default, references > >> a different module that just dumps directly to a file? Users can > >> configure it to put rsyslog errors through the actual logging engine > >> and then manage it with typical syslog selector/action lines. > > > > All internally-generated messages are run through a specific interface. > > For the "diag plugin", I will define a hook, where a diag server can > > register to receive these messages. Maybe a good compromise would be to > > add this $RsyslogErrors directive and make it point to a file. So, > > still, we do not necessarily have it, but package maintainers would > > hopefully include it and point it to a specific file (what still means I > > do not necessarily know where to look). A problem is source tarball > > install, in which case the directive may not be set. But with a proper > > sample rsyslog.conf... > > I think a locally installed sample rsyslog.conf is a solid workaround > for the "different distros, different locations" problem. > > > > >> The HTTP server is an interesting idea, but not something I would make > >> use of. The potential security problems are enormous, and it would > >> introduce too many additional factors to the debugging process: is the > >> firewall right? Was there already a process listening on that port? Is > >> the user's configuration file correct? These are the kinds of things > >> you already have to deal with on rsyslog itself - why force the > >> debugging process to go through them again? > > > > I see (and share) much of your argument. This is why I stayed away for > > such a long time. But if you think a bit more about it, it has *very > > interesting* capabilities. Eg. I may also use it to gather some > > real-time view of internal state, emit commands like HUP and do a number > > of other nice things. Again, security is problematic. > > > > In any case, I'll give it a try. I have started to write a small plugin > > which will dump the error messages within 10 minutes after startup. It > > will not be a real http server but rather a listener that response to > > anyone who connects to it (so that nc can be used as a client ;)). This > > is very rough, but it has two side-effects: > > > > a) it provides something to actually experiment with (and to extend if > > we decide to keep that route) > > b) it helps create the plumbing inside the internal interface that could > > also be used by any other method, so the potential waste of time is > > limited > > It sounds interesting, and I agree that there's a lot of potential > there. Among them, the possibility of adding an XML-RPC API to allow > for greater interface flexibility...but frankly, that scares the hell > out of me. ;) I'll wait to see what you come up with. hehe... We'll see, but I'd say its faaar down the road. One interesting aspect, though, could be a realtime view of syslog data as it is flowing through the system. That could even be integrated into phpLogCon. In any case, I do NOT want to add much (http) application logic / presentation to such a system. But a few html (or xml ;)) pages generated at appropriate times could be really useful. Just think about things like a "granular HUP", like being able to tell to close some files specifically etc etc. But that's too far advanced. For me personally, it would be quite useful to get access to state variables (like mem utilization, object instances, error count, frame rate) of the running system - not under a debugger but in a real live system. From the developers (and troubleshooter's point of view), that would be very interesting. I could even hold the last 2000 or so messages of the debug log in memory, plus the initial first 1000. Together, I guess, they can solve over 90% of all things I have used debug logs for. But this can only be done if I have dynamic runtime access to the live instance. And this obviously also means there is a big security issue ;) Please keep the thoughts flowing, we make good progess :) Rainer > > -HKS > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Mon Jul 28 17:54:04 2008 From: hks.private at gmail.com ((private) HKS) Date: Mon, 28 Jul 2008 11:54:04 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> Message-ID: I don't have much more to add to this discussion - I think we're pretty much on the same page. I'm looking forward to seeing the results of your work. -HKS On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards wrote: > > > Please keep the thoughts flowing, we make good progess :) > Rainer From rory at ooma.com Tue Jul 29 01:11:52 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 16:11:52 -0700 Subject: [rsyslog] error with phplogcon Message-ID: <488E5238.9070606@ooma.com> The installation instructions say I need php, apache and mysql. When I run the installation for phplogcon, it finishes and then I get an error: Errordetails: Error, MYSQL Extensions are not enabled! Function 'mysql_connect' does not exist. There are several things I could install here... I installed the mysql-connector-odbc rpm, but this may or may not be the right thing. It still doesn't work, and docs on this are pretty scarce. So, what am I missing? Which rpm should I be using to get the mysql_connect function? From samuel at dragonboricua.net Tue Jul 29 02:06:53 2008 From: samuel at dragonboricua.net (Elisamuel Resto) Date: Mon, 28 Jul 2008 20:06:53 -0400 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E5238.9070606@ooma.com> References: <488E5238.9070606@ooma.com> Message-ID: <20080728200653.38ae2bed@hades.dragonboricua.com> On Mon, 28 Jul 2008 16:11:52 -0700, Rory Toma wrote: > The installation instructions say I need php, apache and mysql. > > When I run the installation for phplogcon, it finishes and then I get an > error: > > Errordetails: > Error, MYSQL Extensions are not enabled! Function 'mysql_connect' does > not exist. > > There are several things I could install here... I installed the > mysql-connector-odbc rpm, but this may or may not be the right thing. It > still doesn't work, and docs on this are pretty scarce. So, what am I > missing? Which rpm should I be using to get the mysql_connect function? I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, depending on your installed php version). ODBC connector is quite a different thing. -- Elisamuel Resto Source Mage General Guru / http://sourcemage.org GPG ID: 18615F19/1024D / http://simplysam.us From rory at ooma.com Tue Jul 29 02:48:52 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 17:48:52 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080728200653.38ae2bed@hades.dragonboricua.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> Message-ID: <488E68F4.20903@ooma.com> Yes, turns out after random google searching that I needed php-mysql. I am past this stage now. However... I now get at error "No syslog records found. - Error Details Unknown or unhandled error occurred" I can connect manually and do a select * from the SystemEvents db and see records, using the user/passwd combo. One thing I couldn't find... What is supposed to go into the field "Table type"? I put in "rsyslog" but I'm not sure what is supposed to go there. thx > > I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, > depending on your installed php version). ODBC connector is quite a different > thing. > > From mic at npgx.com.au Tue Jul 29 02:56:11 2008 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 29 Jul 2008 11:56:11 +1100 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E68F4.20903@ooma.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> Message-ID: <20080729005501.M56815@npgx.com.au> Hi, > Yes, turns out after random google searching that I needed php- > mysql. I am past this stage now. > > However... > > I now get at error > > "No syslog records found. - Error Details > Unknown or unhandled error occurred" I get this error all the time when I install phplogcon. It's just the case of the "systemevents" name in the config file, I change it to "SystemEvents" and it works. Maybe it's an idea to post your config to the list (just the last bit at the bottom) so we can see what you're doing. Regards, Michael. > I can connect manually and do a select * from the SystemEvents db > and see records, using the user/passwd combo. > > One thing I couldn't find... What is supposed to go into the field > "Table type"? I put in "rsyslog" but I'm not sure what is supposed > to go there. > > thx > > > > I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, > > depending on your installed php version). ODBC connector is quite a different > > thing. > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From rory at ooma.com Tue Jul 29 03:02:19 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 18:02:19 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080729005501.M56815@npgx.com.au> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> Message-ID: <488E6C1B.8000208@ooma.com> Here ya go, thx. $CFG['Sources']['Source1']['ID'] = 'Source1'; $CFG['Sources']['Source1']['Name'] = 'rsyslog'; $CFG['Sources']['Source1']['SourceType'] = 2; $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; $CFG['Sources']['Source1']['DBType'] = '0'; $CFG['Sources']['Source1']['DBServer'] = 'localhost'; $CFG['Sources']['Source1']['DBName'] = 'Syslog'; $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; From mic at npgx.com.au Tue Jul 29 03:58:11 2008 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 29 Jul 2008 12:58:11 +1100 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E6C1B.8000208@ooma.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> <488E6C1B.8000208@ooma.com> Message-ID: <20080729015636.M15076@npgx.com.au> Hi, > Here ya go, thx. > > $CFG['Sources']['Source1']['ID'] = 'Source1'; > $CFG['Sources']['Source1']['Name'] = 'rsyslog'; > $CFG['Sources']['Source1']['SourceType'] = 2; > $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; > $CFG['Sources']['Source1']['DBType'] = '0'; > $CFG['Sources']['Source1']['DBServer'] = 'localhost'; > $CFG['Sources']['Source1']['DBName'] = 'Syslog'; > $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; > $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; > $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; > $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; This is mine: $CFG['DefaultSourceID'] = 'Source1'; $CFG['Sources']['Source1']['ID'] = 'Source1'; $CFG['Sources']['Source1']['Name'] = 'somename'; $CFG['Sources']['Source1']['ViewID'] = 'SYSLOG'; $CFG['Sources']['Source1']['SourceType'] = SOURCE_DB; $CFG['Sources']['Source1']['DBTableType'] = 'monitorware'; $CFG['Sources']['Source1']['DBServer'] = 'somedbserver'; $CFG['Sources']['Source1']['DBName'] = 'somedbname'; $CFG['Sources']['Source1']['DBUser'] = 'someuser'; $CFG['Sources']['Source1']['DBPassword'] = 'longstringofcharacters'; $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; Regards, Michael. From rory at ooma.com Tue Jul 29 04:02:54 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 19:02:54 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080729015636.M15076@npgx.com.au> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> <488E6C1B.8000208@ooma.com> <20080729015636.M15076@npgx.com.au> Message-ID: <488E7A4E.3030103@ooma.com> Thx, that fixed it. I didn't iterate over each difference, but... thx again. Michael Mansour wrote: > Hi, > > >> Here ya go, thx. >> >> $CFG['Sources']['Source1']['ID'] = 'Source1'; >> $CFG['Sources']['Source1']['Name'] = 'rsyslog'; >> $CFG['Sources']['Source1']['SourceType'] = 2; >> $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; >> $CFG['Sources']['Source1']['DBType'] = '0'; >> $CFG['Sources']['Source1']['DBServer'] = 'localhost'; >> $CFG['Sources']['Source1']['DBName'] = 'Syslog'; >> $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; >> $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; >> $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; >> $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; >> > > This is mine: > > $CFG['DefaultSourceID'] = 'Source1'; > > $CFG['Sources']['Source1']['ID'] = 'Source1'; > $CFG['Sources']['Source1']['Name'] = 'somename'; > $CFG['Sources']['Source1']['ViewID'] = 'SYSLOG'; > $CFG['Sources']['Source1']['SourceType'] = SOURCE_DB; > $CFG['Sources']['Source1']['DBTableType'] = 'monitorware'; > $CFG['Sources']['Source1']['DBServer'] = 'somedbserver'; > $CFG['Sources']['Source1']['DBName'] = 'somedbname'; > $CFG['Sources']['Source1']['DBUser'] = 'someuser'; > $CFG['Sources']['Source1']['DBPassword'] = 'longstringofcharacters'; > $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; > $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; > > Regards, > > Michael. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Tue Jul 29 10:10:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 10:10:45 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> I have implemented much yesterday. It is available via git right now: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05b 76862ea2e211cc09b30fe I will create a testbed for it today and hopefully be able to release it some time this afternoon. Unfortunately, it's a busy day... Feedback is appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Monday, July 28, 2008 5:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > I don't have much more to add to this discussion - I think we're > pretty much on the same page. I'm looking forward to seeing the > results of your work. > > -HKS > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > wrote: > > > > > > Please keep the thoughts flowing, we make good progess :) > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 29 15:57:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 15:57:31 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> OK, I have now a preview tarball available. I'd appreciate if you could give it a try. Use -N1 on the command line to make it do a config check, only. It is available here: http://download.rsyslog.com/download/rsyslog-3.21.1.tar.gz md5sum: I will replace that file once the "official" 3.21.1 is out. I have one problem with autotools: a "make distcheck" fails, as far as I can see because rsyslogd is not built inside the tools subdirectory. If someone has a suggestion for a fix (or what could cause the problem), I would appreciate that. Actually this "make distcheck" problem is what made me hold back the official release. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 29, 2008 10:11 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > I have implemented much yesterday. It is available via git right now: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05 > b > 76862ea2e211cc09b30fe > > I will create a testbed for it today and hopefully be able to release > it > some time this afternoon. Unfortunately, it's a busy day... > > Feedback is appreciated. > Rainer > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of (private) HKS > > Sent: Monday, July 28, 2008 5:54 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog error messages > > > > I don't have much more to add to this discussion - I think we're > > pretty much on the same page. I'm looking forward to seeing the > > results of your work. > > > > -HKS > > > > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > > wrote: > > > > > > > > > Please keep the thoughts flowing, we make good progess :) > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 29 15:58:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 15:58:36 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEDA@grfint2.intern.adiscon.com> It would have been better if I had actually included the md5sum ;). It is b9ca041c00d093981e6c6e35d360e6e9 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 29, 2008 3:58 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > OK, I have now a preview tarball available. I'd appreciate if you could > give it a try. > > Use -N1 on the command line to make it do a config check, only. > > It is available here: > > http://download.rsyslog.com/download/rsyslog-3.21.1.tar.gz > md5sum: > > I will replace that file once the "official" 3.21.1 is out. > > I have one problem with autotools: a "make distcheck" fails, as far as > I > can see because rsyslogd is not built inside the tools subdirectory. If > someone has a suggestion for a fix (or what could cause the problem), I > would appreciate that. Actually this "make distcheck" problem is what > made me hold back the official release. > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Tuesday, July 29, 2008 10:11 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog error messages > > > > I have implemented much yesterday. It is available via git right now: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05 > > b > > 76862ea2e211cc09b30fe > > > > I will create a testbed for it today and hopefully be able to release > > it > > some time this afternoon. Unfortunately, it's a busy day... > > > > Feedback is appreciated. > > Rainer > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of (private) HKS > > > Sent: Monday, July 28, 2008 5:54 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog error messages > > > > > > I don't have much more to add to this discussion - I think we're > > > pretty much on the same page. I'm looking forward to seeing the > > > results of your work. > > > > > > -HKS > > > > > > > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > > > wrote: > > > > > > > > > > > > Please keep the thoughts flowing, we make good progess :) > > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Tue Jul 29 16:14:08 2008 From: hks.private at gmail.com ((private) HKS) Date: Tue, 29 Jul 2008 10:14:08 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> Message-ID: I'll test this as soon as I can, but it may be a couple days... -HKS On Tue, Jul 29, 2008 at 4:10 AM, Rainer Gerhards wrote: > I have implemented much yesterday. It is available via git right now: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05b > 76862ea2e211cc09b30fe > > I will create a testbed for it today and hopefully be able to release it > some time this afternoon. Unfortunately, it's a busy day... > > Feedback is appreciated. > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of (private) HKS >> Sent: Monday, July 28, 2008 5:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog error messages >> >> I don't have much more to add to this discussion - I think we're >> pretty much on the same page. I'm looking forward to seeing the >> results of your work. >> >> -HKS >> >> >> On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards >> wrote: >> > >> > >> > Please keep the thoughts flowing, we make good progess :) >> > Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mattjhell at gmail.com Wed Jul 30 00:28:43 2008 From: mattjhell at gmail.com (Matt Hellman) Date: Tue, 29 Jul 2008 17:28:43 -0500 Subject: [rsyslog] Thinking about syslog forwarding infrastructure Message-ID: In the interest of brevity, I'm leaving out details...if you need more, just ask. We have a security event monitoring system that processes probably in the neighborhood of 100 million syslog messages per day (I know precisely how many events it has processed, but it doesn't break them down by protocol). In some of our WAN sites, I would like to implement a local system that will receive all the local syslog messages and ship some/all back to the main collector on our LAN. The main collector on the LAN would receive the bulk of its message from other LAN systems. Then the main collect would ship some/all to the SEM environment. I'm leaning towards using rsyslog for this task and have a few questions: 1) What kind of system [rough estimate] would I need for the main collector if assume 200 million syslog messages per day and peak that is triple that average rate (~7000 eps)? 2) Can I enable rate limiting in a way that will: A1) start dropping messages beyond a given threshold A2) start intelligently dropping messages beyond a given threshold (i.e. start dropping events matching this regex) B) allow me to alert someone that this is occurring (is written to log file, etc) From rory at ooma.com Wed Jul 30 03:14:52 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 18:14:52 -0700 Subject: [rsyslog] mysql inserts and message queues Message-ID: <488FC08C.4080201@ooma.com> I have enabled mysql and disk message queues as per below: $ModLoad immark.so # provides --MARK-- message capability $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # kernel logging (formerly provided by rklogd) $ModLoad ommysql.so $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port 514 $WorkDirectory /var/rsyslog $ActionQueueType LinkedList $ActionQueueFileName dbq $ActionResumeRetryCount -1 *.* :ommysql:127.0.0.1,Syslog,syslogwriter,writeme # *.* /var/log/messages The question I have is... does the disk queue drain w/o me having to intervene? I currently have 323 queue files (I was adding some keys and playing with tuning mysql... kind of slowed it down) and the number of queue files is not going down. thx From rgerhards at hq.adiscon.com Wed Jul 30 07:45:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 07:45:38 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <488FC08C.4080201@ooma.com> References: <488FC08C.4080201@ooma.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> I don't see any obvious error and, yes, the queue files should be deleted. Actually, you should only see any files at all if the inserts take too long. What you have may be an artifact of a previous failure. To get down to what it is, we need to check how things progress. Is your database populated? In any case, a debug log (rsyslogd with -dn additional options run interactively) will tell us what is there. Also, there should be a .qi file. Let us know its contents. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rory Toma > Sent: Wednesday, July 30, 2008 3:15 AM > To: rsyslog-users > Subject: [rsyslog] mysql inserts and message queues > > I have enabled mysql and disk message queues as per below: > > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging (e.g. > via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > $ModLoad ommysql.so > $ModLoad imudp.so # provides UDP syslog reception > $UDPServerRun 514 # start a UDP syslog server at standard port 514 > > $WorkDirectory /var/rsyslog > > $ActionQueueType LinkedList > $ActionQueueFileName dbq > $ActionResumeRetryCount -1 > > *.* :ommysql:127.0.0.1,Syslog,syslogwriter,writeme > # *.* /var/log/messages > > > The question I have is... does the disk queue drain w/o me having to > intervene? I currently have 323 queue files (I was adding some keys and > playing with tuning mysql... kind of slowed it down) and the number of > queue files is not going down. > > thx > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rory at ooma.com Wed Jul 30 07:58:32 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 22:58:32 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> Message-ID: <48900308.3070805@ooma.com> Rainer Gerhards wrote: > I don't see any obvious error and, yes, the queue files should be > deleted. Actually, you should only see any files at all if the inserts > take too long. What you have may be an artifact of a previous failure. > > To get down to what it is, we need to check how things progress. Is your > database populated? In any case, a debug log (rsyslogd with -dn > additional options run interactively) will tell us what is there. Also, > there should be a .qi file. Let us know its contents. > > Thanks, > Rainer > > the db is getting populated and there is no .qi file. If I run in with "-dn" there is a ton of output, so I need to know what to look for. I've been running for about a day, and I'm at 20+ million records. I do get an error -2040 when accessing on-disk files. Should I just nuke these? From rory at ooma.com Wed Jul 30 08:07:59 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 23:07:59 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <48900308.3070805@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> Message-ID: <4890053F.40606@ooma.com> As a side note, I stopped mysql, saw the number of queue files increase, restarted mysql and they went back down to the original 300 or so. So, maybe I just have some horked queue files. Rory Toma wrote: > Rainer Gerhards wrote: > >> I don't see any obvious error and, yes, the queue files should be >> deleted. Actually, you should only see any files at all if the inserts >> take too long. What you have may be an artifact of a previous failure. >> >> To get down to what it is, we need to check how things progress. Is your >> database populated? In any case, a debug log (rsyslogd with -dn >> additional options run interactively) will tell us what is there. Also, >> there should be a .qi file. Let us know its contents. >> >> Thanks, >> Rainer >> >> >> > the db is getting populated and there is no .qi file. If I run in with > "-dn" there is a ton of output, so I need to know what to look for. I've > been running for about a day, and I'm at 20+ million records. > > I do get an error -2040 when accessing on-disk files. Should I just nuke > these? > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Wed Jul 30 08:30:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 08:30:28 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <48900308.3070805@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> Message-ID: <1217399428.7184.260.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-29 at 22:58 -0700, Rory Toma wrote: > Rainer Gerhards wrote: > > I don't see any obvious error and, yes, the queue files should be > > deleted. Actually, you should only see any files at all if the inserts > > take too long. What you have may be an artifact of a previous failure. > > > > To get down to what it is, we need to check how things progress. Is your > > database populated? In any case, a debug log (rsyslogd with -dn > > additional options run interactively) will tell us what is there. Also, > > there should be a .qi file. Let us know its contents. > > > > Thanks, > > Rainer > > > > > the db is getting populated and there is no .qi file. No .qi file is a good indication (though not sufficient) that the queue files are artifacts of a previous failure. > If I run in with > "-dn" there is a ton of output, so I need to know what to look for. I've > been running for about a day, and I'm at 20+ million records. I usually know only when I see. Based on this case, I'd be interested in the first 2000 lines and another 1000 lines while it is processing records. That should at least provide enough of a clue to ask for something more specific. > > I do get an error -2040 when accessing on-disk files. Should I just nuke > these? This can be OK. I think you see the -2040 (file not found) when it is looking for the .qi files. Rainer > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 30 08:31:08 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 08:31:08 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <4890053F.40606@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> <4890053F.40606@ooma.com> Message-ID: <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-29 at 23:07 -0700, Rory Toma wrote: > As a side note, I stopped mysql, saw the number of queue files increase, > restarted mysql and they went back down to the original 300 or so. > > So, maybe I just have some horked queue files. that's now even more probable. But let's verify with the debug log. Rainer > > > Rory Toma wrote: > > Rainer Gerhards wrote: > > > >> I don't see any obvious error and, yes, the queue files should be > >> deleted. Actually, you should only see any files at all if the inserts > >> take too long. What you have may be an artifact of a previous failure. > >> > >> To get down to what it is, we need to check how things progress. Is your > >> database populated? In any case, a debug log (rsyslogd with -dn > >> additional options run interactively) will tell us what is there. Also, > >> there should be a .qi file. Let us know its contents. > >> > >> Thanks, > >> Rainer > >> > >> > >> > > the db is getting populated and there is no .qi file. If I run in with > > "-dn" there is a ton of output, so I need to know what to look for. I've > > been running for about a day, and I'm at 20+ million records. > > > > I do get an error -2040 when accessing on-disk files. Should I just nuke > > these? > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From friedl at hq.adiscon.com Wed Jul 30 17:07:22 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 30 Jul 2008 17:07:22 +0200 Subject: [rsyslog] rsyslog 3.21.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEE8@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.21.1, a development branch version. It contains some bug fixes and support for improved config file error checking. Most importantly, rsyslogd now has a "config file verification option", which makes it verify the config file but not start up or interfere with a running configuration. That should make troubleshooting config problems much easier. The version now also writes all internally-generated messages to stderr (if not disabled). Thanks to HKS for suggesting the new features. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-123.phtml Changelog: http://www.rsyslog.com/Article262.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Wed Jul 30 17:13:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 17:13:35 +0200 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: References: Message-ID: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Hi Matt, On Tue, 2008-07-29 at 17:28 -0500, Matt Hellman wrote: > In the interest of brevity, I'm leaving out details...if you need > more, just ask. > > We have a security event monitoring system that processes probably in > the neighborhood of 100 million syslog messages per day (I know > precisely how many events it has processed, but it doesn't break them > down by protocol). In some of our WAN sites, I would like to > implement a local system that will receive all the local syslog > messages and ship some/all back to the main collector on our LAN. The > main collector on the LAN would receive the bulk of its message from > other LAN systems. Then the main collect would ship some/all to the > SEM environment. I'm leaning towards using rsyslog for this task and > have a few questions: > > 1) What kind of system [rough estimate] would I need for the main > collector if assume 200 million syslog messages per day and peak that > is triple that average rate (~7000 eps)? Quite honestly: I don't know. Which rules you carry out has a big effect. But I have no real good big deployment numbers. The old game: everyone is interested in them, no-one conveys them (hint: let me know if you have some ;)). > > 2) Can I enable rate limiting in a way that will: > A1) start dropping messages beyond a given threshold you can do that > > A2) start intelligently dropping messages beyond a given threshold > (i.e. start dropping events matching this regex) not yet, but an interesting idea > > B) allow me to alert someone that this is occurring (is written to > log file, etc) mmhhh... not really. That's another interesting idea, and it should be simple to enable. It conveys that to the debug log, but does not emit a user message. In any case, I think there are a couple of docs you need to read and *understand* for this scale of deployment. Ask if you do not understand them - I have written them and may have left too much out just out of habit ;) First and foremost, you must understand rsyslog queues. They handle all queueing and rate limiting. The relevant doc is: http://www.rsyslog.com/doc-queues.html Then, I suggest to have a quick look at some use cases. I probably is a good idea to read the queue doc once, go over the use cases and the re-read the queue doc with the use cases on your mind: http://www.rsyslog.com/doc-queues.html http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html http://wiki.rsyslog.com/index.php/OffPeakHours Finally, IMHO you need to think about syslog reliability and the service level you can expect. This is not only true for rsyslog, but the other folks don't tell you about the reliability issues. Read this: http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html I hope this gets you started. Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 30 17:15:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 17:15:32 +0200 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> References: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEE9@grfint2.intern.adiscon.com> I was too quick. A small yet important thing to add: > > A2) start intelligently dropping messages beyond a given > threshold > > (i.e. start dropping events matching this regex) > > not yet, but an interesting idea Well, it is semi-intelligent. Messages are discarded based on severity level. But you can not use any other property. The details are in the queue doc. Rainer From mattjhell at gmail.com Wed Jul 30 19:33:14 2008 From: mattjhell at gmail.com (Matt Hellman) Date: Wed, 30 Jul 2008 12:33:14 -0500 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> References: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Message-ID: Thanks for the reply Rainer. >> 1) What kind of system [rough estimate] would I need for the main >> collector if assume 200 million syslog messages per day and peak that >> is triple that average rate (~7000 eps)? > > Quite honestly: I don't know. Which rules you carry out has a big > effect. But I have no real good big deployment numbers. The old game: > everyone is interested in them, no-one conveys them (hint: let me know > if you have some ;)). crap. well, I can probably test this easily enough myself. I just feel better knowing that someone has already done it. >> A2) start intelligently dropping messages beyond a given threshold >> (i.e. start dropping events matching this regex) > > not yet, but an interesting idea well, regex wouldn't be the only "intelligent" way to drop messages. I suppose anything that isn't arbitrary might be considered intelligent. Currently this is done based on priority, which won't work well for us because we use a product (Snare) that converts windows events into syslog that all have the same priority. FWIW, this is a common way for SEM products to collect Windows events. >> B) allow me to alert someone that this is occurring (is written to >> log file, etc) > > mmhhh... not really. That's another interesting idea, and it should be > simple to enable. It conveys that to the debug log, but does not emit a > user message. I was thinking about this and I don't necessarily need the product to emit something directly to a user, if that's what you mean. I plan to buffer to disk. Can I create a process to monitor the queue files or something --warning: I have printed but at best skimmed many of the docs you reference;-) > In any case, I think there are a couple of docs you need to read and > *understand* for this scale of deployment. Ask if you do not understand > them - I have written them and may have left too much out just out of > habit ;) re: doc links. Thanks. I was being lazy and trying to avoid having to read them prematurely;-) I think I'm too the point where I believe rsyslog can theoretically deliver on my requirements though. It's time to dig in. From rory at ooma.com Wed Jul 30 20:30:43 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 11:30:43 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> <4890053F.40606@ooma.com> <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> Message-ID: <4890B353.6000607@ooma.com> Rainer Gerhards wrote: > On Tue, 2008-07-29 at 23:07 -0700, Rory Toma wrote: > >> As a side note, I stopped mysql, saw the number of queue files increase, >> restarted mysql and they went back down to the original 300 or so. >> >> So, maybe I just have some horked queue files. >> > > that's now even more probable. But let's verify with the debug log. > > OK, here's the log file, about 20-30 seconds of runtime... http://www.colinburns.com/rsyslog/rsyslog.txt.gz thx From julianokyap at gmail.com Thu Jul 31 00:18:09 2008 From: julianokyap at gmail.com (Julian Yap) Date: Wed, 30 Jul 2008 12:18:09 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: Is there a way to set an Alert when multiple repeated lines are found in a log? I want to spawn an email Alert if a message is received 3 times. Example log lines: Jul 30 04:19:29 localhost program: Error detected Jul 30 05:19:29 localhost program: Error detected Jul 30 06:19:29 localhost program: Error detected Thanks, Julian From rory at ooma.com Thu Jul 31 02:15:41 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 17:15:41 -0700 Subject: [rsyslog] tips for managing data Message-ID: <4891042D.9040805@ooma.com> So, my current mysql rsyslog drops about 20 million rows of data per day. Over time, this gets slow as tables grow. I'm not a dba, so I was wondering if anyone had some suggestions for keeping performance still on the order of seconds, and not minutes or hours. thx I did add a key for EventSource, as that is commonly searched. However, using PhpLogCon, it seems that if I search using the web interface (i.e. I click on a host entry and hit the available searches) it is relatively quick. However, changing the text field that is generated and hitting the "search" button is slow. Do these two methods use the same query, or is something else going on? thx From rory at ooma.com Thu Jul 31 04:09:52 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 19:09:52 -0700 Subject: [rsyslog] tips for managing data In-Reply-To: <4891042D.9040805@ooma.com> References: <4891042D.9040805@ooma.com> Message-ID: <48911EF0.1010004@ooma.com> OK, so it seems that doing a query from the query line does a LIKE, which can take significantly longer (sample query 8 seconds vs. 50 msecs...) So, replacing the LIKE % in logstreamdb.class.db with an = speeds things up quite a but, but I lose some flexibility. Is there some kind of search syntax where I can differentiate between LIKE and =? If not, I'm thinking something like: source:foo.bar.com # would be using = ~source:foo # would be using LIKE Rory Toma wrote: > So, my current mysql rsyslog drops about 20 million rows of data per day. > > Over time, this gets slow as tables grow. > > I'm not a dba, so I was wondering if anyone had some suggestions for > keeping performance still on the order of seconds, and not minutes or hours. > > thx > > I did add a key for EventSource, as that is commonly searched. However, > using PhpLogCon, it seems that if I search using the web interface (i.e. > I click on a host entry and hit the available searches) it is relatively > quick. However, changing the text field that is generated and hitting > the "search" button is slow. Do these two methods use the same query, or > is something else going on? > > thx > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From alorbach at ro1.adiscon.com Thu Jul 31 10:15:23 2008 From: alorbach at ro1.adiscon.com (Andre Lorbach) Date: Thu, 31 Jul 2008 10:15:23 +0200 Subject: [rsyslog] tips for managing data In-Reply-To: <48911EF0.1010004@ooma.com> References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com> Message-ID: Hi, the like query can indeed have quiet an impact on performance when doing queries on large databases. But I think we can expand the syntax, so you can either search by part of a string (LIKE '%search%') or the whole string (= 'search'). This should be rather easy to implement. I will put this on my todolist, if it is as easy as I think, the next minor update of the devel branch will contain this new feature. Best regards, Andre Lorbach > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rory Toma > Sent: Thursday, July 31, 2008 4:10 AM > To: rsyslog-users > Subject: Re: [rsyslog] tips for managing data > > OK, so it seems that doing a query from the query line does a LIKE, > which can take significantly longer (sample query 8 seconds vs. 50 msecs...) > > So, replacing the LIKE % in logstreamdb.class.db with an = speeds things > up quite a but, but I lose some flexibility. Is there some kind of > search syntax where I can differentiate between LIKE and =? > > If not, I'm thinking something like: > > source:foo.bar.com # would be using = > > ~source:foo # would be using LIKE > > > > Rory Toma wrote: > > So, my current mysql rsyslog drops about 20 million rows of data per day. > > > > Over time, this gets slow as tables grow. > > > > I'm not a dba, so I was wondering if anyone had some suggestions for > > keeping performance still on the order of seconds, and not minutes or hours. > > > > thx > > > > I did add a key for EventSource, as that is commonly searched. However, > > using PhpLogCon, it seems that if I search using the web interface (i.e. > > I click on a host entry and hit the available searches) it is relatively > > quick. However, changing the text field that is generated and hitting > > the "search" button is slow. Do these two methods use the same query, or > > is something else going on? > > > > thx > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From ml at darville.vm.bytemark.co.uk Thu Jul 31 13:11:22 2008 From: ml at darville.vm.bytemark.co.uk (David Darville) Date: Thu, 31 Jul 2008 12:11:22 +0100 Subject: [rsyslog] Changing hostname field Message-ID: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Hello everyone I am trying to configure rsyslog to service a number of chroot jails in addition to the host itself. But I need to change the hostname field of the syslog messages from the different jails, so that I place them in the right log file on the central logging host. My current rsyslog.conf is as follows: $ModLoad imuxsock $ModLoad imklog $ModLoad immark $ModLoad omrelp $AddUnixListenSocket /jail/1/dev/log $AddUnixListenSocket /jail/2/dev/log *.* :omrelp:10.0.0.4:2514 Can anyone please advice me on how to do that? --- David Darville From hks.private at gmail.com Thu Jul 31 15:48:37 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 31 Jul 2008 09:48:37 -0400 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: References: Message-ID: Not in rsyslogd itself, but you could do this with Swatch, Nagios, or some other monitoring-type software. -HKS On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: > Is there a way to set an Alert when multiple repeated lines are found in a log? > > I want to spawn an email Alert if a message is received 3 times. > > Example log lines: > Jul 30 04:19:29 localhost program: Error detected > Jul 30 05:19:29 localhost program: Error detected > Jul 30 06:19:29 localhost program: Error detected > > Thanks, > Julian > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From hks.private at gmail.com Thu Jul 31 16:00:09 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 31 Jul 2008 10:00:09 -0400 Subject: [rsyslog] Changing hostname field In-Reply-To: <20080731111122.GA24000@darville.vm.bytemark.co.uk> References: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Message-ID: Do the jails all share the same hostname and IP? If not, you should be able to use the %hostname% or %fromhost% properties. If so, are they each running their own instance of (r)syslogd? -HKS On Thu, Jul 31, 2008 at 7:11 AM, David Darville wrote: > Hello everyone > > I am trying to configure rsyslog to service a number of chroot jails in > addition to the host itself. > > But I need to change the hostname field of the syslog messages from the > different jails, so that I place them in the right log file on the central > logging host. > > My current rsyslog.conf is as follows: > > $ModLoad imuxsock > $ModLoad imklog > $ModLoad immark > $ModLoad omrelp > > $AddUnixListenSocket /jail/1/dev/log > $AddUnixListenSocket /jail/2/dev/log > > *.* :omrelp:10.0.0.4:2514 > > > Can anyone please advice me on how to do that? > > > --- > > David Darville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rory at ooma.com Thu Jul 31 16:32:11 2008 From: rory at ooma.com (Rory Toma) Date: Thu, 31 Jul 2008 07:32:11 -0700 Subject: [rsyslog] tips for managing data In-Reply-To: References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com> Message-ID: <4891CCEB.50905@ooma.com> Cool. For me, it seems that using LIKE is most useful when searching the message text. So, something like: source:foo ~bar would produce where fromhost = 'foo' and message LIKE '%bar%' thx Andre Lorbach wrote: > Hi, > > the like query can indeed have quiet an impact on performance when doing > queries on large databases. > But I think we can expand the syntax, so you can either search by part > of a string (LIKE '%search%') or the whole string (= 'search'). This > should be rather easy to implement. I will put this on my todolist, if > it is as easy as I think, the next minor update of the devel branch will > contain this new feature. > > Best regards, > Andre Lorbach > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rory Toma >> Sent: Thursday, July 31, 2008 4:10 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] tips for managing data >> >> OK, so it seems that doing a query from the query line does a LIKE, >> which can take significantly longer (sample query 8 seconds vs. 50 >> > msecs...) > >> So, replacing the LIKE % in logstreamdb.class.db with an = speeds >> > things > >> up quite a but, but I lose some flexibility. Is there some kind of >> search syntax where I can differentiate between LIKE and =? >> >> If not, I'm thinking something like: >> >> source:foo.bar.com # would be using = >> >> ~source:foo # would be using LIKE >> >> >> >> Rory Toma wrote: >> >>> So, my current mysql rsyslog drops about 20 million rows of data per >>> > day. > >>> Over time, this gets slow as tables grow. >>> >>> I'm not a dba, so I was wondering if anyone had some suggestions for >>> keeping performance still on the order of seconds, and not minutes >>> > or hours. > >>> thx >>> >>> I did add a key for EventSource, as that is commonly searched. >>> > However, > >>> using PhpLogCon, it seems that if I search using the web interface >>> > (i.e. > >>> I click on a host entry and hit the available searches) it is >>> > relatively > >>> quick. However, changing the text field that is generated and >>> > hitting > >>> the "search" button is slow. Do these two methods use the same >>> > query, or > >>> is something else going on? >>> >>> thx >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From ml at darville.vm.bytemark.co.uk Thu Jul 31 16:46:49 2008 From: ml at darville.vm.bytemark.co.uk (David Darville) Date: Thu, 31 Jul 2008 15:46:49 +0100 Subject: [rsyslog] Changing hostname field In-Reply-To: References: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Message-ID: <20080731144649.GA24505@darville.vm.bytemark.co.uk> The jails all have their own unique hostname (and IP), but all share an rsyslogd instance running on the main host, and the %hostname% and %fromhost% in all the log messages from the jails are set to the hostname of the main host. And that is what I want to change. On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote: > Do the jails all share the same hostname and IP? If not, you should be > able to use the %hostname% or %fromhost% properties. > > If so, are they each running their own instance of (r)syslogd? > > -HKS > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville > wrote: > > Hello everyone > > > > I am trying to configure rsyslog to service a number of chroot jails in > > addition to the host itself. > > > > But I need to change the hostname field of the syslog messages from the > > different jails, so that I place them in the right log file on the central > > logging host. > > > > My current rsyslog.conf is as follows: > > > > $ModLoad imuxsock > > $ModLoad imklog > > $ModLoad immark > > $ModLoad omrelp > > > > $AddUnixListenSocket /jail/1/dev/log > > $AddUnixListenSocket /jail/2/dev/log > > > > *.* :omrelp:10.0.0.4:2514 > > > > > > Can anyone please advice me on how to do that? > > > > > > --- > > > > David Darville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 31 17:04:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 17:04:18 +0200 Subject: [rsyslog] Changing hostname field Message-ID: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com> Use a template with fixed name. --- Urspr?ngliche Nachricht --- Von: "David Darville" Betreff: Re: [rsyslog] Changing hostname field Datum: 31. Juli 2008 Uhrzeit: 16:46:59 The jails all have their own unique hostname (and IP), but all share an rsyslogd instance running on the main host, and the %hostname% and %fromhost% in all the log messages from the jails are set to the hostname of the main host. And that is what I want to change. On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote: > Do the jails all share the same hostname and IP? If not, you should be > able to use the %hostname% or %fromhost% properties. > > If so, are they each running their own instance of (r)syslogd? > > -HKS > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville > wrote: > > Hello everyone > > > > I am trying to configure rsyslog to service a number of chroot jails in > > addition to the host itself. > > > > But I need to change the hostname field of the syslog messages from the > > different jails, so that I place them in the right log file on the central > > logging host. > > > > My current rsyslog.conf is as follows: > > > > $ModLoad imuxsock > > $ModLoad imklog > > $ModLoad immark > > $ModLoad omrelp > > > > $AddUnixListenSocket /jail/1/dev/log > > $AddUnixListenSocket /jail/2/dev/log > > > > *.* :omrelp:10.0.0.4:2514 > > > > > > Can anyone please advice me on how to do that? > > > > > > --- > > > > David Darville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From julianokyap at gmail.com Thu Jul 31 20:27:14 2008 From: julianokyap at gmail.com (Julian Yap) Date: Thu, 31 Jul 2008 08:27:14 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: References: Message-ID: Hmm, Nagios is a pain to set up. Looking for something more light weight... Was hoping that I could have consolidated lots of Alerts under Rsyslog. Any other suggestions besides Swatch? On 7/31/08, (private) HKS wrote: > Not in rsyslogd itself, but you could do this with Swatch, Nagios, or > some other monitoring-type software. > > -HKS > > On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >> Is there a way to set an Alert when multiple repeated lines are found in a >> log? >> >> I want to spawn an email Alert if a message is received 3 times. >> >> Example log lines: >> Jul 30 04:19:29 localhost program: Error detected >> Jul 30 05:19:29 localhost program: Error detected >> Jul 30 06:19:29 localhost program: Error detected >> >> Thanks, >> Julian >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 20:37:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 20:37:05 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> What exactly do you need to do except the "three in a row" alert? ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Gesendet: 31.07.08 20:27 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found Hmm, Nagios is a pain to set up. Looking for something more light weight... Was hoping that I could have consolidated lots of Alerts under Rsyslog. Any other suggestions besides Swatch? On 7/31/08, (private) HKS wrote: > Not in rsyslogd itself, but you could do this with Swatch, Nagios, or > some other monitoring-type software. > > -HKS > > On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >> Is there a way to set an Alert when multiple repeated lines are found in a >> log? >> >> I want to spawn an email Alert if a message is received 3 times. >> >> Example log lines: >> Jul 30 04:19:29 localhost program: Error detected >> Jul 30 05:19:29 localhost program: Error detected >> Jul 30 06:19:29 localhost program: Error detected >> >> Thanks, >> Julian >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From julianokyap at gmail.com Thu Jul 31 21:59:51 2008 From: julianokyap at gmail.com (Julian Yap) Date: Thu, 31 Jul 2008 09:59:51 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> References: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> Message-ID: That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 22:38:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 22:38:38 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com> To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence: 1. a - a - a - b 2. a - a - b - a Result: in case 1 an alert is triggered, in case 2 not. Is this understanding correct? rainer ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Cc: "rgerhards at hq.adiscon.com" ; "hks.private at gmail.com" Gesendet: 31.07.08 21:59 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 23:19:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 23:19:35 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001601c8f353$363e638d$060013ac@intern.adiscon.com> Oh, and one thing i forgot: what makes an event identical? Same message except timestamp - or what (eg same host, same tag, ...) rainer ----- Urspr?ngliche Nachricht ----- Von: "Rainer Gerhards" An: "Julian Yap" Cc: "rsyslog at lists.adiscon.com" Gesendet: 31.07.08 22:39 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence: 1. a - a - a - b 2. a - a - b - a Result: in case 1 an alert is triggered, in case 2 not. Is this understanding correct? rainer ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Cc: "rgerhards at hq.adiscon.com" ; "hks.private at gmail.com" Gesendet: 31.07.08 21:59 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 1 15:36:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 1 Jul 2008 15:36:11 +0200 Subject: [rsyslog] rsyslog 3.19.8 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092F2@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.8, a new release of the development branch. Most importantly, it adds a bugfix for TLS mode. A TLS session was lost if the OS returned a retry request via the API. This happened frequently on some platforms. The situation is now properly handled. Feature-wise error number have been added to most rsyslog messages, making it easier to troubleshoot problems. There are also some other minor improvements and bug fixes. This is a recommended update for all devel branch users. Change Log: http://www.rsyslog.com/Article247.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-116.phtml I hope this release is useful. Feedback is appreciated. Rainer Gerhards From mycleanjunk at gmail.com Wed Jul 2 05:16:41 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Tue, 1 Jul 2008 20:16:41 -0700 Subject: [rsyslog] configuration file question and error message question Message-ID: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> Hi, I am using rsyslog 3.16.2. When I specify the script of command to issue when a log reaches its max size, how do I specify the command with parameters? This is what I did: $outchannel ch_common,/var/log/messages/1048675,\ mv -f /var/log/messages /var/log/messages.0 Do I use a quotation mark or a single quote or something else? Your help says to specify a script but does that line except a single word for the script, meaning it can not take parameters? To clarify, if I have a script that is named rotate and I want to use it for multiple logs, I would like to say "rotate /var/log/message". Is this possible? I am getting this error message: rsyslogd: omfwd.c:318: doTryResume: Assertion `0' failed. What does this mean? Thanks for your help!! I really like how rsyslogd has a lot of configurable properties. Scott From rgerhards at hq.adiscon.com Wed Jul 2 08:28:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 2 Jul 2008 08:28:59 +0200 Subject: [rsyslog] configuration file question and error message question In-Reply-To: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> References: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092F7@grfint2.intern.adiscon.com> Hi, > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Wednesday, July 02, 2008 5:17 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] configuration file question and error message > question > > Hi, > > I am using rsyslog 3.16.2. > > When I specify the script of command to issue when a log reaches its > max size, how do I specify the command with parameters? The command can have EXACTLY one parameter. > This is what I > did: > > $outchannel ch_common,/var/log/messages/1048675,\ > mv -f /var/log/messages /var/log/messages.0 > > Do I use a quotation mark or a single quote or something else? Your > help says to specify a script but does that line except a single word > for the script, meaning it can not take parameters? To clarify, if I > have a script that is named rotate and I want to use it for multiple > logs, I would like to say "rotate /var/log/message". Is this possible? Yes, but you can not specify more than one parameter. The outchannel code is scheduled to be superseded by something new that works along the lines of the regular log file (just with an additional parameter). However, until this is done, we need to live with the current restriction. > > > I am getting this error message: > rsyslogd: omfwd.c:318: doTryResume: Assertion `0' failed. > > What does this mean? Something goes really wrong ;) I have checked that assertion and there seems to be some problem with the retry log. Can you reproduce it? If so, can you provide a full debug log (run interactively with -dn option)? > > Thanks for your help!! I really like how rsyslogd has a lot of > configurable properties. :) Rainer > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Wed Jul 2 10:15:07 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 01:15:07 -0700 Subject: [rsyslog] rsyslog threads questions Message-ID: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Hi, I have 3.16.2 which was recently released. I see that under certain conditions rsyslogd spawns a lot of threads: 5949 root 11216 S rsyslogd 5950 root 11216 S rsyslogd 5951 root 11216 S rsyslogd 5952 root 11216 S rsyslogd 5953 root 11216 S rsyslogd 5954 root 11216 S rsyslogd 5985 root Z [rsyslogd] 6445 root Z [rsyslogd] I had to kill the rsyslogd and restart it. The first invocation had a pid of 219 before it had to be killed. The second invocation of pid which you see above starts with 5949. The difference is the amount of zombie threads that were invoked by rsyslogd before I had to kill the first invocation of it. The question is under what conditions does rsyslogd spawn a new thread/process and why was it a zombie? I am running rsyslogd in an embedded environment and not a regular laptop/desktop. In addition, I am using busybox and I believe the syslog buffer size is set to something very low or perhaps none at all. Would this be a factor? Furthermore, I ran rsyslogd with -c3 and also without -c3 and both cases happen. Are these issues already known and fixed in a later version? Sorry, if I am asking the same questions or have the same issues as previous people but without the ability to search (or at least, I don't know how to) the archive, I don't know if my problem/questions has already been seen and/or resolved. Thank you very much for your support. Scott From rgerhards at hq.adiscon.com Wed Jul 2 10:27:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 02 Jul 2008 10:27:18 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Message-ID: <1214987239.7184.13.camel@rgf9dev.intern.adiscon.com> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > Hi, > > I have 3.16.2 which was recently released. I see that under certain > conditions rsyslogd spawns a lot of threads: > 5949 root 11216 S rsyslogd > 5950 root 11216 S rsyslogd > 5951 root 11216 S rsyslogd > 5952 root 11216 S rsyslogd > 5953 root 11216 S rsyslogd > 5954 root 11216 S rsyslogd > 5985 root Z [rsyslogd] > 6445 root Z [rsyslogd] > > I had to kill the rsyslogd and restart it. The first invocation had a > pid of 219 before it had to be killed. The second invocation of pid > which you see above starts with 5949. The difference is the amount of > zombie threads that were invoked by rsyslogd before I had to kill the > first invocation of it. I have no explanation yet for the zombies. They should not happen and so far I have never seen them. We may need to go through a debug log (which will become very large) to find out what's going on. > The question is under what conditions does rsyslogd spawn a new > thread/process and why was it a zombie? Unfortunately, there is no quick answer. A quick one may be: when it needs them, based on queue watermark settings and based on you configuration. But to really understand it, you need to read this doc: http://www.rsyslog.com/doc-queues.html The doc also describes all the knobs that you can use to control thread creation. There are many ;) > I am running rsyslogd in an > embedded environment and not a regular laptop/desktop. Interesting use case... > In addition, I > am using busybox and I believe the syslog buffer size is set to what do yo mean by "syslog buffer size"? The length of a receive buffer? It is 2K, thus single messages up to 2K are supported. It can be changed by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) support only up to 1K. > something very low or perhaps none at all. Would this be a factor? > Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > cases happen. The compatibility modes do not affect queue operation. > Are these issues already known and fixed in a later version? Sorry, if > I am asking the same questions or have the same issues as previous > people but without the ability to search (or at least, I don't know > how to) the archive, I don't know if my problem/questions has already > been seen and/or resolved. If we need to find out about the zombies, we need to move on to the current devel version. So I would give that a try in any case. 3.16.2 will (most probably) be replaced by 3.18.0 (based on the current beta) next week. So I won't touch it any longer. Looking forward to your feedback, Rainer > > Thank you very much for your support. > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 2 11:07:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 2 Jul 2008 11:07:28 +0200 Subject: [rsyslog] rsyslog error message repository Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092FA@grfint2.intern.adiscon.com> Hi all, starting with 3.19.8, rsyslog finally offers specific error codes as part of the syslog tag. For example, rsyslogd-2040 means that a file could not be found. I have added these tags both to facilitate log parsing as well as easy troubleshooting. But a tag is only as good as the information that it helps to find. Consequently, I have started to describe error cases inside the knowledge base's event repository: http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog.html So far, there is only a limited set of messages available (to phrase it politely ;)), but I plan to increase it over time. Note that there is an interactive feature where questions to the message can directly be posted to the forum. I hope this is useful. If you run into an error message that is not-yet described, let us know and we'll add an entry. In the long term, the new knowledge base part should be able to solve most problems. Feedback is appreciated, Rainer From mycleanjunk at gmail.com Wed Jul 2 21:04:23 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 12:04:23 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Message-ID: <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> Hi Rainer, Thanks for your reply. Looking at the default settings (from the online help's configuration page), they are what I wanted. The main messages queue is set to fix sized array with 1 worker thread created at maximum and action queues are direct mode which according to the queue document page, means that there will not be a worker thread created. Is my understanding correct? If yes, how do I quickly check without using the -d option if the defaults are set correctly? Or what do I look for in the debug messages that gets printed out to ensure this? You also mentioned that version 3.18.0 is probably going to be released as the stable version next week. I see on the webpage there is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? Also, how come I did not get your reply in my email inbox? My account settings look correct. Thanks, Scott Phuong As for the syslog buffer size, that applies to syslogd and does not apply to rsyslog. My configuration files do not change the Action queue or Worker queue parameters at all. Looking at On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > Hi, > > I have 3.16.2 which was recently released. I see that under certain > conditions rsyslogd spawns a lot of threads: > 5949 root 11216 S rsyslogd > 5950 root 11216 S rsyslogd > 5951 root 11216 S rsyslogd > 5952 root 11216 S rsyslogd > 5953 root 11216 S rsyslogd > 5954 root 11216 S rsyslogd > 5985 root Z [rsyslogd] > 6445 root Z [rsyslogd] > > I had to kill the rsyslogd and restart it. The first invocation had a > pid of 219 before it had to be killed. The second invocation of pid > which you see above starts with 5949. The difference is the amount of > zombie threads that were invoked by rsyslogd before I had to kill the > first invocation of it. I have no explanation yet for the zombies. They should not happen and so far I have never seen them. We may need to go through a debug log (which will become very large) to find out what's going on. > The question is under what conditions does rsyslogd spawn a new > thread/process and why was it a zombie? Unfortunately, there is no quick answer. A quick one may be: when it needs them, based on queue watermark settings and based on you configuration. But to really understand it, you need to read this doc: http://www.rsyslog.com/doc-queues.html The doc also describes all the knobs that you can use to control thread creation. There are many ;) > I am running rsyslogd in an > embedded environment and not a regular laptop/desktop. Interesting use case... > In addition, I > am using busybox and I believe the syslog buffer size is set to what do yo mean by "syslog buffer size"? The length of a receive buffer? It is 2K, thus single messages up to 2K are supported. It can be changed by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) support only up to 1K. > something very low or perhaps none at all. Would this be a factor? > Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > cases happen. The compatibility modes do not affect queue operation. > Are these issues already known and fixed in a later version? Sorry, if > I am asking the same questions or have the same issues as previous > people but without the ability to search (or at least, I don't know > how to) the archive, I don't know if my problem/questions has already > been seen and/or resolved. If we need to find out about the zombies, we need to move on to the current devel version. So I would give that a try in any case. 3.16.2 will (most probably) be replaced by 3.18.0 (based on the current beta) next week. So I won't touch it any longer. Looking forward to your feedback, Rainer > > Thank you very much for your support. > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Thu Jul 3 08:48:02 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 23:48:02 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Message-ID: <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> Sorry for sending a lot of emails. I believe I have root-cause the issue in 3.17.5. It appears that when the rsyslog.conf file does something like this: emerg.* *;mytemplate The code executes wallmsg which does forks. This is how I got a lot of processes were being created in my scenario. Although one would hope that a system doesn't misbehave this awful, nonetheless it does happen. First, it seems that the emergency messages should go to all those that are logged in via whatever mechanism (i.e. telnet, console, stty, ssh, etc...), this message should appear on all those "screens". I see that it does not even if I just logged 1 message at this serverity. I believe this is how syslog described setting up a wall message and what it is. Is this a bug? Lastly, when the child of the fork exits, it does not appear to be removed from Linux and still continues to eat up memory and is reported as a zombie. It appears that when the workerthread has been "destroyed/deconstructed" does things clear up. I am not sure if this is a rsyslogd problem or a linux or gcc problem. Any ideas? Thanks, Scott On Wed, Jul 2, 2008 at 5:57 PM, Scott Phuong wrote: > Hi, > > I've attached four files. Two of which are debug dumps, one is the > conf file and the last one is a test case scenario that constantly > fails on my end. I hope this gives a little more information. > Furthermore, the dumps are from 3.17.5 which is the "closest" version > to 3.18.0 that I was able to find. > > Both failed scenarios occur when lots of messages were being flooded > to rsyslogd at a very fast rate (look at logtest.c) The > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads > that it took up so much memory that executing any command as simple as > 'ls -l' would not work from the command line. I think the number of > threads grew as much as the number of messages. In the latter > scenario, after killing logtest.c, it didn't look like the those > zombies threads went away until I did a CTRL+C to the rsyslogd which > was running in the foreground since I use the "-dn" option. > > This is on an embedded system that runs significantly slower than a > desktop or laptop so maybe it would be harder to reproduce on a > regular computer. I looked at all the parameters that I believe could > affect this and believe for the most part the defaults are more than > adequate. The main message queue never looked like it hit the high > water mark but it did hit the lower one. So, I don't think messages > were being dropped (not sure) or an overflow condition occurred. > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > compiled using GCC and the standard GNU C libraries version 3.4.5. > Rsyslog source code is cross-compiled using the following configure > line: > > ./configure --disable-zlib --disable-largefile > --enable-share=yes > --prefix=/ > --host=arm-unknown-linux-gnu > ac_cv_func_malloc_0_nonnull=yes > ac_cv_func_realloc_0_nonnull=yes > ac_cv_func_lstat_dereferences_slashed_symlink=yes > ac_cv_func_stat_empty_string_bug=no > enable_debug=no > enable_rtinst=no > > Lastly, the logtest was executed with just the "-s" parameter. It is a > simple C file that I came up with. > > I took a look at the debug messages and it does not appear that new > threads are created via calls to wtpStartWrkr in wtp.c. > > Any help I can bring to solve this issue, please let me know. I hope I > am not doing anything wrong here. > > Thanks, > > Scott >> >> On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong wrote: >>> Hi Rainer, >>> >>> Thanks for your reply. Looking at the default settings (from the >>> online help's configuration page), they are what I wanted. The main >>> messages queue is set to fix sized array with 1 worker thread created >>> at maximum and action queues are direct mode which according to the >>> queue document page, means that there will not be a worker thread >>> created. Is my understanding correct? If yes, how do I quickly check >>> without using the -d option if the defaults are set correctly? Or what >>> do I look for in the debug messages that gets printed out to ensure >>> this? >>> >>> You also mentioned that version 3.18.0 is probably going to be >>> released as the stable version next week. I see on the webpage there >>> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >>> >>> Also, how come I did not get your reply in my email inbox? My account >>> settings look correct. >>> >>> Thanks, >>> >>> Scott Phuong >>> >>> As for the syslog buffer size, that applies to syslogd and does not >>> apply to rsyslog. >>> >>> >>> >>> My configuration files do not change the Action queue or Worker queue >>> parameters at all. Looking at >>> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >>>> Hi, >>>> >>>> I have 3.16.2 which was recently released. I see that under certain >>>> conditions rsyslogd spawns a lot of threads: >>>> 5949 root 11216 S rsyslogd >>>> 5950 root 11216 S rsyslogd >>>> 5951 root 11216 S rsyslogd >>>> 5952 root 11216 S rsyslogd >>>> 5953 root 11216 S rsyslogd >>>> 5954 root 11216 S rsyslogd >>>> 5985 root Z [rsyslogd] >>>> 6445 root Z [rsyslogd] >>>> >>>> I had to kill the rsyslogd and restart it. The first invocation had a >>>> pid of 219 before it had to be killed. The second invocation of pid >>>> which you see above starts with 5949. The difference is the amount of >>>> zombie threads that were invoked by rsyslogd before I had to kill the >>>> first invocation of it. >>> >>> I have no explanation yet for the zombies. They should not happen and so >>> far I have never seen them. We may need to go through a debug log (which >>> will become very large) to find out what's going on. >>> >>>> The question is under what conditions does rsyslogd spawn a new >>>> thread/process and why was it a zombie? >>> >>> Unfortunately, there is no quick answer. A quick one may be: when it >>> needs them, based on queue watermark settings and based on you >>> configuration. But to really understand it, you need to read this doc: >>> >>> http://www.rsyslog.com/doc-queues.html >>> >>> The doc also describes all the knobs that you can use to control thread >>> creation. There are many ;) >>> >>>> I am running rsyslogd in an >>>> embedded environment and not a regular laptop/desktop. >>> >>> Interesting use case... >>> >>>> In addition, I >>>> am using busybox and I believe the syslog buffer size is set to >>> >>> what do yo mean by "syslog buffer size"? The length of a receive buffer? >>> It is 2K, thus single messages up to 2K are supported. It can be changed >>> by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) >>> support only up to 1K. >>> >>>> something very low or perhaps none at all. Would this be a factor? >>>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >>>> cases happen. >>> >>> The compatibility modes do not affect queue operation. >>> >>>> Are these issues already known and fixed in a later version? Sorry, if >>>> I am asking the same questions or have the same issues as previous >>>> people but without the ability to search (or at least, I don't know >>>> how to) the archive, I don't know if my problem/questions has already >>>> been seen and/or resolved. >>> >>> If we need to find out about the zombies, we need to move on to the >>> current devel version. So I would give that a try in any case. 3.16.2 >>> will (most probably) be replaced by 3.18.0 (based on the current beta) >>> next week. So I won't touch it any longer. >>> >>> Looking forward to your feedback, >>> Rainer >>> >>>> >>>> Thank you very much for your support. >>>> >>>> Scott >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > From rgerhards at hq.adiscon.com Thu Jul 3 08:59:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 3 Jul 2008 08:59:26 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com><17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309307@grfint2.intern.adiscon.com> Hi Scott, will reply in more detail later today, but one thing quickly. This is very good information. The wall code is ooooooooooold, stems back to sysklogd and has only been slightly modified. I have to admit it is not much on my testing radar. So I assume you found the bug. I think I'll need to re-design it. With the queue engine, forking off should not really be necessary. I need to see if that goes into the v3-stable soon or is kept at the devel tree. I am a bit hesitant to move it to stable, as it involves a good chance for additional bugs... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Thursday, July 03, 2008 8:48 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog threads questions > > Sorry for sending a lot of emails. I believe I have root-cause the > issue in 3.17.5. It appears that when the rsyslog.conf file does > something like this: > emerg.* *;mytemplate > > The code executes wallmsg which does forks. This is how I got a lot of > processes were being created in my scenario. Although one would hope > that a system doesn't misbehave this awful, nonetheless it does > happen. First, it seems that the emergency messages should go to all > those that are logged in via whatever mechanism (i.e. telnet, console, > stty, ssh, etc...), this message should appear on all those "screens". > I see that it does not even if I just logged 1 message at this > serverity. I believe this is how syslog described setting up a wall > message and what it is. Is this a bug? > > Lastly, when the child of the fork exits, it does not appear to be > removed from Linux and still continues to eat up memory and is > reported as a zombie. It appears that when the workerthread has been > "destroyed/deconstructed" does things clear up. I am not sure if this > is a rsyslogd problem or a linux or gcc problem. Any ideas? > > Thanks, > > Scott > > > > On Wed, Jul 2, 2008 at 5:57 PM, Scott Phuong > wrote: > > Hi, > > > > I've attached four files. Two of which are debug dumps, one is the > > conf file and the last one is a test case scenario that constantly > > fails on my end. I hope this gives a little more information. > > Furthermore, the dumps are from 3.17.5 which is the "closest" version > > to 3.18.0 that I was able to find. > > > > Both failed scenarios occur when lots of messages were being flooded > > to rsyslogd at a very fast rate (look at logtest.c) The > > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" > threads > > that it took up so much memory that executing any command as simple > as > > 'ls -l' would not work from the command line. I think the number of > > threads grew as much as the number of messages. In the latter > > scenario, after killing logtest.c, it didn't look like the those > > zombies threads went away until I did a CTRL+C to the rsyslogd which > > was running in the foreground since I use the "-dn" option. > > > > This is on an embedded system that runs significantly slower than a > > desktop or laptop so maybe it would be harder to reproduce on a > > regular computer. I looked at all the parameters that I believe could > > affect this and believe for the most part the defaults are more than > > adequate. The main message queue never looked like it hit the high > > water mark but it did hit the lower one. So, I don't think messages > > were being dropped (not sure) or an overflow condition occurred. > > > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > > compiled using GCC and the standard GNU C libraries version 3.4.5. > > Rsyslog source code is cross-compiled using the following configure > > line: > > > > ./configure --disable-zlib --disable-largefile > > --enable-share=yes > > --prefix=/ > > --host=arm-unknown-linux-gnu > > ac_cv_func_malloc_0_nonnull=yes > > ac_cv_func_realloc_0_nonnull=yes > > > ac_cv_func_lstat_dereferences_slashed_symlink=yes > > ac_cv_func_stat_empty_string_bug=no > > enable_debug=no > > enable_rtinst=no > > > > Lastly, the logtest was executed with just the "-s" parameter. It is > a > > simple C file that I came up with. > > > > I took a look at the debug messages and it does not appear that new > > threads are created via calls to wtpStartWrkr in wtp.c. > > > > Any help I can bring to solve this issue, please let me know. I hope > I > > am not doing anything wrong here. > > > > Thanks, > > > > Scott > >> > >> On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > wrote: > >>> Hi Rainer, > >>> > >>> Thanks for your reply. Looking at the default settings (from the > >>> online help's configuration page), they are what I wanted. The main > >>> messages queue is set to fix sized array with 1 worker thread > created > >>> at maximum and action queues are direct mode which according to the > >>> queue document page, means that there will not be a worker thread > >>> created. Is my understanding correct? If yes, how do I quickly > check > >>> without using the -d option if the defaults are set correctly? Or > what > >>> do I look for in the debug messages that gets printed out to ensure > >>> this? > >>> > >>> You also mentioned that version 3.18.0 is probably going to be > >>> released as the stable version next week. I see on the webpage > there > >>> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? > >>> > >>> Also, how come I did not get your reply in my email inbox? My > account > >>> settings look correct. > >>> > >>> Thanks, > >>> > >>> Scott Phuong > >>> > >>> As for the syslog buffer size, that applies to syslogd and does not > >>> apply to rsyslog. > >>> > >>> > >>> > >>> My configuration files do not change the Action queue or Worker > queue > >>> parameters at all. Looking at > >>> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >>>> Hi, > >>>> > >>>> I have 3.16.2 which was recently released. I see that under > certain > >>>> conditions rsyslogd spawns a lot of threads: > >>>> 5949 root 11216 S rsyslogd > >>>> 5950 root 11216 S rsyslogd > >>>> 5951 root 11216 S rsyslogd > >>>> 5952 root 11216 S rsyslogd > >>>> 5953 root 11216 S rsyslogd > >>>> 5954 root 11216 S rsyslogd > >>>> 5985 root Z [rsyslogd] > >>>> 6445 root Z [rsyslogd] > >>>> > >>>> I had to kill the rsyslogd and restart it. The first invocation > had a > >>>> pid of 219 before it had to be killed. The second invocation of > pid > >>>> which you see above starts with 5949. The difference is the amount > of > >>>> zombie threads that were invoked by rsyslogd before I had to kill > the > >>>> first invocation of it. > >>> > >>> I have no explanation yet for the zombies. They should not happen > and so > >>> far I have never seen them. We may need to go through a debug log > (which > >>> will become very large) to find out what's going on. > >>> > >>>> The question is under what conditions does rsyslogd spawn a new > >>>> thread/process and why was it a zombie? > >>> > >>> Unfortunately, there is no quick answer. A quick one may be: when > it > >>> needs them, based on queue watermark settings and based on you > >>> configuration. But to really understand it, you need to read this > doc: > >>> > >>> http://www.rsyslog.com/doc-queues.html > >>> > >>> The doc also describes all the knobs that you can use to control > thread > >>> creation. There are many ;) > >>> > >>>> I am running rsyslogd in an > >>>> embedded environment and not a regular laptop/desktop. > >>> > >>> Interesting use case... > >>> > >>>> In addition, I > >>>> am using busybox and I believe the syslog buffer size is set to > >>> > >>> what do yo mean by "syslog buffer size"? The length of a receive > buffer? > >>> It is 2K, thus single messages up to 2K are supported. It can be > changed > >>> by modifying the MAXLINE define. Note that stock syslogd (and > RFC3164) > >>> support only up to 1K. > >>> > >>>> something very low or perhaps none at all. Would this be a factor? > >>>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > >>>> cases happen. > >>> > >>> The compatibility modes do not affect queue operation. > >>> > >>>> Are these issues already known and fixed in a later version? > Sorry, if > >>>> I am asking the same questions or have the same issues as previous > >>>> people but without the ability to search (or at least, I don't > know > >>>> how to) the archive, I don't know if my problem/questions has > already > >>>> been seen and/or resolved. > >>> > >>> If we need to find out about the zombies, we need to move on to the > >>> current devel version. So I would give that a try in any case. > 3.16.2 > >>> will (most probably) be replaced by 3.18.0 (based on the current > beta) > >>> next week. So I won't touch it any longer. > >>> > >>> Looking forward to your feedback, > >>> Rainer > >>> > >>>> > >>>> Thank you very much for your support. > >>>> > >>>> Scott > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Thu Jul 3 02:57:39 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 17:57:39 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> Message-ID: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Hi, I've attached four files. Two of which are debug dumps, one is the conf file and the last one is a test case scenario that constantly fails on my end. I hope this gives a little more information. Furthermore, the dumps are from 3.17.5 which is the "closest" version to 3.18.0 that I was able to find. Both failed scenarios occur when lots of messages were being flooded to rsyslogd at a very fast rate (look at logtest.c) The my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads that it took up so much memory that executing any command as simple as 'ls -l' would not work from the command line. I think the number of threads grew as much as the number of messages. In the latter scenario, after killing logtest.c, it didn't look like the those zombies threads went away until I did a CTRL+C to the rsyslogd which was running in the foreground since I use the "-dn" option. This is on an embedded system that runs significantly slower than a desktop or laptop so maybe it would be harder to reproduce on a regular computer. I looked at all the parameters that I believe could affect this and believe for the most part the defaults are more than adequate. The main message queue never looked like it hit the high water mark but it did hit the lower one. So, I don't think messages were being dropped (not sure) or an overflow condition occurred. The processor is ARM-based and it is using Linux kernel 2.6.16.12 and compiled using GCC and the standard GNU C libraries version 3.4.5. Rsyslog source code is cross-compiled using the following configure line: ./configure --disable-zlib --disable-largefile --enable-share=yes --prefix=/ --host=arm-unknown-linux-gnu ac_cv_func_malloc_0_nonnull=yes ac_cv_func_realloc_0_nonnull=yes ac_cv_func_lstat_dereferences_slashed_symlink=yes ac_cv_func_stat_empty_string_bug=no enable_debug=no enable_rtinst=no Lastly, the logtest was executed with just the "-s" parameter. It is a simple C file that I came up with. I took a look at the debug messages and it does not appear that new threads are created via calls to wtpStartWrkr in wtp.c. Any help I can bring to solve this issue, please let me know. I hope I am not doing anything wrong here. Thanks, Scott > > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong wrote: >> Hi Rainer, >> >> Thanks for your reply. Looking at the default settings (from the >> online help's configuration page), they are what I wanted. The main >> messages queue is set to fix sized array with 1 worker thread created >> at maximum and action queues are direct mode which according to the >> queue document page, means that there will not be a worker thread >> created. Is my understanding correct? If yes, how do I quickly check >> without using the -d option if the defaults are set correctly? Or what >> do I look for in the debug messages that gets printed out to ensure >> this? >> >> You also mentioned that version 3.18.0 is probably going to be >> released as the stable version next week. I see on the webpage there >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >> >> Also, how come I did not get your reply in my email inbox? My account >> settings look correct. >> >> Thanks, >> >> Scott Phuong >> >> As for the syslog buffer size, that applies to syslogd and does not >> apply to rsyslog. >> >> >> >> My configuration files do not change the Action queue or Worker queue >> parameters at all. Looking at >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >>> Hi, >>> >>> I have 3.16.2 which was recently released. I see that under certain >>> conditions rsyslogd spawns a lot of threads: >>> 5949 root 11216 S rsyslogd >>> 5950 root 11216 S rsyslogd >>> 5951 root 11216 S rsyslogd >>> 5952 root 11216 S rsyslogd >>> 5953 root 11216 S rsyslogd >>> 5954 root 11216 S rsyslogd >>> 5985 root Z [rsyslogd] >>> 6445 root Z [rsyslogd] >>> >>> I had to kill the rsyslogd and restart it. The first invocation had a >>> pid of 219 before it had to be killed. The second invocation of pid >>> which you see above starts with 5949. The difference is the amount of >>> zombie threads that were invoked by rsyslogd before I had to kill the >>> first invocation of it. >> >> I have no explanation yet for the zombies. They should not happen and so >> far I have never seen them. We may need to go through a debug log (which >> will become very large) to find out what's going on. >> >>> The question is under what conditions does rsyslogd spawn a new >>> thread/process and why was it a zombie? >> >> Unfortunately, there is no quick answer. A quick one may be: when it >> needs them, based on queue watermark settings and based on you >> configuration. But to really understand it, you need to read this doc: >> >> http://www.rsyslog.com/doc-queues.html >> >> The doc also describes all the knobs that you can use to control thread >> creation. There are many ;) >> >>> I am running rsyslogd in an >>> embedded environment and not a regular laptop/desktop. >> >> Interesting use case... >> >>> In addition, I >>> am using busybox and I believe the syslog buffer size is set to >> >> what do yo mean by "syslog buffer size"? The length of a receive buffer? >> It is 2K, thus single messages up to 2K are supported. It can be changed >> by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) >> support only up to 1K. >> >>> something very low or perhaps none at all. Would this be a factor? >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >>> cases happen. >> >> The compatibility modes do not affect queue operation. >> >>> Are these issues already known and fixed in a later version? Sorry, if >>> I am asking the same questions or have the same issues as previous >>> people but without the ability to search (or at least, I don't know >>> how to) the archive, I don't know if my problem/questions has already >>> been seen and/or resolved. >> >> If we need to find out about the zombies, we need to move on to the >> current devel version. So I would give that a try in any case. 3.16.2 >> will (most probably) be replaced by 3.18.0 (based on the current beta) >> next week. So I won't touch it any longer. >> >> Looking forward to your feedback, >> Rainer >> >>> >>> Thank you very much for your support. >>> >>> Scott >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > From niallel at gmail.com Thu Jul 3 23:59:53 2008 From: niallel at gmail.com (niall el-assaad) Date: Thu, 3 Jul 2008 22:59:53 +0100 Subject: [rsyslog] Problem matching localhost Message-ID: Hi, I'm running 2.0.0-11 (version included with redhat 5.2) I want to filter all the messages from external syslog devices to one file and all messages from the localhost to another file. However even with the -x option turned on when a local service (such as crond) sends a message to the log the hostname is set to the domain name of the server. So I can't use the following to match: :HOSTNAME, isequal, "localhost" /var/log/messages :HOSTNAME, !isequal, "localhost" /var/log/externalsyslog I could replace "localhost" with "dnsname" to get it to work, but I would like a generic method that will work on all the syslog servers I have. Is there some switch that will cause rsyslog to report the local services as sending from localhost or 127.0.0.1 rather than the hostname of the localhost. thanks, niall From rgerhards at hq.adiscon.com Fri Jul 4 08:12:39 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 08:12:39 +0200 Subject: [rsyslog] Problem matching localhost In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30932A@grfint2.intern.adiscon.com> I replied to your forum post: http://kb.monitorware.com/post13018.html#p13018 I suggest we keep discussing it in the forum to reduce double work. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of niall el-assaad > Sent: Friday, July 04, 2008 12:00 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Problem matching localhost > > Hi, > > I'm running 2.0.0-11 (version included with redhat 5.2) > > I want to filter all the messages from external syslog devices to one > file > and all messages from the localhost to another file. > > However even with the -x option turned on when a local service (such as > crond) sends a message to the log the hostname is set to the domain > name of > the server. > > So I can't use the following to match: > :HOSTNAME, isequal, "localhost" /var/log/messages > :HOSTNAME, !isequal, "localhost" /var/log/externalsyslog > > I could replace "localhost" with "dnsname" to get it to work, but I > would > like a generic method that will work on all the syslog servers I have. > Is there some switch that will cause rsyslog to report the local > services as > sending from localhost or 127.0.0.1 rather than the hostname of the > localhost. > > thanks, > > niall > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 4 10:31:08 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 10:31:08 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> Scott, I have rewritten the omusrmsg plugin. This is currently part of the development tree, but will become part of the (new) beta when we do a version number shift next week. I'd appreciate if you could give it a try. It is available here: http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz I am hesitant to put this in the upcoming new v3-stable. There were lots of changes and one rule for a stable is that there should be no non-fix type of changes for at least 4 weeks before it turns into a stable. It of course is debatable if the change I made is a fix or not. In some sense it is, but on the other hand it is a rewrite that changed a lot. So given the fact that nobody on a "regular" machine saw an issue the past year, I would not like to bring the new code directly into the stable. If that causes problems for you, you may want to try simply using the omusrmsg.c code together with the v3-stable. I haven't tried it, but it should work well. All changes were just to that file, and there are no dependencies to anything external. OK... but now let's see first if it fixes the issue ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Thursday, July 03, 2008 2:58 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog threads questions > > Hi, > > I've attached four files. Two of which are debug dumps, one is the > conf file and the last one is a test case scenario that constantly > fails on my end. I hope this gives a little more information. > Furthermore, the dumps are from 3.17.5 which is the "closest" version > to 3.18.0 that I was able to find. > > Both failed scenarios occur when lots of messages were being flooded > to rsyslogd at a very fast rate (look at logtest.c) The > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads > that it took up so much memory that executing any command as simple as > 'ls -l' would not work from the command line. I think the number of > threads grew as much as the number of messages. In the latter > scenario, after killing logtest.c, it didn't look like the those > zombies threads went away until I did a CTRL+C to the rsyslogd which > was running in the foreground since I use the "-dn" option. > > This is on an embedded system that runs significantly slower than a > desktop or laptop so maybe it would be harder to reproduce on a > regular computer. I looked at all the parameters that I believe could > affect this and believe for the most part the defaults are more than > adequate. The main message queue never looked like it hit the high > water mark but it did hit the lower one. So, I don't think messages > were being dropped (not sure) or an overflow condition occurred. > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > compiled using GCC and the standard GNU C libraries version 3.4.5. > Rsyslog source code is cross-compiled using the following configure > line: > > ./configure --disable-zlib --disable-largefile > --enable-share=yes > --prefix=/ > --host=arm-unknown-linux-gnu > ac_cv_func_malloc_0_nonnull=yes > ac_cv_func_realloc_0_nonnull=yes > > ac_cv_func_lstat_dereferences_slashed_symlink=yes > ac_cv_func_stat_empty_string_bug=no > enable_debug=no > enable_rtinst=no > > Lastly, the logtest was executed with just the "-s" parameter. It is a > simple C file that I came up with. > > I took a look at the debug messages and it does not appear that new > threads are created via calls to wtpStartWrkr in wtp.c. > > Any help I can bring to solve this issue, please let me know. I hope I > am not doing anything wrong here. > > Thanks, > > Scott > > > > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > wrote: > >> Hi Rainer, > >> > >> Thanks for your reply. Looking at the default settings (from the > >> online help's configuration page), they are what I wanted. The main > >> messages queue is set to fix sized array with 1 worker thread > created > >> at maximum and action queues are direct mode which according to the > >> queue document page, means that there will not be a worker thread > >> created. Is my understanding correct? If yes, how do I quickly > check > >> without using the -d option if the defaults are set correctly? Or > what > >> do I look for in the debug messages that gets printed out to ensure > >> this? > >> > >> You also mentioned that version 3.18.0 is probably going to be > >> released as the stable version next week. I see on the webpage there > >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? > >> > >> Also, how come I did not get your reply in my email inbox? My > account > >> settings look correct. > >> > >> Thanks, > >> > >> Scott Phuong > >> > >> As for the syslog buffer size, that applies to syslogd and does not > >> apply to rsyslog. > >> > >> > >> > >> My configuration files do not change the Action queue or Worker > queue > >> parameters at all. Looking at > >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >>> Hi, > >>> > >>> I have 3.16.2 which was recently released. I see that under certain > >>> conditions rsyslogd spawns a lot of threads: > >>> 5949 root 11216 S rsyslogd > >>> 5950 root 11216 S rsyslogd > >>> 5951 root 11216 S rsyslogd > >>> 5952 root 11216 S rsyslogd > >>> 5953 root 11216 S rsyslogd > >>> 5954 root 11216 S rsyslogd > >>> 5985 root Z [rsyslogd] > >>> 6445 root Z [rsyslogd] > >>> > >>> I had to kill the rsyslogd and restart it. The first invocation had > a > >>> pid of 219 before it had to be killed. The second invocation of pid > >>> which you see above starts with 5949. The difference is the amount > of > >>> zombie threads that were invoked by rsyslogd before I had to kill > the > >>> first invocation of it. > >> > >> I have no explanation yet for the zombies. They should not happen > and so > >> far I have never seen them. We may need to go through a debug log > (which > >> will become very large) to find out what's going on. > >> > >>> The question is under what conditions does rsyslogd spawn a new > >>> thread/process and why was it a zombie? > >> > >> Unfortunately, there is no quick answer. A quick one may be: when it > >> needs them, based on queue watermark settings and based on you > >> configuration. But to really understand it, you need to read this > doc: > >> > >> http://www.rsyslog.com/doc-queues.html > >> > >> The doc also describes all the knobs that you can use to control > thread > >> creation. There are many ;) > >> > >>> I am running rsyslogd in an > >>> embedded environment and not a regular laptop/desktop. > >> > >> Interesting use case... > >> > >>> In addition, I > >>> am using busybox and I believe the syslog buffer size is set to > >> > >> what do yo mean by "syslog buffer size"? The length of a receive > buffer? > >> It is 2K, thus single messages up to 2K are supported. It can be > changed > >> by modifying the MAXLINE define. Note that stock syslogd (and > RFC3164) > >> support only up to 1K. > >> > >>> something very low or perhaps none at all. Would this be a factor? > >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > >>> cases happen. > >> > >> The compatibility modes do not affect queue operation. > >> > >>> Are these issues already known and fixed in a later version? Sorry, > if > >>> I am asking the same questions or have the same issues as previous > >>> people but without the ability to search (or at least, I don't know > >>> how to) the archive, I don't know if my problem/questions has > already > >>> been seen and/or resolved. > >> > >> If we need to find out about the zombies, we need to move on to the > >> current devel version. So I would give that a try in any case. > 3.16.2 > >> will (most probably) be replaced by 3.18.0 (based on the current > beta) > >> next week. So I won't touch it any longer. > >> > >> Looking forward to your feedback, > >> Rainer > >> > >>> > >>> Thank you very much for your support. > >>> > >>> Scott > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > From rgerhards at hq.adiscon.com Fri Jul 4 11:24:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 11:24:29 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> Hi list, I am posting a question here in the hope that some of the subscribers may be able to lend me a helping hand. Recently, I have begun to add a testbench to rsyslog. The idea is that over time the project should have canned tests which are easy to run on each version (as part of make distcheck at latest), increasing the overall code quality. "All" (so far two ;)) tests are located in the tests subdirectory. One test fails if compiled in release mode, that is rscript-parse.c. I have tracked down the failure cause and it is different struct member alignment in the ctok_token_t structure. The alignment is different in the rsyslog runtime and this test tool (I have compared the offsets computed for pToken->tok and they are different). So far, so good. But I do not find the cause of this misalignment. I checked the make output, and as it looks both the runtime as well as the tool are compiled with the same compiler options. Also, in debug mode it works OK, but in release mode (--disable-debug) it fails. I am sure the problem is something quite simple, but I have run out of ideas of what it may be (or, more probable, I simply overlook something). Any help would be deeply appreciated. Thanks, Rainer From mycleanjunk at gmail.com Fri Jul 4 12:28:58 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:28:58 -0700 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> Message-ID: <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> Here is my analysis: In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a #define and is defined to obj_t objData which is the same for both NDEBUG defined and not defined. But obj_t is a typedef of struct obj_s. In the structure definition for obj_s in obj-types.h are the following lines: #ifndef NDEBUG unsigned int iObjCooCKiE; #endif NDEBUG is defined in config.h which was included in the runtime for rsyslog but not for test-parse.c I didn't compile the code to test my theory. I just did a quick check. Hope that solved your problem. Scott I think this may be why the you are getting this structure misalignment? On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards wrote: > Hi list, > > I am posting a question here in the hope that some of the subscribers > may be able to lend me a helping hand. > > Recently, I have begun to add a testbench to rsyslog. The idea is that > over time the project should have canned tests which are easy to run on > each version (as part of make distcheck at latest), increasing the > overall code quality. > > "All" (so far two ;)) tests are located in the tests subdirectory. One > test fails if compiled in release mode, that is rscript-parse.c. I have > tracked down the failure cause and it is different struct member > alignment in the ctok_token_t structure. The alignment is different in > the rsyslog runtime and this test tool (I have compared the offsets > computed for pToken->tok and they are different). So far, so good. But I > do not find the cause of this misalignment. I checked the make output, > and as it looks both the runtime as well as the tool are compiled with > the same compiler options. Also, in debug mode it works OK, but in > release mode (--disable-debug) it fails. > > I am sure the problem is something quite simple, but I have run out of > ideas of what it may be (or, more probable, I simply overlook > something). > > Any help would be deeply appreciated. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mycleanjunk at gmail.com Fri Jul 4 12:31:45 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:31:45 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> Message-ID: <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> Hi Rainer, Thanks!! I'll give this a try next week and let you know the results. If it works, I think it is okay not to include in the upcoming v3 stable since it is due to release very soon. But if it does work, it would be great to have it in the following stable release. Scott On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards wrote: > Scott, > > I have rewritten the omusrmsg plugin. This is currently part of the > development tree, but will become part of the (new) beta when we do a > version number shift next week. I'd appreciate if you could give it a > try. It is available here: > > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz > > I am hesitant to put this in the upcoming new v3-stable. There were lots > of changes and one rule for a stable is that there should be no non-fix > type of changes for at least 4 weeks before it turns into a stable. It > of course is debatable if the change I made is a fix or not. In some > sense it is, but on the other hand it is a rewrite that changed a lot. > So given the fact that nobody on a "regular" machine saw an issue the > past year, I would not like to bring the new code directly into the > stable. If that causes problems for you, you may want to try simply > using the omusrmsg.c code together with the v3-stable. I haven't tried > it, but it should work well. All changes were just to that file, and > there are no dependencies to anything external. > > OK... but now let's see first if it fixes the issue ;) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Thursday, July 03, 2008 2:58 AM >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] rsyslog threads questions >> >> Hi, >> >> I've attached four files. Two of which are debug dumps, one is the >> conf file and the last one is a test case scenario that constantly >> fails on my end. I hope this gives a little more information. >> Furthermore, the dumps are from 3.17.5 which is the "closest" version >> to 3.18.0 that I was able to find. >> >> Both failed scenarios occur when lots of messages were being flooded >> to rsyslogd at a very fast rate (look at logtest.c) The >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads >> that it took up so much memory that executing any command as simple as >> 'ls -l' would not work from the command line. I think the number of >> threads grew as much as the number of messages. In the latter >> scenario, after killing logtest.c, it didn't look like the those >> zombies threads went away until I did a CTRL+C to the rsyslogd which >> was running in the foreground since I use the "-dn" option. >> >> This is on an embedded system that runs significantly slower than a >> desktop or laptop so maybe it would be harder to reproduce on a >> regular computer. I looked at all the parameters that I believe could >> affect this and believe for the most part the defaults are more than >> adequate. The main message queue never looked like it hit the high >> water mark but it did hit the lower one. So, I don't think messages >> were being dropped (not sure) or an overflow condition occurred. >> >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 and >> compiled using GCC and the standard GNU C libraries version 3.4.5. >> Rsyslog source code is cross-compiled using the following configure >> line: >> >> ./configure --disable-zlib --disable-largefile >> --enable-share=yes >> --prefix=/ >> --host=arm-unknown-linux-gnu >> ac_cv_func_malloc_0_nonnull=yes >> ac_cv_func_realloc_0_nonnull=yes >> >> ac_cv_func_lstat_dereferences_slashed_symlink=yes >> ac_cv_func_stat_empty_string_bug=no >> enable_debug=no >> enable_rtinst=no >> >> Lastly, the logtest was executed with just the "-s" parameter. It is a >> simple C file that I came up with. >> >> I took a look at the debug messages and it does not appear that new >> threads are created via calls to wtpStartWrkr in wtp.c. >> >> Any help I can bring to solve this issue, please let me know. I hope I >> am not doing anything wrong here. >> >> Thanks, >> >> Scott >> > >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > >> wrote: >> >> Hi Rainer, >> >> >> >> Thanks for your reply. Looking at the default settings (from the >> >> online help's configuration page), they are what I wanted. The main >> >> messages queue is set to fix sized array with 1 worker thread >> created >> >> at maximum and action queues are direct mode which according to the >> >> queue document page, means that there will not be a worker thread >> >> created. Is my understanding correct? If yes, how do I quickly >> check >> >> without using the -d option if the defaults are set correctly? Or >> what >> >> do I look for in the debug messages that gets printed out to ensure >> >> this? >> >> >> >> You also mentioned that version 3.18.0 is probably going to be >> >> released as the stable version next week. I see on the webpage > there >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >> >> >> >> Also, how come I did not get your reply in my email inbox? My >> account >> >> settings look correct. >> >> >> >> Thanks, >> >> >> >> Scott Phuong >> >> >> >> As for the syslog buffer size, that applies to syslogd and does not >> >> apply to rsyslog. >> >> >> >> >> >> >> >> My configuration files do not change the Action queue or Worker >> queue >> >> parameters at all. Looking at >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >> >>> Hi, >> >>> >> >>> I have 3.16.2 which was recently released. I see that under > certain >> >>> conditions rsyslogd spawns a lot of threads: >> >>> 5949 root 11216 S rsyslogd >> >>> 5950 root 11216 S rsyslogd >> >>> 5951 root 11216 S rsyslogd >> >>> 5952 root 11216 S rsyslogd >> >>> 5953 root 11216 S rsyslogd >> >>> 5954 root 11216 S rsyslogd >> >>> 5985 root Z [rsyslogd] >> >>> 6445 root Z [rsyslogd] >> >>> >> >>> I had to kill the rsyslogd and restart it. The first invocation > had >> a >> >>> pid of 219 before it had to be killed. The second invocation of > pid >> >>> which you see above starts with 5949. The difference is the amount >> of >> >>> zombie threads that were invoked by rsyslogd before I had to kill >> the >> >>> first invocation of it. >> >> >> >> I have no explanation yet for the zombies. They should not happen >> and so >> >> far I have never seen them. We may need to go through a debug log >> (which >> >> will become very large) to find out what's going on. >> >> >> >>> The question is under what conditions does rsyslogd spawn a new >> >>> thread/process and why was it a zombie? >> >> >> >> Unfortunately, there is no quick answer. A quick one may be: when > it >> >> needs them, based on queue watermark settings and based on you >> >> configuration. But to really understand it, you need to read this >> doc: >> >> >> >> http://www.rsyslog.com/doc-queues.html >> >> >> >> The doc also describes all the knobs that you can use to control >> thread >> >> creation. There are many ;) >> >> >> >>> I am running rsyslogd in an >> >>> embedded environment and not a regular laptop/desktop. >> >> >> >> Interesting use case... >> >> >> >>> In addition, I >> >>> am using busybox and I believe the syslog buffer size is set to >> >> >> >> what do yo mean by "syslog buffer size"? The length of a receive >> buffer? >> >> It is 2K, thus single messages up to 2K are supported. It can be >> changed >> >> by modifying the MAXLINE define. Note that stock syslogd (and >> RFC3164) >> >> support only up to 1K. >> >> >> >>> something very low or perhaps none at all. Would this be a factor? >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >> >>> cases happen. >> >> >> >> The compatibility modes do not affect queue operation. >> >> >> >>> Are these issues already known and fixed in a later version? > Sorry, >> if >> >>> I am asking the same questions or have the same issues as previous >> >>> people but without the ability to search (or at least, I don't > know >> >>> how to) the archive, I don't know if my problem/questions has >> already >> >>> been seen and/or resolved. >> >> >> >> If we need to find out about the zombies, we need to move on to the >> >> current devel version. So I would give that a try in any case. >> 3.16.2 >> >> will (most probably) be replaced by 3.18.0 (based on the current >> beta) >> >> next week. So I won't touch it any longer. >> >> >> >> Looking forward to your feedback, >> >> Rainer >> >> >> >>> >> >>> Thank you very much for your support. >> >>> >> >>> Scott >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Fri Jul 4 12:32:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:32:03 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> Hi Scott, thanks for the help, I'll further investigate. BUT: both the runtime AND the tool are compiled in non-debug mode. So in this case, neither of them should have the object cookie, thus the alignment should be the same. But, as I said, I am grateful for any clue and will check if there is some issue along these lines :) In any case, it seems to be inconsistent and should be fixed :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:29 PM > To: rsyslog-users > Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > > Here is my analysis: > In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a > #define and is defined to obj_t objData which is the same for both > NDEBUG defined and not defined. But obj_t is a typedef of struct > obj_s. In the structure definition for obj_s in obj-types.h are the > following lines: > #ifndef NDEBUG > unsigned int iObjCooCKiE; > #endif > > NDEBUG is defined in config.h which was included in the runtime for > rsyslog but not for test-parse.c > > I didn't compile the code to test my theory. I just did a quick check. > > Hope that solved your problem. > > Scott > > > I think this may be why the you are getting this structure > misalignment? > > > On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards > wrote: > > Hi list, > > > > I am posting a question here in the hope that some of the subscribers > > may be able to lend me a helping hand. > > > > Recently, I have begun to add a testbench to rsyslog. The idea is > that > > over time the project should have canned tests which are easy to run > on > > each version (as part of make distcheck at latest), increasing the > > overall code quality. > > > > "All" (so far two ;)) tests are located in the tests subdirectory. > One > > test fails if compiled in release mode, that is rscript-parse.c. I > have > > tracked down the failure cause and it is different struct member > > alignment in the ctok_token_t structure. The alignment is different > in > > the rsyslog runtime and this test tool (I have compared the offsets > > computed for pToken->tok and they are different). So far, so good. > But I > > do not find the cause of this misalignment. I checked the make > output, > > and as it looks both the runtime as well as the tool are compiled > with > > the same compiler options. Also, in debug mode it works OK, but in > > release mode (--disable-debug) it fails. > > > > I am sure the problem is something quite simple, but I have run out > of > > ideas of what it may be (or, more probable, I simply overlook > > something). > > > > Any help would be deeply appreciated. > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 4 12:35:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:35:23 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com><17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> Hi Scott, just for your understanding: once a release is stable, it will only receive fixes. So the fix we are looking it needs to become part of the devel, be migrated to beta and then be migrated to stable. In general, one migration step takes between 4 and 12 weeks, depending on what was done. So far, the average seems on the average, that is two month. So two cycles mean roughly four month. HOWEVER, I intend to put it into the next beta, which will bring us to just one cycle. So it would be available in roughly two month. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog threads questions > > Hi Rainer, > > Thanks!! I'll give this a try next week and let you know the results. > If it works, I think it is okay not to include in the upcoming v3 > stable since it is due to release very soon. But if it does work, it > would be great to have it in the following stable release. > > Scott > > On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards > wrote: > > Scott, > > > > I have rewritten the omusrmsg plugin. This is currently part of the > > development tree, but will become part of the (new) beta when we do a > > version number shift next week. I'd appreciate if you could give it a > > try. It is available here: > > > > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz > > > > I am hesitant to put this in the upcoming new v3-stable. There were > lots > > of changes and one rule for a stable is that there should be no non- > fix > > type of changes for at least 4 weeks before it turns into a stable. > It > > of course is debatable if the change I made is a fix or not. In some > > sense it is, but on the other hand it is a rewrite that changed a > lot. > > So given the fact that nobody on a "regular" machine saw an issue the > > past year, I would not like to bring the new code directly into the > > stable. If that causes problems for you, you may want to try simply > > using the omusrmsg.c code together with the v3-stable. I haven't > tried > > it, but it should work well. All changes were just to that file, and > > there are no dependencies to anything external. > > > > OK... but now let's see first if it fixes the issue ;) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong > >> Sent: Thursday, July 03, 2008 2:58 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: Re: [rsyslog] rsyslog threads questions > >> > >> Hi, > >> > >> I've attached four files. Two of which are debug dumps, one is the > >> conf file and the last one is a test case scenario that constantly > >> fails on my end. I hope this gives a little more information. > >> Furthermore, the dumps are from 3.17.5 which is the "closest" > version > >> to 3.18.0 that I was able to find. > >> > >> Both failed scenarios occur when lots of messages were being flooded > >> to rsyslogd at a very fast rate (look at logtest.c) The > >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" > threads > >> that it took up so much memory that executing any command as simple > as > >> 'ls -l' would not work from the command line. I think the number of > >> threads grew as much as the number of messages. In the latter > >> scenario, after killing logtest.c, it didn't look like the those > >> zombies threads went away until I did a CTRL+C to the rsyslogd which > >> was running in the foreground since I use the "-dn" option. > >> > >> This is on an embedded system that runs significantly slower than a > >> desktop or laptop so maybe it would be harder to reproduce on a > >> regular computer. I looked at all the parameters that I believe > could > >> affect this and believe for the most part the defaults are more than > >> adequate. The main message queue never looked like it hit the high > >> water mark but it did hit the lower one. So, I don't think messages > >> were being dropped (not sure) or an overflow condition occurred. > >> > >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 > and > >> compiled using GCC and the standard GNU C libraries version 3.4.5. > >> Rsyslog source code is cross-compiled using the following configure > >> line: > >> > >> ./configure --disable-zlib --disable-largefile > >> --enable-share=yes > >> --prefix=/ > >> --host=arm-unknown-linux-gnu > >> ac_cv_func_malloc_0_nonnull=yes > >> ac_cv_func_realloc_0_nonnull=yes > >> > >> ac_cv_func_lstat_dereferences_slashed_symlink=yes > >> ac_cv_func_stat_empty_string_bug=no > >> enable_debug=no > >> enable_rtinst=no > >> > >> Lastly, the logtest was executed with just the "-s" parameter. It is > a > >> simple C file that I came up with. > >> > >> I took a look at the debug messages and it does not appear that new > >> threads are created via calls to wtpStartWrkr in wtp.c. > >> > >> Any help I can bring to solve this issue, please let me know. I hope > I > >> am not doing anything wrong here. > >> > >> Thanks, > >> > >> Scott > >> > > >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > > > >> wrote: > >> >> Hi Rainer, > >> >> > >> >> Thanks for your reply. Looking at the default settings (from the > >> >> online help's configuration page), they are what I wanted. The > main > >> >> messages queue is set to fix sized array with 1 worker thread > >> created > >> >> at maximum and action queues are direct mode which according to > the > >> >> queue document page, means that there will not be a worker thread > >> >> created. Is my understanding correct? If yes, how do I quickly > >> check > >> >> without using the -d option if the defaults are set correctly? Or > >> what > >> >> do I look for in the debug messages that gets printed out to > ensure > >> >> this? > >> >> > >> >> You also mentioned that version 3.18.0 is probably going to be > >> >> released as the stable version next week. I see on the webpage > > there > >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to > 3.18.0? > >> >> > >> >> Also, how come I did not get your reply in my email inbox? My > >> account > >> >> settings look correct. > >> >> > >> >> Thanks, > >> >> > >> >> Scott Phuong > >> >> > >> >> As for the syslog buffer size, that applies to syslogd and does > not > >> >> apply to rsyslog. > >> >> > >> >> > >> >> > >> >> My configuration files do not change the Action queue or Worker > >> queue > >> >> parameters at all. Looking at > >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >> >>> Hi, > >> >>> > >> >>> I have 3.16.2 which was recently released. I see that under > > certain > >> >>> conditions rsyslogd spawns a lot of threads: > >> >>> 5949 root 11216 S rsyslogd > >> >>> 5950 root 11216 S rsyslogd > >> >>> 5951 root 11216 S rsyslogd > >> >>> 5952 root 11216 S rsyslogd > >> >>> 5953 root 11216 S rsyslogd > >> >>> 5954 root 11216 S rsyslogd > >> >>> 5985 root Z [rsyslogd] > >> >>> 6445 root Z [rsyslogd] > >> >>> > >> >>> I had to kill the rsyslogd and restart it. The first invocation > > had > >> a > >> >>> pid of 219 before it had to be killed. The second invocation of > > pid > >> >>> which you see above starts with 5949. The difference is the > amount > >> of > >> >>> zombie threads that were invoked by rsyslogd before I had to > kill > >> the > >> >>> first invocation of it. > >> >> > >> >> I have no explanation yet for the zombies. They should not happen > >> and so > >> >> far I have never seen them. We may need to go through a debug log > >> (which > >> >> will become very large) to find out what's going on. > >> >> > >> >>> The question is under what conditions does rsyslogd spawn a new > >> >>> thread/process and why was it a zombie? > >> >> > >> >> Unfortunately, there is no quick answer. A quick one may be: when > > it > >> >> needs them, based on queue watermark settings and based on you > >> >> configuration. But to really understand it, you need to read this > >> doc: > >> >> > >> >> http://www.rsyslog.com/doc-queues.html > >> >> > >> >> The doc also describes all the knobs that you can use to control > >> thread > >> >> creation. There are many ;) > >> >> > >> >>> I am running rsyslogd in an > >> >>> embedded environment and not a regular laptop/desktop. > >> >> > >> >> Interesting use case... > >> >> > >> >>> In addition, I > >> >>> am using busybox and I believe the syslog buffer size is set to > >> >> > >> >> what do yo mean by "syslog buffer size"? The length of a receive > >> buffer? > >> >> It is 2K, thus single messages up to 2K are supported. It can be > >> changed > >> >> by modifying the MAXLINE define. Note that stock syslogd (and > >> RFC3164) > >> >> support only up to 1K. > >> >> > >> >>> something very low or perhaps none at all. Would this be a > factor? > >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and > both > >> >>> cases happen. > >> >> > >> >> The compatibility modes do not affect queue operation. > >> >> > >> >>> Are these issues already known and fixed in a later version? > > Sorry, > >> if > >> >>> I am asking the same questions or have the same issues as > previous > >> >>> people but without the ability to search (or at least, I don't > > know > >> >>> how to) the archive, I don't know if my problem/questions has > >> already > >> >>> been seen and/or resolved. > >> >> > >> >> If we need to find out about the zombies, we need to move on to > the > >> >> current devel version. So I would give that a try in any case. > >> 3.16.2 > >> >> will (most probably) be replaced by 3.18.0 (based on the current > >> beta) > >> >> next week. So I won't touch it any longer. > >> >> > >> >> Looking forward to your feedback, > >> >> Rainer > >> >> > >> >>> > >> >>> Thank you very much for your support. > >> >>> > >> >>> Scott > >> >>> _______________________________________________ > >> >>> rsyslog mailing list > >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > >> > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Fri Jul 4 12:38:12 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:38:12 -0700 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> Message-ID: <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> Hi Rainer, But in release mode, NDEBUG is intentionally defined in the config.h. This is what I recalled from looking at 3.16.1 and 3.17.5. So from the perspective of rsyslog, NDEBUG is defined and the opposite for the test-parse.c (because config.h is not included). Scott On Fri, Jul 4, 2008 at 3:32 AM, Rainer Gerhards wrote: > Hi Scott, > > thanks for the help, I'll further investigate. BUT: both the runtime AND > the tool are compiled in non-debug mode. So in this case, neither of > them should have the object cookie, thus the alignment should be the > same. But, as I said, I am grateful for any clue and will check if there > is some issue along these lines :) In any case, it seems to be > inconsistent and should be fixed :) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Friday, July 04, 2008 12:29 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] struct alignment problem - who can help? ;) >> >> Here is my analysis: >> In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a >> #define and is defined to obj_t objData which is the same for both >> NDEBUG defined and not defined. But obj_t is a typedef of struct >> obj_s. In the structure definition for obj_s in obj-types.h are the >> following lines: >> #ifndef NDEBUG >> unsigned int iObjCooCKiE; >> #endif >> >> NDEBUG is defined in config.h which was included in the runtime for >> rsyslog but not for test-parse.c >> >> I didn't compile the code to test my theory. I just did a quick check. >> >> Hope that solved your problem. >> >> Scott >> >> >> I think this may be why the you are getting this structure >> misalignment? >> >> >> On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards >> wrote: >> > Hi list, >> > >> > I am posting a question here in the hope that some of the > subscribers >> > may be able to lend me a helping hand. >> > >> > Recently, I have begun to add a testbench to rsyslog. The idea is >> that >> > over time the project should have canned tests which are easy to run >> on >> > each version (as part of make distcheck at latest), increasing the >> > overall code quality. >> > >> > "All" (so far two ;)) tests are located in the tests subdirectory. >> One >> > test fails if compiled in release mode, that is rscript-parse.c. I >> have >> > tracked down the failure cause and it is different struct member >> > alignment in the ctok_token_t structure. The alignment is different >> in >> > the rsyslog runtime and this test tool (I have compared the offsets >> > computed for pToken->tok and they are different). So far, so good. >> But I >> > do not find the cause of this misalignment. I checked the make >> output, >> > and as it looks both the runtime as well as the tool are compiled >> with >> > the same compiler options. Also, in debug mode it works OK, but in >> > release mode (--disable-debug) it fails. >> > >> > I am sure the problem is something quite simple, but I have run out >> of >> > ideas of what it may be (or, more probable, I simply overlook >> > something). >> > >> > Any help would be deeply appreciated. >> > >> > Thanks, >> > Rainer >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mycleanjunk at gmail.com Fri Jul 4 12:39:32 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:39:32 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> Message-ID: <17945960807040339y44e5f75cgde6b4593e5c99584@mail.gmail.com> Hi Rainer, Thanks for the information. I didn't know that. Scott On Fri, Jul 4, 2008 at 3:35 AM, Rainer Gerhards wrote: > Hi Scott, > > just for your understanding: once a release is stable, it will only > receive fixes. So the fix we are looking it needs to become part of the > devel, be migrated to beta and then be migrated to stable. In general, > one migration step takes between 4 and 12 weeks, depending on what was > done. So far, the average seems on the average, that is two month. So > two cycles mean roughly four month. HOWEVER, I intend to put it into the > next beta, which will bring us to just one cycle. So it would be > available in roughly two month. > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Friday, July 04, 2008 12:32 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog threads questions >> >> Hi Rainer, >> >> Thanks!! I'll give this a try next week and let you know the results. >> If it works, I think it is okay not to include in the upcoming v3 >> stable since it is due to release very soon. But if it does work, it >> would be great to have it in the following stable release. >> >> Scott >> >> On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards >> wrote: >> > Scott, >> > >> > I have rewritten the omusrmsg plugin. This is currently part of the >> > development tree, but will become part of the (new) beta when we do > a >> > version number shift next week. I'd appreciate if you could give it > a >> > try. It is available here: >> > >> > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz >> > >> > I am hesitant to put this in the upcoming new v3-stable. There were >> lots >> > of changes and one rule for a stable is that there should be no non- >> fix >> > type of changes for at least 4 weeks before it turns into a stable. >> It >> > of course is debatable if the change I made is a fix or not. In some >> > sense it is, but on the other hand it is a rewrite that changed a >> lot. >> > So given the fact that nobody on a "regular" machine saw an issue > the >> > past year, I would not like to bring the new code directly into the >> > stable. If that causes problems for you, you may want to try simply >> > using the omusrmsg.c code together with the v3-stable. I haven't >> tried >> > it, but it should work well. All changes were just to that file, and >> > there are no dependencies to anything external. >> > >> > OK... but now let's see first if it fixes the issue ;) >> > >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> >> Sent: Thursday, July 03, 2008 2:58 AM >> >> To: rsyslog at lists.adiscon.com >> >> Subject: Re: [rsyslog] rsyslog threads questions >> >> >> >> Hi, >> >> >> >> I've attached four files. Two of which are debug dumps, one is the >> >> conf file and the last one is a test case scenario that constantly >> >> fails on my end. I hope this gives a little more information. >> >> Furthermore, the dumps are from 3.17.5 which is the "closest" >> version >> >> to 3.18.0 that I was able to find. >> >> >> >> Both failed scenarios occur when lots of messages were being > flooded >> >> to rsyslogd at a very fast rate (look at logtest.c) The >> >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while >> >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" >> threads >> >> that it took up so much memory that executing any command as simple >> as >> >> 'ls -l' would not work from the command line. I think the number of >> >> threads grew as much as the number of messages. In the latter >> >> scenario, after killing logtest.c, it didn't look like the those >> >> zombies threads went away until I did a CTRL+C to the rsyslogd > which >> >> was running in the foreground since I use the "-dn" option. >> >> >> >> This is on an embedded system that runs significantly slower than a >> >> desktop or laptop so maybe it would be harder to reproduce on a >> >> regular computer. I looked at all the parameters that I believe >> could >> >> affect this and believe for the most part the defaults are more > than >> >> adequate. The main message queue never looked like it hit the high >> >> water mark but it did hit the lower one. So, I don't think messages >> >> were being dropped (not sure) or an overflow condition occurred. >> >> >> >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 >> and >> >> compiled using GCC and the standard GNU C libraries version 3.4.5. >> >> Rsyslog source code is cross-compiled using the following configure >> >> line: >> >> >> >> ./configure --disable-zlib --disable-largefile >> >> --enable-share=yes >> >> --prefix=/ >> >> --host=arm-unknown-linux-gnu >> >> ac_cv_func_malloc_0_nonnull=yes >> >> ac_cv_func_realloc_0_nonnull=yes >> >> >> >> ac_cv_func_lstat_dereferences_slashed_symlink=yes >> >> ac_cv_func_stat_empty_string_bug=no >> >> enable_debug=no >> >> enable_rtinst=no >> >> >> >> Lastly, the logtest was executed with just the "-s" parameter. It > is >> a >> >> simple C file that I came up with. >> >> >> >> I took a look at the debug messages and it does not appear that new >> >> threads are created via calls to wtpStartWrkr in wtp.c. >> >> >> >> Any help I can bring to solve this issue, please let me know. I > hope >> I >> >> am not doing anything wrong here. >> >> >> >> Thanks, >> >> >> >> Scott >> >> > >> >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong >> > >> >> wrote: >> >> >> Hi Rainer, >> >> >> >> >> >> Thanks for your reply. Looking at the default settings (from > the >> >> >> online help's configuration page), they are what I wanted. The >> main >> >> >> messages queue is set to fix sized array with 1 worker thread >> >> created >> >> >> at maximum and action queues are direct mode which according to >> the >> >> >> queue document page, means that there will not be a worker > thread >> >> >> created. Is my understanding correct? If yes, how do I quickly >> >> check >> >> >> without using the -d option if the defaults are set correctly? > Or >> >> what >> >> >> do I look for in the debug messages that gets printed out to >> ensure >> >> >> this? >> >> >> >> >> >> You also mentioned that version 3.18.0 is probably going to be >> >> >> released as the stable version next week. I see on the webpage >> > there >> >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to >> 3.18.0? >> >> >> >> >> >> Also, how come I did not get your reply in my email inbox? My >> >> account >> >> >> settings look correct. >> >> >> >> >> >> Thanks, >> >> >> >> >> >> Scott Phuong >> >> >> >> >> >> As for the syslog buffer size, that applies to syslogd and does >> not >> >> >> apply to rsyslog. >> >> >> >> >> >> >> >> >> >> >> >> My configuration files do not change the Action queue or Worker >> >> queue >> >> >> parameters at all. Looking at >> >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >> >> >>> Hi, >> >> >>> >> >> >>> I have 3.16.2 which was recently released. I see that under >> > certain >> >> >>> conditions rsyslogd spawns a lot of threads: >> >> >>> 5949 root 11216 S rsyslogd >> >> >>> 5950 root 11216 S rsyslogd >> >> >>> 5951 root 11216 S rsyslogd >> >> >>> 5952 root 11216 S rsyslogd >> >> >>> 5953 root 11216 S rsyslogd >> >> >>> 5954 root 11216 S rsyslogd >> >> >>> 5985 root Z [rsyslogd] >> >> >>> 6445 root Z [rsyslogd] >> >> >>> >> >> >>> I had to kill the rsyslogd and restart it. The first invocation >> > had >> >> a >> >> >>> pid of 219 before it had to be killed. The second invocation of >> > pid >> >> >>> which you see above starts with 5949. The difference is the >> amount >> >> of >> >> >>> zombie threads that were invoked by rsyslogd before I had to >> kill >> >> the >> >> >>> first invocation of it. >> >> >> >> >> >> I have no explanation yet for the zombies. They should not > happen >> >> and so >> >> >> far I have never seen them. We may need to go through a debug > log >> >> (which >> >> >> will become very large) to find out what's going on. >> >> >> >> >> >>> The question is under what conditions does rsyslogd spawn a new >> >> >>> thread/process and why was it a zombie? >> >> >> >> >> >> Unfortunately, there is no quick answer. A quick one may be: > when >> > it >> >> >> needs them, based on queue watermark settings and based on you >> >> >> configuration. But to really understand it, you need to read > this >> >> doc: >> >> >> >> >> >> http://www.rsyslog.com/doc-queues.html >> >> >> >> >> >> The doc also describes all the knobs that you can use to control >> >> thread >> >> >> creation. There are many ;) >> >> >> >> >> >>> I am running rsyslogd in an >> >> >>> embedded environment and not a regular laptop/desktop. >> >> >> >> >> >> Interesting use case... >> >> >> >> >> >>> In addition, I >> >> >>> am using busybox and I believe the syslog buffer size is set to >> >> >> >> >> >> what do yo mean by "syslog buffer size"? The length of a receive >> >> buffer? >> >> >> It is 2K, thus single messages up to 2K are supported. It can be >> >> changed >> >> >> by modifying the MAXLINE define. Note that stock syslogd (and >> >> RFC3164) >> >> >> support only up to 1K. >> >> >> >> >> >>> something very low or perhaps none at all. Would this be a >> factor? >> >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and >> both >> >> >>> cases happen. >> >> >> >> >> >> The compatibility modes do not affect queue operation. >> >> >> >> >> >>> Are these issues already known and fixed in a later version? >> > Sorry, >> >> if >> >> >>> I am asking the same questions or have the same issues as >> previous >> >> >>> people but without the ability to search (or at least, I don't >> > know >> >> >>> how to) the archive, I don't know if my problem/questions has >> >> already >> >> >>> been seen and/or resolved. >> >> >> >> >> >> If we need to find out about the zombies, we need to move on to >> the >> >> >> current devel version. So I would give that a try in any case. >> >> 3.16.2 >> >> >> will (most probably) be replaced by 3.18.0 (based on the current >> >> beta) >> >> >> next week. So I won't touch it any longer. >> >> >> >> >> >> Looking forward to your feedback, >> >> >> Rainer >> >> >> >> >> >>> >> >> >>> Thank you very much for your support. >> >> >>> >> >> >>> Scott >> >> >>> _______________________________________________ >> >> >>> rsyslog mailing list >> >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> >> >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Fri Jul 4 12:44:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:44:24 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com><17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933D@grfint2.intern.adiscon.com> ROFL... you got it! I have forgotten to #include "config.h"! I knew that it must have been something simple. Thanks a lot, you saved me from pulling out the rest of my hair ;) Rainer PS: and, yes, it looks like I need to reevaluate mode-specific structure members now that we aim at a real runtime. Doesn't look so smart any longer... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > > Hi Rainer, > > But in release mode, NDEBUG is intentionally defined in the config.h. > This is what I recalled from looking at 3.16.1 and 3.17.5. So from the > perspective of rsyslog, NDEBUG is defined and the opposite for the > test-parse.c (because config.h is not included). > > Scott > > On Fri, Jul 4, 2008 at 3:32 AM, Rainer Gerhards > wrote: > > Hi Scott, > > > > thanks for the help, I'll further investigate. BUT: both the runtime > AND > > the tool are compiled in non-debug mode. So in this case, neither of > > them should have the object cookie, thus the alignment should be the > > same. But, as I said, I am grateful for any clue and will check if > there > > is some issue along these lines :) In any case, it seems to be > > inconsistent and should be fixed :) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong > >> Sent: Friday, July 04, 2008 12:29 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > >> > >> Here is my analysis: > >> In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a > >> #define and is defined to obj_t objData which is the same for both > >> NDEBUG defined and not defined. But obj_t is a typedef of struct > >> obj_s. In the structure definition for obj_s in obj-types.h are the > >> following lines: > >> #ifndef NDEBUG > >> unsigned int iObjCooCKiE; > >> #endif > >> > >> NDEBUG is defined in config.h which was included in the runtime for > >> rsyslog but not for test-parse.c > >> > >> I didn't compile the code to test my theory. I just did a quick > check. > >> > >> Hope that solved your problem. > >> > >> Scott > >> > >> > >> I think this may be why the you are getting this structure > >> misalignment? > >> > >> > >> On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards > >> wrote: > >> > Hi list, > >> > > >> > I am posting a question here in the hope that some of the > > subscribers > >> > may be able to lend me a helping hand. > >> > > >> > Recently, I have begun to add a testbench to rsyslog. The idea is > >> that > >> > over time the project should have canned tests which are easy to > run > >> on > >> > each version (as part of make distcheck at latest), increasing the > >> > overall code quality. > >> > > >> > "All" (so far two ;)) tests are located in the tests subdirectory. > >> One > >> > test fails if compiled in release mode, that is rscript-parse.c. I > >> have > >> > tracked down the failure cause and it is different struct member > >> > alignment in the ctok_token_t structure. The alignment is > different > >> in > >> > the rsyslog runtime and this test tool (I have compared the > offsets > >> > computed for pToken->tok and they are different). So far, so good. > >> But I > >> > do not find the cause of this misalignment. I checked the make > >> output, > >> > and as it looks both the runtime as well as the tool are compiled > >> with > >> > the same compiler options. Also, in debug mode it works OK, but in > >> > release mode (--disable-debug) it fails. > >> > > >> > I am sure the problem is something quite simple, but I have run > out > >> of > >> > ideas of what it may be (or, more probable, I simply overlook > >> > something). > >> > > >> > Any help would be deeply appreciated. > >> > > >> > Thanks, > >> > Rainer > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From friedl at hq.adiscon.com Mon Jul 7 16:02:21 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 7 Jul 2008 16:02:21 +0200 Subject: [rsyslog] rsyslog 3.19.9 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED69@grfint2.intern.adiscon.com> Hi all, Rsyslog 3.19.9, a member of the development branch, has been released. Documentation on how to setup TLS syslog has been greatly improved. Also, omusrmsg has been rewritten to utilize rsyslog's threading instead of forking of children to send the messages. This is expected to also solve some issues with zombie processes. Finally, an issue with TLS anonymous mode, which still required client certificates, and a problem with the RainerScript parser, which did not detect every syntax error, was solved. This is a recommended update for all development branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-117.phtml Changelog: http://www.rsyslog.com/Article250.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rfujita at redhat.com Wed Jul 9 10:34:27 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 17:34:27 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. Message-ID: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> Hi List, $subject And I guess that documents are managed with Wiki. Do you have a plan to do i10n/i18n with rsyslog? Best Rio. ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 10:46:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 9 Jul 2008 10:46:34 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> Hi Rio-san, many thanks for your help! Much appreciated. As far as the documentation goes, there are actually two places. One is the wiki ( http://wiki.rsyslog.com ) , but then there is also a set (called "the core set") of html documentation. The reasoning is as follows: The wiki is a great resource and easy to edit. However, it does not provide the versioning we need. For example, there are changes in the way things can be configured between v2 and v3. In the wiki, version-specifc pages are a bit hard to find. Also, the wiki is only available online. To solve this, we have the core set: html files which document rsyslog's config directives as well as generic use cases. This is contained in git and under version control. An exactly matching version of the documentation is distributed with each source tarball. I think it would be very useful to have a Japanese version of (at least some) documents of the core set. If you are interested in that, I would add your translations to git and the distribution set. > Do you have a plan to do i10n/i18n with rsyslog? Could you elaborate a little bit on this? Unfortunately, I do neither speak Japanse nor any other language with a non-western script. I asked around several times and some folks from Japan told me that rsyslog works well with Japanese characters. But I am always interested in enhancing this support - I just need some guidance what is needed. Looking forward to your relpy, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Wednesday, July 09, 2008 10:34 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] I'd like to translate documents into Japanese. > > Hi List, > > $subject > And I guess that documents are managed with Wiki. > Do you have a plan to do i10n/i18n with rsyslog? > > Best Rio. > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Wed Jul 9 11:33:29 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 18:33:29 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> Message-ID: <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> Hi Rainer-san, Thank you for your quick response! Too quick, I was surprised. I understand the status of the project and what I should do. #The reason why I asked how the contributors collaborate for rsyslog was that I cloned the repository with git and I couldn't find any docbook or .po files. As soon as possible, I'll start to translate docs into Japanese. BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. But I can't find the source file of this version in the git tree and Wiki. Why? Best Rio. On 2008/07/09, at 17:46, Rainer Gerhards wrote: > Hi Rio-san, > > many thanks for your help! Much appreciated. > > As far as the documentation goes, there are actually two places. One > is > the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > (called "the core set") of html documentation. The reasoning is as > follows: The wiki is a great resource and easy to edit. However, it > does > not provide the versioning we need. For example, there are changes in > the way things can be configured between v2 and v3. In the wiki, > version-specifc pages are a bit hard to find. Also, the wiki is only > available online. > > To solve this, we have the core set: html files which document > rsyslog's > config directives as well as generic use cases. This is contained in > git > and under version control. An exactly matching version of the > documentation is distributed with each source tarball. > > I think it would be very useful to have a Japanese version of (at > least > some) documents of the core set. If you are interested in that, I > would > add your translations to git and the distribution set. > >> Do you have a plan to do i10n/i18n with rsyslog? > > Could you elaborate a little bit on this? Unfortunately, I do neither > speak Japanse nor any other language with a non-western script. I > asked > around several times and some folks from Japan told me that rsyslog > works well with Japanese characters. But I am always interested in > enhancing this support - I just need some guidance what is needed. > > Looking forward to your relpy, > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Wednesday, July 09, 2008 10:34 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] I'd like to translate documents into Japanese. >> >> Hi List, >> >> $subject >> And I guess that documents are managed with Wiki. >> Do you have a plan to do i10n/i18n with rsyslog? >> >> Best Rio. >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 11:43:17 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 11:43:17 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> Message-ID: <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: > Hi Rainer-san, > > Thank you for your quick response! > Too quick, I was surprised. I am trying my best - doesn't always work :) > > I understand the status of the project and what I should do. > > #The reason why I asked how the contributors collaborate for rsyslog > was that I cloned the repository with git and I couldn't find any > docbook or .po files. The doc set is in the ./doc subfolder, with .html files (obviously not what you looked for :)). I think it would be a good time now to create a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the files and I would integrate them as soon as I have them availalbe. > > As soon as possible, I'll start to translate docs into Japanese. > > BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. > But I can't find the source file of this version in the git tree and > Wiki. > Why? I restructured the directory structure a bit in 3.19. It is now in the ./tools directory, because it belongs to the rsyslogd tool. Here is a quick link: http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD All the best, Rainer > > Best Rio. > > On 2008/07/09, at 17:46, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > many thanks for your help! Much appreciated. > > > > As far as the documentation goes, there are actually two places. One > > is > > the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > > (called "the core set") of html documentation. The reasoning is as > > follows: The wiki is a great resource and easy to edit. However, it > > does > > not provide the versioning we need. For example, there are changes in > > the way things can be configured between v2 and v3. In the wiki, > > version-specifc pages are a bit hard to find. Also, the wiki is only > > available online. > > > > To solve this, we have the core set: html files which document > > rsyslog's > > config directives as well as generic use cases. This is contained in > > git > > and under version control. An exactly matching version of the > > documentation is distributed with each source tarball. > > > > I think it would be very useful to have a Japanese version of (at > > least > > some) documents of the core set. If you are interested in that, I > > would > > add your translations to git and the distribution set. > > > >> Do you have a plan to do i10n/i18n with rsyslog? > > > > Could you elaborate a little bit on this? Unfortunately, I do neither > > speak Japanse nor any other language with a non-western script. I > > asked > > around several times and some folks from Japan told me that rsyslog > > works well with Japanese characters. But I am always interested in > > enhancing this support - I just need some guidance what is needed. > > > > Looking forward to your relpy, > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Wednesday, July 09, 2008 10:34 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] I'd like to translate documents into Japanese. > >> > >> Hi List, > >> > >> $subject > >> And I guess that documents are managed with Wiki. > >> Do you have a plan to do i10n/i18n with rsyslog? > >> > >> Best Rio. > >> > >> > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Wed Jul 9 11:59:48 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 18:59:48 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Message-ID: Hi Rainer-san, On 2008/07/09, at 18:43, Rainer Gerhards wrote: > I am trying my best - doesn't always work :) I'm working for Red Hat's Tokyo office. Do you live in Germany? If so, we can co-work in the daytime! > The doc set is in the ./doc subfolder, with .html files (obviously not > what you looked for :)). I think it would be a good time now to create > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > files and I would integrate them as soon as I have them availalbe. I agree with you totally. Because Fedora 9 has included rsyslog, lots Japanese and other non English/German speakers have started using it. Sooner or later, we'll need to create ./doc/other_lang directories :) > I restructured the directory structure a bit in 3.19. It is now in > the ./tools directory, because it belongs to the rsyslogd tool. > > Here is a quick link: > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD Oh, I find the file rsyslogd.8 in tools directory. I'll start translating it first. Best Rio. On 2008/07/09, at 18:43, Rainer Gerhards wrote: > Hi Rio-san, > > On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: >> Hi Rainer-san, >> >> Thank you for your quick response! >> Too quick, I was surprised. > > I am trying my best - doesn't always work :) > >> >> I understand the status of the project and what I should do. >> >> #The reason why I asked how the contributors collaborate for rsyslog >> was that I cloned the repository with git and I couldn't find any >> docbook or .po files. > > The doc set is in the ./doc subfolder, with .html files (obviously not > what you looked for :)). I think it would be a good time now to create > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > files and I would integrate them as soon as I have them availalbe. >> >> As soon as possible, I'll start to translate docs into Japanese. >> >> BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. >> But I can't find the source file of this version in the git tree and >> Wiki. >> Why? > > I restructured the directory structure a bit in 3.19. It is now in > the ./tools directory, because it belongs to the rsyslogd tool. > > Here is a quick link: > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > All the best, > Rainer >> >> Best Rio. >> >> On 2008/07/09, at 17:46, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> many thanks for your help! Much appreciated. >>> >>> As far as the documentation goes, there are actually two places. One >>> is >>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a set >>> (called "the core set") of html documentation. The reasoning is as >>> follows: The wiki is a great resource and easy to edit. However, it >>> does >>> not provide the versioning we need. For example, there are changes >>> in >>> the way things can be configured between v2 and v3. In the wiki, >>> version-specifc pages are a bit hard to find. Also, the wiki is only >>> available online. >>> >>> To solve this, we have the core set: html files which document >>> rsyslog's >>> config directives as well as generic use cases. This is contained in >>> git >>> and under version control. An exactly matching version of the >>> documentation is distributed with each source tarball. >>> >>> I think it would be very useful to have a Japanese version of (at >>> least >>> some) documents of the core set. If you are interested in that, I >>> would >>> add your translations to git and the distribution set. >>> >>>> Do you have a plan to do i10n/i18n with rsyslog? >>> >>> Could you elaborate a little bit on this? Unfortunately, I do >>> neither >>> speak Japanse nor any other language with a non-western script. I >>> asked >>> around several times and some folks from Japan told me that rsyslog >>> works well with Japanese characters. But I am always interested in >>> enhancing this support - I just need some guidance what is needed. >>> >>> Looking forward to your relpy, >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Wednesday, July 09, 2008 10:34 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] I'd like to translate documents into Japanese. >>>> >>>> Hi List, >>>> >>>> $subject >>>> And I guess that documents are managed with Wiki. >>>> Do you have a plan to do i10n/i18n with rsyslog? >>>> >>>> Best Rio. >>>> >>>> >>> ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> ######################################################################## >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> ######################################################################## >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 12:06:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 12:06:11 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Message-ID: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Wed, 2008-07-09 at 18:59 +0900, Ryo Fujita wrote: > Hi Rainer-san, > > On 2008/07/09, at 18:43, Rainer Gerhards wrote: > > > I am trying my best - doesn't always work :) > > > I'm working for Red Hat's Tokyo office. > Do you live in Germany? > If so, we can co-work in the daytime! indeed, I do. So we can at least try. > > > The doc set is in the ./doc subfolder, with .html files (obviously not > > what you looked for :)). I think it would be a good time now to create > > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > > files and I would integrate them as soon as I have them availalbe. > > I agree with you totally. > Because Fedora 9 has included rsyslog, > lots Japanese and other non English/German speakers have started using > it. > Sooner or later, we'll need to create ./doc/other_lang directories :) I'll begin this when I start the next devel branch. I scheduled it for this week, but I got a bug report for the beta, which I would like to fix first. But in any case... soon! > > > I restructured the directory structure a bit in 3.19. It is now in > > the ./tools directory, because it belongs to the rsyslogd tool. > > > > Here is a quick link: > > > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > Oh, I find the file rsyslogd.8 in tools directory. > I'll start translating it first. Excellent. If you go through ./doc, you'll also notice that there is room for improvement. I think the structure is not very good today. Suggestions are welcome :) Rainer > > Best Rio. > > On 2008/07/09, at 18:43, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: > >> Hi Rainer-san, > >> > >> Thank you for your quick response! > >> Too quick, I was surprised. > > > > I am trying my best - doesn't always work :) > > > >> > >> I understand the status of the project and what I should do. > >> > >> #The reason why I asked how the contributors collaborate for rsyslog > >> was that I cloned the repository with git and I couldn't find any > >> docbook or .po files. > > > > The doc set is in the ./doc subfolder, with .html files (obviously not > > what you looked for :)). I think it would be a good time now to create > > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > > files and I would integrate them as soon as I have them availalbe. > >> > >> As soon as possible, I'll start to translate docs into Japanese. > >> > >> BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. > >> But I can't find the source file of this version in the git tree and > >> Wiki. > >> Why? > > > > I restructured the directory structure a bit in 3.19. It is now in > > the ./tools directory, because it belongs to the rsyslogd tool. > > > > Here is a quick link: > > > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > > > > All the best, > > Rainer > >> > >> Best Rio. > >> > >> On 2008/07/09, at 17:46, Rainer Gerhards wrote: > >> > >>> Hi Rio-san, > >>> > >>> many thanks for your help! Much appreciated. > >>> > >>> As far as the documentation goes, there are actually two places. One > >>> is > >>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > >>> (called "the core set") of html documentation. The reasoning is as > >>> follows: The wiki is a great resource and easy to edit. However, it > >>> does > >>> not provide the versioning we need. For example, there are changes > >>> in > >>> the way things can be configured between v2 and v3. In the wiki, > >>> version-specifc pages are a bit hard to find. Also, the wiki is only > >>> available online. > >>> > >>> To solve this, we have the core set: html files which document > >>> rsyslog's > >>> config directives as well as generic use cases. This is contained in > >>> git > >>> and under version control. An exactly matching version of the > >>> documentation is distributed with each source tarball. > >>> > >>> I think it would be very useful to have a Japanese version of (at > >>> least > >>> some) documents of the core set. If you are interested in that, I > >>> would > >>> add your translations to git and the distribution set. > >>> > >>>> Do you have a plan to do i10n/i18n with rsyslog? > >>> > >>> Could you elaborate a little bit on this? Unfortunately, I do > >>> neither > >>> speak Japanse nor any other language with a non-western script. I > >>> asked > >>> around several times and some folks from Japan told me that rsyslog > >>> works well with Japanese characters. But I am always interested in > >>> enhancing this support - I just need some guidance what is needed. > >>> > >>> Looking forward to your relpy, > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>>> Sent: Wednesday, July 09, 2008 10:34 AM > >>>> To: rsyslog at lists.adiscon.com > >>>> Subject: [rsyslog] I'd like to translate documents into Japanese. > >>>> > >>>> Hi List, > >>>> > >>>> $subject > >>>> And I guess that documents are managed with Wiki. > >>>> Do you have a plan to do i10n/i18n with rsyslog? > >>>> > >>>> Best Rio. > >>>> > >>>> > >>> ####################################################################### > >>>> # > >>>> Ryo Fujita > >>>> Senior Solution Architect, RHCE > >>>> Red Hat K.K. > >>>> TEL +81-3-5798-8500 > >>>> FAX +81-3-5798-8599 > >>>> Ebisu Neonato 8F > >>>> 4-1-18 Ebisu, Shibuya-ku, > >>>> Tokyo Japan 1500013 > >>>> > >>> ####################################################################### > >>>> # > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> ######################################################################## > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> ######################################################################## > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 9 12:32:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 12:32:38 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Message-ID: <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> Hi Rio-san and others, one question comes up on my mind: > > Oh, I find the file rsyslogd.8 in tools directory. > > I'll start translating it first. The man file is inside the tools directory, but not at all language specific. Do you think it makes sense to create ./tools/mans// (with lan being the ISO 2-char language code) Or is there any other suggestion? Feedback appreciated, Rainer From rfujita at redhat.com Wed Jul 9 17:10:46 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 00:10:46 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Message-ID: <4FD7940E-8E4F-41BB-AC31-578082706F55@redhat.com> Hi Rainer-san, > I'll begin this when I start the next devel branch. I scheduled it for > this week, but I got a bug report for the beta, which I would like to > fix first. But in any case... soon! It's a good timing. I need some time to investigate the best way to include translation stuff into the tree. > Excellent. If you go through ./doc, you'll also notice that there is > room for improvement. I think the structure is not very good today. > Suggestions are welcome :) My colleagues are the best to ask how to restructure the tree in order to translate docs. And one of my colleagues, Noriko Mizumoto-san is the editor of fedora- trans ja project. http://docs.fedoraproject.org/translation-quick-start-guide/ja_JP/index.html I'll ask her tomorrow. Best Rio. On 2008/07/09, at 19:06, Rainer Gerhards wrote: > Hi Rio-san, > > On Wed, 2008-07-09 at 18:59 +0900, Ryo Fujita wrote: >> Hi Rainer-san, >> >> On 2008/07/09, at 18:43, Rainer Gerhards wrote: >> >>> I am trying my best - doesn't always work :) >> >> >> I'm working for Red Hat's Tokyo office. >> Do you live in Germany? >> If so, we can co-work in the daytime! > > indeed, I do. So we can at least try. > >> >>> The doc set is in the ./doc subfolder, with .html files (obviously >>> not >>> what you looked for :)). I think it would be a good time now to >>> create >>> a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over >>> the >>> files and I would integrate them as soon as I have them availalbe. >> >> I agree with you totally. >> Because Fedora 9 has included rsyslog, >> lots Japanese and other non English/German speakers have started >> using >> it. >> Sooner or later, we'll need to create ./doc/other_lang directories :) > > I'll begin this when I start the next devel branch. I scheduled it for > this week, but I got a bug report for the beta, which I would like to > fix first. But in any case... soon! > >> >>> I restructured the directory structure a bit in 3.19. It is now in >>> the ./tools directory, because it belongs to the rsyslogd tool. >>> >>> Here is a quick link: >>> >>> http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD >> >> >> Oh, I find the file rsyslogd.8 in tools directory. >> I'll start translating it first. > > Excellent. If you go through ./doc, you'll also notice that there is > room for improvement. I think the structure is not very good today. > Suggestions are welcome :) > > Rainer >> >> Best Rio. >> >> On 2008/07/09, at 18:43, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: >>>> Hi Rainer-san, >>>> >>>> Thank you for your quick response! >>>> Too quick, I was surprised. >>> >>> I am trying my best - doesn't always work :) >>> >>>> >>>> I understand the status of the project and what I should do. >>>> >>>> #The reason why I asked how the contributors collaborate for >>>> rsyslog >>>> was that I cloned the repository with git and I couldn't find any >>>> docbook or .po files. >>> >>> The doc set is in the ./doc subfolder, with .html files (obviously >>> not >>> what you looked for :)). I think it would be a good time now to >>> create >>> a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over >>> the >>> files and I would integrate them as soon as I have them availalbe. >>>> >>>> As soon as possible, I'll start to translate docs into Japanese. >>>> >>>> BTW, Fedora 9 has the man page, rsyslogd(8) with the version >>>> 3.14.0. >>>> But I can't find the source file of this version in the git tree >>>> and >>>> Wiki. >>>> Why? >>> >>> I restructured the directory structure a bit in 3.19. It is now in >>> the ./tools directory, because it belongs to the rsyslogd tool. >>> >>> Here is a quick link: >>> >>> http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD >>> >>> >>> All the best, >>> Rainer >>>> >>>> Best Rio. >>>> >>>> On 2008/07/09, at 17:46, Rainer Gerhards wrote: >>>> >>>>> Hi Rio-san, >>>>> >>>>> many thanks for your help! Much appreciated. >>>>> >>>>> As far as the documentation goes, there are actually two places. >>>>> One >>>>> is >>>>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a >>>>> set >>>>> (called "the core set") of html documentation. The reasoning is as >>>>> follows: The wiki is a great resource and easy to edit. However, >>>>> it >>>>> does >>>>> not provide the versioning we need. For example, there are changes >>>>> in >>>>> the way things can be configured between v2 and v3. In the wiki, >>>>> version-specifc pages are a bit hard to find. Also, the wiki is >>>>> only >>>>> available online. >>>>> >>>>> To solve this, we have the core set: html files which document >>>>> rsyslog's >>>>> config directives as well as generic use cases. This is >>>>> contained in >>>>> git >>>>> and under version control. An exactly matching version of the >>>>> documentation is distributed with each source tarball. >>>>> >>>>> I think it would be very useful to have a Japanese version of (at >>>>> least >>>>> some) documents of the core set. If you are interested in that, I >>>>> would >>>>> add your translations to git and the distribution set. >>>>> >>>>>> Do you have a plan to do i10n/i18n with rsyslog? >>>>> >>>>> Could you elaborate a little bit on this? Unfortunately, I do >>>>> neither >>>>> speak Japanse nor any other language with a non-western script. I >>>>> asked >>>>> around several times and some folks from Japan told me that >>>>> rsyslog >>>>> works well with Japanese characters. But I am always interested in >>>>> enhancing this support - I just need some guidance what is needed. >>>>> >>>>> Looking forward to your relpy, >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>>> Sent: Wednesday, July 09, 2008 10:34 AM >>>>>> To: rsyslog at lists.adiscon.com >>>>>> Subject: [rsyslog] I'd like to translate documents into Japanese. >>>>>> >>>>>> Hi List, >>>>>> >>>>>> $subject >>>>>> And I guess that documents are managed with Wiki. >>>>>> Do you have a plan to do i10n/i18n with rsyslog? >>>>>> >>>>>> Best Rio. >>>>>> >>>>>> >>>>> ####################################################################### >>>>>> # >>>>>> Ryo Fujita >>>>>> Senior Solution Architect, RHCE >>>>>> Red Hat K.K. >>>>>> TEL +81-3-5798-8500 >>>>>> FAX +81-3-5798-8599 >>>>>> Ebisu Neonato 8F >>>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>>> Tokyo Japan 1500013 >>>>>> >>>>> ####################################################################### >>>>>> # >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> ######################################################################## >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> ######################################################################## >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> ######################################################################## >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> ######################################################################## >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rfujita at redhat.com Wed Jul 9 17:15:01 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 00:15:01 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> Message-ID: <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> Hi List, +1 But I'll ask it to my colleagues who contribute to translation matter tomorrow. I'm sure that they know the smartest way to do it. Best Rio. On 2008/07/09, at 19:32, Rainer Gerhards wrote: > Hi Rio-san and others, > > one question comes up on my mind: > >>> Oh, I find the file rsyslogd.8 in tools directory. >>> I'll start translating it first. > > The man file is inside the tools directory, but not at all language > specific. Do you think it makes sense to create > > ./tools/mans// (with lan being the ISO 2-char language code) > > Or is there any other suggestion? > > Feedback appreciated, > Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 17:16:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 9 Jul 2008 17:16:01 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com><5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com><1215596597.7184.29.camel@rgf9dev.intern.adiscon.com><1215597971.7184.33.camel@rgf9dev.intern.adiscon.com><1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED92@grfint2.intern.adiscon.com> Thanks, I am standing by for more votes and the insight! :) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Wednesday, July 09, 2008 5:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] I'd like to translate documents into Japanese. > > Hi List, > > +1 > > But I'll ask it to my colleagues who contribute to translation matter > tomorrow. > I'm sure that they know the smartest way to do it. > > Best Rio. > > On 2008/07/09, at 19:32, Rainer Gerhards wrote: > > > Hi Rio-san and others, > > > > one question comes up on my mind: > > > >>> Oh, I find the file rsyslogd.8 in tools directory. > >>> I'll start translating it first. > > > > The man file is inside the tools directory, but not at all language > > specific. Do you think it makes sense to create > > > > ./tools/mans// (with lan being the ISO 2-char language code) > > > > Or is there any other suggestion? > > > > Feedback appreciated, > > Rainer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Thu Jul 10 08:51:21 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 15:51:21 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure Message-ID: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> Hi List, I consulted with my colleagues about the smartest way to reconstruct the documents of rsyslog. 1.Work Flow matter The attached image is just an idea for rational flowchart to make translation easier. #If this ML forbids me to attach any files, you can see it on my site. http://rio.tc/2008/07/10-144007.php I'm worried that this flow will give the contributers much trouble. - HTML(from Wiki) to DocBook conversion 'tidy' will help us a little, but exporter of media Wiki is poor to export 'well-formed HTML'. We need to edit and check them manually. Of course, I willingly do this. But it may take a number of days. - Writers need to edit DocBook XML after the reconstruction. As you know, HTML format is not good for a source of multi-output. DocBook XML is a perfect one except for edit :) - Automake related matter There can be two streams to generate man and html from DocBook XML. One : When you edit DocBook, you commit DocBook, HTML and man files to git tree. Of course this method requires us to prepare an environment where you can use xsltproc/docbook2man. Two : By adding some sequences into Makefile, HTML and man files can be generated automatically. All persons who want to build rsyslog from tar ball need to prepare the environment. And we need to put some check routines for dependencies into Makefile. 2.git tree reconstruct only with 'doc' directory `-- doc |-- Makefile.am |-- conf | |-- en | | `-- rsyslog-example.conf | |-- OTHER_LANGUAGES(iso code) | `-- jp | `-- rsyslog-example.conf # annotations are translated. |-- html | |-- en | | |-- bugs.html | | |-- OTHER_HTMLS | | `-- version_naming.html | |-- OTHER_LANGUAGES(iso code) | `-- jp | |-- bugs.html | |-- OTHER_HTMLS | `-- version_naming.html |-- images | |-- gssapi.png | |-- OTHER_IMAGES | `-- tls_cert_ca.jpg |-- man | |-- en | | |-- man5 | | | `-- rsyslog.conf.5 | | `-- man8 | | `-- rsyslogd.8 | |-- OTHER_LANGUAGES(iso code) | `-- jp | |-- man5 | | `-- rsyslog.conf.5 | `-- man8 | `-- rsyslogd.8 `-- src |-- dias | |-- classes.dia | |-- OTHER_DIA_FILES | `-- tls_cert_ca.dia `-- docbook |-- en | |-- bugs.xml | |-- rsyslog.conf.5.xml | |-- rsyslogd.8.xml | |-- OTHER_XMLS | `-- version_naming.xml `-- ja |-- bugs.html |-- rsyslog.conf.5.xml |-- rsyslogd.8.xml |-- OTHER_XMLS `-- version_naming.html # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ directory. Your suggestions will be highly appreciated!! Best Rio. ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Thu Jul 10 09:03:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 10 Jul 2008 09:03:18 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> Hi Rio-san, I need to leave soon, but a quick feedback: the proposal looks *very* interesting. I need to digest it a bit. I have no experience with DocBook, so I probably need to check that out, first. That may take a short while (should you happen to know a good "getting started guide", I'd grateful if you send me a link ;)). From first thought, I would prefer to generate the html et al outputs from a single source (where only that source is in git). Feedback from other list members is also appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Thursday, July 10, 2008 8:51 AM > To: rsyslog-users > Subject: [rsyslog] Suggestion for Documentation Reconstructure > > Hi List, > > I consulted with my colleagues about the smartest way to reconstruct > the documents of rsyslog. > > 1.Work Flow matter > The attached image is just an idea for rational flowchart to make > translation easier. > #If this ML forbids me to attach any files, you can see it on my site. > http://rio.tc/2008/07/10-144007.php > > I'm worried that this flow will give the contributers much trouble. > > - HTML(from Wiki) to DocBook conversion > 'tidy' will help us a little, but exporter of media Wiki is poor to > export 'well-formed HTML'. > We need to edit and check them manually. > Of course, I willingly do this. But it may take a number of days. > > - Writers need to edit DocBook XML after the reconstruction. > As you know, HTML format is not good for a source of multi-output. > DocBook XML is a perfect one except for edit :) > > - Automake related matter > There can be two streams to generate man and html from DocBook XML. > One : When you edit DocBook, you commit DocBook, HTML and man files > to git tree. > Of course this method requires us to prepare an > environment where you can use xsltproc/docbook2man. > Two : By adding some sequences into Makefile, HTML and man files > can be generated automatically. > All persons who want to build rsyslog from tar ball need > to prepare the environment. > And we need to put some check routines for dependencies > into Makefile. > > 2.git tree reconstruct only with 'doc' directory > > `-- doc > |-- Makefile.am > |-- conf > | |-- en > | | `-- rsyslog-example.conf > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | `-- rsyslog-example.conf # annotations are translated. > |-- html > | |-- en > | | |-- bugs.html > | | |-- OTHER_HTMLS > | | `-- version_naming.html > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | |-- bugs.html > | |-- OTHER_HTMLS > | `-- version_naming.html > |-- images > | |-- gssapi.png > | |-- OTHER_IMAGES > | `-- tls_cert_ca.jpg > |-- man > | |-- en > | | |-- man5 > | | | `-- rsyslog.conf.5 > | | `-- man8 > | | `-- rsyslogd.8 > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | |-- man5 > | | `-- rsyslog.conf.5 > | `-- man8 > | `-- rsyslogd.8 > `-- src > |-- dias > | |-- classes.dia > | |-- OTHER_DIA_FILES > | `-- tls_cert_ca.dia > `-- docbook > |-- en > | |-- bugs.xml > | |-- rsyslog.conf.5.xml > | |-- rsyslogd.8.xml > | |-- OTHER_XMLS > | `-- version_naming.xml > `-- ja > |-- bugs.html > |-- rsyslog.conf.5.xml > |-- rsyslogd.8.xml > |-- OTHER_XMLS > `-- version_naming.html > > # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ > directory. > > Your suggestions will be highly appreciated!! > > Best Rio. > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # From rfujita at redhat.com Thu Jul 10 09:39:37 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 16:39:37 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> Message-ID: <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Hi Rainer-san, DocBook.org is here. http://www.docbook.org/ And IBM has a good guide for beginners. http://www.ibm.com/developerworks/xml/library/xml-matters3.html Recently, Red Hat uses DocBook format to generate web pages ,PDFs and so on. For example, https://www.redhat.com/docs/manuals/enterprise/ A document having both html and PDF is generated from DocBook. gnome-doc-utils and kdesdk-utils include very convenient tools to handle DocBook and related formats. Best Rio. On 2008/07/10, at 16:03, Rainer Gerhards wrote: > Hi Rio-san, > > I need to leave soon, but a quick feedback: the proposal looks *very* > interesting. I need to digest it a bit. I have no experience with > DocBook, so I probably need to check that out, first. That may take a > short while (should you happen to know a good "getting started guide", > I'd grateful if you send me a link ;)). From first thought, I would > prefer to generate the html et al outputs from a single source (where > only that source is in git). > > Feedback from other list members is also appreciated. > > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Thursday, July 10, 2008 8:51 AM >> To: rsyslog-users >> Subject: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi List, >> >> I consulted with my colleagues about the smartest way to reconstruct >> the documents of rsyslog. >> >> 1.Work Flow matter >> The attached image is just an idea for rational flowchart to make >> translation easier. >> #If this ML forbids me to attach any files, you can see it on my >> site. >> http://rio.tc/2008/07/10-144007.php >> >> I'm worried that this flow will give the contributers much trouble. >> >> - HTML(from Wiki) to DocBook conversion >> 'tidy' will help us a little, but exporter of media Wiki is poor to >> export 'well-formed HTML'. >> We need to edit and check them manually. >> Of course, I willingly do this. But it may take a number of days. >> >> - Writers need to edit DocBook XML after the reconstruction. >> As you know, HTML format is not good for a source of multi-output. >> DocBook XML is a perfect one except for edit :) >> >> - Automake related matter >> There can be two streams to generate man and html from DocBook XML. >> One : When you edit DocBook, you commit DocBook, HTML and man files >> to git tree. >> Of course this method requires us to prepare an >> environment where you can use xsltproc/docbook2man. >> Two : By adding some sequences into Makefile, HTML and man files >> can be generated automatically. >> All persons who want to build rsyslog from tar ball need >> to prepare the environment. >> And we need to put some check routines for dependencies >> into Makefile. >> >> 2.git tree reconstruct only with 'doc' directory >> >> `-- doc >> |-- Makefile.am >> |-- conf >> | |-- en >> | | `-- rsyslog-example.conf >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | `-- rsyslog-example.conf # annotations are translated. >> |-- html >> | |-- en >> | | |-- bugs.html >> | | |-- OTHER_HTMLS >> | | `-- version_naming.html >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | |-- bugs.html >> | |-- OTHER_HTMLS >> | `-- version_naming.html >> |-- images >> | |-- gssapi.png >> | |-- OTHER_IMAGES >> | `-- tls_cert_ca.jpg >> |-- man >> | |-- en >> | | |-- man5 >> | | | `-- rsyslog.conf.5 >> | | `-- man8 >> | | `-- rsyslogd.8 >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | |-- man5 >> | | `-- rsyslog.conf.5 >> | `-- man8 >> | `-- rsyslogd.8 >> `-- src >> |-- dias >> | |-- classes.dia >> | |-- OTHER_DIA_FILES >> | `-- tls_cert_ca.dia >> `-- docbook >> |-- en >> | |-- bugs.xml >> | |-- rsyslog.conf.5.xml >> | |-- rsyslogd.8.xml >> | |-- OTHER_XMLS >> | `-- version_naming.xml >> `-- ja >> |-- bugs.html >> |-- rsyslog.conf.5.xml >> |-- rsyslogd.8.xml >> |-- OTHER_XMLS >> `-- version_naming.html >> >> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ >> directory. >> >> Your suggestions will be highly appreciated!! >> >> Best Rio. >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 11 16:35:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:35:59 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. It includes all features and bug-fixes from the beta, among others the ability to natively send email. There is one minor bugfix for 3.17.5 included, where RainerScript did not always properly detect syntax errors. This is a recommended update for all v3-stable branch users. Change Log: http://www.rsyslog.com/Article254.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml Please note that I also created a 3.16.3 branch, immediately outdated, for those that prefer to keep the previous version for a couple of days. That version contains an important bugfix which prevents a big memory leak when running the queue in one of the disk modes. This patch is of course also included in 3.18.0. Find 3.16.3 here: http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml Just to make sure I am not misunderstood: 3.18.0 is the current v3-stable. Other v3-stables are not officially supported (except, of course, via a commercial support contract). 3.16.3 will not receive any patches in the future. So I suggest moving over to 3.18.0. It has been tested for over two month and looks quite ready for prime time. I would also like to express my sincere thanks to all of you who provided bug reports and patches. This is immensely helpful. I hope the release is useful. As always, feedback is appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Fri Jul 11 16:42:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:42:04 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> HI all, Michael Biebl just told me that some html files are missing in the 3.18.0 tarball. Please hold download, I am updating it right now... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, July 11, 2008 4:36 PM > To: rsyslog-users > Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > Hi all, > > we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. > It includes all features and bug-fixes from the beta, among others the > ability to natively send email. There is one minor bugfix for 3.17.5 > included, where RainerScript did not always properly detect syntax > errors. This is a recommended update for all v3-stable branch users. > > Change Log: > > http://www.rsyslog.com/Article254.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > Please note that I also created a 3.16.3 branch, immediately outdated, > for those that prefer to keep the previous version for a couple of > days. > That version contains an important bugfix which prevents a big memory > leak when running the queue in one of the disk modes. This patch is of > course also included in 3.18.0. Find 3.16.3 here: > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > Just to make sure I am not misunderstood: 3.18.0 is the current > v3-stable. Other v3-stables are not officially supported (except, of > course, via a commercial support contract). 3.16.3 will not receive any > patches in the future. So I suggest moving over to 3.18.0. It has been > tested for over two month and looks quite ready for prime time. > > I would also like to express my sincere thanks to all of you who > provided bug reports and patches. This is immensely helpful. > > I hope the release is useful. As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 11 16:57:27 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:57:27 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDBC@grfint2.intern.adiscon.com> ...and now this issue is fixed. The new tarball is online. If in doubt which one you loaded, please check the md5sum. The issue was that some documentation files were not properly included inside the tarball. The automatic checks did not detect that. The problem seems to have lurked for quite a while. Many thanks to Michael Biebl for pointing out the issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, July 11, 2008 4:42 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > HI all, > > Michael Biebl just told me that some html files are missing in the > 3.18.0 tarball. Please hold download, I am updating it right now... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Friday, July 11, 2008 4:36 PM > > To: rsyslog-users > > Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > > > Hi all, > > > > we have just released rsyslog 3.18.0, a new v3-stable version. > Rsyslog > > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta > branch. > > It includes all features and bug-fixes from the beta, among others > the > > ability to natively send email. There is one minor bugfix for 3.17.5 > > included, where RainerScript did not always properly detect syntax > > errors. This is a recommended update for all v3-stable branch users. > > > > Change Log: > > > > http://www.rsyslog.com/Article254.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > > > Please note that I also created a 3.16.3 branch, immediately > outdated, > > for those that prefer to keep the previous version for a couple of > > days. > > That version contains an important bugfix which prevents a big memory > > leak when running the queue in one of the disk modes. This patch is > of > > course also included in 3.18.0. Find 3.16.3 here: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > > > Just to make sure I am not misunderstood: 3.18.0 is the current > > v3-stable. Other v3-stables are not officially supported (except, of > > course, via a commercial support contract). 3.16.3 will not receive > any > > patches in the future. So I suggest moving over to 3.18.0. It has > been > > tested for over two month and looks quite ready for prime time. > > > > I would also like to express my sincere thanks to all of you who > > provided bug reports and patches. This is immensely helpful. > > > > I hope the release is useful. As always, feedback is appreciated. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Jul 14 12:06:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 14 Jul 2008 12:06:00 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Hi all, Michael Biebl noticed a small inconsistency in kernel log format: 3.16.x (and before): Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. 3.18.0: Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg started. Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. As you can see, there is a space after kernel: in the pre-3.18.0 messages. This also brings us down to a general inconsistency: Unfortunately, RFC 3164 defines the after TAG not as a delimiter but as part of the CONTENT of the message. This leads to some inconsistency in practice. Depending on who generated the message, there may be a space after the TAG or not. If it is, that space must be submitted as part of the message itself. In versions up to 3.16.x, kernel log messages were generated by old code, which did not know about tag and simply generated a non-structured message. With 3.17.x and above, the klog code is rewritten and correctly fills in all properties. This leads to the "missing" space, because the space is neither part of the tag nor part of the message. I have now three options: 1. leave as is (without a space) In that case, some log parsers may run into troubles 2. add a at the end of the TAG That will probably lead to even more confusion, as matches against TAG would need to include that space. 3. add a in front of each kernel message This comes close to the previous mode, but is "a bit" clumpsy and hackish. It will also make all database tables etc have messages starting with . Similar as 2, MSG matching rules are affected (but I consider this less severe, as matches inside the MSG are usually driven by searches and not absolute positions). I think that real solutions are numbers 1 and 3, with 3 probably having the best arguments. However, I have to admit I do not like this option very much at all. A fourth option would be to add two new property replacer argument "add at end if none is already there" and "drop first if there is one". I could then modify the default templates to use the first one with Tag and the later with MSG (instead of the regular ones). That would be a good fix, probably my favorite, BUT it would also cause format change, now in other messages. Plus, it is a little bit to code and I prefer to do as few code changes to a stable as possible. This issue has unfortunately been lurking ever since the beginning of rsyslog (as it is rooted in rfc 3164) and I should probably have it addressed earlier. But as you see: it is ugly... ;) So question now: what do the list members think? Feedback is highly appreciated. Thanks, Rainer From pascal at impressionet.ch Mon Jul 14 13:31:57 2008 From: pascal at impressionet.ch (Pascal Mainini) Date: Mon, 14 Jul 2008 13:31:57 +0200 Subject: [rsyslog] MARK frequency, too Message-ID: <487B392D.3090408@impressionet.ch> Hi all I'm refering to this mail: http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html I'm having the same issue as described in the Mail above, $MarkMessagePeriod gets completly ignored. I tried putting it at various places in the config-file, with tabs or whitespaces etc., without any result. I'm using the package from debian etch (on debian etch, of course). Find below my configfile (with hostnames anonymized). /etc/rsyslog.d is empty, so no further config should get included... Also, options for rsyslogd are "-c3 -s ". Also, I was wondering: I have a selector *.* which directs anything into a logfile - but not the mark-messages. Why is it that way? And are there maybe other messages which are ignored by a *.* selector? Help would be very much appreciated! Thanks a lot in advance, kind regards, Pascal ---- # /etc/rsyslog.conf Configuration file for rsyslog v3. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) $ModLoad immark # provides --MARK-- message capability $MarkMessagePeriod 300 # provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use default timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # # Set the default permissions for all log files. # $FileOwner root $FileGroup adm $FileCreateMode 0640 # # Include all config files in /etc/rsyslog.d/ # $IncludeConfig /etc/rsyslog.d/*.conf ############### #### RULES #### ############### +hostA *.* -/var/log/remote/hostA.log & -/var/log/full -hostA +hostB *.* -/var/log/remote/hostB.log & -/var/log/full -hostB +hostC *.* -/var/log/remote/hostC.log & -/var/log/full -hostC ####################### #### LOCAL LOGGING #### ####################### +localhostname *.* -/var/log/full # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #*.* -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # #daemon.*;mail.*;\ # news.err;\ # *.=debug;*.=info;\ # *.=notice;*.=warn |/dev/xconsole # From rgerhards at hq.adiscon.com Mon Jul 14 13:43:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 14 Jul 2008 13:43:42 +0200 Subject: [rsyslog] MARK frequency, too In-Reply-To: <487B392D.3090408@impressionet.ch> References: <487B392D.3090408@impressionet.ch> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> Hi, looks like I overlooked hte other post you mention. I'll have a look into the issue, but need to finish some other things first. I'd appreciate if you could file a bug report, so that it will not be forgotten. You can do that at http://bugzilla.adiscon.com Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Pascal Mainini > Sent: Monday, July 14, 2008 1:32 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] MARK frequency, too > > Hi all > > I'm refering to this mail: > > http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html > > I'm having the same issue as described in the Mail above, > $MarkMessagePeriod gets completly ignored. I tried putting it at > various places in the config-file, with tabs or whitespaces etc., > without any result. I'm using the package from debian etch (on > debian etch, of course). > > Find below my configfile (with hostnames anonymized). > /etc/rsyslog.d is empty, so no further config should get included... > Also, options for rsyslogd are "-c3 -s ". > > Also, I was wondering: I have a selector *.* which directs anything > into a logfile - but not the mark-messages. Why is it that way? > And are there maybe other messages which are ignored by a *.* selector? > > Help would be very much appreciated! > > Thanks a lot in advance, kind regards, > > Pascal > > ---- > > # /etc/rsyslog.conf Configuration file for rsyslog v3. > # > # For more information see > # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html > > > ################# > #### MODULES #### > ################# > > $ModLoad imuxsock # provides support for local system logging > $ModLoad imklog # provides kernel logging support (previously done by > rklogd) > > $ModLoad immark # provides --MARK-- message capability > $MarkMessagePeriod 300 > > # provides UDP syslog reception > $ModLoad imudp > $UDPServerRun 514 > > # provides TCP syslog reception > #$ModLoad imtcp > #$InputTCPServerRun 514 > > > ########################### > #### GLOBAL DIRECTIVES #### > ########################### > > # > # Use default timestamp format. > # To enable high precision timestamps, comment out the following line. > # > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # > # Set the default permissions for all log files. > # > $FileOwner root > $FileGroup adm > $FileCreateMode 0640 > > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/*.conf > > > ############### > #### RULES #### > ############### > > +hostA > *.* -/var/log/remote/hostA.log > & -/var/log/full > -hostA > > +hostB > *.* -/var/log/remote/hostB.log > & -/var/log/full > -hostB > > +hostC > *.* -/var/log/remote/hostC.log > & -/var/log/full > -hostC > > ####################### > #### LOCAL LOGGING #### > ####################### > > +localhostname > > *.* -/var/log/full > # > # First some standard log files. Log by facility. > # > auth,authpriv.* /var/log/auth.log > *.*;auth,authpriv.none -/var/log/syslog > #*.* -/var/log/syslog > #cron.* /var/log/cron.log > daemon.* -/var/log/daemon.log > kern.* -/var/log/kern.log > lpr.* -/var/log/lpr.log > mail.* -/var/log/mail.log > user.* -/var/log/user.log > > # > # Logging for the mail system. Split it up so that > # it is easy to write scripts to parse these files. > # > mail.info -/var/log/mail.info > mail.warn -/var/log/mail.warn > mail.err /var/log/mail.err > > # > # Logging for INN news system. > # > news.crit /var/log/news/news.crit > news.err /var/log/news/news.err > news.notice -/var/log/news/news.notice > > # > # Some "catch-all" log files. > # > *.=debug;\ > auth,authpriv.none;\ > news.none;mail.none -/var/log/debug > *.=info;*.=notice;*.=warn;\ > auth,authpriv.none;\ > cron,daemon.none;\ > mail,news.none -/var/log/messages > > # > # Emergencies are sent to everybody logged in. > # > *.emerg * > > # > # I like to have messages displayed on the console, but only on a > virtual > # console I usually leave idle. > # > #daemon,mail.*;\ > # news.=crit;news.=err;news.=notice;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn /dev/tty8 > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use > it, > # you must invoke `xconsole' with the `-file' option: > # > # $ xconsole -file /dev/xconsole [...] > # > # NOTE: adjust the list below, or you'll go crazy if you have a > reasonably > # busy site.. > # > #daemon.*;mail.*;\ > # news.err;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn |/dev/xconsole > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From aoz.syn at gmail.com Mon Jul 14 16:17:31 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 14 Jul 2008 08:17:31 -0600 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: <4255c2570807140717h1a118d86n6dccc31777a98111@mail.gmail.com> > So question now: what do the list members think? I think I may be partially re-stating #4 by saying this, but I think the most pragmatic approach would be to implement strict RFC compliance, then add non-default options to break that in one way or another for backwards compatibility. Make it explicit (in the option name and/or documentation) that by doing so they're breaking RFC 3164; you could even request a courtesy contact to you or the list so you could roughly track its use and thereby when you can drop it (if ever). Of course, that makes more code and more complexity; I'd also support leaving people with inflexible/broken parsers out in the cold, but I'm pretty heartless like that. From mbiebl at gmail.com Tue Jul 15 14:54:48 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Tue, 15 Jul 2008 14:54:48 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: 2008/7/14 Rainer Gerhards : > Hi all, > > Michael Biebl noticed a small inconsistency in kernel log format: > > 3.16.x (and before): > Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): > wlan0: link becomes ready > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > 3.18.0: > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > started. > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. another example: Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down See the <6> after the kernel: tag. It's also new. Is this some kind of log level, set by the kernel? I have never seen, that other syslogger (like syslog-ng, sysklogd set this). > > As you can see, there is a space after kernel: in the pre-3.18.0 > messages. This also brings us down to a general inconsistency: > > Unfortunately, RFC 3164 defines the after TAG not as a delimiter > but as part of the CONTENT of the message. This leads to some > inconsistency in practice. Depending on who generated the message, there > may be a space after the TAG or not. If it is, that space must be > submitted as part of the message itself. > > In versions up to 3.16.x, kernel log messages were generated by old > code, which did not know about tag and simply generated a non-structured > message. With 3.17.x and above, the klog code is rewritten and correctly > fills in all properties. This leads to the "missing" space, because the > space is neither part of the tag nor part of the message. > > I have now three options: > > 1. leave as is (without a space) > In that case, some log parsers may run into troubles This was a rather vague concern of mine. I don't actually know, if other software is affected by this change. > 2. add a at the end of the TAG > That will probably lead to even more confusion, as matches against TAG > would need to include that space. > > 3. add a in front of each kernel message > This comes close to the previous mode, but is "a bit" clumpsy and > hackish. It will also make all database tables etc have messages > starting with . Similar as 2, MSG matching rules are affected (but I > consider this less severe, as matches inside the MSG are usually driven > by searches and not absolute positions). Just curious: For other log messages, that are not generated by the kernel, you have a space after the programname: tag, e.g. Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 Is the space after dhclient: part of the message or part of the :programname tag? Imho, if there is no clear definition in the RFC, how the message should be written, it's best to stay backwards compatible. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Tue Jul 15 15:03:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 15:03:42 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > 2008/7/14 Rainer Gerhards : > > Hi all, > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > 3.16.x (and before): > > Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): > > wlan0: link becomes ready > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > 3.18.0: > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > > started. > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > another example: > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > See the <6> after the kernel: tag. It's also new. > Is this some kind of log level, set by the kernel? This does not look correct. Under FreeBSD, it is typical to have a PRI inside kernel messages. But for linux kernels I tested with, I did not see it and (I think) so I did not support it. But if you see it on Debian, it seems to happen. A cure is to use the same logic that is used for BSD. > I have never seen, that other syslogger (like syslog-ng, sysklogd set this). > > > > > As you can see, there is a space after kernel: in the pre-3.18.0 > > messages. This also brings us down to a general inconsistency: > > > > Unfortunately, RFC 3164 defines the after TAG not as a delimiter > > but as part of the CONTENT of the message. This leads to some > > inconsistency in practice. Depending on who generated the message, there > > may be a space after the TAG or not. If it is, that space must be > > submitted as part of the message itself. > > > > In versions up to 3.16.x, kernel log messages were generated by old > > code, which did not know about tag and simply generated a non-structured > > message. With 3.17.x and above, the klog code is rewritten and correctly > > fills in all properties. This leads to the "missing" space, because the > > space is neither part of the tag nor part of the message. > > > > I have now three options: > > > > 1. leave as is (without a space) > > In that case, some log parsers may run into troubles > > This was a rather vague concern of mine. I don't actually know, if > other software is affected by this change. I share this concern, but also have no additional information. I wanted to point out the options. I have to admit I am still undecided. But that nobody violently objected so far makes this look like it is not so important... > > > 2. add a at the end of the TAG > > That will probably lead to even more confusion, as matches against TAG > > would need to include that space. > > > > 3. add a in front of each kernel message > > This comes close to the previous mode, but is "a bit" clumpsy and > > hackish. It will also make all database tables etc have messages > > starting with . Similar as 2, MSG matching rules are affected (but I > > consider this less severe, as matches inside the MSG are usually driven > > by searches and not absolute positions). > > Just curious: For other log messages, that are not generated by the > kernel, you have a space after the programname: tag, e.g. > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > Is the space after dhclient: part of the message or part of the > :programname tag? In our private communication, I thought it is part of TAG. But that is wrong. It actually is part of MSG - at least for legacy/RFC3164 syslog. For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is well-defined and part of neither. There, it is a delimiter (as one would expect). I you use rsyslog only with syslog-protocol messages (and the syslog-protocol templates), this is more or less no issue. But who does today...? ;) The real problem is that legacy syslog has many different interpretations and we break one or the other in either way... > > Imho, if there is no clear definition in the RFC, how the message > should be written, it's best to stay backwards compatible. Is that a vote for option 3, stick a space in front of each kernel message? ;) > > > Cheers, > Michael > > From rgerhards at hq.adiscon.com Tue Jul 15 16:10:20 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:10:20 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2and 3.18.0 In-Reply-To: <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> Michael, I have reviewed some of the code in question. It looks like a bug. It's actually not a klog driver issue, the PRI parsing is done at an upper klog layer and thus should work for linux as well. I would like to generate a version with some additional instrumentation. Could you run it and report the results back (maybe via private mail to keep the list free of noise). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 15, 2008 3:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] slight kernel log format inconsistency between > 3.16.2and 3.18.0 > > On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > > 2008/7/14 Rainer Gerhards : > > > Hi all, > > > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > > > 3.16.x (and before): > > > Jul 11 17:33:28 pluto kernel: [14177.432349] > ADDRCONF(NETDEV_CHANGE): > > > wlan0: link becomes ready > > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > > > > 3.18.0: > > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > > > started. > > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > > > another example: > > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > > See the <6> after the kernel: tag. It's also new. > > Is this some kind of log level, set by the kernel? > > This does not look correct. Under FreeBSD, it is typical to have a PRI > inside kernel messages. But for linux kernels I tested with, I did not > see it and (I think) so I did not support it. But if you see it on > Debian, it seems to happen. A cure is to use the same logic that is > used > for BSD. > > > I have never seen, that other syslogger (like syslog-ng, sysklogd set > this). > > > > > > > > As you can see, there is a space after kernel: in the pre- > 3.18.0 > > > messages. This also brings us down to a general inconsistency: > > > > > > Unfortunately, RFC 3164 defines the after TAG not as a > delimiter > > > but as part of the CONTENT of the message. This leads to some > > > inconsistency in practice. Depending on who generated the message, > there > > > may be a space after the TAG or not. If it is, that space must be > > > submitted as part of the message itself. > > > > > > In versions up to 3.16.x, kernel log messages were generated by old > > > code, which did not know about tag and simply generated a non- > structured > > > message. With 3.17.x and above, the klog code is rewritten and > correctly > > > fills in all properties. This leads to the "missing" space, because > the > > > space is neither part of the tag nor part of the message. > > > > > > I have now three options: > > > > > > 1. leave as is (without a space) > > > In that case, some log parsers may run into troubles > > > > This was a rather vague concern of mine. I don't actually know, if > > other software is affected by this change. > > I share this concern, but also have no additional information. I wanted > to point out the options. I have to admit I am still undecided. But > that > nobody violently objected so far makes this look like it is not so > important... > > > > > > 2. add a at the end of the TAG > > > That will probably lead to even more confusion, as matches against > TAG > > > would need to include that space. > > > > > > 3. add a in front of each kernel message > > > This comes close to the previous mode, but is "a bit" clumpsy and > > > hackish. It will also make all database tables etc have messages > > > starting with . Similar as 2, MSG matching rules are affected > (but I > > > consider this less severe, as matches inside the MSG are usually > driven > > > by searches and not absolute positions). > > > > Just curious: For other log messages, that are not generated by the > > kernel, you have a space after the programname: tag, e.g. > > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > > > Is the space after dhclient: part of the message or part of the > > :programname tag? > > In our private communication, I thought it is part of TAG. But that is > wrong. It actually is part of MSG - at least for legacy/RFC3164 syslog. > > For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is > well-defined and part of neither. There, it is a delimiter (as one > would > expect). I you use rsyslog only with syslog-protocol messages (and the > syslog-protocol templates), this is more or less no issue. But who does > today...? ;) > > The real problem is that legacy syslog has many different > interpretations and we break one or the other in either way... > > > > > Imho, if there is no clear definition in the RFC, how the message > > should be written, it's best to stay backwards compatible. > > Is that a vote for option 3, stick a space in front of each kernel > message? ;) > > > > > > > Cheers, > > Michael > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 15 16:22:07 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:22:07 +0200 Subject: [rsyslog] MARK frequency, too In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> References: <487B392D.3090408@impressionet.ch> <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDC@grfint2.intern.adiscon.com> FYI: I have found and fixed this bug: http://bugzilla.adiscon.com/process_bug.cgi Thanks for reporting :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, July 14, 2008 1:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] MARK frequency, too > > Hi, > > looks like I overlooked hte other post you mention. I'll have a look > into the issue, but need to finish some other things first. I'd > appreciate if you could file a bug report, so that it will not be > forgotten. You can do that at > > http://bugzilla.adiscon.com > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Pascal Mainini > > Sent: Monday, July 14, 2008 1:32 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] MARK frequency, too > > > > Hi all > > > > I'm refering to this mail: > > > > http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html > > > > I'm having the same issue as described in the Mail above, > > $MarkMessagePeriod gets completly ignored. I tried putting it at > > various places in the config-file, with tabs or whitespaces etc., > > without any result. I'm using the package from debian etch (on > > debian etch, of course). > > > > Find below my configfile (with hostnames anonymized). > > /etc/rsyslog.d is empty, so no further config should get included... > > Also, options for rsyslogd are "-c3 -s ". > > > > Also, I was wondering: I have a selector *.* which directs anything > > into a logfile - but not the mark-messages. Why is it that way? > > And are there maybe other messages which are ignored by a *.* > selector? > > > > Help would be very much appreciated! > > > > Thanks a lot in advance, kind regards, > > > > Pascal > > > > ---- > > > > # /etc/rsyslog.conf Configuration file for rsyslog v3. > > # > > # For more information see > > # > /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html > > > > > > ################# > > #### MODULES #### > > ################# > > > > $ModLoad imuxsock # provides support for local system logging > > $ModLoad imklog # provides kernel logging support (previously done > by > > rklogd) > > > > $ModLoad immark # provides --MARK-- message capability > > $MarkMessagePeriod 300 > > > > # provides UDP syslog reception > > $ModLoad imudp > > $UDPServerRun 514 > > > > # provides TCP syslog reception > > #$ModLoad imtcp > > #$InputTCPServerRun 514 > > > > > > ########################### > > #### GLOBAL DIRECTIVES #### > > ########################### > > > > # > > # Use default timestamp format. > > # To enable high precision timestamps, comment out the following > line. > > # > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > > > # > > # Set the default permissions for all log files. > > # > > $FileOwner root > > $FileGroup adm > > $FileCreateMode 0640 > > > > # > > # Include all config files in /etc/rsyslog.d/ > > # > > $IncludeConfig /etc/rsyslog.d/*.conf > > > > > > ############### > > #### RULES #### > > ############### > > > > +hostA > > *.* -/var/log/remote/hostA.log > > & -/var/log/full > > -hostA > > > > +hostB > > *.* -/var/log/remote/hostB.log > > & -/var/log/full > > -hostB > > > > +hostC > > *.* -/var/log/remote/hostC.log > > & -/var/log/full > > -hostC > > > > ####################### > > #### LOCAL LOGGING #### > > ####################### > > > > +localhostname > > > > *.* -/var/log/full > > # > > # First some standard log files. Log by facility. > > # > > auth,authpriv.* /var/log/auth.log > > *.*;auth,authpriv.none -/var/log/syslog > > #*.* -/var/log/syslog > > #cron.* /var/log/cron.log > > daemon.* -/var/log/daemon.log > > kern.* -/var/log/kern.log > > lpr.* -/var/log/lpr.log > > mail.* -/var/log/mail.log > > user.* -/var/log/user.log > > > > # > > # Logging for the mail system. Split it up so that > > # it is easy to write scripts to parse these files. > > # > > mail.info -/var/log/mail.info > > mail.warn -/var/log/mail.warn > > mail.err /var/log/mail.err > > > > # > > # Logging for INN news system. > > # > > news.crit /var/log/news/news.crit > > news.err /var/log/news/news.err > > news.notice -/var/log/news/news.notice > > > > # > > # Some "catch-all" log files. > > # > > *.=debug;\ > > auth,authpriv.none;\ > > news.none;mail.none -/var/log/debug > > *.=info;*.=notice;*.=warn;\ > > auth,authpriv.none;\ > > cron,daemon.none;\ > > mail,news.none -/var/log/messages > > > > # > > # Emergencies are sent to everybody logged in. > > # > > *.emerg * > > > > # > > # I like to have messages displayed on the console, but only on a > > virtual > > # console I usually leave idle. > > # > > #daemon,mail.*;\ > > # news.=crit;news.=err;news.=notice;\ > > # *.=debug;*.=info;\ > > # *.=notice;*.=warn /dev/tty8 > > > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use > > it, > > # you must invoke `xconsole' with the `-file' option: > > # > > # $ xconsole -file /dev/xconsole [...] > > # > > # NOTE: adjust the list below, or you'll go crazy if you have a > > reasonably > > # busy site.. > > # > > #daemon.*;mail.*;\ > > # news.err;\ > > # *.=debug;*.=info;\ > > # *.=notice;*.=warn |/dev/xconsole > > # > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 15 16:38:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:38:46 +0200 Subject: [rsyslog] rsyslog 3.19.10 (beta) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDF@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.10. This starts a new iteration of the beta branch, bringing all the great features of the previous development version to it. It contains all features from 3.19.9 and also a number of bug fixes. The most important one is for a bad memory leak that occurred if queues were running in one of the disk-queuing modes. Also, a number of cross-platform problems on FreeBSD were sorted out. This is a suggested update for all beta branch users. Just in case you wonder: at this time there is NO development branch version available. One will be announced in the not so distant future. ChangeLog: http://www.rsyslog.com/Article256.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-120.phtml As always, feedback is appreciated. I hope this release is useful. Rainer Gerhards From hks.private at gmail.com Tue Jul 15 20:14:39 2008 From: hks.private at gmail.com ((private) HKS) Date: Tue, 15 Jul 2008 14:14:39 -0400 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: FYI, the links at the rsyslog status page are a bit funky: http://www.rsyslog.com/doc-status.html Beta 3.19.10's download link takes you to a page describing 3.17.3 beta with a download link for 3.19.10. v3 stable 3.18.0's download link takes you to a page describing 3.19.10 beta with a download link for the same. v2 stable 2.0.5's download link takes you to a page mostly describing 2.0.5 but with the rsyslog version misreported (and perhaps md5 checksum?). -HKS On Fri, Jul 11, 2008 at 10:35 AM, Rainer Gerhards wrote: > Hi all, > > we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. > It includes all features and bug-fixes from the beta, among others the > ability to natively send email. There is one minor bugfix for 3.17.5 > included, where RainerScript did not always properly detect syntax > errors. This is a recommended update for all v3-stable branch users. > > Change Log: > > http://www.rsyslog.com/Article254.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > Please note that I also created a 3.16.3 branch, immediately outdated, > for those that prefer to keep the previous version for a couple of days. > That version contains an important bugfix which prevents a big memory > leak when running the queue in one of the disk modes. This patch is of > course also included in 3.18.0. Find 3.16.3 here: > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > Just to make sure I am not misunderstood: 3.18.0 is the current > v3-stable. Other v3-stables are not officially supported (except, of > course, via a commercial support contract). 3.16.3 will not receive any > patches in the future. So I suggest moving over to 3.18.0. It has been > tested for over two month and looks quite ready for prime time. > > I would also like to express my sincere thanks to all of you who > provided bug reports and patches. This is immensely helpful. > > I hope the release is useful. As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Wed Jul 16 08:30:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 16 Jul 2008 08:30:56 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDE3@grfint2.intern.adiscon.com> Hi HKS, Thanks, looks like I had a bad time with the links... Corrected now. I have also checked 2.0.5, the version was incorrect (copy&paste error), but the md5sum is correct. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Tuesday, July 15, 2008 8:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > FYI, the links at the rsyslog status page are a bit funky: > http://www.rsyslog.com/doc-status.html > > Beta 3.19.10's download link takes you to a page describing 3.17.3 > beta with a download link for 3.19.10. > > v3 stable 3.18.0's download link takes you to a page describing > 3.19.10 beta with a download link for the same. > > v2 stable 2.0.5's download link takes you to a page mostly describing > 2.0.5 but with the rsyslog version misreported (and perhaps md5 > checksum?). > > -HKS > > On Fri, Jul 11, 2008 at 10:35 AM, Rainer Gerhards > wrote: > > Hi all, > > > > we have just released rsyslog 3.18.0, a new v3-stable version. > Rsyslog > > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta > branch. > > It includes all features and bug-fixes from the beta, among others > the > > ability to natively send email. There is one minor bugfix for 3.17.5 > > included, where RainerScript did not always properly detect syntax > > errors. This is a recommended update for all v3-stable branch users. > > > > Change Log: > > > > http://www.rsyslog.com/Article254.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > > > Please note that I also created a 3.16.3 branch, immediately > outdated, > > for those that prefer to keep the previous version for a couple of > days. > > That version contains an important bugfix which prevents a big memory > > leak when running the queue in one of the disk modes. This patch is > of > > course also included in 3.18.0. Find 3.16.3 here: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > > > Just to make sure I am not misunderstood: 3.18.0 is the current > > v3-stable. Other v3-stables are not officially supported (except, of > > course, via a commercial support contract). 3.16.3 will not receive > any > > patches in the future. So I suggest moving over to 3.18.0. It has > been > > tested for over two month and looks quite ready for prime time. > > > > I would also like to express my sincere thanks to all of you who > > provided bug reports and patches. This is immensely helpful. > > > > I hope the release is useful. As always, feedback is appreciated. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 17 12:43:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 12:43:54 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Hi all, I just got an idea for a feature inspired by the ramlog project. I have blogged about it: http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h tml If you have some minutes to spare, please review and provide feedback. Thanks, Rainer From mic at npgx.com.au Thu Jul 17 13:09:27 2008 From: mic at npgx.com.au (Michael Mansour) Date: Thu, 17 Jul 2008 22:09:27 +1100 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <20080717110306.M99550@npgx.com.au> Hi Rainer, > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. I've read and honestly have no opinion on it :) The last time I tried delayed writes with rsyslog the machines I set them up on basically died (load averages above 30, so they were complete unresponsive). I log rsyslog to a MySQL DB (for maillogs only) and it took me a while to realise it was the rsyslog changes I made that killed the mail servers. They even hung on boot ie. would not complete a boot process. When I got into single user mode and didn't start up any services, I was able to change the rsyslog.conf file back to my older one, and everything was fine again. This was some versions back (but still the 3.x stable series) but because these were production servers, I didn't bother analysing what went wrong or why it happened, I was just glad I had my mail servers back again and working. I might give this a second attempt at some stage this year, because I like the feature for queuing entries if the DB is down, but until then, I really don't have much to say about the delayed writes stuff :) Michael. > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From rgerhards at hq.adiscon.com Thu Jul 17 13:45:49 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 13:45:49 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <20080717110306.M99550@npgx.com.au> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <20080717110306.M99550@npgx.com.au> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE12@grfint2.intern.adiscon.com> Hi Michael, I think you were hit by this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=86 interestingly, nobody who experienced it bothered to report, until recently. Thus it was for quite a while inside the core engine ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Mansour > Sent: Thursday, July 17, 2008 1:09 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback request: in-memory buffering messages > > Hi Rainer, > > > Hi all, > > > > I just got an idea for a feature inspired by the ramlog project. I > have > > blogged about it: > > > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram- > buffer.h > > tml > > > > If you have some minutes to spare, please review and provide > feedback. > > I've read and honestly have no opinion on it :) > > The last time I tried delayed writes with rsyslog the machines I set > them up > on basically died (load averages above 30, so they were complete > unresponsive). > > I log rsyslog to a MySQL DB (for maillogs only) and it took me a while > to > realise it was the rsyslog changes I made that killed the mail servers. > They > even hung on boot ie. would not complete a boot process. When I got > into > single user mode and didn't start up any services, I was able to change > the > rsyslog.conf file back to my older one, and everything was fine again. > > This was some versions back (but still the 3.x stable series) but > because > these were production servers, I didn't bother analysing what went > wrong or > why it happened, I was just glad I had my mail servers back again and > working. > > I might give this a second attempt at some stage this year, because I > like the > feature for queuing entries if the DB is down, but until then, I really > don't > have much to say about the delayed writes stuff :) > > Michael. > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > ------- End of Original Message ------- > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Thu Jul 17 16:13:10 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 17 Jul 2008 23:13:10 +0900 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <5646AC09-7597-4549-8193-BD6C187DEA7E@redhat.com> Hi Rainer-san, +1 How to use logging facilities varies according to circumstances. For example, RHEL is used for from edge to mission critical. Our customers want robust logging functions, like writing to MySQL or transporting via TCP/RELP. But some users want 'greener' machines. It sounds interesting to combine the feature to write log messages to RAM and the powertop project. http://www.lesswatts.org/projects/powertop/ Best Rio. On 2008/07/17, at 19:43, Rainer Gerhards wrote: > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I > have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From hks.private at gmail.com Thu Jul 17 16:19:11 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 17 Jul 2008 10:19:11 -0400 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: This is certainly something I would use. I've got a number of mission critical devices running off of flash drives (OpenBSD firewalls), and the less they have to write to disk the better. This would also be handy on a high-use database machine, where excessive disk writes could have a significant impact on service delivery. I doubt it would see very widely used, but it would make rsyslog appealing to another niche. -HKS On Thu, Jul 17, 2008 at 6:43 AM, Rainer Gerhards wrote: > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From aoz.syn at gmail.com Thu Jul 17 16:20:07 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 08:20:07 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml Prior art exists and I think it's well worth the effort, particularly for busy servers on reliable power. If I read your post right, you're considering something rather similar to the syslog-ng options 'flush_lines' and 'flush_timeout'? From rgerhards at hq.adiscon.com Thu Jul 17 16:22:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 16:22:43 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> > Prior art exists and I think it's well worth the effort, particularly > for busy servers on reliable power. If I read your post right, you're > considering something rather similar to the syslog-ng options > 'flush_lines' and 'flush_timeout'? I actually (really ;)) do not know syslog-ng and keep myself somewhat away from it to prevent accidental steal of "whatever" from there. But now that you raise the point: could you quickly describe how it works. >From your mail, it sounds like what I am thinking about... Rainer From mbiebl at gmail.com Thu Jul 17 16:32:08 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Thu, 17 Jul 2008 16:32:08 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Message-ID: Hi, I just wanted to say, that I like the idea of using docbook/xml to generate the (html) documentation (and possibly the man pages). Imo this would make the documentation more consistent (consistent font sizes, headers, footers etc.), more usable (You'd get something like a TOC and possibly better cross-linking) and perhaps easier keeping up-to-date. Cheers, Michael 2008/7/10 Ryo Fujita : > Hi Rainer-san, > > DocBook.org is here. > http://www.docbook.org/ > > And IBM has a good guide for beginners. > http://www.ibm.com/developerworks/xml/library/xml-matters3.html > > Recently, Red Hat uses DocBook format to generate web pages ,PDFs and > so on. > For example, > https://www.redhat.com/docs/manuals/enterprise/ > A document having both html and PDF is generated from DocBook. > > gnome-doc-utils and kdesdk-utils include very convenient tools to > handle DocBook and related formats. > > Best Rio. > > On 2008/07/10, at 16:03, Rainer Gerhards wrote: > >> Hi Rio-san, >> >> I need to leave soon, but a quick feedback: the proposal looks *very* >> interesting. I need to digest it a bit. I have no experience with >> DocBook, so I probably need to check that out, first. That may take a >> short while (should you happen to know a good "getting started guide", >> I'd grateful if you send me a link ;)). From first thought, I would >> prefer to generate the html et al outputs from a single source (where >> only that source is in git). >> >> Feedback from other list members is also appreciated. >> >> Rainer >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>> Sent: Thursday, July 10, 2008 8:51 AM >>> To: rsyslog-users >>> Subject: [rsyslog] Suggestion for Documentation Reconstructure >>> >>> Hi List, >>> >>> I consulted with my colleagues about the smartest way to reconstruct >>> the documents of rsyslog. >>> >>> 1.Work Flow matter >>> The attached image is just an idea for rational flowchart to make >>> translation easier. >>> #If this ML forbids me to attach any files, you can see it on my >>> site. >>> http://rio.tc/2008/07/10-144007.php >>> >>> I'm worried that this flow will give the contributers much trouble. >>> >>> - HTML(from Wiki) to DocBook conversion >>> 'tidy' will help us a little, but exporter of media Wiki is poor to >>> export 'well-formed HTML'. >>> We need to edit and check them manually. >>> Of course, I willingly do this. But it may take a number of days. >>> >>> - Writers need to edit DocBook XML after the reconstruction. >>> As you know, HTML format is not good for a source of multi-output. >>> DocBook XML is a perfect one except for edit :) >>> >>> - Automake related matter >>> There can be two streams to generate man and html from DocBook XML. >>> One : When you edit DocBook, you commit DocBook, HTML and man files >>> to git tree. >>> Of course this method requires us to prepare an >>> environment where you can use xsltproc/docbook2man. >>> Two : By adding some sequences into Makefile, HTML and man files >>> can be generated automatically. >>> All persons who want to build rsyslog from tar ball need >>> to prepare the environment. >>> And we need to put some check routines for dependencies >>> into Makefile. >>> >>> 2.git tree reconstruct only with 'doc' directory >>> >>> `-- doc >>> |-- Makefile.am >>> |-- conf >>> | |-- en >>> | | `-- rsyslog-example.conf >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | `-- rsyslog-example.conf # annotations are translated. >>> |-- html >>> | |-- en >>> | | |-- bugs.html >>> | | |-- OTHER_HTMLS >>> | | `-- version_naming.html >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | |-- bugs.html >>> | |-- OTHER_HTMLS >>> | `-- version_naming.html >>> |-- images >>> | |-- gssapi.png >>> | |-- OTHER_IMAGES >>> | `-- tls_cert_ca.jpg >>> |-- man >>> | |-- en >>> | | |-- man5 >>> | | | `-- rsyslog.conf.5 >>> | | `-- man8 >>> | | `-- rsyslogd.8 >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | |-- man5 >>> | | `-- rsyslog.conf.5 >>> | `-- man8 >>> | `-- rsyslogd.8 >>> `-- src >>> |-- dias >>> | |-- classes.dia >>> | |-- OTHER_DIA_FILES >>> | `-- tls_cert_ca.dia >>> `-- docbook >>> |-- en >>> | |-- bugs.xml >>> | |-- rsyslog.conf.5.xml >>> | |-- rsyslogd.8.xml >>> | |-- OTHER_XMLS >>> | `-- version_naming.xml >>> `-- ja >>> |-- bugs.html >>> |-- rsyslog.conf.5.xml >>> |-- rsyslogd.8.xml >>> |-- OTHER_XMLS >>> `-- version_naming.html >>> >>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ >>> directory. >>> >>> Your suggestions will be highly appreciated!! >>> >>> Best Rio. >>> >>> >> ####################################################################### >>> # >>> Ryo Fujita >>> Senior Solution Architect, RHCE >>> Red Hat K.K. >>> TEL +81-3-5798-8500 >>> FAX +81-3-5798-8599 >>> Ebisu Neonato 8F >>> 4-1-18 Ebisu, Shibuya-ku, >>> Tokyo Japan 1500013 >>> >> ####################################################################### >>> # >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From aoz.syn at gmail.com Thu Jul 17 17:08:22 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 09:08:22 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> Message-ID: <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> > I actually (really ;)) do not know syslog-ng and keep myself somewhat > away from it to prevent accidental steal of "whatever" from there. But > now that you raise the point: could you quickly describe how it works. > From your mail, it sounds like what I am thinking about... I 100% agree with and applaud that separation. As you've stated before, they make a good product and don't warrant any interference. Not touching the docs myself (since I know too much of it by heart anyway), flush_lines allows the admin to configure a particular number of log entries to buffer before forcing a flush to disk, whereas flush_timeout configures the maximum interval between disk flushes. Unlike ramlog, it seems to do this with internal allocations and doesn't rely on a ramdisk. I usually set it rather conservatively (20 and 600 respectively), but I can definitely see being more aggressive on a dedicated log collector with critical power or a laptop in ultra-low power mode. This has been a feature of the public version of syslog-ng for as long as I can remember (or four years, whichever is sooner ;). Combined with disk queues I can see a very nice tiered approach to handling extremely high volumes of log data in a rather reliable manner. From rgerhards at hq.adiscon.com Thu Jul 17 18:29:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 18:29:30 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> Message-ID: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> On Thu, 2008-07-17 at 09:08 -0600, RB wrote: > > I actually (really ;)) do not know syslog-ng and keep myself somewhat > > away from it to prevent accidental steal of "whatever" from there. But > > now that you raise the point: could you quickly describe how it works. > > From your mail, it sounds like what I am thinking about... > > I 100% agree with and applaud that separation. As you've stated > before, they make a good product and don't warrant any interference. Definitely. > > Not touching the docs myself (since I know too much of it by heart > anyway), flush_lines allows the admin to configure a particular number > of log entries to buffer before forcing a flush to disk, whereas > flush_timeout configures the maximum interval between disk flushes. > Unlike ramlog, it seems to do this with internal allocations and > doesn't rely on a ramdisk. I usually set it rather conservatively (20 > and 600 respectively), but I can definitely see being more aggressive > on a dedicated log collector with critical power or a laptop in > ultra-low power mode. This begins to make more and more sense. Just one thing to make sure we are talking along the same lines. The ramdisk approach provides a way to save the IO while still being able to look at the log lines as fast as they come in. If, in rsyslog, I create a memory buffer, I can persist lines to disk only after it is "time to write them to disk" (whatever that triggers). So you will not be able to observe the messages while they stick inside rsyslog's queue. I guess for many cases this is not really relevant. And of course, it is something that must be configurable on an action basis, so different files may use different settings. Thinking more about a potential algorithm, I tend to think it would probably useful to write log files in chunks of n bytes instead of n lines. I am thinking along the lines of matching up the output buffer with the disk sector size (or any multiple of it) so that partial writes of the same sector are avoided. Of course, that involves some stating of the file to obtain the size of the first buffer to be written (filesize mod sector [allocation unit] size). I also requires me to find out the allocation unit size for a given file system. It obviously involves writing incomplete lines. It requires additional code. So the best solution may actually be to permit partial writes (as initially thought) but recommend large buffers. The overall effect could justify the performance impact from doing non-optimal writes. But I am in too much detail. The actual questions are: a) is it OK that log data is visible only after the (write) delay? b) does it sound useful to buffer based on allocation unit sizes? Thanks, Rainer > > This has been a feature of the public version of syslog-ng for as long > as I can remember (or four years, whichever is sooner ;). Combined > with disk queues I can see a very nice tiered approach to handling > extremely high volumes of log data in a rather reliable manner. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 17 18:33:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 18:33:38 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Message-ID: <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> I still haven't found the time to have an in-depth look at docbook, but I strongly tend to agree to Michael's conclusion. It looks like a problem solver for some ugly problems I am facing. I will probably re-evaluate the release goal for 3.21.x. It may happen that I push back the script engine evolution to after summer to do some cleanup, performance improvement, doc work and maybe a rewrite of the file output handler. I find it a bit hard to think what is more important, but I think we can stay with the non-scriptable config for two more month. Maybe I can add custom functions as a smaller improvement sometime while doing the other things. In any case, I am happy to have Rio-san over here and as such someone to ask for solutions to the doc problem. Thanks a lot! Rainer On Thu, 2008-07-17 at 16:32 +0200, Michael Biebl wrote: > Hi, > > I just wanted to say, that I like the idea of using docbook/xml to > generate the (html) documentation (and possibly the man pages). > Imo this would make the documentation more consistent (consistent font > sizes, headers, footers etc.), more usable (You'd get something like a > TOC and possibly better cross-linking) and perhaps easier keeping > up-to-date. > > Cheers, > Michael > > > 2008/7/10 Ryo Fujita : > > Hi Rainer-san, > > > > DocBook.org is here. > > http://www.docbook.org/ > > > > And IBM has a good guide for beginners. > > http://www.ibm.com/developerworks/xml/library/xml-matters3.html > > > > Recently, Red Hat uses DocBook format to generate web pages ,PDFs and > > so on. > > For example, > > https://www.redhat.com/docs/manuals/enterprise/ > > A document having both html and PDF is generated from DocBook. > > > > gnome-doc-utils and kdesdk-utils include very convenient tools to > > handle DocBook and related formats. > > > > Best Rio. > > > > On 2008/07/10, at 16:03, Rainer Gerhards wrote: > > > >> Hi Rio-san, > >> > >> I need to leave soon, but a quick feedback: the proposal looks *very* > >> interesting. I need to digest it a bit. I have no experience with > >> DocBook, so I probably need to check that out, first. That may take a > >> short while (should you happen to know a good "getting started guide", > >> I'd grateful if you send me a link ;)). From first thought, I would > >> prefer to generate the html et al outputs from a single source (where > >> only that source is in git). > >> > >> Feedback from other list members is also appreciated. > >> > >> Rainer > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>> Sent: Thursday, July 10, 2008 8:51 AM > >>> To: rsyslog-users > >>> Subject: [rsyslog] Suggestion for Documentation Reconstructure > >>> > >>> Hi List, > >>> > >>> I consulted with my colleagues about the smartest way to reconstruct > >>> the documents of rsyslog. > >>> > >>> 1.Work Flow matter > >>> The attached image is just an idea for rational flowchart to make > >>> translation easier. > >>> #If this ML forbids me to attach any files, you can see it on my > >>> site. > >>> http://rio.tc/2008/07/10-144007.php > >>> > >>> I'm worried that this flow will give the contributers much trouble. > >>> > >>> - HTML(from Wiki) to DocBook conversion > >>> 'tidy' will help us a little, but exporter of media Wiki is poor to > >>> export 'well-formed HTML'. > >>> We need to edit and check them manually. > >>> Of course, I willingly do this. But it may take a number of days. > >>> > >>> - Writers need to edit DocBook XML after the reconstruction. > >>> As you know, HTML format is not good for a source of multi-output. > >>> DocBook XML is a perfect one except for edit :) > >>> > >>> - Automake related matter > >>> There can be two streams to generate man and html from DocBook XML. > >>> One : When you edit DocBook, you commit DocBook, HTML and man files > >>> to git tree. > >>> Of course this method requires us to prepare an > >>> environment where you can use xsltproc/docbook2man. > >>> Two : By adding some sequences into Makefile, HTML and man files > >>> can be generated automatically. > >>> All persons who want to build rsyslog from tar ball need > >>> to prepare the environment. > >>> And we need to put some check routines for dependencies > >>> into Makefile. > >>> > >>> 2.git tree reconstruct only with 'doc' directory > >>> > >>> `-- doc > >>> |-- Makefile.am > >>> |-- conf > >>> | |-- en > >>> | | `-- rsyslog-example.conf > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | `-- rsyslog-example.conf # annotations are translated. > >>> |-- html > >>> | |-- en > >>> | | |-- bugs.html > >>> | | |-- OTHER_HTMLS > >>> | | `-- version_naming.html > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | |-- bugs.html > >>> | |-- OTHER_HTMLS > >>> | `-- version_naming.html > >>> |-- images > >>> | |-- gssapi.png > >>> | |-- OTHER_IMAGES > >>> | `-- tls_cert_ca.jpg > >>> |-- man > >>> | |-- en > >>> | | |-- man5 > >>> | | | `-- rsyslog.conf.5 > >>> | | `-- man8 > >>> | | `-- rsyslogd.8 > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | |-- man5 > >>> | | `-- rsyslog.conf.5 > >>> | `-- man8 > >>> | `-- rsyslogd.8 > >>> `-- src > >>> |-- dias > >>> | |-- classes.dia > >>> | |-- OTHER_DIA_FILES > >>> | `-- tls_cert_ca.dia > >>> `-- docbook > >>> |-- en > >>> | |-- bugs.xml > >>> | |-- rsyslog.conf.5.xml > >>> | |-- rsyslogd.8.xml > >>> | |-- OTHER_XMLS > >>> | `-- version_naming.xml > >>> `-- ja > >>> |-- bugs.html > >>> |-- rsyslog.conf.5.xml > >>> |-- rsyslogd.8.xml > >>> |-- OTHER_XMLS > >>> `-- version_naming.html > >>> > >>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ > >>> directory. > >>> > >>> Your suggestions will be highly appreciated!! > >>> > >>> Best Rio. > >>> > >>> > >> ####################################################################### > >>> # > >>> Ryo Fujita > >>> Senior Solution Architect, RHCE > >>> Red Hat K.K. > >>> TEL +81-3-5798-8500 > >>> FAX +81-3-5798-8599 > >>> Ebisu Neonato 8F > >>> 4-1-18 Ebisu, Shibuya-ku, > >>> Tokyo Japan 1500013 > >>> > >> ####################################################################### > >>> # > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > ######################################################################## > > Ryo Fujita > > Senior Solution Architect, RHCE > > Red Hat K.K. > > TEL +81-3-5798-8500 > > FAX +81-3-5798-8599 > > Ebisu Neonato 8F > > 4-1-18 Ebisu, Shibuya-ku, > > Tokyo Japan 1500013 > > ######################################################################## > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > From martinmie at PartyGaming.com Thu Jul 17 18:41:19 2008 From: martinmie at PartyGaming.com (Martin Mielke) Date: Thu, 17 Jul 2008 18:41:19 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com><4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com><4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: <0B1B9163138571439EAF804646F3F6AA050C3078@GIBSVWIN004X.partygaming.local> Hi all, [ big snip ] > > The actual questions are: > > a) is it OK that log data is visible only after the (write) delay? Hmmm.. no. This would eliminate the "magic" of watching system events in real-time as in the commonly used 'tail -n0 -f /var/log/messages' which is often used to troubleshoot problems... Unless you provide some tools to implement such functionality by watching the log events in memory... huh... My 2 cents... Martin > b) does it sound useful to buffer based on allocation unit sizes? > > Thanks, > Rainer > > > > This has been a feature of the public version of syslog-ng for as long > > as I can remember (or four years, whichever is sooner ;). Combined > > with disk queues I can see a very nice tiered approach to handling > > extremely high volumes of log data in a rather reliable manner. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Thu Jul 17 20:16:03 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 17 Jul 2008 14:16:03 -0400 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: > The actual questions are: > > a) is it OK that log data is visible only after the (write) delay? > b) does it sound useful to buffer based on allocation unit sizes? a) It depends on the situation. If you're using this to conserve power or flash-disk writes, then it's going to be a pain. If you're using this to tune performance in a high log-volume environment, then this is an acceptable tradeoff. Perhaps a secondary script that would send a signal to rsyslogd and print the lines to STDOUT would be an acceptable compromise? b) It's more granular than many (most?) users will need, including myself, but I'm sure some people will find this very useful. -HKS From aoz.syn at gmail.com Thu Jul 17 21:11:24 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 13:11:24 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: <4255c2570807171211q76742249o6ae3917feec70a62@mail.gmail.com> > a) is it OK that log data is visible only after the (write) delay? For my purposes it is; I know some of the other responses have been that it's not, but perhaps setting up a signal (maybe USR2) for triggering the flush mechanism would help those situations. Of course, people wanting "real-time" data would probably either leave off the configurations (knowing they're doing so at the expense of performance) or change it at runtime and HUP. > b) does it sound useful to buffer based on allocation unit sizes? I had actually considered suggesting this as a third option (differentiating from other implementations), but abandoned it since I was unsure of the value and felt the implementation may be overly complex. If you implement byte/allocation-aligned flushes, my preference would be to have it in addition to but mutually (configuration) exclusive with record-aligned ones. Partial-sector writes are less of a concern to me in most cases than getting the whole message to disk, but I can again see the usefulness of allocation alignment in a very high-volume environment or one where disk performance is constrained. Regardless, it's a fascinating development and one that will come in very handy. From rfujita at redhat.com Fri Jul 18 04:19:53 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 11:19:53 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> Message-ID: <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Hi, As I found an excellent template in the SRPM of docbook2X package, I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. First of all, let me finish the work related to man pages. I hope Rainer-san and other contributors devote yourself to make rsyslog better :) Best Rio. On 2008/07/18, at 1:33, Rainer Gerhards wrote: > I still haven't found the time to have an in-depth look at docbook, > but > I strongly tend to agree to Michael's conclusion. It looks like a > problem solver for some ugly problems I am facing. > > I will probably re-evaluate the release goal for 3.21.x. It may happen > that I push back the script engine evolution to after summer to do > some > cleanup, performance improvement, doc work and maybe a rewrite of the > file output handler. I find it a bit hard to think what is more > important, but I think we can stay with the non-scriptable config for > two more month. Maybe I can add custom functions as a smaller > improvement sometime while doing the other things. > > In any case, I am happy to have Rio-san over here and as such > someone to > ask for solutions to the doc problem. Thanks a lot! > > Rainer > > On Thu, 2008-07-17 at 16:32 +0200, Michael Biebl wrote: >> Hi, >> >> I just wanted to say, that I like the idea of using docbook/xml to >> generate the (html) documentation (and possibly the man pages). >> Imo this would make the documentation more consistent (consistent >> font >> sizes, headers, footers etc.), more usable (You'd get something >> like a >> TOC and possibly better cross-linking) and perhaps easier keeping >> up-to-date. >> >> Cheers, >> Michael >> >> >> 2008/7/10 Ryo Fujita : >>> Hi Rainer-san, >>> >>> DocBook.org is here. >>> http://www.docbook.org/ >>> >>> And IBM has a good guide for beginners. >>> http://www.ibm.com/developerworks/xml/library/xml-matters3.html >>> >>> Recently, Red Hat uses DocBook format to generate web pages ,PDFs >>> and >>> so on. >>> For example, >>> https://www.redhat.com/docs/manuals/enterprise/ >>> A document having both html and PDF is generated from DocBook. >>> >>> gnome-doc-utils and kdesdk-utils include very convenient tools to >>> handle DocBook and related formats. >>> >>> Best Rio. >>> >>> On 2008/07/10, at 16:03, Rainer Gerhards wrote: >>> >>>> Hi Rio-san, >>>> >>>> I need to leave soon, but a quick feedback: the proposal looks >>>> *very* >>>> interesting. I need to digest it a bit. I have no experience with >>>> DocBook, so I probably need to check that out, first. That may >>>> take a >>>> short while (should you happen to know a good "getting started >>>> guide", >>>> I'd grateful if you send me a link ;)). From first thought, I would >>>> prefer to generate the html et al outputs from a single source >>>> (where >>>> only that source is in git). >>>> >>>> Feedback from other list members is also appreciated. >>>> >>>> Rainer >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>> Sent: Thursday, July 10, 2008 8:51 AM >>>>> To: rsyslog-users >>>>> Subject: [rsyslog] Suggestion for Documentation Reconstructure >>>>> >>>>> Hi List, >>>>> >>>>> I consulted with my colleagues about the smartest way to >>>>> reconstruct >>>>> the documents of rsyslog. >>>>> >>>>> 1.Work Flow matter >>>>> The attached image is just an idea for rational flowchart to make >>>>> translation easier. >>>>> #If this ML forbids me to attach any files, you can see it on my >>>>> site. >>>>> http://rio.tc/2008/07/10-144007.php >>>>> >>>>> I'm worried that this flow will give the contributers much >>>>> trouble. >>>>> >>>>> - HTML(from Wiki) to DocBook conversion >>>>> 'tidy' will help us a little, but exporter of media Wiki is >>>>> poor to >>>>> export 'well-formed HTML'. >>>>> We need to edit and check them manually. >>>>> Of course, I willingly do this. But it may take a number of days. >>>>> >>>>> - Writers need to edit DocBook XML after the reconstruction. >>>>> As you know, HTML format is not good for a source of multi- >>>>> output. >>>>> DocBook XML is a perfect one except for edit :) >>>>> >>>>> - Automake related matter >>>>> There can be two streams to generate man and html from DocBook >>>>> XML. >>>>> One : When you edit DocBook, you commit DocBook, HTML and man >>>>> files >>>>> to git tree. >>>>> Of course this method requires us to prepare an >>>>> environment where you can use xsltproc/docbook2man. >>>>> Two : By adding some sequences into Makefile, HTML and man files >>>>> can be generated automatically. >>>>> All persons who want to build rsyslog from tar ball >>>>> need >>>>> to prepare the environment. >>>>> And we need to put some check routines for >>>>> dependencies >>>>> into Makefile. >>>>> >>>>> 2.git tree reconstruct only with 'doc' directory >>>>> >>>>> `-- doc >>>>> |-- Makefile.am >>>>> |-- conf >>>>> | |-- en >>>>> | | `-- rsyslog-example.conf >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | `-- rsyslog-example.conf # annotations are translated. >>>>> |-- html >>>>> | |-- en >>>>> | | |-- bugs.html >>>>> | | |-- OTHER_HTMLS >>>>> | | `-- version_naming.html >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | |-- bugs.html >>>>> | |-- OTHER_HTMLS >>>>> | `-- version_naming.html >>>>> |-- images >>>>> | |-- gssapi.png >>>>> | |-- OTHER_IMAGES >>>>> | `-- tls_cert_ca.jpg >>>>> |-- man >>>>> | |-- en >>>>> | | |-- man5 >>>>> | | | `-- rsyslog.conf.5 >>>>> | | `-- man8 >>>>> | | `-- rsyslogd.8 >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | |-- man5 >>>>> | | `-- rsyslog.conf.5 >>>>> | `-- man8 >>>>> | `-- rsyslogd.8 >>>>> `-- src >>>>> |-- dias >>>>> | |-- classes.dia >>>>> | |-- OTHER_DIA_FILES >>>>> | `-- tls_cert_ca.dia >>>>> `-- docbook >>>>> |-- en >>>>> | |-- bugs.xml >>>>> | |-- rsyslog.conf.5.xml >>>>> | |-- rsyslogd.8.xml >>>>> | |-- OTHER_XMLS >>>>> | `-- version_naming.xml >>>>> `-- ja >>>>> |-- bugs.html >>>>> |-- rsyslog.conf.5.xml >>>>> |-- rsyslogd.8.xml >>>>> |-- OTHER_XMLS >>>>> `-- version_naming.html >>>>> >>>>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./ >>>>> tools/ >>>>> directory. >>>>> >>>>> Your suggestions will be highly appreciated!! >>>>> >>>>> Best Rio. >>>>> >>>>> >>>> ####################################################################### >>>>> # >>>>> Ryo Fujita >>>>> Senior Solution Architect, RHCE >>>>> Red Hat K.K. >>>>> TEL +81-3-5798-8500 >>>>> FAX +81-3-5798-8599 >>>>> Ebisu Neonato 8F >>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>> Tokyo Japan 1500013 >>>>> >>>> ####################################################################### >>>>> # >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> ######################################################################## >>> Ryo Fujita >>> Senior Solution Architect, RHCE >>> Red Hat K.K. >>> TEL +81-3-5798-8500 >>> FAX +81-3-5798-8599 >>> Ebisu Neonato 8F >>> 4-1-18 Ebisu, Shibuya-ku, >>> Tokyo Japan 1500013 >>> ######################################################################## >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From mbiebl at gmail.com Fri Jul 18 04:38:31 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 18 Jul 2008 04:38:31 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Message-ID: 2008/7/18 Ryo Fujita : > Hi, > > As I found an excellent template in the SRPM of docbook2X package, > I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. Do you have a public git repo somewhere and a separate branch, where you make these changes? It would be interesting to track the progress. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rfujita at redhat.com Fri Jul 18 05:17:36 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 12:17:36 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Message-ID: <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> Hi Mike-san, I have no git repo now. So, I sent the files I converted to Rainer-san directly. But I recently feel a necessity to build a public repo. If I have a time to do it this weekend, I'll try it :) Best Rio. On 2008/07/18, at 11:38, Michael Biebl wrote: > 2008/7/18 Ryo Fujita : >> Hi, >> >> As I found an excellent template in the SRPM of docbook2X package, >> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > > Do you have a public git repo somewhere and a separate branch, where > you make these changes? > > It would be interesting to track the progress. > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 08:14:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 08:14:26 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com> <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Hi Rio-san, I would really appreciate if you could build a git repro. I think that would be a very valuable resource and help streamline the integration process :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 5:18 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi Mike-san, > > I have no git repo now. > So, I sent the files I converted to Rainer-san directly. > But I recently feel a necessity to build a public repo. > If I have a time to do it this weekend, I'll try it :) > > Best Rio. > > On 2008/07/18, at 11:38, Michael Biebl wrote: > > > 2008/7/18 Ryo Fujita : > >> Hi, > >> > >> As I found an excellent template in the SRPM of docbook2X package, > >> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > > > > Do you have a public git repo somewhere and a separate branch, where > > you make these changes? > > > > It would be interesting to track the progress. > > > > Cheers, > > Michael > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 08:43:33 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 15:43:33 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com> <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Message-ID: Hi all, Please check the following git repo. http://rio.tc/git/rsyslog Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? # It's my first time to build my own git repo :) On 2008/07/18, at 15:14, Rainer Gerhards wrote: > Hi Rio-san, > > I would really appreciate if you could build a git repro. I think that > would be a very valuable resource and help streamline the integration > process :) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 5:18 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi Mike-san, >> >> I have no git repo now. >> So, I sent the files I converted to Rainer-san directly. >> But I recently feel a necessity to build a public repo. >> If I have a time to do it this weekend, I'll try it :) >> >> Best Rio. >> >> On 2008/07/18, at 11:38, Michael Biebl wrote: >> >>> 2008/7/18 Ryo Fujita : >>>> Hi, >>>> >>>> As I found an excellent template in the SRPM of docbook2X package, >>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>> >>> Do you have a public git repo somewhere and a separate branch, where >>> you make these changes? >>> >>> It would be interesting to track the progress. >>> >>> Cheers, >>> Michael >>> >>> -- >>> Why is it that all of the instruments seeking intelligent life in > the >>> universe are pointed away from Earth? >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 09:24:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 09:24:19 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> Hi Rio-san, I get a "file not found" error. Unfortunately, I am far from being a git-wizard, so I can not provide much advise :( Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 8:44 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi all, > > Please check the following git repo. > http://rio.tc/git/rsyslog > Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? > > # It's my first time to build my own git repo :) > > On 2008/07/18, at 15:14, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > I would really appreciate if you could build a git repro. I think > that > > would be a very valuable resource and help streamline the integration > > process :) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Friday, July 18, 2008 5:18 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >> > >> Hi Mike-san, > >> > >> I have no git repo now. > >> So, I sent the files I converted to Rainer-san directly. > >> But I recently feel a necessity to build a public repo. > >> If I have a time to do it this weekend, I'll try it :) > >> > >> Best Rio. > >> > >> On 2008/07/18, at 11:38, Michael Biebl wrote: > >> > >>> 2008/7/18 Ryo Fujita : > >>>> Hi, > >>>> > >>>> As I found an excellent template in the SRPM of docbook2X package, > >>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > >>> > >>> Do you have a public git repo somewhere and a separate branch, > where > >>> you make these changes? > >>> > >>> It would be interesting to track the progress. > >>> > >>> Cheers, > >>> Michael > >>> > >>> -- > >>> Why is it that all of the instruments seeking intelligent life in > > the > >>> universe are pointed away from Earth? > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 09:32:59 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 16:32:59 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> Message-ID: <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> Hi Rainer-san, Did you access with git? Access with web browser may fail or get 404 not found. I succeeded as follows. $ git clone http://rio.tc/git/rsyslog Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ Getting alternates list for http://rio.tc/git/rsyslog Getting pack list for http://rio.tc/git/rsyslog Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 $ cd rsyslog $ git branch -r origin/HEAD origin/beta origin/master origin/queuememleak origin/rfujita origin/rscript origin/tests origin/v1-stable origin/v2-stable origin/v3-stable origin/v3stable-klogbug origin/wall $ git-pull http://rio.tc/git/rsyslog rfujita Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http Updating 37bac26..9d841de Fast forward man/Makefile.am | 101 ++++ man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ man/en/man8/rsyslogd.8 | 303 +++++++++++ man/entities/xinclude.dtd | 31 + man/ja/man8/rsyslogd.8 | 319 +++++++++++ man/xml-source/en/rsyslog.conf.5.xml | 994 +++++++++++++++++++++++++ ++++++++ man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ man/xml-source/ja/rsyslog.conf.5.xml | 997 +++++++++++++++++++++++++ +++++++++ man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ man/xslt/docbook.xsl | 127 +++++ man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ man/xslt/man.xsl | 70 +++ man/xslt/settings.xsl | 11 + 13 files changed, 5464 insertions(+), 0 deletions(-) create mode 100644 man/Makefile.am create mode 100644 man/en/man5/rsyslog.conf.5 create mode 100644 man/en/man8/rsyslogd.8 create mode 100644 man/entities/xinclude.dtd create mode 100644 man/ja/man8/rsyslogd.8 create mode 100644 man/xml-source/en/rsyslog.conf.5.xml create mode 100644 man/xml-source/en/rsyslogd.8.xml create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml create mode 100644 man/xml-source/ja/rsyslogd.8.xml create mode 100644 man/xslt/docbook.xsl create mode 100644 man/xslt/expand-sambadoc.xsl create mode 100644 man/xslt/man.xsl create mode 100644 man/xslt/settings.xsl On 2008/07/18, at 16:24, Rainer Gerhards wrote: > Hi Rio-san, > > I get a "file not found" error. Unfortunately, I am far from being a > git-wizard, so I can not provide much advise :( > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 8:44 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi all, >> >> Please check the following git repo. >> http://rio.tc/git/rsyslog >> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? >> >> # It's my first time to build my own git repo :) >> >> On 2008/07/18, at 15:14, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> I would really appreciate if you could build a git repro. I think >> that >>> would be a very valuable resource and help streamline the > integration >>> process :) >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Friday, July 18, 2008 5:18 AM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >>>> >>>> Hi Mike-san, >>>> >>>> I have no git repo now. >>>> So, I sent the files I converted to Rainer-san directly. >>>> But I recently feel a necessity to build a public repo. >>>> If I have a time to do it this weekend, I'll try it :) >>>> >>>> Best Rio. >>>> >>>> On 2008/07/18, at 11:38, Michael Biebl wrote: >>>> >>>>> 2008/7/18 Ryo Fujita : >>>>>> Hi, >>>>>> >>>>>> As I found an excellent template in the SRPM of docbook2X > package, >>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>>>> >>>>> Do you have a public git repo somewhere and a separate branch, >> where >>>>> you make these changes? >>>>> >>>>> It would be interesting to track the progress. >>>>> >>>>> Cheers, >>>>> Michael >>>>> >>>>> -- >>>>> Why is it that all of the instruments seeking intelligent life in >>> the >>>>> universe are pointed away from Earth? >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> >> > ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> >> > ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 09:46:33 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 09:46:33 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> Hi Rio-san, doh ;) Yes, I was after the web interface. I now pulled the repo with git and everything works fine :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 9:33 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi Rainer-san, > > Did you access with git? > Access with web browser may fail or get 404 not found. > > I succeeded as follows. > > $ git clone http://rio.tc/git/rsyslog > Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ > Getting alternates list for http://rio.tc/git/rsyslog > Getting pack list for http://rio.tc/git/rsyslog > Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b > Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b > which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 > > > $ cd rsyslog > $ git branch -r > origin/HEAD > origin/beta > origin/master > origin/queuememleak > origin/rfujita > origin/rscript > origin/tests > origin/v1-stable > origin/v2-stable > origin/v3-stable > origin/v3stable-klogbug > origin/wall > > $ git-pull http://rio.tc/git/rsyslog rfujita > Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http > Updating 37bac26..9d841de > > Fast forward > man/Makefile.am | 101 ++++ > man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ > man/en/man8/rsyslogd.8 | 303 +++++++++++ > man/entities/xinclude.dtd | 31 + > man/ja/man8/rsyslogd.8 | 319 +++++++++++ > man/xml-source/en/rsyslog.conf.5.xml | 994 +++++++++++++++++++++++++ > ++++++++ > man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ > man/xml-source/ja/rsyslog.conf.5.xml | 997 +++++++++++++++++++++++++ > +++++++++ > man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ > man/xslt/docbook.xsl | 127 +++++ > man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ > man/xslt/man.xsl | 70 +++ > man/xslt/settings.xsl | 11 + > 13 files changed, 5464 insertions(+), 0 deletions(-) > create mode 100644 man/Makefile.am > create mode 100644 man/en/man5/rsyslog.conf.5 > create mode 100644 man/en/man8/rsyslogd.8 > create mode 100644 man/entities/xinclude.dtd > create mode 100644 man/ja/man8/rsyslogd.8 > create mode 100644 man/xml-source/en/rsyslog.conf.5.xml > create mode 100644 man/xml-source/en/rsyslogd.8.xml > create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml > create mode 100644 man/xml-source/ja/rsyslogd.8.xml > create mode 100644 man/xslt/docbook.xsl > create mode 100644 man/xslt/expand-sambadoc.xsl > create mode 100644 man/xslt/man.xsl > create mode 100644 man/xslt/settings.xsl > > On 2008/07/18, at 16:24, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > I get a "file not found" error. Unfortunately, I am far from being a > > git-wizard, so I can not provide much advise :( > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Friday, July 18, 2008 8:44 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >> > >> Hi all, > >> > >> Please check the following git repo. > >> http://rio.tc/git/rsyslog > >> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? > >> > >> # It's my first time to build my own git repo :) > >> > >> On 2008/07/18, at 15:14, Rainer Gerhards wrote: > >> > >>> Hi Rio-san, > >>> > >>> I would really appreciate if you could build a git repro. I think > >> that > >>> would be a very valuable resource and help streamline the > > integration > >>> process :) > >>> > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>>> Sent: Friday, July 18, 2008 5:18 AM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >>>> > >>>> Hi Mike-san, > >>>> > >>>> I have no git repo now. > >>>> So, I sent the files I converted to Rainer-san directly. > >>>> But I recently feel a necessity to build a public repo. > >>>> If I have a time to do it this weekend, I'll try it :) > >>>> > >>>> Best Rio. > >>>> > >>>> On 2008/07/18, at 11:38, Michael Biebl wrote: > >>>> > >>>>> 2008/7/18 Ryo Fujita : > >>>>>> Hi, > >>>>>> > >>>>>> As I found an excellent template in the SRPM of docbook2X > > package, > >>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > >>>>> > >>>>> Do you have a public git repo somewhere and a separate branch, > >> where > >>>>> you make these changes? > >>>>> > >>>>> It would be interesting to track the progress. > >>>>> > >>>>> Cheers, > >>>>> Michael > >>>>> > >>>>> -- > >>>>> Why is it that all of the instruments seeking intelligent life in > >>> the > >>>>> universe are pointed away from Earth? > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>> > >> > > > ####################################################################### > >>>> # > >>>> Ryo Fujita > >>>> Senior Solution Architect, RHCE > >>>> Red Hat K.K. > >>>> TEL +81-3-5798-8500 > >>>> FAX +81-3-5798-8599 > >>>> Ebisu Neonato 8F > >>>> 4-1-18 Ebisu, Shibuya-ku, > >>>> Tokyo Japan 1500013 > >>>> > >>> > >> > > > ####################################################################### > >>>> # > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 09:50:28 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 16:50:28 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> Message-ID: Fine! I don't know how to publish git directory for web browser. If I find the way to do, all you can access with browser :) Best Rio. On 2008/07/18, at 16:46, Rainer Gerhards wrote: > Hi Rio-san, > > doh ;) Yes, I was after the web interface. I now pulled the repo with > git and everything works fine :) > > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 9:33 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi Rainer-san, >> >> Did you access with git? >> Access with web browser may fail or get 404 not found. >> >> I succeeded as follows. >> >> $ git clone http://rio.tc/git/rsyslog >> Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ >> Getting alternates list for http://rio.tc/git/rsyslog >> Getting pack list for http://rio.tc/git/rsyslog >> Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b >> Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b >> which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 >> >> >> $ cd rsyslog >> $ git branch -r >> origin/HEAD >> origin/beta >> origin/master >> origin/queuememleak >> origin/rfujita >> origin/rscript >> origin/tests >> origin/v1-stable >> origin/v2-stable >> origin/v3-stable >> origin/v3stable-klogbug >> origin/wall >> >> $ git-pull http://rio.tc/git/rsyslog rfujita >> Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http >> Updating 37bac26..9d841de >> >> Fast forward >> man/Makefile.am | 101 ++++ >> man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ >> man/en/man8/rsyslogd.8 | 303 +++++++++++ >> man/entities/xinclude.dtd | 31 + >> man/ja/man8/rsyslogd.8 | 319 +++++++++++ >> man/xml-source/en/rsyslog.conf.5.xml | 994 > +++++++++++++++++++++++++ >> ++++++++ >> man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ >> man/xml-source/ja/rsyslog.conf.5.xml | 997 > +++++++++++++++++++++++++ >> +++++++++ >> man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ >> man/xslt/docbook.xsl | 127 +++++ >> man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ >> man/xslt/man.xsl | 70 +++ >> man/xslt/settings.xsl | 11 + >> 13 files changed, 5464 insertions(+), 0 deletions(-) >> create mode 100644 man/Makefile.am >> create mode 100644 man/en/man5/rsyslog.conf.5 >> create mode 100644 man/en/man8/rsyslogd.8 >> create mode 100644 man/entities/xinclude.dtd >> create mode 100644 man/ja/man8/rsyslogd.8 >> create mode 100644 man/xml-source/en/rsyslog.conf.5.xml >> create mode 100644 man/xml-source/en/rsyslogd.8.xml >> create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml >> create mode 100644 man/xml-source/ja/rsyslogd.8.xml >> create mode 100644 man/xslt/docbook.xsl >> create mode 100644 man/xslt/expand-sambadoc.xsl >> create mode 100644 man/xslt/man.xsl >> create mode 100644 man/xslt/settings.xsl >> >> On 2008/07/18, at 16:24, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> I get a "file not found" error. Unfortunately, I am far from being a >>> git-wizard, so I can not provide much advise :( >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Friday, July 18, 2008 8:44 AM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >>>> >>>> Hi all, >>>> >>>> Please check the following git repo. >>>> http://rio.tc/git/rsyslog >>>> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? >>>> >>>> # It's my first time to build my own git repo :) >>>> >>>> On 2008/07/18, at 15:14, Rainer Gerhards wrote: >>>> >>>>> Hi Rio-san, >>>>> >>>>> I would really appreciate if you could build a git repro. I think >>>> that >>>>> would be a very valuable resource and help streamline the >>> integration >>>>> process :) >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>>> Sent: Friday, July 18, 2008 5:18 AM >>>>>> To: rsyslog-users >>>>>> Subject: Re: [rsyslog] Suggestion for Documentation > Reconstructure >>>>>> >>>>>> Hi Mike-san, >>>>>> >>>>>> I have no git repo now. >>>>>> So, I sent the files I converted to Rainer-san directly. >>>>>> But I recently feel a necessity to build a public repo. >>>>>> If I have a time to do it this weekend, I'll try it :) >>>>>> >>>>>> Best Rio. >>>>>> >>>>>> On 2008/07/18, at 11:38, Michael Biebl wrote: >>>>>> >>>>>>> 2008/7/18 Ryo Fujita : >>>>>>>> Hi, >>>>>>>> >>>>>>>> As I found an excellent template in the SRPM of docbook2X >>> package, >>>>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>>>>>> >>>>>>> Do you have a public git repo somewhere and a separate branch, >>>> where >>>>>>> you make these changes? >>>>>>> >>>>>>> It would be interesting to track the progress. >>>>>>> >>>>>>> Cheers, >>>>>>> Michael >>>>>>> >>>>>>> -- >>>>>>> Why is it that all of the instruments seeking intelligent life > in >>>>> the >>>>>>> universe are pointed away from Earth? >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> >>>>>> >>>>> >>>> >>> >> > ####################################################################### >>>>>> # >>>>>> Ryo Fujita >>>>>> Senior Solution Architect, RHCE >>>>>> Red Hat K.K. >>>>>> TEL +81-3-5798-8500 >>>>>> FAX +81-3-5798-8599 >>>>>> Ebisu Neonato 8F >>>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>>> Tokyo Japan 1500013 >>>>>> >>>>> >>>> >>> >> > ####################################################################### >>>>>> # >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> >> > ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> >> > ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 14:34:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 14:34:41 +0200 Subject: [rsyslog] slight kernel log format inconsistencybetween 3.16.2and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com><1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE2F@grfint2.intern.adiscon.com> FYI: I have now taken action against the format problem. I think the changelog entries tells it all: - added a new property replacer option "sp-if-no-1st-sp" to cover a problem with RFC 3164 based interpretation of tag separation. While it is a generic approach, it fixes a format problem introduced in 3.18.0, where kernel messages no longer had a space after the tag. This is done by a modification of the default templates. Please note that this may affect some messages where there intentionally is no space between the tag and the first character of the message content. If so, this needs to be worked around via a specific template. However, we consider this scenario to be quite remote and, even if it exists, it is not expected that it will actually cause problems with log parsers (instead, we assume the new default template behavior may fix previous problems with log parsers due to the missing space). It is not a perfect solution, but I hope a pragmatic and good-enough one. Other than format issues, I had also some performance concerns. What I have now implemented affects performance very little. Most importantly, it enables us to stay away from copying large strings in memory just to prepend a space. If some has a concern, please voice it. This patch will be part of 3.21.0 (as it looks to be released today) and, if no hard objection is received, 3.18.1. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 15, 2008 4:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] slight kernel log format inconsistencybetween > 3.16.2and 3.18.0 > > Michael, > > I have reviewed some of the code in question. It looks like a bug. It's > actually not a klog driver issue, the PRI parsing is done at an upper > klog layer and thus should work for linux as well. I would like to > generate a version with some additional instrumentation. Could you run > it and report the results back (maybe via private mail to keep the list > free of noise). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Tuesday, July 15, 2008 3:04 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] slight kernel log format inconsistency between > > 3.16.2and 3.18.0 > > > > On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > > > 2008/7/14 Rainer Gerhards : > > > > Hi all, > > > > > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > > > > > 3.16.x (and before): > > > > Jul 11 17:33:28 pluto kernel: [14177.432349] > > ADDRCONF(NETDEV_CHANGE): > > > > wlan0: link becomes ready > > > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > > > > > > > 3.18.0: > > > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = > /proc/kmsg > > > > started. > > > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > > > > > another example: > > > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > > > See the <6> after the kernel: tag. It's also new. > > > Is this some kind of log level, set by the kernel? > > > > This does not look correct. Under FreeBSD, it is typical to have a > PRI > > inside kernel messages. But for linux kernels I tested with, I did > not > > see it and (I think) so I did not support it. But if you see it on > > Debian, it seems to happen. A cure is to use the same logic that is > > used > > for BSD. > > > > > I have never seen, that other syslogger (like syslog-ng, sysklogd > set > > this). > > > > > > > > > > > As you can see, there is a space after kernel: in the pre- > > 3.18.0 > > > > messages. This also brings us down to a general inconsistency: > > > > > > > > Unfortunately, RFC 3164 defines the after TAG not as a > > delimiter > > > > but as part of the CONTENT of the message. This leads to some > > > > inconsistency in practice. Depending on who generated the > message, > > there > > > > may be a space after the TAG or not. If it is, that space must be > > > > submitted as part of the message itself. > > > > > > > > In versions up to 3.16.x, kernel log messages were generated by > old > > > > code, which did not know about tag and simply generated a non- > > structured > > > > message. With 3.17.x and above, the klog code is rewritten and > > correctly > > > > fills in all properties. This leads to the "missing" space, > because > > the > > > > space is neither part of the tag nor part of the message. > > > > > > > > I have now three options: > > > > > > > > 1. leave as is (without a space) > > > > In that case, some log parsers may run into troubles > > > > > > This was a rather vague concern of mine. I don't actually know, if > > > other software is affected by this change. > > > > I share this concern, but also have no additional information. I > wanted > > to point out the options. I have to admit I am still undecided. But > > that > > nobody violently objected so far makes this look like it is not so > > important... > > > > > > > > > 2. add a at the end of the TAG > > > > That will probably lead to even more confusion, as matches > against > > TAG > > > > would need to include that space. > > > > > > > > 3. add a in front of each kernel message > > > > This comes close to the previous mode, but is "a bit" clumpsy and > > > > hackish. It will also make all database tables etc have messages > > > > starting with . Similar as 2, MSG matching rules are affected > > (but I > > > > consider this less severe, as matches inside the MSG are usually > > driven > > > > by searches and not absolute positions). > > > > > > Just curious: For other log messages, that are not generated by the > > > kernel, you have a space after the programname: tag, e.g. > > > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > > > > > Is the space after dhclient: part of the message or part of the > > > :programname tag? > > > > In our private communication, I thought it is part of TAG. But that > is > > wrong. It actually is part of MSG - at least for legacy/RFC3164 > syslog. > > > > For the new IETF syslog RFC series (syslog-protocol, -tls et al) it > is > > well-defined and part of neither. There, it is a delimiter (as one > > would > > expect). I you use rsyslog only with syslog-protocol messages (and > the > > syslog-protocol templates), this is more or less no issue. But who > does > > today...? ;) > > > > The real problem is that legacy syslog has many different > > interpretations and we break one or the other in either way... > > > > > > > > Imho, if there is no clear definition in the RFC, how the message > > > should be written, it's best to stay backwards compatible. > > > > Is that a vote for option 3, stick a space in front of each kernel > > message? ;) > > > > > > > > > > > Cheers, > > > Michael > > > > > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 18 14:42:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 14:42:12 +0200 Subject: [rsyslog] preview of 3.18.1 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE30@grfint2.intern.adiscon.com> Hi all, as you are probably aware, 3.18.1 contains a "fix" for a format issue with kernel messages. This fix has a somewhat broader scope. In an ideal world, I would have preferred to run it through the beta phase and not apply it directly to stable. But this is counter-productive, as the format issue may affect users of v3-stable *right now*. So I am doing the second best thing: I have prepared a preview tarball and hope that some of you will implement it and tell me if it works OK. I do not expect any problems, but, hey, there is always a difference between lab and practice. I have pushed back 3.18.1 in the hopes to get some feedback. I need to release 3.18.1 soon, preferably by next Monday or Tuesday. So I would appreciate any feedback you can offer (including "no problem experienced" feedback). The tarball is available at: http://download.rsyslog.com/rsyslog/rsyslog-3.18.1-Test1.tar.gz This most probably is the official 3.18.1, except for some changes to cover the new release number (except, of course, feedback requires changes). Thanks, Rainer From friedl at hq.adiscon.com Fri Jul 18 17:25:23 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 18 Jul 2008 17:25:23 +0200 Subject: [rsyslog] rsyslog 3.21.0 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE36@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.21.0, the first release of the new 3.21.x devel branch. It includes all fixes and additions of the current v3-stable and beta plus an improved testbench. Please note that this includes some vital bugfixes from earlier releases. 3.21.0 lays the foundation for the next round of rsyslog enhancements. It is a recommended update for all devel branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-121.phtml Changelog: http://www.rsyslog.com/Article258.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Mon Jul 21 18:16:24 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 21 Jul 2008 18:16:24 +0200 Subject: [rsyslog] rsyslog 3.18.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE53@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.18.1. It is a v3-stable branch version. It offers a number of bug fixes and solves portability issues on FreeBSD. Most importantly, a format difference in kernel logs between 3.16.x and 3.18.0 has been corrected. This is a recommended update for all v3-stable branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-122.phtml Changelog: http://www.rsyslog.com/Article260.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From martinmie at PartyGaming.com Tue Jul 22 10:38:32 2008 From: martinmie at PartyGaming.com (Martin Mielke) Date: Tue, 22 Jul 2008 10:38:32 +0200 Subject: [rsyslog] Whitepapers on perfomance Message-ID: <0B1B9163138571439EAF804646F3F6AA051AD910@GIBSVWIN004X.partygaming.local> Hi list, Can someone point me to a official document covering rsyslog's performance under heavy load? TIA, Martin From rgerhards at hq.adiscon.com Tue Jul 22 12:16:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 22 Jul 2008 12:16:28 +0200 Subject: [rsyslog] rsyslog scheduled as new default syslogd on Debian Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> Hi folks, I thought I share the good news. I have just blogged about the details: http://blog.gerhards.net/2008/07/rsyslog-will-become-debians-default.htm l Special thanks go to Michael Biebl, who is the driving force behind this development. Rainer From rfujita at redhat.com Wed Jul 23 19:10:56 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 24 Jul 2008 02:10:56 +0900 Subject: [rsyslog] rsyslog scheduled as new default syslogd on Debian In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> Message-ID: Hi Rainer-san, Congrats! And the news cheer me up, too! Best Rio. On 2008/07/22, at 19:16, Rainer Gerhards wrote: > Hi folks, > > I thought I share the good news. I have just blogged about the > details: > > http://blog.gerhards.net/2008/07/rsyslog-will-become-debians-default.htm > l > > Special thanks go to Michael Biebl, who is the driving force behind > this > development. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From mmcgrath at redhat.com Wed Jul 23 21:21:46 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 14:21:46 -0500 (CDT) Subject: [rsyslog] rsyslog dropping logs Message-ID: I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central logging server. When running tcpdump I'm seeing all the udp packets coming in but many of them are not getting logged. And we're talking like 10% or so getting logged (maybe less) and the rest are just lost. I've attached my config file. (side note, if I'm doing something stupid in the config please correct me) -Mike -------------- next part -------------- #rsyslog v3 config file #### GLOBAL DIRECTIVES #### # Use default timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # File syncing capability is disabled by default. This feature is usually not required, # not useful and an extreme performance hit #$ActionFileEnableSync on # An "In-Memory Queue" is created for remote logging. # $WorkDirectory /var/spool/rsyslog # where to place spool files # $ActionQueueFileName queue # unique name prefix for spool files # $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) # $ActionQueueSaveOnShutdown on # save messages to disk on shutdown # $ActionQueueType LinkedList # run asynchronously # $ActionResumeRetryCount -1 # infinety retries if host is down #### MODULES #### $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # provides kernel logging support (previously done by rklogd) #$ModLoad immark.so # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp.so $UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp.so #$InputTCPServerRun 514 $FileGroup sysadmin $FileCreateMode 0640 $DirGroup sysadmin $DirCreateMode 0750 #### RULES #### # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;local1.none;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log $template RawMessage,"%msg%\n" $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" if $app-name contains 'access.log' then -?HttpAccessTemplate;RawMessage $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" if $app-name contains 'error.log' then -?HttpErrorTemplate;RawMessage $template DynaFile,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/messages.log" *.*;local1.none -?DynaFile From Samuel.Kielek at marriott.com Wed Jul 23 21:44:25 2008 From: Samuel.Kielek at marriott.com (Kielek, Samuel) Date: Wed, 23 Jul 2008 15:44:25 -0400 Subject: [rsyslog] rsyslog dropping logs In-Reply-To: References: Message-ID: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> Mike, You're using some v3 features that will not work with v2 such as the conditional filtering. I'll email you off list with a copy of my config (also using RHEL 5.2 here). -Sam -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Mike McGrath Sent: Wednesday, July 23, 2008 3:22 PM To: rsyslog at lists.adiscon.com Subject: [rsyslog] rsyslog dropping logs I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central logging server. When running tcpdump I'm seeing all the udp packets coming in but many of them are not getting logged. And we're talking like 10% or so getting logged (maybe less) and the rest are just lost. I've attached my config file. (side note, if I'm doing something stupid in the config please correct me) -Mike From mmcgrath at redhat.com Wed Jul 23 23:04:24 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 16:04:24 -0500 (CDT) Subject: [rsyslog] Leading whitespace Message-ID: Right now I'm doing live logging of my http servers to my rsyslog host. I've got this code on the rsyslog server: $template RawMessage,"%msg%\n" $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" On the http servers I've got: CustomLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-access.log" combined ErrorLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-error.log" It works great except for one thing. The logs have an additional whitespace in front of each line. for example: 111.111.111.11 - - [23/Jul/2008:21:01:32 +0000] "GET /static/images/fedora-logo.png HTTP/1.1" 200 5354 "http://fedoraproject.org/static/css/fedora.css" "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.15) Gecko/20080702Fedora/2.0.0.15-1.fc8 Firefox/2.0.0.15" Any ideas? -Mike From mmcgrath at redhat.com Wed Jul 23 23:06:24 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 16:06:24 -0500 (CDT) Subject: [rsyslog] rsyslog dropping logs In-Reply-To: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> References: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> Message-ID: On Wed, 23 Jul 2008, Kielek, Samuel wrote: > Mike, > > You're using some v3 features that will not work with v2 such as the > conditional filtering. I'll email you off list with a copy of my config > (also using RHEL 5.2 here). > Thanks, I upgraded to a v3 version with my old config, same issue. Used the config provided off list and it works great. even in v3. -Mike From rgerhards at hq.adiscon.com Thu Jul 24 12:27:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 24 Jul 2008 12:27:05 +0200 Subject: [rsyslog] Leading whitespace In-Reply-To: References: Message-ID: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> This is rooted in RFC 3164. A description can be found here: http://lists.adiscon.net/pipermail/rsyslog/2008-July/000893.html If you want to consistently get rid of that SP, you can remove it via the property replacer. I think this works: ?$template RawMessage,"%msg:2%\n" if not, then this ;) ??$template RawMessage,"%msg:2:2048%\n" Let us know if that cured the problem. Rainer On Wed, 2008-07-23 at 16:04 -0500, Mike McGrath wrote: > Right now I'm doing live logging of my http servers to my rsyslog host. > I've got this code on the rsyslog server: > > $template RawMessage,"%msg%\n" > $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" > $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" > > > On the http servers I've got: > > CustomLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-access.log" combined > ErrorLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-error.log" > > > It works great except for one thing. The logs have an additional > whitespace in front of each line. for example: > > 111.111.111.11 - - [23/Jul/2008:21:01:32 +0000] "GET /static/images/fedora-logo.png HTTP/1.1" 200 5354 "http://fedoraproject.org/static/css/fedora.css" "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.15) Gecko/20080702Fedora/2.0.0.15-1.fc8 Firefox/2.0.0.15" > > Any ideas? > > -Mike > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 24 12:40:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 24 Jul 2008 12:40:24 +0200 Subject: [rsyslog] rsyslog dropping logs In-Reply-To: References: Message-ID: <1216896024.7184.189.camel@rgf9dev.intern.adiscon.com> (I am not commenting on v2 vs. v3 as this is already done) First of all, we need to keep in mind that UDP is inherently lossy. Even when a frame is seen received by the local stack, it does not mean that it will eventually be forwarded to the application. If message bursts come in very quickly and the OS scheduler does not schedule the app fast enough to receive this messages (or the app is too slow in itself! ;)) new frames may overwrite frames inside the stack's receive buffers. So it is always a good idea to avoid UDP if that's possible. HOWEVER, I, too, find it somewhat unusual that around 90% of all incoming frames are discarded before the rsyslog receiver could process them. One explanation I have is that you have bursts (or volume in general) that outperforms the configured actions. Having seen the config file, and seeing it does not include any database writer, it is hard to imagine this should happen, assuming reasonable hardware sizing is used. A cause could be excessive synchronous writes. Many rules do not put a dash in front of the file name and without it (in v2), every write is immediately synced. This is very costly. But still, I have never seen that this alone outperforms a system. To dig deeper into what is happening, a debug log would be most useful, together with the information which frames have been seen in tcpdump but NOT in one of the log files. You can enable debug mode via -dn command line switch and is recommended to run rsyslog interactively while doing so. Then, you can simply capture its output via stdout redirection. Please note that debug mode generates considerable output, and requires considerable additional processing time. In any case, though, it should show us where the bottleneck is. Please note that I need a consistent excerpt from the debug log that shows how things began and how it worked during the fault conditions. Usually, this means I need everything ;) Debug logs may also reveal sensitive information, even passwords, so you should be careful in what you do. I am used to log files around the size of 1GB. With reasonable compression, the transfer is usually not a problem (but I suggest you place them on a server for me to download). Download links and/or smaller logs you can email me privately at rgerhards at gmail.com (please NOT at my primary, adiscon, email address). I hope this helps and I am looking forward for the additional information. Rainer On Wed, 2008-07-23 at 14:21 -0500, Mike McGrath wrote: > I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central > logging server. When running tcpdump I'm seeing all the udp packets > coming in but many of them are not getting logged. And we're talking > like 10% or so getting logged (maybe less) and the rest are just lost. > I've attached my config file. > > (side note, if I'm doing something stupid in the config please correct me) > > -Mike > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From mmcgrath at redhat.com Thu Jul 24 15:41:59 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 24 Jul 2008 08:41:59 -0500 (CDT) Subject: [rsyslog] Leading whitespace In-Reply-To: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> References: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> Message-ID: On Thu, 24 Jul 2008, Rainer Gerhards wrote: > This is rooted in RFC 3164. A description can be found here: > > http://lists.adiscon.net/pipermail/rsyslog/2008-July/000893.html > > If you want to consistently get rid of that SP, you can remove it via > the property replacer. I think this works: > > ?$template RawMessage,"%msg:2%\n" > > if not, then this ;) > > ??$template RawMessage,"%msg:2:2048%\n" > > Let us know if that cured the problem. > %msg:2:2048% seems to work perfectly, thank you. -Mike From Kevin.Goutos at budget.state.ny.us Thu Jul 24 21:50:04 2008 From: Kevin.Goutos at budget.state.ny.us (Goutos, Kevin (DOB)) Date: Thu, 24 Jul 2008 15:50:04 -0400 Subject: [rsyslog] Help with Ommail Configuration Message-ID: Hello all, First off, I am not very Linux savvy. I don't have a lot of experience other then a basic course. This is probably way past my knowledge, but I really need to get it done. Appreciate any help you guys have to offer. I am working on a Red Hat Enterprise 4 box and I am running the latest edition of rsyslog. I currently have rsyslog configured to receive messages remotely via UDP. I am trying to set it up so that it will send out E-mail messages to the system Admin's based on the severity level of the log files I am receiving. I would like it so that any device that sends a log with a critical, alert, or emergency level facility will send out an e-mail to a specific address. Here is my rsyslog.conf file. I used the sample code from Rainer Gerhards configuration page. I tried sending a test syslog with 'hard disk fatal failure' in it, but it is not sending out mail. Also tried without the templates below thinking it would just send me an email for every syslog that I received, but it doesn't appear to be sending mail. Any thoughts on what I am doing wrong. I'm sure there is a lot I need to do, so please let me know. Thanks! $template mailSubject,"disk problem on %hostname%" $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' I edited out the 3 lines below in the code for security reasons.. $ActionMailSMTPServer $ActionMailFrom $ActionMailTo Here is my code from rsyslog.conf below. # if you experience problems, check # http://www.rsyslog.com/troubleshoot for assistance # rsyslog v3: load input modules # If you do not load inputs, nothing happens! # You may need to set the module load path if modules are not found. $ModLoad immark.so # provides --MARK-- message capability $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # kernel logging (formerly provided by rklogd) $ModLoad ommail $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none -/var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* -/var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit -/var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log #Catch all incoming syslog messages *.* /var/log/catchall;TraditionalFormatWithPRI # Remote Logging (we use TCP for reliable delivery) # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. $WorkDirectory /rsyslog/spool # where to place spool files $ActionQueueFileName uniqName # unique name prefix for spool files $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueType LinkedList # run asynchronously $ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional *.* @10.57.106.140:514 $ModLoad ommail $ActionMailSMTPServer $ActionMailFrom $ActionMailTo $template mailSubject,"disk problem on %hostname%" $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" $ActionMailSubject mailSubject # make sure we receive a mail only once in six # hours (21,600 seconds ;)) $ActionExecOnlyOnceEveryInterval 21600 # the if ... then ... mailBody mus be on one line! if $msg contains 'hard disk fatal failure' then :ommail:;mailBody # ######### Receiving Messages from Remote Hosts ########## # TCP Syslog Server: # provides TCP syslog reception and GSS-API (if compiled to support it) $ModLoad imtcp.so # load module $InputTCPServerRun 514 # start up TCP listener at port 514 # UDP Syslog Server: $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port -------------------------------------------------------- This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. From hks.private at gmail.com Thu Jul 24 23:01:42 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 24 Jul 2008 17:01:42 -0400 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: A few things to try: - You load ommail.so twice, once at the top and once right above your $ActionMail... lines. I don't think this will break it, but it's unnecessary - delete the second one. - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going to attempt sending a message once every 6 hours. For testing purposes, this will be obnoxious. Until you're ready for production, change it to: $ActionExecOnlyOnceEveryInterval 30 - Send everything to make sure you're not missing it based on something in the property-replacer/templates/whatever. Replace "if $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: *.* :ommail:;mailBody Try again. Try logging a few messages from the localhost first (with RHEL you can just run "logger test" to log a user.notice message with content "test") and see if you get them. If you don't, check the mail logs on your mail server to see if it ever received the message. If not, it's time to break out tcpdump and see if the packets are ever being generated. Hope that helps. -HKS On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) wrote: > Hello all, > > First off, I am not very Linux savvy. I don't have a lot of experience > other then a basic course. This is probably way past my knowledge, but I > really need to get it done. Appreciate any help you guys have to offer. > > I am working on a Red Hat Enterprise 4 box and I am running the latest > edition of rsyslog. I currently have rsyslog configured to receive > messages remotely via UDP. I am trying to set it up so that it will send > out E-mail messages to the system Admin's based on the severity level of > the log files I am receiving. I would like it so that any device that > sends a log with a critical, alert, or emergency level facility will > send out an e-mail to a specific address. > > Here is my rsyslog.conf file. I used the sample code from Rainer > Gerhards configuration page. I tried sending a test syslog with 'hard > disk fatal failure' in it, but it is not sending out mail. Also tried > without the templates below thinking it would just send me an email for > every syslog that I received, but it doesn't appear to be sending mail. > Any thoughts on what I am doing wrong. I'm sure there is a lot I need to > do, so please let me know. Thanks! > > $template mailSubject,"disk problem on %hostname%" > $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > > I edited out the 3 lines below in the code for security reasons.. > $ActionMailSMTPServer > $ActionMailFrom > $ActionMailTo > > > > Here is my code from rsyslog.conf below. > > > > # if you experience problems, check > # http://www.rsyslog.com/troubleshoot for assistance > > # rsyslog v3: load input modules > # If you do not load inputs, nothing happens! > # You may need to set the module load path if modules are not found. > > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging (e.g. > via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > $ModLoad ommail > > $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > > # Log all kernel messages to the console. > # Logging much else clutters up the screen. > #kern.* /dev/console > > # Log anything (except mail) of level info or higher. > # Don't log private authentication messages! > *.info;mail.none;authpriv.none;cron.none > -/var/log/messages > > # The authpriv file has restricted access. > authpriv.* /var/log/secure > > # Log all the mail messages in one place. > mail.* > -/var/log/maillog > > # Log cron stuff > cron.* -/var/log/cron > > # Everybody gets emergency messages > *.emerg * > > # Save news errors of level crit and higher in a special file. > uucp,news.crit > -/var/log/spooler > > # Save boot messages also to boot.log > local7.* > /var/log/boot.log > > #Catch all incoming syslog messages > *.* > /var/log/catchall;TraditionalFormatWithPRI > > # Remote Logging (we use TCP for reliable delivery) > # An on-disk queue is created for this action. If the remote host is > # down, messages are spooled to disk and sent when it is up again. > $WorkDirectory /rsyslog/spool # where to place spool files > $ActionQueueFileName uniqName # unique name prefix for spool files > $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > possible) > $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > $ActionQueueType LinkedList # run asynchronously > $ActionResumeRetryCount -1 # infinite retries if host is down > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > *.* @10.57.106.140:514 > > $ModLoad ommail > $ActionMailSMTPServer > $ActionMailFrom > $ActionMailTo > $template mailSubject,"disk problem on %hostname%" > $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > $ActionMailSubject mailSubject > # make sure we receive a mail only once in six > # hours (21,600 seconds ;)) > $ActionExecOnlyOnceEveryInterval 21600 > # the if ... then ... mailBody mus be on one line! > if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > > > # ######### Receiving Messages from Remote Hosts ########## > # TCP Syslog Server: > # provides TCP syslog reception and GSS-API (if compiled to support it) > $ModLoad imtcp.so # load module > $InputTCPServerRun 514 # start up TCP listener at port 514 > > # UDP Syslog Server: > $ModLoad imudp.so # provides UDP syslog reception > $UDPServerRun 514 # start a UDP syslog server at standard port > -------------------------------------------------------- > This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From hks.private at gmail.com Thu Jul 24 23:35:39 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 24 Jul 2008 17:35:39 -0400 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: Oh - one other possibility. rsyslogd has to have mail enabled at compile time to work with it at runtime. check your catchall logfile - if it has messages like this, you didn't compile it in: rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object file: No such file or directory To fix this, you'll need to reinstall. This time, when you run ./configure be sure to include --enable-mail. make install clean and you should be good to go... -HKS On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS wrote: > A few things to try: > > - You load ommail.so twice, once at the top and once right above your > $ActionMail... lines. I don't think this will break it, but it's > unnecessary - delete the second one. > > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > to attempt sending a message once every 6 hours. For testing purposes, > this will be obnoxious. Until you're ready for production, change it > to: > $ActionExecOnlyOnceEveryInterval 30 > > - Send everything to make sure you're not missing it based on > something in the property-replacer/templates/whatever. Replace "if > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: > *.* :ommail:;mailBody > > Try again. Try logging a few messages from the localhost first (with > RHEL you can just run "logger test" to log a user.notice message with > content "test") and see if you get them. > > If you don't, check the mail logs on your mail server to see if it > ever received the message. If not, it's time to break out tcpdump and > see if the packets are ever being generated. > > Hope that helps. > > -HKS > > > > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > wrote: >> Hello all, >> >> First off, I am not very Linux savvy. I don't have a lot of experience >> other then a basic course. This is probably way past my knowledge, but I >> really need to get it done. Appreciate any help you guys have to offer. >> >> I am working on a Red Hat Enterprise 4 box and I am running the latest >> edition of rsyslog. I currently have rsyslog configured to receive >> messages remotely via UDP. I am trying to set it up so that it will send >> out E-mail messages to the system Admin's based on the severity level of >> the log files I am receiving. I would like it so that any device that >> sends a log with a critical, alert, or emergency level facility will >> send out an e-mail to a specific address. >> >> Here is my rsyslog.conf file. I used the sample code from Rainer >> Gerhards configuration page. I tried sending a test syslog with 'hard >> disk fatal failure' in it, but it is not sending out mail. Also tried >> without the templates below thinking it would just send me an email for >> every syslog that I received, but it doesn't appear to be sending mail. >> Any thoughts on what I am doing wrong. I'm sure there is a lot I need to >> do, so please let me know. Thanks! >> >> $template mailSubject,"disk problem on %hostname%" >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >> >> I edited out the 3 lines below in the code for security reasons.. >> $ActionMailSMTPServer >> $ActionMailFrom >> $ActionMailTo >> >> >> >> Here is my code from rsyslog.conf below. >> >> >> >> # if you experience problems, check >> # http://www.rsyslog.com/troubleshoot for assistance >> >> # rsyslog v3: load input modules >> # If you do not load inputs, nothing happens! >> # You may need to set the module load path if modules are not found. >> >> $ModLoad immark.so # provides --MARK-- message capability >> $ModLoad imuxsock.so # provides support for local system logging (e.g. >> via logger command) >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >> $ModLoad ommail >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >> >> # Log all kernel messages to the console. >> # Logging much else clutters up the screen. >> #kern.* /dev/console >> >> # Log anything (except mail) of level info or higher. >> # Don't log private authentication messages! >> *.info;mail.none;authpriv.none;cron.none >> -/var/log/messages >> >> # The authpriv file has restricted access. >> authpriv.* /var/log/secure >> >> # Log all the mail messages in one place. >> mail.* >> -/var/log/maillog >> >> # Log cron stuff >> cron.* -/var/log/cron >> >> # Everybody gets emergency messages >> *.emerg * >> >> # Save news errors of level crit and higher in a special file. >> uucp,news.crit >> -/var/log/spooler >> >> # Save boot messages also to boot.log >> local7.* >> /var/log/boot.log >> >> #Catch all incoming syslog messages >> *.* >> /var/log/catchall;TraditionalFormatWithPRI >> >> # Remote Logging (we use TCP for reliable delivery) >> # An on-disk queue is created for this action. If the remote host is >> # down, messages are spooled to disk and sent when it is up again. >> $WorkDirectory /rsyslog/spool # where to place spool files >> $ActionQueueFileName uniqName # unique name prefix for spool files >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >> possible) >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> $ActionQueueType LinkedList # run asynchronously >> $ActionResumeRetryCount -1 # infinite retries if host is down >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> *.* @10.57.106.140:514 >> >> $ModLoad ommail >> $ActionMailSMTPServer >> $ActionMailFrom >> $ActionMailTo >> $template mailSubject,"disk problem on %hostname%" >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >> $ActionMailSubject mailSubject >> # make sure we receive a mail only once in six >> # hours (21,600 seconds ;)) >> $ActionExecOnlyOnceEveryInterval 21600 >> # the if ... then ... mailBody mus be on one line! >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >> >> >> # ######### Receiving Messages from Remote Hosts ########## >> # TCP Syslog Server: >> # provides TCP syslog reception and GSS-API (if compiled to support it) >> $ModLoad imtcp.so # load module >> $InputTCPServerRun 514 # start up TCP listener at port 514 >> >> # UDP Syslog Server: >> $ModLoad imudp.so # provides UDP syslog reception >> $UDPServerRun 514 # start a UDP syslog server at standard port >> -------------------------------------------------------- >> This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > From rgerhards at hq.adiscon.com Fri Jul 25 11:41:50 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 11:41:50 +0200 Subject: [rsyslog] the philosophy behind rsyslog, phplogcon et al... Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEA0@grfint2.intern.adiscon.com> Hi all, This page is for those of you interested in the philosophy behind rsyslog and its sister projects: http://blog.gerhards.net/2008/07/what-is-event-and-what-event-log.html Be warned, it is a bit technical, but I think quite useful to look behind the curtain. Feedback is welcome. Thanks, Rainer From rgerhards at hq.adiscon.com Fri Jul 25 12:31:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 12:31:51 +0200 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEA4@grfint2.intern.adiscon.com> Thanks, HKS, for the good answers. There is one thing I would like to bring to your attention: I often see that people do not check for rsyslog error messages themselves. That complicates setup. I do not blame anyone, but would like to make you aware of the problem. I have just blogged about a potential (partial) solution and will try to implement it: http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht ml Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Thursday, July 24, 2008 11:36 PM > To: rsyslog-users > Subject: Re: [rsyslog] Help with Ommail Configuration > > Oh - one other possibility. rsyslogd has to have mail enabled at > compile time to work with it at runtime. check your catchall logfile - > if it has messages like this, you didn't compile it in: > > rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > file: No such file or directory > > To fix this, you'll need to reinstall. This time, when you run > ./configure be sure to include --enable-mail. make install clean and > you should be good to go... > > -HKS > > On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS > wrote: > > A few things to try: > > > > - You load ommail.so twice, once at the top and once right above > your > > $ActionMail... lines. I don't think this will break it, but it's > > unnecessary - delete the second one. > > > > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > > to attempt sending a message once every 6 hours. For testing > purposes, > > this will be obnoxious. Until you're ready for production, change it > > to: > > $ActionExecOnlyOnceEveryInterval 30 > > > > - Send everything to make sure you're not missing it based on > > something in the property-replacer/templates/whatever. Replace "if > > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: > > *.* :ommail:;mailBody > > > > Try again. Try logging a few messages from the localhost first (with > > RHEL you can just run "logger test" to log a user.notice message with > > content "test") and see if you get them. > > > > If you don't, check the mail logs on your mail server to see if it > > ever received the message. If not, it's time to break out tcpdump and > > see if the packets are ever being generated. > > > > Hope that helps. > > > > -HKS > > > > > > > > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > > wrote: > >> Hello all, > >> > >> First off, I am not very Linux savvy. I don't have a lot of > experience > >> other then a basic course. This is probably way past my knowledge, > but I > >> really need to get it done. Appreciate any help you guys have to > offer. > >> > >> I am working on a Red Hat Enterprise 4 box and I am running the > latest > >> edition of rsyslog. I currently have rsyslog configured to receive > >> messages remotely via UDP. I am trying to set it up so that it will > send > >> out E-mail messages to the system Admin's based on the severity > level of > >> the log files I am receiving. I would like it so that any device > that > >> sends a log with a critical, alert, or emergency level facility will > >> send out an e-mail to a specific address. > >> > >> Here is my rsyslog.conf file. I used the sample code from Rainer > >> Gerhards configuration page. I tried sending a test syslog with > 'hard > >> disk fatal failure' in it, but it is not sending out mail. Also > tried > >> without the templates below thinking it would just send me an email > for > >> every syslog that I received, but it doesn't appear to be sending > mail. > >> Any thoughts on what I am doing wrong. I'm sure there is a lot I > need to > >> do, so please let me know. Thanks! > >> > >> $template mailSubject,"disk problem on %hostname%" > >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >> > >> I edited out the 3 lines below in the code for security reasons.. > >> $ActionMailSMTPServer > >> $ActionMailFrom > >> $ActionMailTo > >> > >> > >> > >> Here is my code from rsyslog.conf below. > >> > >> > >> > >> # if you experience problems, check > >> # http://www.rsyslog.com/troubleshoot for assistance > >> > >> # rsyslog v3: load input modules > >> # If you do not load inputs, nothing happens! > >> # You may need to set the module load path if modules are not found. > >> > >> $ModLoad immark.so # provides --MARK-- message capability > >> $ModLoad imuxsock.so # provides support for local system logging > (e.g. > >> via logger command) > >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >> $ModLoad ommail > >> > >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >> > >> # Log all kernel messages to the console. > >> # Logging much else clutters up the screen. > >> #kern.* /dev/console > >> > >> # Log anything (except mail) of level info or higher. > >> # Don't log private authentication messages! > >> *.info;mail.none;authpriv.none;cron.none > >> -/var/log/messages > >> > >> # The authpriv file has restricted access. > >> authpriv.* > /var/log/secure > >> > >> # Log all the mail messages in one place. > >> mail.* > >> -/var/log/maillog > >> > >> # Log cron stuff > >> cron.* - > /var/log/cron > >> > >> # Everybody gets emergency messages > >> *.emerg * > >> > >> # Save news errors of level crit and higher in a special file. > >> uucp,news.crit > >> -/var/log/spooler > >> > >> # Save boot messages also to boot.log > >> local7.* > >> /var/log/boot.log > >> > >> #Catch all incoming syslog messages > >> *.* > >> /var/log/catchall;TraditionalFormatWithPRI > >> > >> # Remote Logging (we use TCP for reliable delivery) > >> # An on-disk queue is created for this action. If the remote host is > >> # down, messages are spooled to disk and sent when it is up again. > >> $WorkDirectory /rsyslog/spool # where to place spool files > >> $ActionQueueFileName uniqName # unique name prefix for spool files > >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >> possible) > >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >> $ActionQueueType LinkedList # run asynchronously > >> $ActionResumeRetryCount -1 # infinite retries if host is down > >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> *.* @10.57.106.140:514 > >> > >> $ModLoad ommail > >> $ActionMailSMTPServer > >> $ActionMailFrom > >> $ActionMailTo > >> $template mailSubject,"disk problem on %hostname%" > >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >> $ActionMailSubject mailSubject > >> # make sure we receive a mail only once in six > >> # hours (21,600 seconds ;)) > >> $ActionExecOnlyOnceEveryInterval 21600 > >> # the if ... then ... mailBody mus be on one line! > >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >> > >> > >> # ######### Receiving Messages from Remote Hosts ########## > >> # TCP Syslog Server: > >> # provides TCP syslog reception and GSS-API (if compiled to support > it) > >> $ModLoad imtcp.so # load module > >> $InputTCPServerRun 514 # start up TCP listener at port 514 > >> > >> # UDP Syslog Server: > >> $ModLoad imudp.so # provides UDP syslog reception > >> $UDPServerRun 514 # start a UDP syslog server at standard port > >> -------------------------------------------------------- > >> This e-mail, including any attachments, may be confidential, > privileged or otherwise legally protected. If you have received this e- > mail in error, or from someone who was not authorized to send it to > you, do not disseminate, copy or otherwise use this e-mail or its > attachments. Please notify the sender immediately if you have received > this e-mail by mistake, and delete it from your system. > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Fri Jul 25 16:33:26 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 25 Jul 2008 10:33:26 -0400 Subject: [rsyslog] rsyslog error messages Message-ID: Hi Rainer, Interesting post. The tendency to ignore errors/diagnostics is not limited to rsyslog, it's something that plagues pretty much every piece of software ever created anywhere. Some users are ignorant of basic diagnostic tools like syslog, --help flags, tcpdump, and man pages. Others just need a bit of prodding before they slap themselves and realize that they forgot some basic troubleshooting. Some are lazy slobs. However, rsyslog lends itself to error reports that lack troubleshooting for one huge reason: errors are not reliably reported in the default configuration. It's happened a few times that I start rsyslogd at the command line, and yet there's nothing in the process table, and nothing dumped to my logs. I've been able to track down the problem with -dn switches, but that involves weeding through a lot of irrelevant information. There are three modifications I'd love to see that would help resolve this: 1 - Configuration errors should be fatal 2 - Configuration and other runtime errors should be printed to STDERR 3 - By default, rsyslogd should log all of its own errors (fatal or not) to /var/log/rsyslog.log. This should also be user-configurable My reasoning: 1 - If rsyslog doesn't understand your config file, obviously it won't be doing what you intend it to. There's little benefit in running with a horked configuration, but if you don't test thoroughly, the cost could be devastating. Some users won't know they're not logging things until they need them - then, if they keep their jobs, they probably won't keep rsyslog ("This software doesn't even log ssh attempts!"). 2 - This will avoid a lot of silly questions without any supporting information. At least when you get a complaint like "Rsyslogd won't start!" it will generally include a "It just says 'configuration file invalid: broken syntax at line 135.'" For most users, this kind of error reporting will be enough to lead them to a solution. If nothing else, it removes the need to explain how to find the errors. 3 - For those more subtle problems, and things you might miss upon bootup. This also gives you a simple, easily accessible debugging source with minimal security implications and additional code and no required user configuration. I'm not sure how the code is written, but I'd prefer to see this happen even if the configuration file can't be read. Perhaps an $RsyslogErrors directive that, by default, references a different module that just dumps directly to a file? Users can configure it to put rsyslog errors through the actual logging engine and then manage it with typical syslog selector/action lines. The HTTP server is an interesting idea, but not something I would make use of. The potential security problems are enormous, and it would introduce too many additional factors to the debugging process: is the firewall right? Was there already a process listening on that port? Is the user's configuration file correct? These are the kinds of things you already have to deal with on rsyslog itself - why force the debugging process to go through them again? Hope that helps, and sorry for the long email. -HKS On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards wrote: > Thanks, HKS, for the good answers. There is one thing I would like to > bring to your attention: I often see that people do not check for > rsyslog error messages themselves. That complicates setup. I do not > blame anyone, but would like to make you aware of the problem. > > I have just blogged about a potential (partial) solution and will try to > implement it: > > http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > ml > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of (private) HKS >> Sent: Thursday, July 24, 2008 11:36 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Help with Ommail Configuration >> >> Oh - one other possibility. rsyslogd has to have mail enabled at >> compile time to work with it at runtime. check your catchall logfile - >> if it has messages like this, you didn't compile it in: >> >> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', >> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object >> file: No such file or directory >> >> To fix this, you'll need to reinstall. This time, when you run >> ./configure be sure to include --enable-mail. make install clean and >> you should be good to go... >> >> -HKS >> >> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >> wrote: >> > A few things to try: >> > >> > - You load ommail.so twice, once at the top and once right above >> your >> > $ActionMail... lines. I don't think this will break it, but it's >> > unnecessary - delete the second one. >> > >> > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going >> > to attempt sending a message once every 6 hours. For testing >> purposes, >> > this will be obnoxious. Until you're ready for production, change it >> > to: >> > $ActionExecOnlyOnceEveryInterval 30 >> > >> > - Send everything to make sure you're not missing it based on >> > something in the property-replacer/templates/whatever. Replace "if >> > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > with: >> > *.* :ommail:;mailBody >> > >> > Try again. Try logging a few messages from the localhost first (with >> > RHEL you can just run "logger test" to log a user.notice message > with >> > content "test") and see if you get them. >> > >> > If you don't, check the mail logs on your mail server to see if it >> > ever received the message. If not, it's time to break out tcpdump > and >> > see if the packets are ever being generated. >> > >> > Hope that helps. >> > >> > -HKS >> > >> > >> > >> > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) >> > wrote: >> >> Hello all, >> >> >> >> First off, I am not very Linux savvy. I don't have a lot of >> experience >> >> other then a basic course. This is probably way past my knowledge, >> but I >> >> really need to get it done. Appreciate any help you guys have to >> offer. >> >> >> >> I am working on a Red Hat Enterprise 4 box and I am running the >> latest >> >> edition of rsyslog. I currently have rsyslog configured to receive >> >> messages remotely via UDP. I am trying to set it up so that it will >> send >> >> out E-mail messages to the system Admin's based on the severity >> level of >> >> the log files I am receiving. I would like it so that any device >> that >> >> sends a log with a critical, alert, or emergency level facility > will >> >> send out an e-mail to a specific address. >> >> >> >> Here is my rsyslog.conf file. I used the sample code from Rainer >> >> Gerhards configuration page. I tried sending a test syslog with >> 'hard >> >> disk fatal failure' in it, but it is not sending out mail. Also >> tried >> >> without the templates below thinking it would just send me an email >> for >> >> every syslog that I received, but it doesn't appear to be sending >> mail. >> >> Any thoughts on what I am doing wrong. I'm sure there is a lot I >> need to >> >> do, so please let me know. Thanks! >> >> >> >> $template mailSubject,"disk problem on %hostname%" >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >> >> >> >> I edited out the 3 lines below in the code for security reasons.. >> >> $ActionMailSMTPServer >> >> $ActionMailFrom >> >> $ActionMailTo >> >> >> >> >> >> >> >> Here is my code from rsyslog.conf below. >> >> >> >> >> >> >> >> # if you experience problems, check >> >> # http://www.rsyslog.com/troubleshoot for assistance >> >> >> >> # rsyslog v3: load input modules >> >> # If you do not load inputs, nothing happens! >> >> # You may need to set the module load path if modules are not > found. >> >> >> >> $ModLoad immark.so # provides --MARK-- message capability >> >> $ModLoad imuxsock.so # provides support for local system logging >> (e.g. >> >> via logger command) >> >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >> >> $ModLoad ommail >> >> >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >> >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >> >> >> >> # Log all kernel messages to the console. >> >> # Logging much else clutters up the screen. >> >> #kern.* > /dev/console >> >> >> >> # Log anything (except mail) of level info or higher. >> >> # Don't log private authentication messages! >> >> *.info;mail.none;authpriv.none;cron.none >> >> -/var/log/messages >> >> >> >> # The authpriv file has restricted access. >> >> authpriv.* >> /var/log/secure >> >> >> >> # Log all the mail messages in one place. >> >> mail.* >> >> -/var/log/maillog >> >> >> >> # Log cron stuff >> >> cron.* - >> /var/log/cron >> >> >> >> # Everybody gets emergency messages >> >> *.emerg * >> >> >> >> # Save news errors of level crit and higher in a special file. >> >> uucp,news.crit >> >> -/var/log/spooler >> >> >> >> # Save boot messages also to boot.log >> >> local7.* >> >> /var/log/boot.log >> >> >> >> #Catch all incoming syslog messages >> >> *.* >> >> /var/log/catchall;TraditionalFormatWithPRI >> >> >> >> # Remote Logging (we use TCP for reliable delivery) >> >> # An on-disk queue is created for this action. If the remote host > is >> >> # down, messages are spooled to disk and sent when it is up again. >> >> $WorkDirectory /rsyslog/spool # where to place spool files >> >> $ActionQueueFileName uniqName # unique name prefix for spool files >> >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >> >> possible) >> >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> >> $ActionQueueType LinkedList # run asynchronously >> >> $ActionResumeRetryCount -1 # infinite retries if host is down >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> >> *.* @10.57.106.140:514 >> >> >> >> $ModLoad ommail >> >> $ActionMailSMTPServer >> >> $ActionMailFrom >> >> $ActionMailTo >> >> $template mailSubject,"disk problem on %hostname%" >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >> >> $ActionMailSubject mailSubject >> >> # make sure we receive a mail only once in six >> >> # hours (21,600 seconds ;)) >> >> $ActionExecOnlyOnceEveryInterval 21600 >> >> # the if ... then ... mailBody mus be on one line! >> >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >> >> >> >> >> >> # ######### Receiving Messages from Remote Hosts ########## >> >> # TCP Syslog Server: >> >> # provides TCP syslog reception and GSS-API (if compiled to support >> it) >> >> $ModLoad imtcp.so # load module >> >> $InputTCPServerRun 514 # start up TCP listener at port 514 >> >> >> >> # UDP Syslog Server: >> >> $ModLoad imudp.so # provides UDP syslog reception >> >> $UDPServerRun 514 # start a UDP syslog server at standard port >> >> -------------------------------------------------------- >> >> This e-mail, including any attachments, may be confidential, >> privileged or otherwise legally protected. If you have received this > e- >> mail in error, or from someone who was not authorized to send it to >> you, do not disseminate, copy or otherwise use this e-mail or its >> attachments. Please notify the sender immediately if you have received >> this e-mail by mistake, and delete it from your system. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rfujita at redhat.com Fri Jul 25 16:56:04 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 25 Jul 2008 23:56:04 +0900 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: Hi HKS-san, +1 > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.' In addition, '... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.' Any ideas? Best Rio. On 2008/07/25, at 23:33, (private) HKS wrote: > Hi Rainer, > > Interesting post. The tendency to ignore errors/diagnostics is not > limited to rsyslog, it's something that plagues pretty much every > piece of software ever created anywhere. Some users are ignorant of > basic diagnostic tools like syslog, --help flags, tcpdump, and man > pages. Others just need a bit of prodding before they slap themselves > and realize that they forgot some basic troubleshooting. Some are lazy > slobs. > > However, rsyslog lends itself to error reports that lack > troubleshooting for one huge reason: errors are not reliably reported > in the default configuration. It's happened a few times that I start > rsyslogd at the command line, and yet there's nothing in the process > table, and nothing dumped to my logs. I've been able to track down the > problem with -dn switches, but that involves weeding through a lot of > irrelevant information. > > There are three modifications I'd love to see that would help > resolve this: > > 1 - Configuration errors should be fatal > 2 - Configuration and other runtime errors should be printed to STDERR > 3 - By default, rsyslogd should log all of its own errors (fatal or > not) to /var/log/rsyslog.log. This should also be user-configurable > > My reasoning: > > 1 - If rsyslog doesn't understand your config file, obviously it won't > be doing what you intend it to. There's little benefit in running with > a horked configuration, but if you don't test thoroughly, the cost > could be devastating. Some users won't know they're not logging things > until they need them - then, if they keep their jobs, they probably > won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.'" For most users, this kind of > error reporting will be enough to lead them to a solution. If nothing > else, it removes the need to explain how to find the errors. > > 3 - For those more subtle problems, and things you might miss upon > bootup. This also gives you a simple, easily accessible debugging > source with minimal security implications and additional code and no > required user configuration. I'm not sure how the code is written, but > I'd prefer to see this happen even if the configuration file can't be > read. Perhaps an $RsyslogErrors directive that, by default, references > a different module that just dumps directly to a file? Users can > configure it to put rsyslog errors through the actual logging engine > and then manage it with typical syslog selector/action lines. > > The HTTP server is an interesting idea, but not something I would make > use of. The potential security problems are enormous, and it would > introduce too many additional factors to the debugging process: is the > firewall right? Was there already a process listening on that port? Is > the user's configuration file correct? These are the kinds of things > you already have to deal with on rsyslog itself - why force the > debugging process to go through them again? > > Hope that helps, and sorry for the long email. > -HKS > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > wrote: >> Thanks, HKS, for the good answers. There is one thing I would like to >> bring to your attention: I often see that people do not check for >> rsyslog error messages themselves. That complicates setup. I do not >> blame anyone, but would like to make you aware of the problem. >> >> I have just blogged about a potential (partial) solution and will >> try to >> implement it: >> >> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht >> ml >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of (private) HKS >>> Sent: Thursday, July 24, 2008 11:36 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] Help with Ommail Configuration >>> >>> Oh - one other possibility. rsyslogd has to have mail enabled at >>> compile time to work with it at runtime. check your catchall >>> logfile - >>> if it has messages like this, you didn't compile it in: >>> >>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', >>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object >>> file: No such file or directory >>> >>> To fix this, you'll need to reinstall. This time, when you run >>> ./configure be sure to include --enable-mail. make install clean and >>> you should be good to go... >>> >>> -HKS >>> >>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >> > >>> wrote: >>>> A few things to try: >>>> >>>> - You load ommail.so twice, once at the top and once right above >>> your >>>> $ActionMail... lines. I don't think this will break it, but it's >>>> unnecessary - delete the second one. >>>> >>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going >>>> to attempt sending a message once every 6 hours. For testing >>> purposes, >>>> this will be obnoxious. Until you're ready for production, change >>>> it >>>> to: >>>> $ActionExecOnlyOnceEveryInterval 30 >>>> >>>> - Send everything to make sure you're not missing it based on >>>> something in the property-replacer/templates/whatever. Replace "if >>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody" >> with: >>>> *.* :ommail:;mailBody >>>> >>>> Try again. Try logging a few messages from the localhost first >>>> (with >>>> RHEL you can just run "logger test" to log a user.notice message >> with >>>> content "test") and see if you get them. >>>> >>>> If you don't, check the mail logs on your mail server to see if it >>>> ever received the message. If not, it's time to break out tcpdump >> and >>>> see if the packets are ever being generated. >>>> >>>> Hope that helps. >>>> >>>> -HKS >>>> >>>> >>>> >>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) >>>> wrote: >>>>> Hello all, >>>>> >>>>> First off, I am not very Linux savvy. I don't have a lot of >>> experience >>>>> other then a basic course. This is probably way past my knowledge, >>> but I >>>>> really need to get it done. Appreciate any help you guys have to >>> offer. >>>>> >>>>> I am working on a Red Hat Enterprise 4 box and I am running the >>> latest >>>>> edition of rsyslog. I currently have rsyslog configured to receive >>>>> messages remotely via UDP. I am trying to set it up so that it >>>>> will >>> send >>>>> out E-mail messages to the system Admin's based on the severity >>> level of >>>>> the log files I am receiving. I would like it so that any device >>> that >>>>> sends a log with a critical, alert, or emergency level facility >> will >>>>> send out an e-mail to a specific address. >>>>> >>>>> Here is my rsyslog.conf file. I used the sample code from Rainer >>>>> Gerhards configuration page. I tried sending a test syslog with >>> 'hard >>>>> disk fatal failure' in it, but it is not sending out mail. Also >>> tried >>>>> without the templates below thinking it would just send me an >>>>> email >>> for >>>>> every syslog that I received, but it doesn't appear to be sending >>> mail. >>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I >>> need to >>>>> do, so please let me know. Thanks! >>>>> >>>>> $template mailSubject,"disk problem on %hostname%" >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >>>>> >>>>> I edited out the 3 lines below in the code for security reasons.. >>>>> $ActionMailSMTPServer >>>>> $ActionMailFrom >>>>> $ActionMailTo >>>>> >>>>> >>>>> >>>>> Here is my code from rsyslog.conf below. >>>>> >>>>> >>>>> >>>>> # if you experience problems, check >>>>> # http://www.rsyslog.com/troubleshoot for assistance >>>>> >>>>> # rsyslog v3: load input modules >>>>> # If you do not load inputs, nothing happens! >>>>> # You may need to set the module load path if modules are not >> found. >>>>> >>>>> $ModLoad immark.so # provides --MARK-- message capability >>>>> $ModLoad imuxsock.so # provides support for local system logging >>> (e.g. >>>>> via logger command) >>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >>>>> $ModLoad ommail >>>>> >>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >>>>> >>>>> # Log all kernel messages to the console. >>>>> # Logging much else clutters up the screen. >>>>> #kern.* >> /dev/console >>>>> >>>>> # Log anything (except mail) of level info or higher. >>>>> # Don't log private authentication messages! >>>>> *.info;mail.none;authpriv.none;cron.none >>>>> -/var/log/messages >>>>> >>>>> # The authpriv file has restricted access. >>>>> authpriv.* >>> /var/log/secure >>>>> >>>>> # Log all the mail messages in one place. >>>>> mail.* >>>>> -/var/log/maillog >>>>> >>>>> # Log cron stuff >>>>> cron.* - >>> /var/log/cron >>>>> >>>>> # Everybody gets emergency messages >>>>> *.emerg * >>>>> >>>>> # Save news errors of level crit and higher in a special file. >>>>> uucp,news.crit >>>>> -/var/log/spooler >>>>> >>>>> # Save boot messages also to boot.log >>>>> local7.* >>>>> /var/log/boot.log >>>>> >>>>> #Catch all incoming syslog messages >>>>> *.* >>>>> /var/log/catchall;TraditionalFormatWithPRI >>>>> >>>>> # Remote Logging (we use TCP for reliable delivery) >>>>> # An on-disk queue is created for this action. If the remote host >> is >>>>> # down, messages are spooled to disk and sent when it is up again. >>>>> $WorkDirectory /rsyslog/spool # where to place spool files >>>>> $ActionQueueFileName uniqName # unique name prefix for spool files >>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >>>>> possible) >>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >>>>> $ActionQueueType LinkedList # run asynchronously >>>>> $ActionResumeRetryCount -1 # infinite retries if host is down >>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port >>>>> optional >>>>> *.* @10.57.106.140:514 >>>>> >>>>> $ModLoad ommail >>>>> $ActionMailSMTPServer >>>>> $ActionMailFrom >>>>> $ActionMailTo >>>>> $template mailSubject,"disk problem on %hostname%" >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >>>>> $ActionMailSubject mailSubject >>>>> # make sure we receive a mail only once in six >>>>> # hours (21,600 seconds ;)) >>>>> $ActionExecOnlyOnceEveryInterval 21600 >>>>> # the if ... then ... mailBody mus be on one line! >>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >>>>> >>>>> >>>>> # ######### Receiving Messages from Remote Hosts ########## >>>>> # TCP Syslog Server: >>>>> # provides TCP syslog reception and GSS-API (if compiled to >>>>> support >>> it) >>>>> $ModLoad imtcp.so # load module >>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 >>>>> >>>>> # UDP Syslog Server: >>>>> $ModLoad imudp.so # provides UDP syslog reception >>>>> $UDPServerRun 514 # start a UDP syslog server at standard port >>>>> -------------------------------------------------------- >>>>> This e-mail, including any attachments, may be confidential, >>> privileged or otherwise legally protected. If you have received this >> e- >>> mail in error, or from someone who was not authorized to send it to >>> you, do not disseminate, copy or otherwise use this e-mail or its >>> attachments. Please notify the sender immediately if you have >>> received >>> this e-mail by mistake, and delete it from your system. >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 25 17:16:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 17:16:12 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> On Fri, 2008-07-25 at 10:33 -0400, (private) HKS wrote: > Hi Rainer, > > Interesting post. The tendency to ignore errors/diagnostics is not > limited to rsyslog, it's something that plagues pretty much every > piece of software ever created anywhere. I totally agree, but I think the situation is even worse for a syslogd. Because it *is* the usual error reporting system. I think we are a bit in the same situation (if you allow that analogy) like HIV which is attacking the human immune system. We have a problem in the troubleshooting system itself, and it may even prevent us from conveying troubleshooting information :( > Some users are ignorant of > basic diagnostic tools like syslog, --help flags, tcpdump, and man > pages. Others just need a bit of prodding before they slap themselves > and realize that they forgot some basic troubleshooting. Some are lazy > slobs. > > However, rsyslog lends itself to error reports that lack > troubleshooting for one huge reason: errors are not reliably reported > in the default configuration. > It's happened a few times that I start > rsyslogd at the command line, and yet there's nothing in the process > table, and nothing dumped to my logs. Could you elaborate a bit about above sentence - I have to admit I do not fully get you... > I've been able to track down the > problem with -dn switches, but that involves weeding through a lot of > irrelevant information. > > There are three modifications I'd love to see that would help resolve this: > > 1 - Configuration errors should be fatal > 2 - Configuration and other runtime errors should be printed to STDERR Does that really help? If so, that would be extremely easy to add. But does somebody see them? > 3 - By default, rsyslogd should log all of its own errors (fatal or > not) to /var/log/rsyslog.log. This should also be user-configurable This default comes into the way of some distros: they use different pathes. I really don't like the idea of embedding any information about the file system into the binary. Just think about embedded systems... > > My reasoning: > > 1 - If rsyslog doesn't understand your config file, obviously it won't > be doing what you intend it to. There's little benefit in running with > a horked configuration, but if you don't test thoroughly, the cost > could be devastating. Some users won't know they're not logging things > until they need them - then, if they keep their jobs, they probably > won't keep rsyslog ("This software doesn't even log ssh attempts!"). That's quite controversal and and does not match rsyslog's root philosophy. Let me explain: The syslogd is an extremely important piece of software. If it is not functioning, a) vital data can be lost b) the system as whole may hang (due to the fact that the log socket gets filled up and so other processes block on it) As such, rsyslogd is written in a sense that it tries to continue to work under almost all circumstances. It even tries to survive a system-wide low memory condition, though I am not 100% convinced it will manage to do that in all cases. At least it is coded careful to survive as much as possible. Now let's assume that a configuration error happens that prevents rsyslogd from executing an otherwise perfect configuration. Should we actually stop it from running? And live with the results? Or should we do whatever we can do and do that as good as possible? What, if it is not a configuration issue but a hard disk failure or an admin error that make part of the config unavailable (e.g. not able to load module, not able to write file)? Assuming we can carry out other actions - should we really abort? This may even come at the price that we can NOT warn users (via wall) of a serious problem. And what with warnings? Things that look wrong, but may be OK. Reason to abort? Or continue run? Tough decision. So I still think the right thing to do is to run as long as there is limited sense in it, because everything else would actually be worse. Please note that rsyslogd even comes with a hardcoded emergency configuration which is used in case the real configuration file can not be obtained. What I could add, however, is an option to make it stop on any config error (not warning). However, I fear that, if turned on, the results can be fatal in some cases... > > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.'" For most users, this kind of > error reporting will be enough to lead them to a solution. If nothing > else, it removes the need to explain how to find the errors. > > 3 - For those more subtle problems, and things you might miss upon > bootup. This also gives you a simple, easily accessible debugging > source with minimal security implications and additional code and no > required user configuration. I'm not sure how the code is written, but > I'd prefer to see this happen even if the configuration file can't be > read. Perhaps an $RsyslogErrors directive that, by default, references > a different module that just dumps directly to a file? Users can > configure it to put rsyslog errors through the actual logging engine > and then manage it with typical syslog selector/action lines. All internally-generated messages are run through a specific interface. For the "diag plugin", I will define a hook, where a diag server can register to receive these messages. Maybe a good compromise would be to add this $RsyslogErrors directive and make it point to a file. So, still, we do not necessarily have it, but package maintainers would hopefully include it and point it to a specific file (what still means I do not necessarily know where to look). A problem is source tarball install, in which case the directive may not be set. But with a proper sample rsyslog.conf... > The HTTP server is an interesting idea, but not something I would make > use of. The potential security problems are enormous, and it would > introduce too many additional factors to the debugging process: is the > firewall right? Was there already a process listening on that port? Is > the user's configuration file correct? These are the kinds of things > you already have to deal with on rsyslog itself - why force the > debugging process to go through them again? I see (and share) much of your argument. This is why I stayed away for such a long time. But if you think a bit more about it, it has *very interesting* capabilities. Eg. I may also use it to gather some real-time view of internal state, emit commands like HUP and do a number of other nice things. Again, security is problematic. In any case, I'll give it a try. I have started to write a small plugin which will dump the error messages within 10 minutes after startup. It will not be a real http server but rather a listener that response to anyone who connects to it (so that nc can be used as a client ;)). This is very rough, but it has two side-effects: a) it provides something to actually experiment with (and to extend if we decide to keep that route) b) it helps create the plumbing inside the internal interface that could also be used by any other method, so the potential waste of time is limited So while I object some suggestions, I am very interested in a discussion. The rsyslog "keep running" philosophy *is* a discussion topic, just expect me to argue strongly in favor of it (pls think about the HIV analogy to see why I think we have a unique situation in case of a syslogd). Thanks for the good post, and I guess I just wrote another long mail ;) Rainer > > Hope that helps, and sorry for the long email. > -HKS > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > wrote: > > Thanks, HKS, for the good answers. There is one thing I would like to > > bring to your attention: I often see that people do not check for > > rsyslog error messages themselves. That complicates setup. I do not > > blame anyone, but would like to make you aware of the problem. > > > > I have just blogged about a potential (partial) solution and will try to > > implement it: > > > > http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > > ml > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of (private) HKS > >> Sent: Thursday, July 24, 2008 11:36 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Help with Ommail Configuration > >> > >> Oh - one other possibility. rsyslogd has to have mail enabled at > >> compile time to work with it at runtime. check your catchall logfile - > >> if it has messages like this, you didn't compile it in: > >> > >> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > >> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > >> file: No such file or directory > >> > >> To fix this, you'll need to reinstall. This time, when you run > >> ./configure be sure to include --enable-mail. make install clean and > >> you should be good to go... > >> > >> -HKS > >> > >> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS > >> wrote: > >> > A few things to try: > >> > > >> > - You load ommail.so twice, once at the top and once right above > >> your > >> > $ActionMail... lines. I don't think this will break it, but it's > >> > unnecessary - delete the second one. > >> > > >> > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > >> > to attempt sending a message once every 6 hours. For testing > >> purposes, > >> > this will be obnoxious. Until you're ready for production, change it > >> > to: > >> > $ActionExecOnlyOnceEveryInterval 30 > >> > > >> > - Send everything to make sure you're not missing it based on > >> > something in the property-replacer/templates/whatever. Replace "if > >> > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > > with: > >> > *.* :ommail:;mailBody > >> > > >> > Try again. Try logging a few messages from the localhost first (with > >> > RHEL you can just run "logger test" to log a user.notice message > > with > >> > content "test") and see if you get them. > >> > > >> > If you don't, check the mail logs on your mail server to see if it > >> > ever received the message. If not, it's time to break out tcpdump > > and > >> > see if the packets are ever being generated. > >> > > >> > Hope that helps. > >> > > >> > -HKS > >> > > >> > > >> > > >> > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > >> > wrote: > >> >> Hello all, > >> >> > >> >> First off, I am not very Linux savvy. I don't have a lot of > >> experience > >> >> other then a basic course. This is probably way past my knowledge, > >> but I > >> >> really need to get it done. Appreciate any help you guys have to > >> offer. > >> >> > >> >> I am working on a Red Hat Enterprise 4 box and I am running the > >> latest > >> >> edition of rsyslog. I currently have rsyslog configured to receive > >> >> messages remotely via UDP. I am trying to set it up so that it will > >> send > >> >> out E-mail messages to the system Admin's based on the severity > >> level of > >> >> the log files I am receiving. I would like it so that any device > >> that > >> >> sends a log with a critical, alert, or emergency level facility > > will > >> >> send out an e-mail to a specific address. > >> >> > >> >> Here is my rsyslog.conf file. I used the sample code from Rainer > >> >> Gerhards configuration page. I tried sending a test syslog with > >> 'hard > >> >> disk fatal failure' in it, but it is not sending out mail. Also > >> tried > >> >> without the templates below thinking it would just send me an email > >> for > >> >> every syslog that I received, but it doesn't appear to be sending > >> mail. > >> >> Any thoughts on what I am doing wrong. I'm sure there is a lot I > >> need to > >> >> do, so please let me know. Thanks! > >> >> > >> >> $template mailSubject,"disk problem on %hostname%" > >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >> >> > >> >> I edited out the 3 lines below in the code for security reasons.. > >> >> $ActionMailSMTPServer > >> >> $ActionMailFrom > >> >> $ActionMailTo > >> >> > >> >> > >> >> > >> >> Here is my code from rsyslog.conf below. > >> >> > >> >> > >> >> > >> >> # if you experience problems, check > >> >> # http://www.rsyslog.com/troubleshoot for assistance > >> >> > >> >> # rsyslog v3: load input modules > >> >> # If you do not load inputs, nothing happens! > >> >> # You may need to set the module load path if modules are not > > found. > >> >> > >> >> $ModLoad immark.so # provides --MARK-- message capability > >> >> $ModLoad imuxsock.so # provides support for local system logging > >> (e.g. > >> >> via logger command) > >> >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >> >> $ModLoad ommail > >> >> > >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >> >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >> >> > >> >> # Log all kernel messages to the console. > >> >> # Logging much else clutters up the screen. > >> >> #kern.* > > /dev/console > >> >> > >> >> # Log anything (except mail) of level info or higher. > >> >> # Don't log private authentication messages! > >> >> *.info;mail.none;authpriv.none;cron.none > >> >> -/var/log/messages > >> >> > >> >> # The authpriv file has restricted access. > >> >> authpriv.* > >> /var/log/secure > >> >> > >> >> # Log all the mail messages in one place. > >> >> mail.* > >> >> -/var/log/maillog > >> >> > >> >> # Log cron stuff > >> >> cron.* - > >> /var/log/cron > >> >> > >> >> # Everybody gets emergency messages > >> >> *.emerg * > >> >> > >> >> # Save news errors of level crit and higher in a special file. > >> >> uucp,news.crit > >> >> -/var/log/spooler > >> >> > >> >> # Save boot messages also to boot.log > >> >> local7.* > >> >> /var/log/boot.log > >> >> > >> >> #Catch all incoming syslog messages > >> >> *.* > >> >> /var/log/catchall;TraditionalFormatWithPRI > >> >> > >> >> # Remote Logging (we use TCP for reliable delivery) > >> >> # An on-disk queue is created for this action. If the remote host > > is > >> >> # down, messages are spooled to disk and sent when it is up again. > >> >> $WorkDirectory /rsyslog/spool # where to place spool files > >> >> $ActionQueueFileName uniqName # unique name prefix for spool files > >> >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >> >> possible) > >> >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >> >> $ActionQueueType LinkedList # run asynchronously > >> >> $ActionResumeRetryCount -1 # infinite retries if host is down > >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> >> *.* @10.57.106.140:514 > >> >> > >> >> $ModLoad ommail > >> >> $ActionMailSMTPServer > >> >> $ActionMailFrom > >> >> $ActionMailTo > >> >> $template mailSubject,"disk problem on %hostname%" > >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >> >> $ActionMailSubject mailSubject > >> >> # make sure we receive a mail only once in six > >> >> # hours (21,600 seconds ;)) > >> >> $ActionExecOnlyOnceEveryInterval 21600 > >> >> # the if ... then ... mailBody mus be on one line! > >> >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >> >> > >> >> > >> >> # ######### Receiving Messages from Remote Hosts ########## > >> >> # TCP Syslog Server: > >> >> # provides TCP syslog reception and GSS-API (if compiled to support > >> it) > >> >> $ModLoad imtcp.so # load module > >> >> $InputTCPServerRun 514 # start up TCP listener at port 514 > >> >> > >> >> # UDP Syslog Server: > >> >> $ModLoad imudp.so # provides UDP syslog reception > >> >> $UDPServerRun 514 # start a UDP syslog server at standard port > >> >> -------------------------------------------------------- > >> >> This e-mail, including any attachments, may be confidential, > >> privileged or otherwise legally protected. If you have received this > > e- > >> mail in error, or from someone who was not authorized to send it to > >> you, do not disseminate, copy or otherwise use this e-mail or its > >> attachments. Please notify the sender immediately if you have received > >> this e-mail by mistake, and delete it from your system. > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 25 17:25:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 17:25:31 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: <1216999531.7184.219.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Fri, 2008-07-25 at 23:56 +0900, Ryo Fujita wrote: > Hi HKS-san, > > +1 > > > 2 - This will avoid a lot of silly questions without any supporting > > information. At least when you get a complaint like "Rsyslogd won't > > start!" it will generally include a "It just says 'configuration file > > invalid: broken syntax at line 135.' > > In addition, > '... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.' Same thought over here :) The recent build even emits live links to troubleshooting information. Log lines look like this: 2008-07-25T17:22:18.523900+02:00 rgf9dev rsyslogd-3003: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] Note the part in the bracket at the end of the message. That repository is far from being complete, but I hope more and more information will be added. I hope this is a useful addition. Together with the error number, we can hopefully most often pinpoint the source of the problem (though the 3003 above is a good example of where it may not be possible to tell the exact cause, but for sure give a good hint). BTW: this is also one thing that makes me think about the scripting engine. It shall provide better error reporting capabilties, but that's harder than it initially sounds ;) Rainer > > Any ideas? > > Best Rio. > > On 2008/07/25, at 23:33, (private) HKS wrote: > > > Hi Rainer, > > > > Interesting post. The tendency to ignore errors/diagnostics is not > > limited to rsyslog, it's something that plagues pretty much every > > piece of software ever created anywhere. Some users are ignorant of > > basic diagnostic tools like syslog, --help flags, tcpdump, and man > > pages. Others just need a bit of prodding before they slap themselves > > and realize that they forgot some basic troubleshooting. Some are lazy > > slobs. > > > > However, rsyslog lends itself to error reports that lack > > troubleshooting for one huge reason: errors are not reliably reported > > in the default configuration. It's happened a few times that I start > > rsyslogd at the command line, and yet there's nothing in the process > > table, and nothing dumped to my logs. I've been able to track down the > > problem with -dn switches, but that involves weeding through a lot of > > irrelevant information. > > > > There are three modifications I'd love to see that would help > > resolve this: > > > > 1 - Configuration errors should be fatal > > 2 - Configuration and other runtime errors should be printed to STDERR > > 3 - By default, rsyslogd should log all of its own errors (fatal or > > not) to /var/log/rsyslog.log. This should also be user-configurable > > > > My reasoning: > > > > 1 - If rsyslog doesn't understand your config file, obviously it won't > > be doing what you intend it to. There's little benefit in running with > > a horked configuration, but if you don't test thoroughly, the cost > > could be devastating. Some users won't know they're not logging things > > until they need them - then, if they keep their jobs, they probably > > won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > > > 2 - This will avoid a lot of silly questions without any supporting > > information. At least when you get a complaint like "Rsyslogd won't > > start!" it will generally include a "It just says 'configuration file > > invalid: broken syntax at line 135.'" For most users, this kind of > > error reporting will be enough to lead them to a solution. If nothing > > else, it removes the need to explain how to find the errors. > > > > 3 - For those more subtle problems, and things you might miss upon > > bootup. This also gives you a simple, easily accessible debugging > > source with minimal security implications and additional code and no > > required user configuration. I'm not sure how the code is written, but > > I'd prefer to see this happen even if the configuration file can't be > > read. Perhaps an $RsyslogErrors directive that, by default, references > > a different module that just dumps directly to a file? Users can > > configure it to put rsyslog errors through the actual logging engine > > and then manage it with typical syslog selector/action lines. > > > > The HTTP server is an interesting idea, but not something I would make > > use of. The potential security problems are enormous, and it would > > introduce too many additional factors to the debugging process: is the > > firewall right? Was there already a process listening on that port? Is > > the user's configuration file correct? These are the kinds of things > > you already have to deal with on rsyslog itself - why force the > > debugging process to go through them again? > > > > Hope that helps, and sorry for the long email. > > -HKS > > > > > > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > > wrote: > >> Thanks, HKS, for the good answers. There is one thing I would like to > >> bring to your attention: I often see that people do not check for > >> rsyslog error messages themselves. That complicates setup. I do not > >> blame anyone, but would like to make you aware of the problem. > >> > >> I have just blogged about a potential (partial) solution and will > >> try to > >> implement it: > >> > >> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > >> ml > >> > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of (private) HKS > >>> Sent: Thursday, July 24, 2008 11:36 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] Help with Ommail Configuration > >>> > >>> Oh - one other possibility. rsyslogd has to have mail enabled at > >>> compile time to work with it at runtime. check your catchall > >>> logfile - > >>> if it has messages like this, you didn't compile it in: > >>> > >>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > >>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > >>> file: No such file or directory > >>> > >>> To fix this, you'll need to reinstall. This time, when you run > >>> ./configure be sure to include --enable-mail. make install clean and > >>> you should be good to go... > >>> > >>> -HKS > >>> > >>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >>> > > >>> wrote: > >>>> A few things to try: > >>>> > >>>> - You load ommail.so twice, once at the top and once right above > >>> your > >>>> $ActionMail... lines. I don't think this will break it, but it's > >>>> unnecessary - delete the second one. > >>>> > >>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > >>>> to attempt sending a message once every 6 hours. For testing > >>> purposes, > >>>> this will be obnoxious. Until you're ready for production, change > >>>> it > >>>> to: > >>>> $ActionExecOnlyOnceEveryInterval 30 > >>>> > >>>> - Send everything to make sure you're not missing it based on > >>>> something in the property-replacer/templates/whatever. Replace "if > >>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > >> with: > >>>> *.* :ommail:;mailBody > >>>> > >>>> Try again. Try logging a few messages from the localhost first > >>>> (with > >>>> RHEL you can just run "logger test" to log a user.notice message > >> with > >>>> content "test") and see if you get them. > >>>> > >>>> If you don't, check the mail logs on your mail server to see if it > >>>> ever received the message. If not, it's time to break out tcpdump > >> and > >>>> see if the packets are ever being generated. > >>>> > >>>> Hope that helps. > >>>> > >>>> -HKS > >>>> > >>>> > >>>> > >>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > >>>> wrote: > >>>>> Hello all, > >>>>> > >>>>> First off, I am not very Linux savvy. I don't have a lot of > >>> experience > >>>>> other then a basic course. This is probably way past my knowledge, > >>> but I > >>>>> really need to get it done. Appreciate any help you guys have to > >>> offer. > >>>>> > >>>>> I am working on a Red Hat Enterprise 4 box and I am running the > >>> latest > >>>>> edition of rsyslog. I currently have rsyslog configured to receive > >>>>> messages remotely via UDP. I am trying to set it up so that it > >>>>> will > >>> send > >>>>> out E-mail messages to the system Admin's based on the severity > >>> level of > >>>>> the log files I am receiving. I would like it so that any device > >>> that > >>>>> sends a log with a critical, alert, or emergency level facility > >> will > >>>>> send out an e-mail to a specific address. > >>>>> > >>>>> Here is my rsyslog.conf file. I used the sample code from Rainer > >>>>> Gerhards configuration page. I tried sending a test syslog with > >>> 'hard > >>>>> disk fatal failure' in it, but it is not sending out mail. Also > >>> tried > >>>>> without the templates below thinking it would just send me an > >>>>> email > >>> for > >>>>> every syslog that I received, but it doesn't appear to be sending > >>> mail. > >>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I > >>> need to > >>>>> do, so please let me know. Thanks! > >>>>> > >>>>> $template mailSubject,"disk problem on %hostname%" > >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >>>>> > >>>>> I edited out the 3 lines below in the code for security reasons.. > >>>>> $ActionMailSMTPServer > >>>>> $ActionMailFrom > >>>>> $ActionMailTo > >>>>> > >>>>> > >>>>> > >>>>> Here is my code from rsyslog.conf below. > >>>>> > >>>>> > >>>>> > >>>>> # if you experience problems, check > >>>>> # http://www.rsyslog.com/troubleshoot for assistance > >>>>> > >>>>> # rsyslog v3: load input modules > >>>>> # If you do not load inputs, nothing happens! > >>>>> # You may need to set the module load path if modules are not > >> found. > >>>>> > >>>>> $ModLoad immark.so # provides --MARK-- message capability > >>>>> $ModLoad imuxsock.so # provides support for local system logging > >>> (e.g. > >>>>> via logger command) > >>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >>>>> $ModLoad ommail > >>>>> > >>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >>>>> > >>>>> # Log all kernel messages to the console. > >>>>> # Logging much else clutters up the screen. > >>>>> #kern.* > >> /dev/console > >>>>> > >>>>> # Log anything (except mail) of level info or higher. > >>>>> # Don't log private authentication messages! > >>>>> *.info;mail.none;authpriv.none;cron.none > >>>>> -/var/log/messages > >>>>> > >>>>> # The authpriv file has restricted access. > >>>>> authpriv.* > >>> /var/log/secure > >>>>> > >>>>> # Log all the mail messages in one place. > >>>>> mail.* > >>>>> -/var/log/maillog > >>>>> > >>>>> # Log cron stuff > >>>>> cron.* - > >>> /var/log/cron > >>>>> > >>>>> # Everybody gets emergency messages > >>>>> *.emerg * > >>>>> > >>>>> # Save news errors of level crit and higher in a special file. > >>>>> uucp,news.crit > >>>>> -/var/log/spooler > >>>>> > >>>>> # Save boot messages also to boot.log > >>>>> local7.* > >>>>> /var/log/boot.log > >>>>> > >>>>> #Catch all incoming syslog messages > >>>>> *.* > >>>>> /var/log/catchall;TraditionalFormatWithPRI > >>>>> > >>>>> # Remote Logging (we use TCP for reliable delivery) > >>>>> # An on-disk queue is created for this action. If the remote host > >> is > >>>>> # down, messages are spooled to disk and sent when it is up again. > >>>>> $WorkDirectory /rsyslog/spool # where to place spool files > >>>>> $ActionQueueFileName uniqName # unique name prefix for spool files > >>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >>>>> possible) > >>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >>>>> $ActionQueueType LinkedList # run asynchronously > >>>>> $ActionResumeRetryCount -1 # infinite retries if host is down > >>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port > >>>>> optional > >>>>> *.* @10.57.106.140:514 > >>>>> > >>>>> $ModLoad ommail > >>>>> $ActionMailSMTPServer > >>>>> $ActionMailFrom > >>>>> $ActionMailTo > >>>>> $template mailSubject,"disk problem on %hostname%" > >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >>>>> $ActionMailSubject mailSubject > >>>>> # make sure we receive a mail only once in six > >>>>> # hours (21,600 seconds ;)) > >>>>> $ActionExecOnlyOnceEveryInterval 21600 > >>>>> # the if ... then ... mailBody mus be on one line! > >>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >>>>> > >>>>> > >>>>> # ######### Receiving Messages from Remote Hosts ########## > >>>>> # TCP Syslog Server: > >>>>> # provides TCP syslog reception and GSS-API (if compiled to > >>>>> support > >>> it) > >>>>> $ModLoad imtcp.so # load module > >>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 > >>>>> > >>>>> # UDP Syslog Server: > >>>>> $ModLoad imudp.so # provides UDP syslog reception > >>>>> $UDPServerRun 514 # start a UDP syslog server at standard port > >>>>> -------------------------------------------------------- > >>>>> This e-mail, including any attachments, may be confidential, > >>> privileged or otherwise legally protected. If you have received this > >> e- > >>> mail in error, or from someone who was not authorized to send it to > >>> you, do not disseminate, copy or otherwise use this e-mail or its > >>> attachments. Please notify the sender immediately if you have > >>> received > >>> this e-mail by mistake, and delete it from your system. > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Fri Jul 25 17:44:18 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 25 Jul 2008 11:44:18 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> Message-ID: Responses inline. > I totally agree, but I think the situation is even worse for a syslogd. > Because it *is* the usual error reporting system. I think we are a bit > in the same situation (if you allow that analogy) like HIV which is > attacking the human immune system. We have a problem in the > troubleshooting system itself, and it may even prevent us from conveying > troubleshooting information :( That's a good way of putting it. This is one of the reasons I think it's so important to print errors to STDERR - if the logging mechanisms are broken for some reason, then at least they're going somewhere. And I sincerely hope that users have the sense to start it from the command line the first time rather than sticking it into their system startup scripts and rebooting. >> It's happened a few times that I start >> rsyslogd at the command line, and yet there's nothing in the process >> table, and nothing dumped to my logs. > > Could you elaborate a bit about above sentence - I have to admit I do > not fully get you... I've messed up my configs bad enough a couple times that when I start rsyslogd, nothing apparently happens. No errors to STDERR, no errors in my logs, and the process isn't running. Rewriting the config or running with -dn is the solution. >> 2 - Configuration and other runtime errors should be printed to STDERR > > Does that really help? If so, that would be extremely easy to add. But > does somebody see them? Absolutely. The first time I configure any daemon, I do it in a test environment and run it by hand until I have the configuration acting exactly as I want. Most users take the "run-by-hand" precaution even if they don't have the luxury of a test setup. These will also pop up when a user tries to restart the daemon after making a config change. I'm not sure if they'll be visible when a HUP signal is sent, but that would make me awfully happy. >> 3 - By default, rsyslogd should log all of its own errors (fatal or >> not) to /var/log/rsyslog.log. This should also be user-configurable > > This default comes into the way of some distros: they use different > pathes. I really don't like the idea of embedding any information about > the file system into the binary. Just think about embedded systems... Good point. >> 1 - If rsyslog doesn't understand your config file, obviously it won't >> be doing what you intend it to. There's little benefit in running with >> a horked configuration, but if you don't test thoroughly, the cost >> could be devastating. Some users won't know they're not logging things >> until they need them - then, if they keep their jobs, they probably >> won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > That's quite controversal and and does not match rsyslog's root > philosophy. Let me explain: > > The syslogd is an extremely important piece of software. If it is not > functioning, > > a) vital data can be lost > b) the system as whole may hang (due to the fact that the log > socket gets filled up and so other processes block on it) > > As such, rsyslogd is written in a sense that it tries to continue to > work under almost all circumstances. It even tries to survive a > system-wide low memory condition, though I am not 100% convinced it will > manage to do that in all cases. At least it is coded careful to survive > as much as possible. > > Now let's assume that a configuration error happens that prevents > rsyslogd from executing an otherwise perfect configuration. Should we > actually stop it from running? And live with the results? Or should we > do whatever we can do and do that as good as possible? What, if it is > not a configuration issue but a hard disk failure or an admin error that > make part of the config unavailable (e.g. not able to load module, not > able to write file)? Assuming we can carry out other actions - should we > really abort? This may even come at the price that we can NOT warn users > (via wall) of a serious problem. And what with warnings? Things that > look wrong, but may be OK. Reason to abort? Or continue run? Tough > decision. So I still think the right thing to do is to run as long as > there is limited sense in it, because everything else would actually be > worse. Please note that rsyslogd even comes with a hardcoded emergency > configuration which is used in case the real configuration file can not > be obtained. > > What I could add, however, is an option to make it stop on any config > error (not warning). However, I fear that, if turned on, the results can > be fatal in some cases... You make a lot of good points here. Does the hardcoded emergency configuration that rsyslogd uses actually log? I only ask because it's easy (if you fail to read the documentation) to write a config file that doesn't do anything. Would it make sense for rsyslog to determine whether the passed configuration actually asks it to do anything, and complaining if not? Printing errors to STDERR would help with this. Another possibility might be adding a flag (-n is typical for a lot of systems, but that's already in use) that will tell rsyslog to parse the configuration, spit out any problems, and quit. That way, administrators can modify a config file on a live system and verify it's syntax without hosing the actual rsyslogd process. >> 3 - For those more subtle problems, and things you might miss upon >> bootup. This also gives you a simple, easily accessible debugging >> source with minimal security implications and additional code and no >> required user configuration. I'm not sure how the code is written, but >> I'd prefer to see this happen even if the configuration file can't be >> read. Perhaps an $RsyslogErrors directive that, by default, references >> a different module that just dumps directly to a file? Users can >> configure it to put rsyslog errors through the actual logging engine >> and then manage it with typical syslog selector/action lines. > > All internally-generated messages are run through a specific interface. > For the "diag plugin", I will define a hook, where a diag server can > register to receive these messages. Maybe a good compromise would be to > add this $RsyslogErrors directive and make it point to a file. So, > still, we do not necessarily have it, but package maintainers would > hopefully include it and point it to a specific file (what still means I > do not necessarily know where to look). A problem is source tarball > install, in which case the directive may not be set. But with a proper > sample rsyslog.conf... I think a locally installed sample rsyslog.conf is a solid workaround for the "different distros, different locations" problem. > >> The HTTP server is an interesting idea, but not something I would make >> use of. The potential security problems are enormous, and it would >> introduce too many additional factors to the debugging process: is the >> firewall right? Was there already a process listening on that port? Is >> the user's configuration file correct? These are the kinds of things >> you already have to deal with on rsyslog itself - why force the >> debugging process to go through them again? > > I see (and share) much of your argument. This is why I stayed away for > such a long time. But if you think a bit more about it, it has *very > interesting* capabilities. Eg. I may also use it to gather some > real-time view of internal state, emit commands like HUP and do a number > of other nice things. Again, security is problematic. > > In any case, I'll give it a try. I have started to write a small plugin > which will dump the error messages within 10 minutes after startup. It > will not be a real http server but rather a listener that response to > anyone who connects to it (so that nc can be used as a client ;)). This > is very rough, but it has two side-effects: > > a) it provides something to actually experiment with (and to extend if > we decide to keep that route) > b) it helps create the plumbing inside the internal interface that could > also be used by any other method, so the potential waste of time is > limited It sounds interesting, and I agree that there's a lot of potential there. Among them, the possibility of adding an XML-RPC API to allow for greater interface flexibility...but frankly, that scares the hell out of me. ;) I'll wait to see what you come up with. -HKS From rgerhards at hq.adiscon.com Fri Jul 25 18:07:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 18:07:51 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> Message-ID: <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> inline... On Fri, 2008-07-25 at 11:44 -0400, (private) HKS wrote: > Responses inline. > > > I totally agree, but I think the situation is even worse for a syslogd. > > Because it *is* the usual error reporting system. I think we are a bit > > in the same situation (if you allow that analogy) like HIV which is > > attacking the human immune system. We have a problem in the > > troubleshooting system itself, and it may even prevent us from conveying > > troubleshooting information :( > > That's a good way of putting it. This is one of the reasons I think > it's so important to print errors to STDERR - if the logging > mechanisms are broken for some reason, then at least they're going > somewhere. And I sincerely hope that users have the sense to start it > from the command line the first time rather than sticking it into > their system startup scripts and rebooting. If I look at some of the support calls I see, I doubt that ;) The problem is that there are a lot of novices who have never ever done any real system administration. But as the rsyslog homepage says: "Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user." Look at the end of the sentence. In any case, I see the utility of what you say, if not in the novice case, then with more experienced users. > >> It's happened a few times that I start > >> rsyslogd at the command line, and yet there's nothing in the process > >> table, and nothing dumped to my logs. > > > > Could you elaborate a bit about above sentence - I have to admit I do > > not fully get you... > > I've messed up my configs bad enough a couple times that when I start > rsyslogd, nothing apparently happens. No errors to STDERR, no errors > in my logs, and the process isn't running. Rewriting the config or > running with -dn is the solution. Ah, ok, now I understand. Thanks. > > > > >> 2 - Configuration and other runtime errors should be printed to STDERR > > > > Does that really help? If so, that would be extremely easy to add. But > > does somebody see them? > > Absolutely. The first time I configure any daemon, I do it in a test > environment and run it by hand until I have the configuration acting > exactly as I want. I understand. That makes a lot of sense. As I said, it should be easy to implement. I'll once again shuffle priorities a bit and make that the top one. Should not take too long to implement. I'll probably have a switch to disable it, but being enabled by default. > Most users take the "run-by-hand" precaution even > if they don't have the luxury of a test setup. These will also pop up > when a user tries to restart the daemon after making a config change. > I'm not sure if they'll be visible when a HUP signal is sent, but that > would make me awfully happy. So expect to be happy some time next week ;) > > > >> 3 - By default, rsyslogd should log all of its own errors (fatal or > >> not) to /var/log/rsyslog.log. This should also be user-configurable > > > > This default comes into the way of some distros: they use different > > pathes. I really don't like the idea of embedding any information about > > the file system into the binary. Just think about embedded systems... > > Good point. > > > >> 1 - If rsyslog doesn't understand your config file, obviously it won't > >> be doing what you intend it to. There's little benefit in running with > >> a horked configuration, but if you don't test thoroughly, the cost > >> could be devastating. Some users won't know they're not logging things > >> until they need them - then, if they keep their jobs, they probably > >> won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > > > That's quite controversal and and does not match rsyslog's root > > philosophy. Let me explain: > > > > The syslogd is an extremely important piece of software. If it is not > > functioning, > > > > a) vital data can be lost > > b) the system as whole may hang (due to the fact that the log > > socket gets filled up and so other processes block on it) > > > > As such, rsyslogd is written in a sense that it tries to continue to > > work under almost all circumstances. It even tries to survive a > > system-wide low memory condition, though I am not 100% convinced it will > > manage to do that in all cases. At least it is coded careful to survive > > as much as possible. > > > > Now let's assume that a configuration error happens that prevents > > rsyslogd from executing an otherwise perfect configuration. Should we > > actually stop it from running? And live with the results? Or should we > > do whatever we can do and do that as good as possible? What, if it is > > not a configuration issue but a hard disk failure or an admin error that > > make part of the config unavailable (e.g. not able to load module, not > > able to write file)? Assuming we can carry out other actions - should we > > really abort? This may even come at the price that we can NOT warn users > > (via wall) of a serious problem. And what with warnings? Things that > > look wrong, but may be OK. Reason to abort? Or continue run? Tough > > decision. So I still think the right thing to do is to run as long as > > there is limited sense in it, because everything else would actually be > > worse. Please note that rsyslogd even comes with a hardcoded emergency > > configuration which is used in case the real configuration file can not > > be obtained. > > > > What I could add, however, is an option to make it stop on any config > > error (not warning). However, I fear that, if turned on, the results can > > be fatal in some cases... > > You make a lot of good points here. Does the hardcoded emergency > configuration that rsyslogd uses actually log? Yes, but in a very limited sense. It puts *.err onto the console, does a wall for *.panic and writes everything else to the controlling tty. I have to admit that this code isn't awfully well tested, usually only when something is done at the config code (and sometimes not even then...). > I only ask because it's > easy (if you fail to read the documentation) to write a config file > that doesn't do anything. Would it make sense for rsyslog to determine > whether the passed configuration actually asks it to do anything, and > complaining if not? That's an interesting idea. I am not sure if this is already checked. I can at least check if there is no action at all, which means there is no useful work to be done. The question remains, as usual, where to complain to. But that could be another situation where the hardcoded config could be activated. I have also thought about an implicit write user, with the user being root by default (and being overwritable via command line and config file). > > Printing errors to STDERR would help with this. Another possibility > might be adding a flag (-n is typical for a lot of systems, but that's > already in use) that will tell rsyslog to parse the configuration, > spit out any problems, and quit. That way, administrators can modify a > config file on a live system and verify it's syntax without hosing the > actual rsyslogd process. That's also a good idea. Rsyslog is quite modular (not yet as much as I would like it to be, but well enough). So I could even generate a dedicated config check tool. That would not touch any running configuration at all. But maybe the switch is easier ;) I need to think on how this affects the testbed I have begun to work on. Looks like this could also drive some automated tests. > > > > >> 3 - For those more subtle problems, and things you might miss upon > >> bootup. This also gives you a simple, easily accessible debugging > >> source with minimal security implications and additional code and no > >> required user configuration. I'm not sure how the code is written, but > >> I'd prefer to see this happen even if the configuration file can't be > >> read. Perhaps an $RsyslogErrors directive that, by default, references > >> a different module that just dumps directly to a file? Users can > >> configure it to put rsyslog errors through the actual logging engine > >> and then manage it with typical syslog selector/action lines. > > > > All internally-generated messages are run through a specific interface. > > For the "diag plugin", I will define a hook, where a diag server can > > register to receive these messages. Maybe a good compromise would be to > > add this $RsyslogErrors directive and make it point to a file. So, > > still, we do not necessarily have it, but package maintainers would > > hopefully include it and point it to a specific file (what still means I > > do not necessarily know where to look). A problem is source tarball > > install, in which case the directive may not be set. But with a proper > > sample rsyslog.conf... > > I think a locally installed sample rsyslog.conf is a solid workaround > for the "different distros, different locations" problem. > > > > >> The HTTP server is an interesting idea, but not something I would make > >> use of. The potential security problems are enormous, and it would > >> introduce too many additional factors to the debugging process: is the > >> firewall right? Was there already a process listening on that port? Is > >> the user's configuration file correct? These are the kinds of things > >> you already have to deal with on rsyslog itself - why force the > >> debugging process to go through them again? > > > > I see (and share) much of your argument. This is why I stayed away for > > such a long time. But if you think a bit more about it, it has *very > > interesting* capabilities. Eg. I may also use it to gather some > > real-time view of internal state, emit commands like HUP and do a number > > of other nice things. Again, security is problematic. > > > > In any case, I'll give it a try. I have started to write a small plugin > > which will dump the error messages within 10 minutes after startup. It > > will not be a real http server but rather a listener that response to > > anyone who connects to it (so that nc can be used as a client ;)). This > > is very rough, but it has two side-effects: > > > > a) it provides something to actually experiment with (and to extend if > > we decide to keep that route) > > b) it helps create the plumbing inside the internal interface that could > > also be used by any other method, so the potential waste of time is > > limited > > It sounds interesting, and I agree that there's a lot of potential > there. Among them, the possibility of adding an XML-RPC API to allow > for greater interface flexibility...but frankly, that scares the hell > out of me. ;) I'll wait to see what you come up with. hehe... We'll see, but I'd say its faaar down the road. One interesting aspect, though, could be a realtime view of syslog data as it is flowing through the system. That could even be integrated into phpLogCon. In any case, I do NOT want to add much (http) application logic / presentation to such a system. But a few html (or xml ;)) pages generated at appropriate times could be really useful. Just think about things like a "granular HUP", like being able to tell to close some files specifically etc etc. But that's too far advanced. For me personally, it would be quite useful to get access to state variables (like mem utilization, object instances, error count, frame rate) of the running system - not under a debugger but in a real live system. From the developers (and troubleshooter's point of view), that would be very interesting. I could even hold the last 2000 or so messages of the debug log in memory, plus the initial first 1000. Together, I guess, they can solve over 90% of all things I have used debug logs for. But this can only be done if I have dynamic runtime access to the live instance. And this obviously also means there is a big security issue ;) Please keep the thoughts flowing, we make good progess :) Rainer > > -HKS > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Mon Jul 28 17:54:04 2008 From: hks.private at gmail.com ((private) HKS) Date: Mon, 28 Jul 2008 11:54:04 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> Message-ID: I don't have much more to add to this discussion - I think we're pretty much on the same page. I'm looking forward to seeing the results of your work. -HKS On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards wrote: > > > Please keep the thoughts flowing, we make good progess :) > Rainer From rory at ooma.com Tue Jul 29 01:11:52 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 16:11:52 -0700 Subject: [rsyslog] error with phplogcon Message-ID: <488E5238.9070606@ooma.com> The installation instructions say I need php, apache and mysql. When I run the installation for phplogcon, it finishes and then I get an error: Errordetails: Error, MYSQL Extensions are not enabled! Function 'mysql_connect' does not exist. There are several things I could install here... I installed the mysql-connector-odbc rpm, but this may or may not be the right thing. It still doesn't work, and docs on this are pretty scarce. So, what am I missing? Which rpm should I be using to get the mysql_connect function? From samuel at dragonboricua.net Tue Jul 29 02:06:53 2008 From: samuel at dragonboricua.net (Elisamuel Resto) Date: Mon, 28 Jul 2008 20:06:53 -0400 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E5238.9070606@ooma.com> References: <488E5238.9070606@ooma.com> Message-ID: <20080728200653.38ae2bed@hades.dragonboricua.com> On Mon, 28 Jul 2008 16:11:52 -0700, Rory Toma wrote: > The installation instructions say I need php, apache and mysql. > > When I run the installation for phplogcon, it finishes and then I get an > error: > > Errordetails: > Error, MYSQL Extensions are not enabled! Function 'mysql_connect' does > not exist. > > There are several things I could install here... I installed the > mysql-connector-odbc rpm, but this may or may not be the right thing. It > still doesn't work, and docs on this are pretty scarce. So, what am I > missing? Which rpm should I be using to get the mysql_connect function? I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, depending on your installed php version). ODBC connector is quite a different thing. -- Elisamuel Resto Source Mage General Guru / http://sourcemage.org GPG ID: 18615F19/1024D / http://simplysam.us From rory at ooma.com Tue Jul 29 02:48:52 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 17:48:52 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080728200653.38ae2bed@hades.dragonboricua.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> Message-ID: <488E68F4.20903@ooma.com> Yes, turns out after random google searching that I needed php-mysql. I am past this stage now. However... I now get at error "No syslog records found. - Error Details Unknown or unhandled error occurred" I can connect manually and do a select * from the SystemEvents db and see records, using the user/passwd combo. One thing I couldn't find... What is supposed to go into the field "Table type"? I put in "rsyslog" but I'm not sure what is supposed to go there. thx > > I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, > depending on your installed php version). ODBC connector is quite a different > thing. > > From mic at npgx.com.au Tue Jul 29 02:56:11 2008 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 29 Jul 2008 11:56:11 +1100 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E68F4.20903@ooma.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> Message-ID: <20080729005501.M56815@npgx.com.au> Hi, > Yes, turns out after random google searching that I needed php- > mysql. I am past this stage now. > > However... > > I now get at error > > "No syslog records found. - Error Details > Unknown or unhandled error occurred" I get this error all the time when I install phplogcon. It's just the case of the "systemevents" name in the config file, I change it to "SystemEvents" and it works. Maybe it's an idea to post your config to the list (just the last bit at the bottom) so we can see what you're doing. Regards, Michael. > I can connect manually and do a select * from the SystemEvents db > and see records, using the user/passwd combo. > > One thing I couldn't find... What is supposed to go into the field > "Table type"? I put in "rsyslog" but I'm not sure what is supposed > to go there. > > thx > > > > I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, > > depending on your installed php version). ODBC connector is quite a different > > thing. > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From rory at ooma.com Tue Jul 29 03:02:19 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 18:02:19 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080729005501.M56815@npgx.com.au> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> Message-ID: <488E6C1B.8000208@ooma.com> Here ya go, thx. $CFG['Sources']['Source1']['ID'] = 'Source1'; $CFG['Sources']['Source1']['Name'] = 'rsyslog'; $CFG['Sources']['Source1']['SourceType'] = 2; $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; $CFG['Sources']['Source1']['DBType'] = '0'; $CFG['Sources']['Source1']['DBServer'] = 'localhost'; $CFG['Sources']['Source1']['DBName'] = 'Syslog'; $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; From mic at npgx.com.au Tue Jul 29 03:58:11 2008 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 29 Jul 2008 12:58:11 +1100 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E6C1B.8000208@ooma.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> <488E6C1B.8000208@ooma.com> Message-ID: <20080729015636.M15076@npgx.com.au> Hi, > Here ya go, thx. > > $CFG['Sources']['Source1']['ID'] = 'Source1'; > $CFG['Sources']['Source1']['Name'] = 'rsyslog'; > $CFG['Sources']['Source1']['SourceType'] = 2; > $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; > $CFG['Sources']['Source1']['DBType'] = '0'; > $CFG['Sources']['Source1']['DBServer'] = 'localhost'; > $CFG['Sources']['Source1']['DBName'] = 'Syslog'; > $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; > $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; > $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; > $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; This is mine: $CFG['DefaultSourceID'] = 'Source1'; $CFG['Sources']['Source1']['ID'] = 'Source1'; $CFG['Sources']['Source1']['Name'] = 'somename'; $CFG['Sources']['Source1']['ViewID'] = 'SYSLOG'; $CFG['Sources']['Source1']['SourceType'] = SOURCE_DB; $CFG['Sources']['Source1']['DBTableType'] = 'monitorware'; $CFG['Sources']['Source1']['DBServer'] = 'somedbserver'; $CFG['Sources']['Source1']['DBName'] = 'somedbname'; $CFG['Sources']['Source1']['DBUser'] = 'someuser'; $CFG['Sources']['Source1']['DBPassword'] = 'longstringofcharacters'; $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; Regards, Michael. From rory at ooma.com Tue Jul 29 04:02:54 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 19:02:54 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080729015636.M15076@npgx.com.au> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> <488E6C1B.8000208@ooma.com> <20080729015636.M15076@npgx.com.au> Message-ID: <488E7A4E.3030103@ooma.com> Thx, that fixed it. I didn't iterate over each difference, but... thx again. Michael Mansour wrote: > Hi, > > >> Here ya go, thx. >> >> $CFG['Sources']['Source1']['ID'] = 'Source1'; >> $CFG['Sources']['Source1']['Name'] = 'rsyslog'; >> $CFG['Sources']['Source1']['SourceType'] = 2; >> $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; >> $CFG['Sources']['Source1']['DBType'] = '0'; >> $CFG['Sources']['Source1']['DBServer'] = 'localhost'; >> $CFG['Sources']['Source1']['DBName'] = 'Syslog'; >> $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; >> $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; >> $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; >> $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; >> > > This is mine: > > $CFG['DefaultSourceID'] = 'Source1'; > > $CFG['Sources']['Source1']['ID'] = 'Source1'; > $CFG['Sources']['Source1']['Name'] = 'somename'; > $CFG['Sources']['Source1']['ViewID'] = 'SYSLOG'; > $CFG['Sources']['Source1']['SourceType'] = SOURCE_DB; > $CFG['Sources']['Source1']['DBTableType'] = 'monitorware'; > $CFG['Sources']['Source1']['DBServer'] = 'somedbserver'; > $CFG['Sources']['Source1']['DBName'] = 'somedbname'; > $CFG['Sources']['Source1']['DBUser'] = 'someuser'; > $CFG['Sources']['Source1']['DBPassword'] = 'longstringofcharacters'; > $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; > $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; > > Regards, > > Michael. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Tue Jul 29 10:10:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 10:10:45 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> I have implemented much yesterday. It is available via git right now: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05b 76862ea2e211cc09b30fe I will create a testbed for it today and hopefully be able to release it some time this afternoon. Unfortunately, it's a busy day... Feedback is appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Monday, July 28, 2008 5:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > I don't have much more to add to this discussion - I think we're > pretty much on the same page. I'm looking forward to seeing the > results of your work. > > -HKS > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > wrote: > > > > > > Please keep the thoughts flowing, we make good progess :) > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 29 15:57:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 15:57:31 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> OK, I have now a preview tarball available. I'd appreciate if you could give it a try. Use -N1 on the command line to make it do a config check, only. It is available here: http://download.rsyslog.com/download/rsyslog-3.21.1.tar.gz md5sum: I will replace that file once the "official" 3.21.1 is out. I have one problem with autotools: a "make distcheck" fails, as far as I can see because rsyslogd is not built inside the tools subdirectory. If someone has a suggestion for a fix (or what could cause the problem), I would appreciate that. Actually this "make distcheck" problem is what made me hold back the official release. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 29, 2008 10:11 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > I have implemented much yesterday. It is available via git right now: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05 > b > 76862ea2e211cc09b30fe > > I will create a testbed for it today and hopefully be able to release > it > some time this afternoon. Unfortunately, it's a busy day... > > Feedback is appreciated. > Rainer > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of (private) HKS > > Sent: Monday, July 28, 2008 5:54 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog error messages > > > > I don't have much more to add to this discussion - I think we're > > pretty much on the same page. I'm looking forward to seeing the > > results of your work. > > > > -HKS > > > > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > > wrote: > > > > > > > > > Please keep the thoughts flowing, we make good progess :) > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 29 15:58:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 15:58:36 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEDA@grfint2.intern.adiscon.com> It would have been better if I had actually included the md5sum ;). It is b9ca041c00d093981e6c6e35d360e6e9 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 29, 2008 3:58 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > OK, I have now a preview tarball available. I'd appreciate if you could > give it a try. > > Use -N1 on the command line to make it do a config check, only. > > It is available here: > > http://download.rsyslog.com/download/rsyslog-3.21.1.tar.gz > md5sum: > > I will replace that file once the "official" 3.21.1 is out. > > I have one problem with autotools: a "make distcheck" fails, as far as > I > can see because rsyslogd is not built inside the tools subdirectory. If > someone has a suggestion for a fix (or what could cause the problem), I > would appreciate that. Actually this "make distcheck" problem is what > made me hold back the official release. > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Tuesday, July 29, 2008 10:11 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog error messages > > > > I have implemented much yesterday. It is available via git right now: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05 > > b > > 76862ea2e211cc09b30fe > > > > I will create a testbed for it today and hopefully be able to release > > it > > some time this afternoon. Unfortunately, it's a busy day... > > > > Feedback is appreciated. > > Rainer > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of (private) HKS > > > Sent: Monday, July 28, 2008 5:54 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog error messages > > > > > > I don't have much more to add to this discussion - I think we're > > > pretty much on the same page. I'm looking forward to seeing the > > > results of your work. > > > > > > -HKS > > > > > > > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > > > wrote: > > > > > > > > > > > > Please keep the thoughts flowing, we make good progess :) > > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Tue Jul 29 16:14:08 2008 From: hks.private at gmail.com ((private) HKS) Date: Tue, 29 Jul 2008 10:14:08 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> Message-ID: I'll test this as soon as I can, but it may be a couple days... -HKS On Tue, Jul 29, 2008 at 4:10 AM, Rainer Gerhards wrote: > I have implemented much yesterday. It is available via git right now: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05b > 76862ea2e211cc09b30fe > > I will create a testbed for it today and hopefully be able to release it > some time this afternoon. Unfortunately, it's a busy day... > > Feedback is appreciated. > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of (private) HKS >> Sent: Monday, July 28, 2008 5:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog error messages >> >> I don't have much more to add to this discussion - I think we're >> pretty much on the same page. I'm looking forward to seeing the >> results of your work. >> >> -HKS >> >> >> On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards >> wrote: >> > >> > >> > Please keep the thoughts flowing, we make good progess :) >> > Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mattjhell at gmail.com Wed Jul 30 00:28:43 2008 From: mattjhell at gmail.com (Matt Hellman) Date: Tue, 29 Jul 2008 17:28:43 -0500 Subject: [rsyslog] Thinking about syslog forwarding infrastructure Message-ID: In the interest of brevity, I'm leaving out details...if you need more, just ask. We have a security event monitoring system that processes probably in the neighborhood of 100 million syslog messages per day (I know precisely how many events it has processed, but it doesn't break them down by protocol). In some of our WAN sites, I would like to implement a local system that will receive all the local syslog messages and ship some/all back to the main collector on our LAN. The main collector on the LAN would receive the bulk of its message from other LAN systems. Then the main collect would ship some/all to the SEM environment. I'm leaning towards using rsyslog for this task and have a few questions: 1) What kind of system [rough estimate] would I need for the main collector if assume 200 million syslog messages per day and peak that is triple that average rate (~7000 eps)? 2) Can I enable rate limiting in a way that will: A1) start dropping messages beyond a given threshold A2) start intelligently dropping messages beyond a given threshold (i.e. start dropping events matching this regex) B) allow me to alert someone that this is occurring (is written to log file, etc) From rory at ooma.com Wed Jul 30 03:14:52 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 18:14:52 -0700 Subject: [rsyslog] mysql inserts and message queues Message-ID: <488FC08C.4080201@ooma.com> I have enabled mysql and disk message queues as per below: $ModLoad immark.so # provides --MARK-- message capability $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # kernel logging (formerly provided by rklogd) $ModLoad ommysql.so $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port 514 $WorkDirectory /var/rsyslog $ActionQueueType LinkedList $ActionQueueFileName dbq $ActionResumeRetryCount -1 *.* :ommysql:127.0.0.1,Syslog,syslogwriter,writeme # *.* /var/log/messages The question I have is... does the disk queue drain w/o me having to intervene? I currently have 323 queue files (I was adding some keys and playing with tuning mysql... kind of slowed it down) and the number of queue files is not going down. thx From rgerhards at hq.adiscon.com Wed Jul 30 07:45:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 07:45:38 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <488FC08C.4080201@ooma.com> References: <488FC08C.4080201@ooma.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> I don't see any obvious error and, yes, the queue files should be deleted. Actually, you should only see any files at all if the inserts take too long. What you have may be an artifact of a previous failure. To get down to what it is, we need to check how things progress. Is your database populated? In any case, a debug log (rsyslogd with -dn additional options run interactively) will tell us what is there. Also, there should be a .qi file. Let us know its contents. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rory Toma > Sent: Wednesday, July 30, 2008 3:15 AM > To: rsyslog-users > Subject: [rsyslog] mysql inserts and message queues > > I have enabled mysql and disk message queues as per below: > > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging (e.g. > via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > $ModLoad ommysql.so > $ModLoad imudp.so # provides UDP syslog reception > $UDPServerRun 514 # start a UDP syslog server at standard port 514 > > $WorkDirectory /var/rsyslog > > $ActionQueueType LinkedList > $ActionQueueFileName dbq > $ActionResumeRetryCount -1 > > *.* :ommysql:127.0.0.1,Syslog,syslogwriter,writeme > # *.* /var/log/messages > > > The question I have is... does the disk queue drain w/o me having to > intervene? I currently have 323 queue files (I was adding some keys and > playing with tuning mysql... kind of slowed it down) and the number of > queue files is not going down. > > thx > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rory at ooma.com Wed Jul 30 07:58:32 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 22:58:32 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> Message-ID: <48900308.3070805@ooma.com> Rainer Gerhards wrote: > I don't see any obvious error and, yes, the queue files should be > deleted. Actually, you should only see any files at all if the inserts > take too long. What you have may be an artifact of a previous failure. > > To get down to what it is, we need to check how things progress. Is your > database populated? In any case, a debug log (rsyslogd with -dn > additional options run interactively) will tell us what is there. Also, > there should be a .qi file. Let us know its contents. > > Thanks, > Rainer > > the db is getting populated and there is no .qi file. If I run in with "-dn" there is a ton of output, so I need to know what to look for. I've been running for about a day, and I'm at 20+ million records. I do get an error -2040 when accessing on-disk files. Should I just nuke these? From rory at ooma.com Wed Jul 30 08:07:59 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 23:07:59 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <48900308.3070805@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> Message-ID: <4890053F.40606@ooma.com> As a side note, I stopped mysql, saw the number of queue files increase, restarted mysql and they went back down to the original 300 or so. So, maybe I just have some horked queue files. Rory Toma wrote: > Rainer Gerhards wrote: > >> I don't see any obvious error and, yes, the queue files should be >> deleted. Actually, you should only see any files at all if the inserts >> take too long. What you have may be an artifact of a previous failure. >> >> To get down to what it is, we need to check how things progress. Is your >> database populated? In any case, a debug log (rsyslogd with -dn >> additional options run interactively) will tell us what is there. Also, >> there should be a .qi file. Let us know its contents. >> >> Thanks, >> Rainer >> >> >> > the db is getting populated and there is no .qi file. If I run in with > "-dn" there is a ton of output, so I need to know what to look for. I've > been running for about a day, and I'm at 20+ million records. > > I do get an error -2040 when accessing on-disk files. Should I just nuke > these? > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Wed Jul 30 08:30:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 08:30:28 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <48900308.3070805@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> Message-ID: <1217399428.7184.260.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-29 at 22:58 -0700, Rory Toma wrote: > Rainer Gerhards wrote: > > I don't see any obvious error and, yes, the queue files should be > > deleted. Actually, you should only see any files at all if the inserts > > take too long. What you have may be an artifact of a previous failure. > > > > To get down to what it is, we need to check how things progress. Is your > > database populated? In any case, a debug log (rsyslogd with -dn > > additional options run interactively) will tell us what is there. Also, > > there should be a .qi file. Let us know its contents. > > > > Thanks, > > Rainer > > > > > the db is getting populated and there is no .qi file. No .qi file is a good indication (though not sufficient) that the queue files are artifacts of a previous failure. > If I run in with > "-dn" there is a ton of output, so I need to know what to look for. I've > been running for about a day, and I'm at 20+ million records. I usually know only when I see. Based on this case, I'd be interested in the first 2000 lines and another 1000 lines while it is processing records. That should at least provide enough of a clue to ask for something more specific. > > I do get an error -2040 when accessing on-disk files. Should I just nuke > these? This can be OK. I think you see the -2040 (file not found) when it is looking for the .qi files. Rainer > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 30 08:31:08 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 08:31:08 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <4890053F.40606@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> <4890053F.40606@ooma.com> Message-ID: <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-29 at 23:07 -0700, Rory Toma wrote: > As a side note, I stopped mysql, saw the number of queue files increase, > restarted mysql and they went back down to the original 300 or so. > > So, maybe I just have some horked queue files. that's now even more probable. But let's verify with the debug log. Rainer > > > Rory Toma wrote: > > Rainer Gerhards wrote: > > > >> I don't see any obvious error and, yes, the queue files should be > >> deleted. Actually, you should only see any files at all if the inserts > >> take too long. What you have may be an artifact of a previous failure. > >> > >> To get down to what it is, we need to check how things progress. Is your > >> database populated? In any case, a debug log (rsyslogd with -dn > >> additional options run interactively) will tell us what is there. Also, > >> there should be a .qi file. Let us know its contents. > >> > >> Thanks, > >> Rainer > >> > >> > >> > > the db is getting populated and there is no .qi file. If I run in with > > "-dn" there is a ton of output, so I need to know what to look for. I've > > been running for about a day, and I'm at 20+ million records. > > > > I do get an error -2040 when accessing on-disk files. Should I just nuke > > these? > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From friedl at hq.adiscon.com Wed Jul 30 17:07:22 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 30 Jul 2008 17:07:22 +0200 Subject: [rsyslog] rsyslog 3.21.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEE8@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.21.1, a development branch version. It contains some bug fixes and support for improved config file error checking. Most importantly, rsyslogd now has a "config file verification option", which makes it verify the config file but not start up or interfere with a running configuration. That should make troubleshooting config problems much easier. The version now also writes all internally-generated messages to stderr (if not disabled). Thanks to HKS for suggesting the new features. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-123.phtml Changelog: http://www.rsyslog.com/Article262.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Wed Jul 30 17:13:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 17:13:35 +0200 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: References: Message-ID: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Hi Matt, On Tue, 2008-07-29 at 17:28 -0500, Matt Hellman wrote: > In the interest of brevity, I'm leaving out details...if you need > more, just ask. > > We have a security event monitoring system that processes probably in > the neighborhood of 100 million syslog messages per day (I know > precisely how many events it has processed, but it doesn't break them > down by protocol). In some of our WAN sites, I would like to > implement a local system that will receive all the local syslog > messages and ship some/all back to the main collector on our LAN. The > main collector on the LAN would receive the bulk of its message from > other LAN systems. Then the main collect would ship some/all to the > SEM environment. I'm leaning towards using rsyslog for this task and > have a few questions: > > 1) What kind of system [rough estimate] would I need for the main > collector if assume 200 million syslog messages per day and peak that > is triple that average rate (~7000 eps)? Quite honestly: I don't know. Which rules you carry out has a big effect. But I have no real good big deployment numbers. The old game: everyone is interested in them, no-one conveys them (hint: let me know if you have some ;)). > > 2) Can I enable rate limiting in a way that will: > A1) start dropping messages beyond a given threshold you can do that > > A2) start intelligently dropping messages beyond a given threshold > (i.e. start dropping events matching this regex) not yet, but an interesting idea > > B) allow me to alert someone that this is occurring (is written to > log file, etc) mmhhh... not really. That's another interesting idea, and it should be simple to enable. It conveys that to the debug log, but does not emit a user message. In any case, I think there are a couple of docs you need to read and *understand* for this scale of deployment. Ask if you do not understand them - I have written them and may have left too much out just out of habit ;) First and foremost, you must understand rsyslog queues. They handle all queueing and rate limiting. The relevant doc is: http://www.rsyslog.com/doc-queues.html Then, I suggest to have a quick look at some use cases. I probably is a good idea to read the queue doc once, go over the use cases and the re-read the queue doc with the use cases on your mind: http://www.rsyslog.com/doc-queues.html http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html http://wiki.rsyslog.com/index.php/OffPeakHours Finally, IMHO you need to think about syslog reliability and the service level you can expect. This is not only true for rsyslog, but the other folks don't tell you about the reliability issues. Read this: http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html I hope this gets you started. Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 30 17:15:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 17:15:32 +0200 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> References: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEE9@grfint2.intern.adiscon.com> I was too quick. A small yet important thing to add: > > A2) start intelligently dropping messages beyond a given > threshold > > (i.e. start dropping events matching this regex) > > not yet, but an interesting idea Well, it is semi-intelligent. Messages are discarded based on severity level. But you can not use any other property. The details are in the queue doc. Rainer From mattjhell at gmail.com Wed Jul 30 19:33:14 2008 From: mattjhell at gmail.com (Matt Hellman) Date: Wed, 30 Jul 2008 12:33:14 -0500 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> References: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Message-ID: Thanks for the reply Rainer. >> 1) What kind of system [rough estimate] would I need for the main >> collector if assume 200 million syslog messages per day and peak that >> is triple that average rate (~7000 eps)? > > Quite honestly: I don't know. Which rules you carry out has a big > effect. But I have no real good big deployment numbers. The old game: > everyone is interested in them, no-one conveys them (hint: let me know > if you have some ;)). crap. well, I can probably test this easily enough myself. I just feel better knowing that someone has already done it. >> A2) start intelligently dropping messages beyond a given threshold >> (i.e. start dropping events matching this regex) > > not yet, but an interesting idea well, regex wouldn't be the only "intelligent" way to drop messages. I suppose anything that isn't arbitrary might be considered intelligent. Currently this is done based on priority, which won't work well for us because we use a product (Snare) that converts windows events into syslog that all have the same priority. FWIW, this is a common way for SEM products to collect Windows events. >> B) allow me to alert someone that this is occurring (is written to >> log file, etc) > > mmhhh... not really. That's another interesting idea, and it should be > simple to enable. It conveys that to the debug log, but does not emit a > user message. I was thinking about this and I don't necessarily need the product to emit something directly to a user, if that's what you mean. I plan to buffer to disk. Can I create a process to monitor the queue files or something --warning: I have printed but at best skimmed many of the docs you reference;-) > In any case, I think there are a couple of docs you need to read and > *understand* for this scale of deployment. Ask if you do not understand > them - I have written them and may have left too much out just out of > habit ;) re: doc links. Thanks. I was being lazy and trying to avoid having to read them prematurely;-) I think I'm too the point where I believe rsyslog can theoretically deliver on my requirements though. It's time to dig in. From rory at ooma.com Wed Jul 30 20:30:43 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 11:30:43 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> <4890053F.40606@ooma.com> <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> Message-ID: <4890B353.6000607@ooma.com> Rainer Gerhards wrote: > On Tue, 2008-07-29 at 23:07 -0700, Rory Toma wrote: > >> As a side note, I stopped mysql, saw the number of queue files increase, >> restarted mysql and they went back down to the original 300 or so. >> >> So, maybe I just have some horked queue files. >> > > that's now even more probable. But let's verify with the debug log. > > OK, here's the log file, about 20-30 seconds of runtime... http://www.colinburns.com/rsyslog/rsyslog.txt.gz thx From julianokyap at gmail.com Thu Jul 31 00:18:09 2008 From: julianokyap at gmail.com (Julian Yap) Date: Wed, 30 Jul 2008 12:18:09 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: Is there a way to set an Alert when multiple repeated lines are found in a log? I want to spawn an email Alert if a message is received 3 times. Example log lines: Jul 30 04:19:29 localhost program: Error detected Jul 30 05:19:29 localhost program: Error detected Jul 30 06:19:29 localhost program: Error detected Thanks, Julian From rory at ooma.com Thu Jul 31 02:15:41 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 17:15:41 -0700 Subject: [rsyslog] tips for managing data Message-ID: <4891042D.9040805@ooma.com> So, my current mysql rsyslog drops about 20 million rows of data per day. Over time, this gets slow as tables grow. I'm not a dba, so I was wondering if anyone had some suggestions for keeping performance still on the order of seconds, and not minutes or hours. thx I did add a key for EventSource, as that is commonly searched. However, using PhpLogCon, it seems that if I search using the web interface (i.e. I click on a host entry and hit the available searches) it is relatively quick. However, changing the text field that is generated and hitting the "search" button is slow. Do these two methods use the same query, or is something else going on? thx From rory at ooma.com Thu Jul 31 04:09:52 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 19:09:52 -0700 Subject: [rsyslog] tips for managing data In-Reply-To: <4891042D.9040805@ooma.com> References: <4891042D.9040805@ooma.com> Message-ID: <48911EF0.1010004@ooma.com> OK, so it seems that doing a query from the query line does a LIKE, which can take significantly longer (sample query 8 seconds vs. 50 msecs...) So, replacing the LIKE % in logstreamdb.class.db with an = speeds things up quite a but, but I lose some flexibility. Is there some kind of search syntax where I can differentiate between LIKE and =? If not, I'm thinking something like: source:foo.bar.com # would be using = ~source:foo # would be using LIKE Rory Toma wrote: > So, my current mysql rsyslog drops about 20 million rows of data per day. > > Over time, this gets slow as tables grow. > > I'm not a dba, so I was wondering if anyone had some suggestions for > keeping performance still on the order of seconds, and not minutes or hours. > > thx > > I did add a key for EventSource, as that is commonly searched. However, > using PhpLogCon, it seems that if I search using the web interface (i.e. > I click on a host entry and hit the available searches) it is relatively > quick. However, changing the text field that is generated and hitting > the "search" button is slow. Do these two methods use the same query, or > is something else going on? > > thx > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From alorbach at ro1.adiscon.com Thu Jul 31 10:15:23 2008 From: alorbach at ro1.adiscon.com (Andre Lorbach) Date: Thu, 31 Jul 2008 10:15:23 +0200 Subject: [rsyslog] tips for managing data In-Reply-To: <48911EF0.1010004@ooma.com> References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com> Message-ID: Hi, the like query can indeed have quiet an impact on performance when doing queries on large databases. But I think we can expand the syntax, so you can either search by part of a string (LIKE '%search%') or the whole string (= 'search'). This should be rather easy to implement. I will put this on my todolist, if it is as easy as I think, the next minor update of the devel branch will contain this new feature. Best regards, Andre Lorbach > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rory Toma > Sent: Thursday, July 31, 2008 4:10 AM > To: rsyslog-users > Subject: Re: [rsyslog] tips for managing data > > OK, so it seems that doing a query from the query line does a LIKE, > which can take significantly longer (sample query 8 seconds vs. 50 msecs...) > > So, replacing the LIKE % in logstreamdb.class.db with an = speeds things > up quite a but, but I lose some flexibility. Is there some kind of > search syntax where I can differentiate between LIKE and =? > > If not, I'm thinking something like: > > source:foo.bar.com # would be using = > > ~source:foo # would be using LIKE > > > > Rory Toma wrote: > > So, my current mysql rsyslog drops about 20 million rows of data per day. > > > > Over time, this gets slow as tables grow. > > > > I'm not a dba, so I was wondering if anyone had some suggestions for > > keeping performance still on the order of seconds, and not minutes or hours. > > > > thx > > > > I did add a key for EventSource, as that is commonly searched. However, > > using PhpLogCon, it seems that if I search using the web interface (i.e. > > I click on a host entry and hit the available searches) it is relatively > > quick. However, changing the text field that is generated and hitting > > the "search" button is slow. Do these two methods use the same query, or > > is something else going on? > > > > thx > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From ml at darville.vm.bytemark.co.uk Thu Jul 31 13:11:22 2008 From: ml at darville.vm.bytemark.co.uk (David Darville) Date: Thu, 31 Jul 2008 12:11:22 +0100 Subject: [rsyslog] Changing hostname field Message-ID: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Hello everyone I am trying to configure rsyslog to service a number of chroot jails in addition to the host itself. But I need to change the hostname field of the syslog messages from the different jails, so that I place them in the right log file on the central logging host. My current rsyslog.conf is as follows: $ModLoad imuxsock $ModLoad imklog $ModLoad immark $ModLoad omrelp $AddUnixListenSocket /jail/1/dev/log $AddUnixListenSocket /jail/2/dev/log *.* :omrelp:10.0.0.4:2514 Can anyone please advice me on how to do that? --- David Darville From hks.private at gmail.com Thu Jul 31 15:48:37 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 31 Jul 2008 09:48:37 -0400 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: References: Message-ID: Not in rsyslogd itself, but you could do this with Swatch, Nagios, or some other monitoring-type software. -HKS On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: > Is there a way to set an Alert when multiple repeated lines are found in a log? > > I want to spawn an email Alert if a message is received 3 times. > > Example log lines: > Jul 30 04:19:29 localhost program: Error detected > Jul 30 05:19:29 localhost program: Error detected > Jul 30 06:19:29 localhost program: Error detected > > Thanks, > Julian > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From hks.private at gmail.com Thu Jul 31 16:00:09 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 31 Jul 2008 10:00:09 -0400 Subject: [rsyslog] Changing hostname field In-Reply-To: <20080731111122.GA24000@darville.vm.bytemark.co.uk> References: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Message-ID: Do the jails all share the same hostname and IP? If not, you should be able to use the %hostname% or %fromhost% properties. If so, are they each running their own instance of (r)syslogd? -HKS On Thu, Jul 31, 2008 at 7:11 AM, David Darville wrote: > Hello everyone > > I am trying to configure rsyslog to service a number of chroot jails in > addition to the host itself. > > But I need to change the hostname field of the syslog messages from the > different jails, so that I place them in the right log file on the central > logging host. > > My current rsyslog.conf is as follows: > > $ModLoad imuxsock > $ModLoad imklog > $ModLoad immark > $ModLoad omrelp > > $AddUnixListenSocket /jail/1/dev/log > $AddUnixListenSocket /jail/2/dev/log > > *.* :omrelp:10.0.0.4:2514 > > > Can anyone please advice me on how to do that? > > > --- > > David Darville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rory at ooma.com Thu Jul 31 16:32:11 2008 From: rory at ooma.com (Rory Toma) Date: Thu, 31 Jul 2008 07:32:11 -0700 Subject: [rsyslog] tips for managing data In-Reply-To: References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com> Message-ID: <4891CCEB.50905@ooma.com> Cool. For me, it seems that using LIKE is most useful when searching the message text. So, something like: source:foo ~bar would produce where fromhost = 'foo' and message LIKE '%bar%' thx Andre Lorbach wrote: > Hi, > > the like query can indeed have quiet an impact on performance when doing > queries on large databases. > But I think we can expand the syntax, so you can either search by part > of a string (LIKE '%search%') or the whole string (= 'search'). This > should be rather easy to implement. I will put this on my todolist, if > it is as easy as I think, the next minor update of the devel branch will > contain this new feature. > > Best regards, > Andre Lorbach > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rory Toma >> Sent: Thursday, July 31, 2008 4:10 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] tips for managing data >> >> OK, so it seems that doing a query from the query line does a LIKE, >> which can take significantly longer (sample query 8 seconds vs. 50 >> > msecs...) > >> So, replacing the LIKE % in logstreamdb.class.db with an = speeds >> > things > >> up quite a but, but I lose some flexibility. Is there some kind of >> search syntax where I can differentiate between LIKE and =? >> >> If not, I'm thinking something like: >> >> source:foo.bar.com # would be using = >> >> ~source:foo # would be using LIKE >> >> >> >> Rory Toma wrote: >> >>> So, my current mysql rsyslog drops about 20 million rows of data per >>> > day. > >>> Over time, this gets slow as tables grow. >>> >>> I'm not a dba, so I was wondering if anyone had some suggestions for >>> keeping performance still on the order of seconds, and not minutes >>> > or hours. > >>> thx >>> >>> I did add a key for EventSource, as that is commonly searched. >>> > However, > >>> using PhpLogCon, it seems that if I search using the web interface >>> > (i.e. > >>> I click on a host entry and hit the available searches) it is >>> > relatively > >>> quick. However, changing the text field that is generated and >>> > hitting > >>> the "search" button is slow. Do these two methods use the same >>> > query, or > >>> is something else going on? >>> >>> thx >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From ml at darville.vm.bytemark.co.uk Thu Jul 31 16:46:49 2008 From: ml at darville.vm.bytemark.co.uk (David Darville) Date: Thu, 31 Jul 2008 15:46:49 +0100 Subject: [rsyslog] Changing hostname field In-Reply-To: References: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Message-ID: <20080731144649.GA24505@darville.vm.bytemark.co.uk> The jails all have their own unique hostname (and IP), but all share an rsyslogd instance running on the main host, and the %hostname% and %fromhost% in all the log messages from the jails are set to the hostname of the main host. And that is what I want to change. On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote: > Do the jails all share the same hostname and IP? If not, you should be > able to use the %hostname% or %fromhost% properties. > > If so, are they each running their own instance of (r)syslogd? > > -HKS > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville > wrote: > > Hello everyone > > > > I am trying to configure rsyslog to service a number of chroot jails in > > addition to the host itself. > > > > But I need to change the hostname field of the syslog messages from the > > different jails, so that I place them in the right log file on the central > > logging host. > > > > My current rsyslog.conf is as follows: > > > > $ModLoad imuxsock > > $ModLoad imklog > > $ModLoad immark > > $ModLoad omrelp > > > > $AddUnixListenSocket /jail/1/dev/log > > $AddUnixListenSocket /jail/2/dev/log > > > > *.* :omrelp:10.0.0.4:2514 > > > > > > Can anyone please advice me on how to do that? > > > > > > --- > > > > David Darville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 31 17:04:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 17:04:18 +0200 Subject: [rsyslog] Changing hostname field Message-ID: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com> Use a template with fixed name. --- Urspr?ngliche Nachricht --- Von: "David Darville" Betreff: Re: [rsyslog] Changing hostname field Datum: 31. Juli 2008 Uhrzeit: 16:46:59 The jails all have their own unique hostname (and IP), but all share an rsyslogd instance running on the main host, and the %hostname% and %fromhost% in all the log messages from the jails are set to the hostname of the main host. And that is what I want to change. On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote: > Do the jails all share the same hostname and IP? If not, you should be > able to use the %hostname% or %fromhost% properties. > > If so, are they each running their own instance of (r)syslogd? > > -HKS > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville > wrote: > > Hello everyone > > > > I am trying to configure rsyslog to service a number of chroot jails in > > addition to the host itself. > > > > But I need to change the hostname field of the syslog messages from the > > different jails, so that I place them in the right log file on the central > > logging host. > > > > My current rsyslog.conf is as follows: > > > > $ModLoad imuxsock > > $ModLoad imklog > > $ModLoad immark > > $ModLoad omrelp > > > > $AddUnixListenSocket /jail/1/dev/log > > $AddUnixListenSocket /jail/2/dev/log > > > > *.* :omrelp:10.0.0.4:2514 > > > > > > Can anyone please advice me on how to do that? > > > > > > --- > > > > David Darville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From julianokyap at gmail.com Thu Jul 31 20:27:14 2008 From: julianokyap at gmail.com (Julian Yap) Date: Thu, 31 Jul 2008 08:27:14 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: References: Message-ID: Hmm, Nagios is a pain to set up. Looking for something more light weight... Was hoping that I could have consolidated lots of Alerts under Rsyslog. Any other suggestions besides Swatch? On 7/31/08, (private) HKS wrote: > Not in rsyslogd itself, but you could do this with Swatch, Nagios, or > some other monitoring-type software. > > -HKS > > On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >> Is there a way to set an Alert when multiple repeated lines are found in a >> log? >> >> I want to spawn an email Alert if a message is received 3 times. >> >> Example log lines: >> Jul 30 04:19:29 localhost program: Error detected >> Jul 30 05:19:29 localhost program: Error detected >> Jul 30 06:19:29 localhost program: Error detected >> >> Thanks, >> Julian >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 20:37:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 20:37:05 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> What exactly do you need to do except the "three in a row" alert? ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Gesendet: 31.07.08 20:27 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found Hmm, Nagios is a pain to set up. Looking for something more light weight... Was hoping that I could have consolidated lots of Alerts under Rsyslog. Any other suggestions besides Swatch? On 7/31/08, (private) HKS wrote: > Not in rsyslogd itself, but you could do this with Swatch, Nagios, or > some other monitoring-type software. > > -HKS > > On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >> Is there a way to set an Alert when multiple repeated lines are found in a >> log? >> >> I want to spawn an email Alert if a message is received 3 times. >> >> Example log lines: >> Jul 30 04:19:29 localhost program: Error detected >> Jul 30 05:19:29 localhost program: Error detected >> Jul 30 06:19:29 localhost program: Error detected >> >> Thanks, >> Julian >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From julianokyap at gmail.com Thu Jul 31 21:59:51 2008 From: julianokyap at gmail.com (Julian Yap) Date: Thu, 31 Jul 2008 09:59:51 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> References: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> Message-ID: That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 22:38:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 22:38:38 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com> To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence: 1. a - a - a - b 2. a - a - b - a Result: in case 1 an alert is triggered, in case 2 not. Is this understanding correct? rainer ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Cc: "rgerhards at hq.adiscon.com" ; "hks.private at gmail.com" Gesendet: 31.07.08 21:59 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 23:19:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 23:19:35 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001601c8f353$363e638d$060013ac@intern.adiscon.com> Oh, and one thing i forgot: what makes an event identical? Same message except timestamp - or what (eg same host, same tag, ...) rainer ----- Urspr?ngliche Nachricht ----- Von: "Rainer Gerhards" An: "Julian Yap" Cc: "rsyslog at lists.adiscon.com" Gesendet: 31.07.08 22:39 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence: 1. a - a - a - b 2. a - a - b - a Result: in case 1 an alert is triggered, in case 2 not. Is this understanding correct? rainer ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Cc: "rgerhards at hq.adiscon.com" ; "hks.private at gmail.com" Gesendet: 31.07.08 21:59 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 1 15:36:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 1 Jul 2008 15:36:11 +0200 Subject: [rsyslog] rsyslog 3.19.8 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092F2@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.8, a new release of the development branch. Most importantly, it adds a bugfix for TLS mode. A TLS session was lost if the OS returned a retry request via the API. This happened frequently on some platforms. The situation is now properly handled. Feature-wise error number have been added to most rsyslog messages, making it easier to troubleshoot problems. There are also some other minor improvements and bug fixes. This is a recommended update for all devel branch users. Change Log: http://www.rsyslog.com/Article247.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-116.phtml I hope this release is useful. Feedback is appreciated. Rainer Gerhards From mycleanjunk at gmail.com Wed Jul 2 05:16:41 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Tue, 1 Jul 2008 20:16:41 -0700 Subject: [rsyslog] configuration file question and error message question Message-ID: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> Hi, I am using rsyslog 3.16.2. When I specify the script of command to issue when a log reaches its max size, how do I specify the command with parameters? This is what I did: $outchannel ch_common,/var/log/messages/1048675,\ mv -f /var/log/messages /var/log/messages.0 Do I use a quotation mark or a single quote or something else? Your help says to specify a script but does that line except a single word for the script, meaning it can not take parameters? To clarify, if I have a script that is named rotate and I want to use it for multiple logs, I would like to say "rotate /var/log/message". Is this possible? I am getting this error message: rsyslogd: omfwd.c:318: doTryResume: Assertion `0' failed. What does this mean? Thanks for your help!! I really like how rsyslogd has a lot of configurable properties. Scott From rgerhards at hq.adiscon.com Wed Jul 2 08:28:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 2 Jul 2008 08:28:59 +0200 Subject: [rsyslog] configuration file question and error message question In-Reply-To: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> References: <17945960807012016t753757aaxd0c1add1d3a12bb1@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092F7@grfint2.intern.adiscon.com> Hi, > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Wednesday, July 02, 2008 5:17 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] configuration file question and error message > question > > Hi, > > I am using rsyslog 3.16.2. > > When I specify the script of command to issue when a log reaches its > max size, how do I specify the command with parameters? The command can have EXACTLY one parameter. > This is what I > did: > > $outchannel ch_common,/var/log/messages/1048675,\ > mv -f /var/log/messages /var/log/messages.0 > > Do I use a quotation mark or a single quote or something else? Your > help says to specify a script but does that line except a single word > for the script, meaning it can not take parameters? To clarify, if I > have a script that is named rotate and I want to use it for multiple > logs, I would like to say "rotate /var/log/message". Is this possible? Yes, but you can not specify more than one parameter. The outchannel code is scheduled to be superseded by something new that works along the lines of the regular log file (just with an additional parameter). However, until this is done, we need to live with the current restriction. > > > I am getting this error message: > rsyslogd: omfwd.c:318: doTryResume: Assertion `0' failed. > > What does this mean? Something goes really wrong ;) I have checked that assertion and there seems to be some problem with the retry log. Can you reproduce it? If so, can you provide a full debug log (run interactively with -dn option)? > > Thanks for your help!! I really like how rsyslogd has a lot of > configurable properties. :) Rainer > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Wed Jul 2 10:15:07 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 01:15:07 -0700 Subject: [rsyslog] rsyslog threads questions Message-ID: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Hi, I have 3.16.2 which was recently released. I see that under certain conditions rsyslogd spawns a lot of threads: 5949 root 11216 S rsyslogd 5950 root 11216 S rsyslogd 5951 root 11216 S rsyslogd 5952 root 11216 S rsyslogd 5953 root 11216 S rsyslogd 5954 root 11216 S rsyslogd 5985 root Z [rsyslogd] 6445 root Z [rsyslogd] I had to kill the rsyslogd and restart it. The first invocation had a pid of 219 before it had to be killed. The second invocation of pid which you see above starts with 5949. The difference is the amount of zombie threads that were invoked by rsyslogd before I had to kill the first invocation of it. The question is under what conditions does rsyslogd spawn a new thread/process and why was it a zombie? I am running rsyslogd in an embedded environment and not a regular laptop/desktop. In addition, I am using busybox and I believe the syslog buffer size is set to something very low or perhaps none at all. Would this be a factor? Furthermore, I ran rsyslogd with -c3 and also without -c3 and both cases happen. Are these issues already known and fixed in a later version? Sorry, if I am asking the same questions or have the same issues as previous people but without the ability to search (or at least, I don't know how to) the archive, I don't know if my problem/questions has already been seen and/or resolved. Thank you very much for your support. Scott From rgerhards at hq.adiscon.com Wed Jul 2 10:27:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 02 Jul 2008 10:27:18 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Message-ID: <1214987239.7184.13.camel@rgf9dev.intern.adiscon.com> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > Hi, > > I have 3.16.2 which was recently released. I see that under certain > conditions rsyslogd spawns a lot of threads: > 5949 root 11216 S rsyslogd > 5950 root 11216 S rsyslogd > 5951 root 11216 S rsyslogd > 5952 root 11216 S rsyslogd > 5953 root 11216 S rsyslogd > 5954 root 11216 S rsyslogd > 5985 root Z [rsyslogd] > 6445 root Z [rsyslogd] > > I had to kill the rsyslogd and restart it. The first invocation had a > pid of 219 before it had to be killed. The second invocation of pid > which you see above starts with 5949. The difference is the amount of > zombie threads that were invoked by rsyslogd before I had to kill the > first invocation of it. I have no explanation yet for the zombies. They should not happen and so far I have never seen them. We may need to go through a debug log (which will become very large) to find out what's going on. > The question is under what conditions does rsyslogd spawn a new > thread/process and why was it a zombie? Unfortunately, there is no quick answer. A quick one may be: when it needs them, based on queue watermark settings and based on you configuration. But to really understand it, you need to read this doc: http://www.rsyslog.com/doc-queues.html The doc also describes all the knobs that you can use to control thread creation. There are many ;) > I am running rsyslogd in an > embedded environment and not a regular laptop/desktop. Interesting use case... > In addition, I > am using busybox and I believe the syslog buffer size is set to what do yo mean by "syslog buffer size"? The length of a receive buffer? It is 2K, thus single messages up to 2K are supported. It can be changed by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) support only up to 1K. > something very low or perhaps none at all. Would this be a factor? > Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > cases happen. The compatibility modes do not affect queue operation. > Are these issues already known and fixed in a later version? Sorry, if > I am asking the same questions or have the same issues as previous > people but without the ability to search (or at least, I don't know > how to) the archive, I don't know if my problem/questions has already > been seen and/or resolved. If we need to find out about the zombies, we need to move on to the current devel version. So I would give that a try in any case. 3.16.2 will (most probably) be replaced by 3.18.0 (based on the current beta) next week. So I won't touch it any longer. Looking forward to your feedback, Rainer > > Thank you very much for your support. > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 2 11:07:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 2 Jul 2008 11:07:28 +0200 Subject: [rsyslog] rsyslog error message repository Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3092FA@grfint2.intern.adiscon.com> Hi all, starting with 3.19.8, rsyslog finally offers specific error codes as part of the syslog tag. For example, rsyslogd-2040 means that a file could not be found. I have added these tags both to facilitate log parsing as well as easy troubleshooting. But a tag is only as good as the information that it helps to find. Consequently, I have started to describe error cases inside the knowledge base's event repository: http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog.html So far, there is only a limited set of messages available (to phrase it politely ;)), but I plan to increase it over time. Note that there is an interactive feature where questions to the message can directly be posted to the forum. I hope this is useful. If you run into an error message that is not-yet described, let us know and we'll add an entry. In the long term, the new knowledge base part should be able to solve most problems. Feedback is appreciated, Rainer From mycleanjunk at gmail.com Wed Jul 2 21:04:23 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 12:04:23 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> Message-ID: <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> Hi Rainer, Thanks for your reply. Looking at the default settings (from the online help's configuration page), they are what I wanted. The main messages queue is set to fix sized array with 1 worker thread created at maximum and action queues are direct mode which according to the queue document page, means that there will not be a worker thread created. Is my understanding correct? If yes, how do I quickly check without using the -d option if the defaults are set correctly? Or what do I look for in the debug messages that gets printed out to ensure this? You also mentioned that version 3.18.0 is probably going to be released as the stable version next week. I see on the webpage there is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? Also, how come I did not get your reply in my email inbox? My account settings look correct. Thanks, Scott Phuong As for the syslog buffer size, that applies to syslogd and does not apply to rsyslog. My configuration files do not change the Action queue or Worker queue parameters at all. Looking at On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > Hi, > > I have 3.16.2 which was recently released. I see that under certain > conditions rsyslogd spawns a lot of threads: > 5949 root 11216 S rsyslogd > 5950 root 11216 S rsyslogd > 5951 root 11216 S rsyslogd > 5952 root 11216 S rsyslogd > 5953 root 11216 S rsyslogd > 5954 root 11216 S rsyslogd > 5985 root Z [rsyslogd] > 6445 root Z [rsyslogd] > > I had to kill the rsyslogd and restart it. The first invocation had a > pid of 219 before it had to be killed. The second invocation of pid > which you see above starts with 5949. The difference is the amount of > zombie threads that were invoked by rsyslogd before I had to kill the > first invocation of it. I have no explanation yet for the zombies. They should not happen and so far I have never seen them. We may need to go through a debug log (which will become very large) to find out what's going on. > The question is under what conditions does rsyslogd spawn a new > thread/process and why was it a zombie? Unfortunately, there is no quick answer. A quick one may be: when it needs them, based on queue watermark settings and based on you configuration. But to really understand it, you need to read this doc: http://www.rsyslog.com/doc-queues.html The doc also describes all the knobs that you can use to control thread creation. There are many ;) > I am running rsyslogd in an > embedded environment and not a regular laptop/desktop. Interesting use case... > In addition, I > am using busybox and I believe the syslog buffer size is set to what do yo mean by "syslog buffer size"? The length of a receive buffer? It is 2K, thus single messages up to 2K are supported. It can be changed by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) support only up to 1K. > something very low or perhaps none at all. Would this be a factor? > Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > cases happen. The compatibility modes do not affect queue operation. > Are these issues already known and fixed in a later version? Sorry, if > I am asking the same questions or have the same issues as previous > people but without the ability to search (or at least, I don't know > how to) the archive, I don't know if my problem/questions has already > been seen and/or resolved. If we need to find out about the zombies, we need to move on to the current devel version. So I would give that a try in any case. 3.16.2 will (most probably) be replaced by 3.18.0 (based on the current beta) next week. So I won't touch it any longer. Looking forward to your feedback, Rainer > > Thank you very much for your support. > > Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Thu Jul 3 08:48:02 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 23:48:02 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Message-ID: <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> Sorry for sending a lot of emails. I believe I have root-cause the issue in 3.17.5. It appears that when the rsyslog.conf file does something like this: emerg.* *;mytemplate The code executes wallmsg which does forks. This is how I got a lot of processes were being created in my scenario. Although one would hope that a system doesn't misbehave this awful, nonetheless it does happen. First, it seems that the emergency messages should go to all those that are logged in via whatever mechanism (i.e. telnet, console, stty, ssh, etc...), this message should appear on all those "screens". I see that it does not even if I just logged 1 message at this serverity. I believe this is how syslog described setting up a wall message and what it is. Is this a bug? Lastly, when the child of the fork exits, it does not appear to be removed from Linux and still continues to eat up memory and is reported as a zombie. It appears that when the workerthread has been "destroyed/deconstructed" does things clear up. I am not sure if this is a rsyslogd problem or a linux or gcc problem. Any ideas? Thanks, Scott On Wed, Jul 2, 2008 at 5:57 PM, Scott Phuong wrote: > Hi, > > I've attached four files. Two of which are debug dumps, one is the > conf file and the last one is a test case scenario that constantly > fails on my end. I hope this gives a little more information. > Furthermore, the dumps are from 3.17.5 which is the "closest" version > to 3.18.0 that I was able to find. > > Both failed scenarios occur when lots of messages were being flooded > to rsyslogd at a very fast rate (look at logtest.c) The > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads > that it took up so much memory that executing any command as simple as > 'ls -l' would not work from the command line. I think the number of > threads grew as much as the number of messages. In the latter > scenario, after killing logtest.c, it didn't look like the those > zombies threads went away until I did a CTRL+C to the rsyslogd which > was running in the foreground since I use the "-dn" option. > > This is on an embedded system that runs significantly slower than a > desktop or laptop so maybe it would be harder to reproduce on a > regular computer. I looked at all the parameters that I believe could > affect this and believe for the most part the defaults are more than > adequate. The main message queue never looked like it hit the high > water mark but it did hit the lower one. So, I don't think messages > were being dropped (not sure) or an overflow condition occurred. > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > compiled using GCC and the standard GNU C libraries version 3.4.5. > Rsyslog source code is cross-compiled using the following configure > line: > > ./configure --disable-zlib --disable-largefile > --enable-share=yes > --prefix=/ > --host=arm-unknown-linux-gnu > ac_cv_func_malloc_0_nonnull=yes > ac_cv_func_realloc_0_nonnull=yes > ac_cv_func_lstat_dereferences_slashed_symlink=yes > ac_cv_func_stat_empty_string_bug=no > enable_debug=no > enable_rtinst=no > > Lastly, the logtest was executed with just the "-s" parameter. It is a > simple C file that I came up with. > > I took a look at the debug messages and it does not appear that new > threads are created via calls to wtpStartWrkr in wtp.c. > > Any help I can bring to solve this issue, please let me know. I hope I > am not doing anything wrong here. > > Thanks, > > Scott >> >> On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong wrote: >>> Hi Rainer, >>> >>> Thanks for your reply. Looking at the default settings (from the >>> online help's configuration page), they are what I wanted. The main >>> messages queue is set to fix sized array with 1 worker thread created >>> at maximum and action queues are direct mode which according to the >>> queue document page, means that there will not be a worker thread >>> created. Is my understanding correct? If yes, how do I quickly check >>> without using the -d option if the defaults are set correctly? Or what >>> do I look for in the debug messages that gets printed out to ensure >>> this? >>> >>> You also mentioned that version 3.18.0 is probably going to be >>> released as the stable version next week. I see on the webpage there >>> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >>> >>> Also, how come I did not get your reply in my email inbox? My account >>> settings look correct. >>> >>> Thanks, >>> >>> Scott Phuong >>> >>> As for the syslog buffer size, that applies to syslogd and does not >>> apply to rsyslog. >>> >>> >>> >>> My configuration files do not change the Action queue or Worker queue >>> parameters at all. Looking at >>> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >>>> Hi, >>>> >>>> I have 3.16.2 which was recently released. I see that under certain >>>> conditions rsyslogd spawns a lot of threads: >>>> 5949 root 11216 S rsyslogd >>>> 5950 root 11216 S rsyslogd >>>> 5951 root 11216 S rsyslogd >>>> 5952 root 11216 S rsyslogd >>>> 5953 root 11216 S rsyslogd >>>> 5954 root 11216 S rsyslogd >>>> 5985 root Z [rsyslogd] >>>> 6445 root Z [rsyslogd] >>>> >>>> I had to kill the rsyslogd and restart it. The first invocation had a >>>> pid of 219 before it had to be killed. The second invocation of pid >>>> which you see above starts with 5949. The difference is the amount of >>>> zombie threads that were invoked by rsyslogd before I had to kill the >>>> first invocation of it. >>> >>> I have no explanation yet for the zombies. They should not happen and so >>> far I have never seen them. We may need to go through a debug log (which >>> will become very large) to find out what's going on. >>> >>>> The question is under what conditions does rsyslogd spawn a new >>>> thread/process and why was it a zombie? >>> >>> Unfortunately, there is no quick answer. A quick one may be: when it >>> needs them, based on queue watermark settings and based on you >>> configuration. But to really understand it, you need to read this doc: >>> >>> http://www.rsyslog.com/doc-queues.html >>> >>> The doc also describes all the knobs that you can use to control thread >>> creation. There are many ;) >>> >>>> I am running rsyslogd in an >>>> embedded environment and not a regular laptop/desktop. >>> >>> Interesting use case... >>> >>>> In addition, I >>>> am using busybox and I believe the syslog buffer size is set to >>> >>> what do yo mean by "syslog buffer size"? The length of a receive buffer? >>> It is 2K, thus single messages up to 2K are supported. It can be changed >>> by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) >>> support only up to 1K. >>> >>>> something very low or perhaps none at all. Would this be a factor? >>>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >>>> cases happen. >>> >>> The compatibility modes do not affect queue operation. >>> >>>> Are these issues already known and fixed in a later version? Sorry, if >>>> I am asking the same questions or have the same issues as previous >>>> people but without the ability to search (or at least, I don't know >>>> how to) the archive, I don't know if my problem/questions has already >>>> been seen and/or resolved. >>> >>> If we need to find out about the zombies, we need to move on to the >>> current devel version. So I would give that a try in any case. 3.16.2 >>> will (most probably) be replaced by 3.18.0 (based on the current beta) >>> next week. So I won't touch it any longer. >>> >>> Looking forward to your feedback, >>> Rainer >>> >>>> >>>> Thank you very much for your support. >>>> >>>> Scott >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > From rgerhards at hq.adiscon.com Thu Jul 3 08:59:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 3 Jul 2008 08:59:26 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com><17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <17945960807022348i50dcb4e9p912059a42c693796@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309307@grfint2.intern.adiscon.com> Hi Scott, will reply in more detail later today, but one thing quickly. This is very good information. The wall code is ooooooooooold, stems back to sysklogd and has only been slightly modified. I have to admit it is not much on my testing radar. So I assume you found the bug. I think I'll need to re-design it. With the queue engine, forking off should not really be necessary. I need to see if that goes into the v3-stable soon or is kept at the devel tree. I am a bit hesitant to move it to stable, as it involves a good chance for additional bugs... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Thursday, July 03, 2008 8:48 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog threads questions > > Sorry for sending a lot of emails. I believe I have root-cause the > issue in 3.17.5. It appears that when the rsyslog.conf file does > something like this: > emerg.* *;mytemplate > > The code executes wallmsg which does forks. This is how I got a lot of > processes were being created in my scenario. Although one would hope > that a system doesn't misbehave this awful, nonetheless it does > happen. First, it seems that the emergency messages should go to all > those that are logged in via whatever mechanism (i.e. telnet, console, > stty, ssh, etc...), this message should appear on all those "screens". > I see that it does not even if I just logged 1 message at this > serverity. I believe this is how syslog described setting up a wall > message and what it is. Is this a bug? > > Lastly, when the child of the fork exits, it does not appear to be > removed from Linux and still continues to eat up memory and is > reported as a zombie. It appears that when the workerthread has been > "destroyed/deconstructed" does things clear up. I am not sure if this > is a rsyslogd problem or a linux or gcc problem. Any ideas? > > Thanks, > > Scott > > > > On Wed, Jul 2, 2008 at 5:57 PM, Scott Phuong > wrote: > > Hi, > > > > I've attached four files. Two of which are debug dumps, one is the > > conf file and the last one is a test case scenario that constantly > > fails on my end. I hope this gives a little more information. > > Furthermore, the dumps are from 3.17.5 which is the "closest" version > > to 3.18.0 that I was able to find. > > > > Both failed scenarios occur when lots of messages were being flooded > > to rsyslogd at a very fast rate (look at logtest.c) The > > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" > threads > > that it took up so much memory that executing any command as simple > as > > 'ls -l' would not work from the command line. I think the number of > > threads grew as much as the number of messages. In the latter > > scenario, after killing logtest.c, it didn't look like the those > > zombies threads went away until I did a CTRL+C to the rsyslogd which > > was running in the foreground since I use the "-dn" option. > > > > This is on an embedded system that runs significantly slower than a > > desktop or laptop so maybe it would be harder to reproduce on a > > regular computer. I looked at all the parameters that I believe could > > affect this and believe for the most part the defaults are more than > > adequate. The main message queue never looked like it hit the high > > water mark but it did hit the lower one. So, I don't think messages > > were being dropped (not sure) or an overflow condition occurred. > > > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > > compiled using GCC and the standard GNU C libraries version 3.4.5. > > Rsyslog source code is cross-compiled using the following configure > > line: > > > > ./configure --disable-zlib --disable-largefile > > --enable-share=yes > > --prefix=/ > > --host=arm-unknown-linux-gnu > > ac_cv_func_malloc_0_nonnull=yes > > ac_cv_func_realloc_0_nonnull=yes > > > ac_cv_func_lstat_dereferences_slashed_symlink=yes > > ac_cv_func_stat_empty_string_bug=no > > enable_debug=no > > enable_rtinst=no > > > > Lastly, the logtest was executed with just the "-s" parameter. It is > a > > simple C file that I came up with. > > > > I took a look at the debug messages and it does not appear that new > > threads are created via calls to wtpStartWrkr in wtp.c. > > > > Any help I can bring to solve this issue, please let me know. I hope > I > > am not doing anything wrong here. > > > > Thanks, > > > > Scott > >> > >> On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > wrote: > >>> Hi Rainer, > >>> > >>> Thanks for your reply. Looking at the default settings (from the > >>> online help's configuration page), they are what I wanted. The main > >>> messages queue is set to fix sized array with 1 worker thread > created > >>> at maximum and action queues are direct mode which according to the > >>> queue document page, means that there will not be a worker thread > >>> created. Is my understanding correct? If yes, how do I quickly > check > >>> without using the -d option if the defaults are set correctly? Or > what > >>> do I look for in the debug messages that gets printed out to ensure > >>> this? > >>> > >>> You also mentioned that version 3.18.0 is probably going to be > >>> released as the stable version next week. I see on the webpage > there > >>> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? > >>> > >>> Also, how come I did not get your reply in my email inbox? My > account > >>> settings look correct. > >>> > >>> Thanks, > >>> > >>> Scott Phuong > >>> > >>> As for the syslog buffer size, that applies to syslogd and does not > >>> apply to rsyslog. > >>> > >>> > >>> > >>> My configuration files do not change the Action queue or Worker > queue > >>> parameters at all. Looking at > >>> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >>>> Hi, > >>>> > >>>> I have 3.16.2 which was recently released. I see that under > certain > >>>> conditions rsyslogd spawns a lot of threads: > >>>> 5949 root 11216 S rsyslogd > >>>> 5950 root 11216 S rsyslogd > >>>> 5951 root 11216 S rsyslogd > >>>> 5952 root 11216 S rsyslogd > >>>> 5953 root 11216 S rsyslogd > >>>> 5954 root 11216 S rsyslogd > >>>> 5985 root Z [rsyslogd] > >>>> 6445 root Z [rsyslogd] > >>>> > >>>> I had to kill the rsyslogd and restart it. The first invocation > had a > >>>> pid of 219 before it had to be killed. The second invocation of > pid > >>>> which you see above starts with 5949. The difference is the amount > of > >>>> zombie threads that were invoked by rsyslogd before I had to kill > the > >>>> first invocation of it. > >>> > >>> I have no explanation yet for the zombies. They should not happen > and so > >>> far I have never seen them. We may need to go through a debug log > (which > >>> will become very large) to find out what's going on. > >>> > >>>> The question is under what conditions does rsyslogd spawn a new > >>>> thread/process and why was it a zombie? > >>> > >>> Unfortunately, there is no quick answer. A quick one may be: when > it > >>> needs them, based on queue watermark settings and based on you > >>> configuration. But to really understand it, you need to read this > doc: > >>> > >>> http://www.rsyslog.com/doc-queues.html > >>> > >>> The doc also describes all the knobs that you can use to control > thread > >>> creation. There are many ;) > >>> > >>>> I am running rsyslogd in an > >>>> embedded environment and not a regular laptop/desktop. > >>> > >>> Interesting use case... > >>> > >>>> In addition, I > >>>> am using busybox and I believe the syslog buffer size is set to > >>> > >>> what do yo mean by "syslog buffer size"? The length of a receive > buffer? > >>> It is 2K, thus single messages up to 2K are supported. It can be > changed > >>> by modifying the MAXLINE define. Note that stock syslogd (and > RFC3164) > >>> support only up to 1K. > >>> > >>>> something very low or perhaps none at all. Would this be a factor? > >>>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > >>>> cases happen. > >>> > >>> The compatibility modes do not affect queue operation. > >>> > >>>> Are these issues already known and fixed in a later version? > Sorry, if > >>>> I am asking the same questions or have the same issues as previous > >>>> people but without the ability to search (or at least, I don't > know > >>>> how to) the archive, I don't know if my problem/questions has > already > >>>> been seen and/or resolved. > >>> > >>> If we need to find out about the zombies, we need to move on to the > >>> current devel version. So I would give that a try in any case. > 3.16.2 > >>> will (most probably) be replaced by 3.18.0 (based on the current > beta) > >>> next week. So I won't touch it any longer. > >>> > >>> Looking forward to your feedback, > >>> Rainer > >>> > >>>> > >>>> Thank you very much for your support. > >>>> > >>>> Scott > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Thu Jul 3 02:57:39 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Wed, 2 Jul 2008 17:57:39 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> Message-ID: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Hi, I've attached four files. Two of which are debug dumps, one is the conf file and the last one is a test case scenario that constantly fails on my end. I hope this gives a little more information. Furthermore, the dumps are from 3.17.5 which is the "closest" version to 3.18.0 that I was able to find. Both failed scenarios occur when lots of messages were being flooded to rsyslogd at a very fast rate (look at logtest.c) The my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads that it took up so much memory that executing any command as simple as 'ls -l' would not work from the command line. I think the number of threads grew as much as the number of messages. In the latter scenario, after killing logtest.c, it didn't look like the those zombies threads went away until I did a CTRL+C to the rsyslogd which was running in the foreground since I use the "-dn" option. This is on an embedded system that runs significantly slower than a desktop or laptop so maybe it would be harder to reproduce on a regular computer. I looked at all the parameters that I believe could affect this and believe for the most part the defaults are more than adequate. The main message queue never looked like it hit the high water mark but it did hit the lower one. So, I don't think messages were being dropped (not sure) or an overflow condition occurred. The processor is ARM-based and it is using Linux kernel 2.6.16.12 and compiled using GCC and the standard GNU C libraries version 3.4.5. Rsyslog source code is cross-compiled using the following configure line: ./configure --disable-zlib --disable-largefile --enable-share=yes --prefix=/ --host=arm-unknown-linux-gnu ac_cv_func_malloc_0_nonnull=yes ac_cv_func_realloc_0_nonnull=yes ac_cv_func_lstat_dereferences_slashed_symlink=yes ac_cv_func_stat_empty_string_bug=no enable_debug=no enable_rtinst=no Lastly, the logtest was executed with just the "-s" parameter. It is a simple C file that I came up with. I took a look at the debug messages and it does not appear that new threads are created via calls to wtpStartWrkr in wtp.c. Any help I can bring to solve this issue, please let me know. I hope I am not doing anything wrong here. Thanks, Scott > > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong wrote: >> Hi Rainer, >> >> Thanks for your reply. Looking at the default settings (from the >> online help's configuration page), they are what I wanted. The main >> messages queue is set to fix sized array with 1 worker thread created >> at maximum and action queues are direct mode which according to the >> queue document page, means that there will not be a worker thread >> created. Is my understanding correct? If yes, how do I quickly check >> without using the -d option if the defaults are set correctly? Or what >> do I look for in the debug messages that gets printed out to ensure >> this? >> >> You also mentioned that version 3.18.0 is probably going to be >> released as the stable version next week. I see on the webpage there >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >> >> Also, how come I did not get your reply in my email inbox? My account >> settings look correct. >> >> Thanks, >> >> Scott Phuong >> >> As for the syslog buffer size, that applies to syslogd and does not >> apply to rsyslog. >> >> >> >> My configuration files do not change the Action queue or Worker queue >> parameters at all. Looking at >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >>> Hi, >>> >>> I have 3.16.2 which was recently released. I see that under certain >>> conditions rsyslogd spawns a lot of threads: >>> 5949 root 11216 S rsyslogd >>> 5950 root 11216 S rsyslogd >>> 5951 root 11216 S rsyslogd >>> 5952 root 11216 S rsyslogd >>> 5953 root 11216 S rsyslogd >>> 5954 root 11216 S rsyslogd >>> 5985 root Z [rsyslogd] >>> 6445 root Z [rsyslogd] >>> >>> I had to kill the rsyslogd and restart it. The first invocation had a >>> pid of 219 before it had to be killed. The second invocation of pid >>> which you see above starts with 5949. The difference is the amount of >>> zombie threads that were invoked by rsyslogd before I had to kill the >>> first invocation of it. >> >> I have no explanation yet for the zombies. They should not happen and so >> far I have never seen them. We may need to go through a debug log (which >> will become very large) to find out what's going on. >> >>> The question is under what conditions does rsyslogd spawn a new >>> thread/process and why was it a zombie? >> >> Unfortunately, there is no quick answer. A quick one may be: when it >> needs them, based on queue watermark settings and based on you >> configuration. But to really understand it, you need to read this doc: >> >> http://www.rsyslog.com/doc-queues.html >> >> The doc also describes all the knobs that you can use to control thread >> creation. There are many ;) >> >>> I am running rsyslogd in an >>> embedded environment and not a regular laptop/desktop. >> >> Interesting use case... >> >>> In addition, I >>> am using busybox and I believe the syslog buffer size is set to >> >> what do yo mean by "syslog buffer size"? The length of a receive buffer? >> It is 2K, thus single messages up to 2K are supported. It can be changed >> by modifying the MAXLINE define. Note that stock syslogd (and RFC3164) >> support only up to 1K. >> >>> something very low or perhaps none at all. Would this be a factor? >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >>> cases happen. >> >> The compatibility modes do not affect queue operation. >> >>> Are these issues already known and fixed in a later version? Sorry, if >>> I am asking the same questions or have the same issues as previous >>> people but without the ability to search (or at least, I don't know >>> how to) the archive, I don't know if my problem/questions has already >>> been seen and/or resolved. >> >> If we need to find out about the zombies, we need to move on to the >> current devel version. So I would give that a try in any case. 3.16.2 >> will (most probably) be replaced by 3.18.0 (based on the current beta) >> next week. So I won't touch it any longer. >> >> Looking forward to your feedback, >> Rainer >> >>> >>> Thank you very much for your support. >>> >>> Scott >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > From niallel at gmail.com Thu Jul 3 23:59:53 2008 From: niallel at gmail.com (niall el-assaad) Date: Thu, 3 Jul 2008 22:59:53 +0100 Subject: [rsyslog] Problem matching localhost Message-ID: Hi, I'm running 2.0.0-11 (version included with redhat 5.2) I want to filter all the messages from external syslog devices to one file and all messages from the localhost to another file. However even with the -x option turned on when a local service (such as crond) sends a message to the log the hostname is set to the domain name of the server. So I can't use the following to match: :HOSTNAME, isequal, "localhost" /var/log/messages :HOSTNAME, !isequal, "localhost" /var/log/externalsyslog I could replace "localhost" with "dnsname" to get it to work, but I would like a generic method that will work on all the syslog servers I have. Is there some switch that will cause rsyslog to report the local services as sending from localhost or 127.0.0.1 rather than the hostname of the localhost. thanks, niall From rgerhards at hq.adiscon.com Fri Jul 4 08:12:39 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 08:12:39 +0200 Subject: [rsyslog] Problem matching localhost In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30932A@grfint2.intern.adiscon.com> I replied to your forum post: http://kb.monitorware.com/post13018.html#p13018 I suggest we keep discussing it in the forum to reduce double work. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of niall el-assaad > Sent: Friday, July 04, 2008 12:00 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Problem matching localhost > > Hi, > > I'm running 2.0.0-11 (version included with redhat 5.2) > > I want to filter all the messages from external syslog devices to one > file > and all messages from the localhost to another file. > > However even with the -x option turned on when a local service (such as > crond) sends a message to the log the hostname is set to the domain > name of > the server. > > So I can't use the following to match: > :HOSTNAME, isequal, "localhost" /var/log/messages > :HOSTNAME, !isequal, "localhost" /var/log/externalsyslog > > I could replace "localhost" with "dnsname" to get it to work, but I > would > like a generic method that will work on all the syslog servers I have. > Is there some switch that will cause rsyslog to report the local > services as > sending from localhost or 127.0.0.1 rather than the hostname of the > localhost. > > thanks, > > niall > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 4 10:31:08 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 10:31:08 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> Scott, I have rewritten the omusrmsg plugin. This is currently part of the development tree, but will become part of the (new) beta when we do a version number shift next week. I'd appreciate if you could give it a try. It is available here: http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz I am hesitant to put this in the upcoming new v3-stable. There were lots of changes and one rule for a stable is that there should be no non-fix type of changes for at least 4 weeks before it turns into a stable. It of course is debatable if the change I made is a fix or not. In some sense it is, but on the other hand it is a rewrite that changed a lot. So given the fact that nobody on a "regular" machine saw an issue the past year, I would not like to bring the new code directly into the stable. If that causes problems for you, you may want to try simply using the omusrmsg.c code together with the v3-stable. I haven't tried it, but it should work well. All changes were just to that file, and there are no dependencies to anything external. OK... but now let's see first if it fixes the issue ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Thursday, July 03, 2008 2:58 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog threads questions > > Hi, > > I've attached four files. Two of which are debug dumps, one is the > conf file and the last one is a test case scenario that constantly > fails on my end. I hope this gives a little more information. > Furthermore, the dumps are from 3.17.5 which is the "closest" version > to 3.18.0 that I was able to find. > > Both failed scenarios occur when lots of messages were being flooded > to rsyslogd at a very fast rate (look at logtest.c) The > my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads > that it took up so much memory that executing any command as simple as > 'ls -l' would not work from the command line. I think the number of > threads grew as much as the number of messages. In the latter > scenario, after killing logtest.c, it didn't look like the those > zombies threads went away until I did a CTRL+C to the rsyslogd which > was running in the foreground since I use the "-dn" option. > > This is on an embedded system that runs significantly slower than a > desktop or laptop so maybe it would be harder to reproduce on a > regular computer. I looked at all the parameters that I believe could > affect this and believe for the most part the defaults are more than > adequate. The main message queue never looked like it hit the high > water mark but it did hit the lower one. So, I don't think messages > were being dropped (not sure) or an overflow condition occurred. > > The processor is ARM-based and it is using Linux kernel 2.6.16.12 and > compiled using GCC and the standard GNU C libraries version 3.4.5. > Rsyslog source code is cross-compiled using the following configure > line: > > ./configure --disable-zlib --disable-largefile > --enable-share=yes > --prefix=/ > --host=arm-unknown-linux-gnu > ac_cv_func_malloc_0_nonnull=yes > ac_cv_func_realloc_0_nonnull=yes > > ac_cv_func_lstat_dereferences_slashed_symlink=yes > ac_cv_func_stat_empty_string_bug=no > enable_debug=no > enable_rtinst=no > > Lastly, the logtest was executed with just the "-s" parameter. It is a > simple C file that I came up with. > > I took a look at the debug messages and it does not appear that new > threads are created via calls to wtpStartWrkr in wtp.c. > > Any help I can bring to solve this issue, please let me know. I hope I > am not doing anything wrong here. > > Thanks, > > Scott > > > > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > wrote: > >> Hi Rainer, > >> > >> Thanks for your reply. Looking at the default settings (from the > >> online help's configuration page), they are what I wanted. The main > >> messages queue is set to fix sized array with 1 worker thread > created > >> at maximum and action queues are direct mode which according to the > >> queue document page, means that there will not be a worker thread > >> created. Is my understanding correct? If yes, how do I quickly > check > >> without using the -d option if the defaults are set correctly? Or > what > >> do I look for in the debug messages that gets printed out to ensure > >> this? > >> > >> You also mentioned that version 3.18.0 is probably going to be > >> released as the stable version next week. I see on the webpage there > >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? > >> > >> Also, how come I did not get your reply in my email inbox? My > account > >> settings look correct. > >> > >> Thanks, > >> > >> Scott Phuong > >> > >> As for the syslog buffer size, that applies to syslogd and does not > >> apply to rsyslog. > >> > >> > >> > >> My configuration files do not change the Action queue or Worker > queue > >> parameters at all. Looking at > >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >>> Hi, > >>> > >>> I have 3.16.2 which was recently released. I see that under certain > >>> conditions rsyslogd spawns a lot of threads: > >>> 5949 root 11216 S rsyslogd > >>> 5950 root 11216 S rsyslogd > >>> 5951 root 11216 S rsyslogd > >>> 5952 root 11216 S rsyslogd > >>> 5953 root 11216 S rsyslogd > >>> 5954 root 11216 S rsyslogd > >>> 5985 root Z [rsyslogd] > >>> 6445 root Z [rsyslogd] > >>> > >>> I had to kill the rsyslogd and restart it. The first invocation had > a > >>> pid of 219 before it had to be killed. The second invocation of pid > >>> which you see above starts with 5949. The difference is the amount > of > >>> zombie threads that were invoked by rsyslogd before I had to kill > the > >>> first invocation of it. > >> > >> I have no explanation yet for the zombies. They should not happen > and so > >> far I have never seen them. We may need to go through a debug log > (which > >> will become very large) to find out what's going on. > >> > >>> The question is under what conditions does rsyslogd spawn a new > >>> thread/process and why was it a zombie? > >> > >> Unfortunately, there is no quick answer. A quick one may be: when it > >> needs them, based on queue watermark settings and based on you > >> configuration. But to really understand it, you need to read this > doc: > >> > >> http://www.rsyslog.com/doc-queues.html > >> > >> The doc also describes all the knobs that you can use to control > thread > >> creation. There are many ;) > >> > >>> I am running rsyslogd in an > >>> embedded environment and not a regular laptop/desktop. > >> > >> Interesting use case... > >> > >>> In addition, I > >>> am using busybox and I believe the syslog buffer size is set to > >> > >> what do yo mean by "syslog buffer size"? The length of a receive > buffer? > >> It is 2K, thus single messages up to 2K are supported. It can be > changed > >> by modifying the MAXLINE define. Note that stock syslogd (and > RFC3164) > >> support only up to 1K. > >> > >>> something very low or perhaps none at all. Would this be a factor? > >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both > >>> cases happen. > >> > >> The compatibility modes do not affect queue operation. > >> > >>> Are these issues already known and fixed in a later version? Sorry, > if > >>> I am asking the same questions or have the same issues as previous > >>> people but without the ability to search (or at least, I don't know > >>> how to) the archive, I don't know if my problem/questions has > already > >>> been seen and/or resolved. > >> > >> If we need to find out about the zombies, we need to move on to the > >> current devel version. So I would give that a try in any case. > 3.16.2 > >> will (most probably) be replaced by 3.18.0 (based on the current > beta) > >> next week. So I won't touch it any longer. > >> > >> Looking forward to your feedback, > >> Rainer > >> > >>> > >>> Thank you very much for your support. > >>> > >>> Scott > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > From rgerhards at hq.adiscon.com Fri Jul 4 11:24:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 11:24:29 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> Hi list, I am posting a question here in the hope that some of the subscribers may be able to lend me a helping hand. Recently, I have begun to add a testbench to rsyslog. The idea is that over time the project should have canned tests which are easy to run on each version (as part of make distcheck at latest), increasing the overall code quality. "All" (so far two ;)) tests are located in the tests subdirectory. One test fails if compiled in release mode, that is rscript-parse.c. I have tracked down the failure cause and it is different struct member alignment in the ctok_token_t structure. The alignment is different in the rsyslog runtime and this test tool (I have compared the offsets computed for pToken->tok and they are different). So far, so good. But I do not find the cause of this misalignment. I checked the make output, and as it looks both the runtime as well as the tool are compiled with the same compiler options. Also, in debug mode it works OK, but in release mode (--disable-debug) it fails. I am sure the problem is something quite simple, but I have run out of ideas of what it may be (or, more probable, I simply overlook something). Any help would be deeply appreciated. Thanks, Rainer From mycleanjunk at gmail.com Fri Jul 4 12:28:58 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:28:58 -0700 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> Message-ID: <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> Here is my analysis: In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a #define and is defined to obj_t objData which is the same for both NDEBUG defined and not defined. But obj_t is a typedef of struct obj_s. In the structure definition for obj_s in obj-types.h are the following lines: #ifndef NDEBUG unsigned int iObjCooCKiE; #endif NDEBUG is defined in config.h which was included in the runtime for rsyslog but not for test-parse.c I didn't compile the code to test my theory. I just did a quick check. Hope that solved your problem. Scott I think this may be why the you are getting this structure misalignment? On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards wrote: > Hi list, > > I am posting a question here in the hope that some of the subscribers > may be able to lend me a helping hand. > > Recently, I have begun to add a testbench to rsyslog. The idea is that > over time the project should have canned tests which are easy to run on > each version (as part of make distcheck at latest), increasing the > overall code quality. > > "All" (so far two ;)) tests are located in the tests subdirectory. One > test fails if compiled in release mode, that is rscript-parse.c. I have > tracked down the failure cause and it is different struct member > alignment in the ctok_token_t structure. The alignment is different in > the rsyslog runtime and this test tool (I have compared the offsets > computed for pToken->tok and they are different). So far, so good. But I > do not find the cause of this misalignment. I checked the make output, > and as it looks both the runtime as well as the tool are compiled with > the same compiler options. Also, in debug mode it works OK, but in > release mode (--disable-debug) it fails. > > I am sure the problem is something quite simple, but I have run out of > ideas of what it may be (or, more probable, I simply overlook > something). > > Any help would be deeply appreciated. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mycleanjunk at gmail.com Fri Jul 4 12:31:45 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:31:45 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> Message-ID: <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> Hi Rainer, Thanks!! I'll give this a try next week and let you know the results. If it works, I think it is okay not to include in the upcoming v3 stable since it is due to release very soon. But if it does work, it would be great to have it in the following stable release. Scott On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards wrote: > Scott, > > I have rewritten the omusrmsg plugin. This is currently part of the > development tree, but will become part of the (new) beta when we do a > version number shift next week. I'd appreciate if you could give it a > try. It is available here: > > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz > > I am hesitant to put this in the upcoming new v3-stable. There were lots > of changes and one rule for a stable is that there should be no non-fix > type of changes for at least 4 weeks before it turns into a stable. It > of course is debatable if the change I made is a fix or not. In some > sense it is, but on the other hand it is a rewrite that changed a lot. > So given the fact that nobody on a "regular" machine saw an issue the > past year, I would not like to bring the new code directly into the > stable. If that causes problems for you, you may want to try simply > using the omusrmsg.c code together with the v3-stable. I haven't tried > it, but it should work well. All changes were just to that file, and > there are no dependencies to anything external. > > OK... but now let's see first if it fixes the issue ;) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Thursday, July 03, 2008 2:58 AM >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] rsyslog threads questions >> >> Hi, >> >> I've attached four files. Two of which are debug dumps, one is the >> conf file and the last one is a test case scenario that constantly >> fails on my end. I hope this gives a little more information. >> Furthermore, the dumps are from 3.17.5 which is the "closest" version >> to 3.18.0 that I was able to find. >> >> Both failed scenarios occur when lots of messages were being flooded >> to rsyslogd at a very fast rate (look at logtest.c) The >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" threads >> that it took up so much memory that executing any command as simple as >> 'ls -l' would not work from the command line. I think the number of >> threads grew as much as the number of messages. In the latter >> scenario, after killing logtest.c, it didn't look like the those >> zombies threads went away until I did a CTRL+C to the rsyslogd which >> was running in the foreground since I use the "-dn" option. >> >> This is on an embedded system that runs significantly slower than a >> desktop or laptop so maybe it would be harder to reproduce on a >> regular computer. I looked at all the parameters that I believe could >> affect this and believe for the most part the defaults are more than >> adequate. The main message queue never looked like it hit the high >> water mark but it did hit the lower one. So, I don't think messages >> were being dropped (not sure) or an overflow condition occurred. >> >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 and >> compiled using GCC and the standard GNU C libraries version 3.4.5. >> Rsyslog source code is cross-compiled using the following configure >> line: >> >> ./configure --disable-zlib --disable-largefile >> --enable-share=yes >> --prefix=/ >> --host=arm-unknown-linux-gnu >> ac_cv_func_malloc_0_nonnull=yes >> ac_cv_func_realloc_0_nonnull=yes >> >> ac_cv_func_lstat_dereferences_slashed_symlink=yes >> ac_cv_func_stat_empty_string_bug=no >> enable_debug=no >> enable_rtinst=no >> >> Lastly, the logtest was executed with just the "-s" parameter. It is a >> simple C file that I came up with. >> >> I took a look at the debug messages and it does not appear that new >> threads are created via calls to wtpStartWrkr in wtp.c. >> >> Any help I can bring to solve this issue, please let me know. I hope I >> am not doing anything wrong here. >> >> Thanks, >> >> Scott >> > >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > >> wrote: >> >> Hi Rainer, >> >> >> >> Thanks for your reply. Looking at the default settings (from the >> >> online help's configuration page), they are what I wanted. The main >> >> messages queue is set to fix sized array with 1 worker thread >> created >> >> at maximum and action queues are direct mode which according to the >> >> queue document page, means that there will not be a worker thread >> >> created. Is my understanding correct? If yes, how do I quickly >> check >> >> without using the -d option if the defaults are set correctly? Or >> what >> >> do I look for in the debug messages that gets printed out to ensure >> >> this? >> >> >> >> You also mentioned that version 3.18.0 is probably going to be >> >> released as the stable version next week. I see on the webpage > there >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to 3.18.0? >> >> >> >> Also, how come I did not get your reply in my email inbox? My >> account >> >> settings look correct. >> >> >> >> Thanks, >> >> >> >> Scott Phuong >> >> >> >> As for the syslog buffer size, that applies to syslogd and does not >> >> apply to rsyslog. >> >> >> >> >> >> >> >> My configuration files do not change the Action queue or Worker >> queue >> >> parameters at all. Looking at >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >> >>> Hi, >> >>> >> >>> I have 3.16.2 which was recently released. I see that under > certain >> >>> conditions rsyslogd spawns a lot of threads: >> >>> 5949 root 11216 S rsyslogd >> >>> 5950 root 11216 S rsyslogd >> >>> 5951 root 11216 S rsyslogd >> >>> 5952 root 11216 S rsyslogd >> >>> 5953 root 11216 S rsyslogd >> >>> 5954 root 11216 S rsyslogd >> >>> 5985 root Z [rsyslogd] >> >>> 6445 root Z [rsyslogd] >> >>> >> >>> I had to kill the rsyslogd and restart it. The first invocation > had >> a >> >>> pid of 219 before it had to be killed. The second invocation of > pid >> >>> which you see above starts with 5949. The difference is the amount >> of >> >>> zombie threads that were invoked by rsyslogd before I had to kill >> the >> >>> first invocation of it. >> >> >> >> I have no explanation yet for the zombies. They should not happen >> and so >> >> far I have never seen them. We may need to go through a debug log >> (which >> >> will become very large) to find out what's going on. >> >> >> >>> The question is under what conditions does rsyslogd spawn a new >> >>> thread/process and why was it a zombie? >> >> >> >> Unfortunately, there is no quick answer. A quick one may be: when > it >> >> needs them, based on queue watermark settings and based on you >> >> configuration. But to really understand it, you need to read this >> doc: >> >> >> >> http://www.rsyslog.com/doc-queues.html >> >> >> >> The doc also describes all the knobs that you can use to control >> thread >> >> creation. There are many ;) >> >> >> >>> I am running rsyslogd in an >> >>> embedded environment and not a regular laptop/desktop. >> >> >> >> Interesting use case... >> >> >> >>> In addition, I >> >>> am using busybox and I believe the syslog buffer size is set to >> >> >> >> what do yo mean by "syslog buffer size"? The length of a receive >> buffer? >> >> It is 2K, thus single messages up to 2K are supported. It can be >> changed >> >> by modifying the MAXLINE define. Note that stock syslogd (and >> RFC3164) >> >> support only up to 1K. >> >> >> >>> something very low or perhaps none at all. Would this be a factor? >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and both >> >>> cases happen. >> >> >> >> The compatibility modes do not affect queue operation. >> >> >> >>> Are these issues already known and fixed in a later version? > Sorry, >> if >> >>> I am asking the same questions or have the same issues as previous >> >>> people but without the ability to search (or at least, I don't > know >> >>> how to) the archive, I don't know if my problem/questions has >> already >> >>> been seen and/or resolved. >> >> >> >> If we need to find out about the zombies, we need to move on to the >> >> current devel version. So I would give that a try in any case. >> 3.16.2 >> >> will (most probably) be replaced by 3.18.0 (based on the current >> beta) >> >> next week. So I won't touch it any longer. >> >> >> >> Looking forward to your feedback, >> >> Rainer >> >> >> >>> >> >>> Thank you very much for your support. >> >>> >> >>> Scott >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Fri Jul 4 12:32:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:32:03 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> Hi Scott, thanks for the help, I'll further investigate. BUT: both the runtime AND the tool are compiled in non-debug mode. So in this case, neither of them should have the object cookie, thus the alignment should be the same. But, as I said, I am grateful for any clue and will check if there is some issue along these lines :) In any case, it seems to be inconsistent and should be fixed :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:29 PM > To: rsyslog-users > Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > > Here is my analysis: > In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a > #define and is defined to obj_t objData which is the same for both > NDEBUG defined and not defined. But obj_t is a typedef of struct > obj_s. In the structure definition for obj_s in obj-types.h are the > following lines: > #ifndef NDEBUG > unsigned int iObjCooCKiE; > #endif > > NDEBUG is defined in config.h which was included in the runtime for > rsyslog but not for test-parse.c > > I didn't compile the code to test my theory. I just did a quick check. > > Hope that solved your problem. > > Scott > > > I think this may be why the you are getting this structure > misalignment? > > > On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards > wrote: > > Hi list, > > > > I am posting a question here in the hope that some of the subscribers > > may be able to lend me a helping hand. > > > > Recently, I have begun to add a testbench to rsyslog. The idea is > that > > over time the project should have canned tests which are easy to run > on > > each version (as part of make distcheck at latest), increasing the > > overall code quality. > > > > "All" (so far two ;)) tests are located in the tests subdirectory. > One > > test fails if compiled in release mode, that is rscript-parse.c. I > have > > tracked down the failure cause and it is different struct member > > alignment in the ctok_token_t structure. The alignment is different > in > > the rsyslog runtime and this test tool (I have compared the offsets > > computed for pToken->tok and they are different). So far, so good. > But I > > do not find the cause of this misalignment. I checked the make > output, > > and as it looks both the runtime as well as the tool are compiled > with > > the same compiler options. Also, in debug mode it works OK, but in > > release mode (--disable-debug) it fails. > > > > I am sure the problem is something quite simple, but I have run out > of > > ideas of what it may be (or, more probable, I simply overlook > > something). > > > > Any help would be deeply appreciated. > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 4 12:35:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:35:23 +0200 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com><17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com><17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com><17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> Hi Scott, just for your understanding: once a release is stable, it will only receive fixes. So the fix we are looking it needs to become part of the devel, be migrated to beta and then be migrated to stable. In general, one migration step takes between 4 and 12 weeks, depending on what was done. So far, the average seems on the average, that is two month. So two cycles mean roughly four month. HOWEVER, I intend to put it into the next beta, which will bring us to just one cycle. So it would be available in roughly two month. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog threads questions > > Hi Rainer, > > Thanks!! I'll give this a try next week and let you know the results. > If it works, I think it is okay not to include in the upcoming v3 > stable since it is due to release very soon. But if it does work, it > would be great to have it in the following stable release. > > Scott > > On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards > wrote: > > Scott, > > > > I have rewritten the omusrmsg plugin. This is currently part of the > > development tree, but will become part of the (new) beta when we do a > > version number shift next week. I'd appreciate if you could give it a > > try. It is available here: > > > > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz > > > > I am hesitant to put this in the upcoming new v3-stable. There were > lots > > of changes and one rule for a stable is that there should be no non- > fix > > type of changes for at least 4 weeks before it turns into a stable. > It > > of course is debatable if the change I made is a fix or not. In some > > sense it is, but on the other hand it is a rewrite that changed a > lot. > > So given the fact that nobody on a "regular" machine saw an issue the > > past year, I would not like to bring the new code directly into the > > stable. If that causes problems for you, you may want to try simply > > using the omusrmsg.c code together with the v3-stable. I haven't > tried > > it, but it should work well. All changes were just to that file, and > > there are no dependencies to anything external. > > > > OK... but now let's see first if it fixes the issue ;) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong > >> Sent: Thursday, July 03, 2008 2:58 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: Re: [rsyslog] rsyslog threads questions > >> > >> Hi, > >> > >> I've attached four files. Two of which are debug dumps, one is the > >> conf file and the last one is a test case scenario that constantly > >> fails on my end. I hope this gives a little more information. > >> Furthermore, the dumps are from 3.17.5 which is the "closest" > version > >> to 3.18.0 that I was able to find. > >> > >> Both failed scenarios occur when lots of messages were being flooded > >> to rsyslogd at a very fast rate (look at logtest.c) The > >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while > >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" > threads > >> that it took up so much memory that executing any command as simple > as > >> 'ls -l' would not work from the command line. I think the number of > >> threads grew as much as the number of messages. In the latter > >> scenario, after killing logtest.c, it didn't look like the those > >> zombies threads went away until I did a CTRL+C to the rsyslogd which > >> was running in the foreground since I use the "-dn" option. > >> > >> This is on an embedded system that runs significantly slower than a > >> desktop or laptop so maybe it would be harder to reproduce on a > >> regular computer. I looked at all the parameters that I believe > could > >> affect this and believe for the most part the defaults are more than > >> adequate. The main message queue never looked like it hit the high > >> water mark but it did hit the lower one. So, I don't think messages > >> were being dropped (not sure) or an overflow condition occurred. > >> > >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 > and > >> compiled using GCC and the standard GNU C libraries version 3.4.5. > >> Rsyslog source code is cross-compiled using the following configure > >> line: > >> > >> ./configure --disable-zlib --disable-largefile > >> --enable-share=yes > >> --prefix=/ > >> --host=arm-unknown-linux-gnu > >> ac_cv_func_malloc_0_nonnull=yes > >> ac_cv_func_realloc_0_nonnull=yes > >> > >> ac_cv_func_lstat_dereferences_slashed_symlink=yes > >> ac_cv_func_stat_empty_string_bug=no > >> enable_debug=no > >> enable_rtinst=no > >> > >> Lastly, the logtest was executed with just the "-s" parameter. It is > a > >> simple C file that I came up with. > >> > >> I took a look at the debug messages and it does not appear that new > >> threads are created via calls to wtpStartWrkr in wtp.c. > >> > >> Any help I can bring to solve this issue, please let me know. I hope > I > >> am not doing anything wrong here. > >> > >> Thanks, > >> > >> Scott > >> > > >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong > > > >> wrote: > >> >> Hi Rainer, > >> >> > >> >> Thanks for your reply. Looking at the default settings (from the > >> >> online help's configuration page), they are what I wanted. The > main > >> >> messages queue is set to fix sized array with 1 worker thread > >> created > >> >> at maximum and action queues are direct mode which according to > the > >> >> queue document page, means that there will not be a worker thread > >> >> created. Is my understanding correct? If yes, how do I quickly > >> check > >> >> without using the -d option if the defaults are set correctly? Or > >> what > >> >> do I look for in the debug messages that gets printed out to > ensure > >> >> this? > >> >> > >> >> You also mentioned that version 3.18.0 is probably going to be > >> >> released as the stable version next week. I see on the webpage > > there > >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to > 3.18.0? > >> >> > >> >> Also, how come I did not get your reply in my email inbox? My > >> account > >> >> settings look correct. > >> >> > >> >> Thanks, > >> >> > >> >> Scott Phuong > >> >> > >> >> As for the syslog buffer size, that applies to syslogd and does > not > >> >> apply to rsyslog. > >> >> > >> >> > >> >> > >> >> My configuration files do not change the Action queue or Worker > >> queue > >> >> parameters at all. Looking at > >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: > >> >>> Hi, > >> >>> > >> >>> I have 3.16.2 which was recently released. I see that under > > certain > >> >>> conditions rsyslogd spawns a lot of threads: > >> >>> 5949 root 11216 S rsyslogd > >> >>> 5950 root 11216 S rsyslogd > >> >>> 5951 root 11216 S rsyslogd > >> >>> 5952 root 11216 S rsyslogd > >> >>> 5953 root 11216 S rsyslogd > >> >>> 5954 root 11216 S rsyslogd > >> >>> 5985 root Z [rsyslogd] > >> >>> 6445 root Z [rsyslogd] > >> >>> > >> >>> I had to kill the rsyslogd and restart it. The first invocation > > had > >> a > >> >>> pid of 219 before it had to be killed. The second invocation of > > pid > >> >>> which you see above starts with 5949. The difference is the > amount > >> of > >> >>> zombie threads that were invoked by rsyslogd before I had to > kill > >> the > >> >>> first invocation of it. > >> >> > >> >> I have no explanation yet for the zombies. They should not happen > >> and so > >> >> far I have never seen them. We may need to go through a debug log > >> (which > >> >> will become very large) to find out what's going on. > >> >> > >> >>> The question is under what conditions does rsyslogd spawn a new > >> >>> thread/process and why was it a zombie? > >> >> > >> >> Unfortunately, there is no quick answer. A quick one may be: when > > it > >> >> needs them, based on queue watermark settings and based on you > >> >> configuration. But to really understand it, you need to read this > >> doc: > >> >> > >> >> http://www.rsyslog.com/doc-queues.html > >> >> > >> >> The doc also describes all the knobs that you can use to control > >> thread > >> >> creation. There are many ;) > >> >> > >> >>> I am running rsyslogd in an > >> >>> embedded environment and not a regular laptop/desktop. > >> >> > >> >> Interesting use case... > >> >> > >> >>> In addition, I > >> >>> am using busybox and I believe the syslog buffer size is set to > >> >> > >> >> what do yo mean by "syslog buffer size"? The length of a receive > >> buffer? > >> >> It is 2K, thus single messages up to 2K are supported. It can be > >> changed > >> >> by modifying the MAXLINE define. Note that stock syslogd (and > >> RFC3164) > >> >> support only up to 1K. > >> >> > >> >>> something very low or perhaps none at all. Would this be a > factor? > >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and > both > >> >>> cases happen. > >> >> > >> >> The compatibility modes do not affect queue operation. > >> >> > >> >>> Are these issues already known and fixed in a later version? > > Sorry, > >> if > >> >>> I am asking the same questions or have the same issues as > previous > >> >>> people but without the ability to search (or at least, I don't > > know > >> >>> how to) the archive, I don't know if my problem/questions has > >> already > >> >>> been seen and/or resolved. > >> >> > >> >> If we need to find out about the zombies, we need to move on to > the > >> >> current devel version. So I would give that a try in any case. > >> 3.16.2 > >> >> will (most probably) be replaced by 3.18.0 (based on the current > >> beta) > >> >> next week. So I won't touch it any longer. > >> >> > >> >> Looking forward to your feedback, > >> >> Rainer > >> >> > >> >>> > >> >>> Thank you very much for your support. > >> >>> > >> >>> Scott > >> >>> _______________________________________________ > >> >>> rsyslog mailing list > >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > >> > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mycleanjunk at gmail.com Fri Jul 4 12:38:12 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:38:12 -0700 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com> <17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> Message-ID: <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> Hi Rainer, But in release mode, NDEBUG is intentionally defined in the config.h. This is what I recalled from looking at 3.16.1 and 3.17.5. So from the perspective of rsyslog, NDEBUG is defined and the opposite for the test-parse.c (because config.h is not included). Scott On Fri, Jul 4, 2008 at 3:32 AM, Rainer Gerhards wrote: > Hi Scott, > > thanks for the help, I'll further investigate. BUT: both the runtime AND > the tool are compiled in non-debug mode. So in this case, neither of > them should have the object cookie, thus the alignment should be the > same. But, as I said, I am grateful for any clue and will check if there > is some issue along these lines :) In any case, it seems to be > inconsistent and should be fixed :) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Friday, July 04, 2008 12:29 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] struct alignment problem - who can help? ;) >> >> Here is my analysis: >> In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a >> #define and is defined to obj_t objData which is the same for both >> NDEBUG defined and not defined. But obj_t is a typedef of struct >> obj_s. In the structure definition for obj_s in obj-types.h are the >> following lines: >> #ifndef NDEBUG >> unsigned int iObjCooCKiE; >> #endif >> >> NDEBUG is defined in config.h which was included in the runtime for >> rsyslog but not for test-parse.c >> >> I didn't compile the code to test my theory. I just did a quick check. >> >> Hope that solved your problem. >> >> Scott >> >> >> I think this may be why the you are getting this structure >> misalignment? >> >> >> On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards >> wrote: >> > Hi list, >> > >> > I am posting a question here in the hope that some of the > subscribers >> > may be able to lend me a helping hand. >> > >> > Recently, I have begun to add a testbench to rsyslog. The idea is >> that >> > over time the project should have canned tests which are easy to run >> on >> > each version (as part of make distcheck at latest), increasing the >> > overall code quality. >> > >> > "All" (so far two ;)) tests are located in the tests subdirectory. >> One >> > test fails if compiled in release mode, that is rscript-parse.c. I >> have >> > tracked down the failure cause and it is different struct member >> > alignment in the ctok_token_t structure. The alignment is different >> in >> > the rsyslog runtime and this test tool (I have compared the offsets >> > computed for pToken->tok and they are different). So far, so good. >> But I >> > do not find the cause of this misalignment. I checked the make >> output, >> > and as it looks both the runtime as well as the tool are compiled >> with >> > the same compiler options. Also, in debug mode it works OK, but in >> > release mode (--disable-debug) it fails. >> > >> > I am sure the problem is something quite simple, but I have run out >> of >> > ideas of what it may be (or, more probable, I simply overlook >> > something). >> > >> > Any help would be deeply appreciated. >> > >> > Thanks, >> > Rainer >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mycleanjunk at gmail.com Fri Jul 4 12:39:32 2008 From: mycleanjunk at gmail.com (Scott Phuong) Date: Fri, 4 Jul 2008 03:39:32 -0700 Subject: [rsyslog] rsyslog threads questions In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> References: <17945960807020115y7ba1d72bp485080214ced80c2@mail.gmail.com> <17945960807021204n38c2e9bfka17e9341035295f3@mail.gmail.com> <17945960807021749q29f89272w74b8c30d45fba97b@mail.gmail.com> <17945960807021757g3a9f8c16le353e03f17b5f18c@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA309331@grfint2.intern.adiscon.com> <17945960807040331u3c0a3deen3cb9d49bb035e665@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30933C@grfint2.intern.adiscon.com> Message-ID: <17945960807040339y44e5f75cgde6b4593e5c99584@mail.gmail.com> Hi Rainer, Thanks for the information. I didn't know that. Scott On Fri, Jul 4, 2008 at 3:35 AM, Rainer Gerhards wrote: > Hi Scott, > > just for your understanding: once a release is stable, it will only > receive fixes. So the fix we are looking it needs to become part of the > devel, be migrated to beta and then be migrated to stable. In general, > one migration step takes between 4 and 12 weeks, depending on what was > done. So far, the average seems on the average, that is two month. So > two cycles mean roughly four month. HOWEVER, I intend to put it into the > next beta, which will bring us to just one cycle. So it would be > available in roughly two month. > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> Sent: Friday, July 04, 2008 12:32 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog threads questions >> >> Hi Rainer, >> >> Thanks!! I'll give this a try next week and let you know the results. >> If it works, I think it is okay not to include in the upcoming v3 >> stable since it is due to release very soon. But if it does work, it >> would be great to have it in the following stable release. >> >> Scott >> >> On Fri, Jul 4, 2008 at 1:31 AM, Rainer Gerhards >> wrote: >> > Scott, >> > >> > I have rewritten the omusrmsg plugin. This is currently part of the >> > development tree, but will become part of the (new) beta when we do > a >> > version number shift next week. I'd appreciate if you could give it > a >> > try. It is available here: >> > >> > http://download.rsyslog.com/download/rsyslog-3.19.9-pre1.tar.gz >> > >> > I am hesitant to put this in the upcoming new v3-stable. There were >> lots >> > of changes and one rule for a stable is that there should be no non- >> fix >> > type of changes for at least 4 weeks before it turns into a stable. >> It >> > of course is debatable if the change I made is a fix or not. In some >> > sense it is, but on the other hand it is a rewrite that changed a >> lot. >> > So given the fact that nobody on a "regular" machine saw an issue > the >> > past year, I would not like to bring the new code directly into the >> > stable. If that causes problems for you, you may want to try simply >> > using the omusrmsg.c code together with the v3-stable. I haven't >> tried >> > it, but it should work well. All changes were just to that file, and >> > there are no dependencies to anything external. >> > >> > OK... but now let's see first if it fixes the issue ;) >> > >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong >> >> Sent: Thursday, July 03, 2008 2:58 AM >> >> To: rsyslog at lists.adiscon.com >> >> Subject: Re: [rsyslog] rsyslog threads questions >> >> >> >> Hi, >> >> >> >> I've attached four files. Two of which are debug dumps, one is the >> >> conf file and the last one is a test case scenario that constantly >> >> fails on my end. I hope this gives a little more information. >> >> Furthermore, the dumps are from 3.17.5 which is the "closest" >> version >> >> to 3.18.0 that I was able to find. >> >> >> >> Both failed scenarios occur when lots of messages were being > flooded >> >> to rsyslogd at a very fast rate (look at logtest.c) The >> >> my_arm_rsyslog_suicide_debug.txt received a sigsegv fault while >> >> my_arm_rsyslog_sh_cannot_fork caused so many "Z [rsyslogd]" >> threads >> >> that it took up so much memory that executing any command as simple >> as >> >> 'ls -l' would not work from the command line. I think the number of >> >> threads grew as much as the number of messages. In the latter >> >> scenario, after killing logtest.c, it didn't look like the those >> >> zombies threads went away until I did a CTRL+C to the rsyslogd > which >> >> was running in the foreground since I use the "-dn" option. >> >> >> >> This is on an embedded system that runs significantly slower than a >> >> desktop or laptop so maybe it would be harder to reproduce on a >> >> regular computer. I looked at all the parameters that I believe >> could >> >> affect this and believe for the most part the defaults are more > than >> >> adequate. The main message queue never looked like it hit the high >> >> water mark but it did hit the lower one. So, I don't think messages >> >> were being dropped (not sure) or an overflow condition occurred. >> >> >> >> The processor is ARM-based and it is using Linux kernel 2.6.16.12 >> and >> >> compiled using GCC and the standard GNU C libraries version 3.4.5. >> >> Rsyslog source code is cross-compiled using the following configure >> >> line: >> >> >> >> ./configure --disable-zlib --disable-largefile >> >> --enable-share=yes >> >> --prefix=/ >> >> --host=arm-unknown-linux-gnu >> >> ac_cv_func_malloc_0_nonnull=yes >> >> ac_cv_func_realloc_0_nonnull=yes >> >> >> >> ac_cv_func_lstat_dereferences_slashed_symlink=yes >> >> ac_cv_func_stat_empty_string_bug=no >> >> enable_debug=no >> >> enable_rtinst=no >> >> >> >> Lastly, the logtest was executed with just the "-s" parameter. It > is >> a >> >> simple C file that I came up with. >> >> >> >> I took a look at the debug messages and it does not appear that new >> >> threads are created via calls to wtpStartWrkr in wtp.c. >> >> >> >> Any help I can bring to solve this issue, please let me know. I > hope >> I >> >> am not doing anything wrong here. >> >> >> >> Thanks, >> >> >> >> Scott >> >> > >> >> > On Wed, Jul 2, 2008 at 12:04 PM, Scott Phuong >> > >> >> wrote: >> >> >> Hi Rainer, >> >> >> >> >> >> Thanks for your reply. Looking at the default settings (from > the >> >> >> online help's configuration page), they are what I wanted. The >> main >> >> >> messages queue is set to fix sized array with 1 worker thread >> >> created >> >> >> at maximum and action queues are direct mode which according to >> the >> >> >> queue document page, means that there will not be a worker > thread >> >> >> created. Is my understanding correct? If yes, how do I quickly >> >> check >> >> >> without using the -d option if the defaults are set correctly? > Or >> >> what >> >> >> do I look for in the debug messages that gets printed out to >> ensure >> >> >> this? >> >> >> >> >> >> You also mentioned that version 3.18.0 is probably going to be >> >> >> released as the stable version next week. I see on the webpage >> > there >> >> >> is a 3.17.4 and 3.17.5. Are these two versions similiar to >> 3.18.0? >> >> >> >> >> >> Also, how come I did not get your reply in my email inbox? My >> >> account >> >> >> settings look correct. >> >> >> >> >> >> Thanks, >> >> >> >> >> >> Scott Phuong >> >> >> >> >> >> As for the syslog buffer size, that applies to syslogd and does >> not >> >> >> apply to rsyslog. >> >> >> >> >> >> >> >> >> >> >> >> My configuration files do not change the Action queue or Worker >> >> queue >> >> >> parameters at all. Looking at >> >> >> On Wed, 2008-07-02 at 01:15 -0700, Scott Phuong wrote: >> >> >>> Hi, >> >> >>> >> >> >>> I have 3.16.2 which was recently released. I see that under >> > certain >> >> >>> conditions rsyslogd spawns a lot of threads: >> >> >>> 5949 root 11216 S rsyslogd >> >> >>> 5950 root 11216 S rsyslogd >> >> >>> 5951 root 11216 S rsyslogd >> >> >>> 5952 root 11216 S rsyslogd >> >> >>> 5953 root 11216 S rsyslogd >> >> >>> 5954 root 11216 S rsyslogd >> >> >>> 5985 root Z [rsyslogd] >> >> >>> 6445 root Z [rsyslogd] >> >> >>> >> >> >>> I had to kill the rsyslogd and restart it. The first invocation >> > had >> >> a >> >> >>> pid of 219 before it had to be killed. The second invocation of >> > pid >> >> >>> which you see above starts with 5949. The difference is the >> amount >> >> of >> >> >>> zombie threads that were invoked by rsyslogd before I had to >> kill >> >> the >> >> >>> first invocation of it. >> >> >> >> >> >> I have no explanation yet for the zombies. They should not > happen >> >> and so >> >> >> far I have never seen them. We may need to go through a debug > log >> >> (which >> >> >> will become very large) to find out what's going on. >> >> >> >> >> >>> The question is under what conditions does rsyslogd spawn a new >> >> >>> thread/process and why was it a zombie? >> >> >> >> >> >> Unfortunately, there is no quick answer. A quick one may be: > when >> > it >> >> >> needs them, based on queue watermark settings and based on you >> >> >> configuration. But to really understand it, you need to read > this >> >> doc: >> >> >> >> >> >> http://www.rsyslog.com/doc-queues.html >> >> >> >> >> >> The doc also describes all the knobs that you can use to control >> >> thread >> >> >> creation. There are many ;) >> >> >> >> >> >>> I am running rsyslogd in an >> >> >>> embedded environment and not a regular laptop/desktop. >> >> >> >> >> >> Interesting use case... >> >> >> >> >> >>> In addition, I >> >> >>> am using busybox and I believe the syslog buffer size is set to >> >> >> >> >> >> what do yo mean by "syslog buffer size"? The length of a receive >> >> buffer? >> >> >> It is 2K, thus single messages up to 2K are supported. It can be >> >> changed >> >> >> by modifying the MAXLINE define. Note that stock syslogd (and >> >> RFC3164) >> >> >> support only up to 1K. >> >> >> >> >> >>> something very low or perhaps none at all. Would this be a >> factor? >> >> >>> Furthermore, I ran rsyslogd with -c3 and also without -c3 and >> both >> >> >>> cases happen. >> >> >> >> >> >> The compatibility modes do not affect queue operation. >> >> >> >> >> >>> Are these issues already known and fixed in a later version? >> > Sorry, >> >> if >> >> >>> I am asking the same questions or have the same issues as >> previous >> >> >>> people but without the ability to search (or at least, I don't >> > know >> >> >>> how to) the archive, I don't know if my problem/questions has >> >> already >> >> >>> been seen and/or resolved. >> >> >> >> >> >> If we need to find out about the zombies, we need to move on to >> the >> >> >> current devel version. So I would give that a try in any case. >> >> 3.16.2 >> >> >> will (most probably) be replaced by 3.18.0 (based on the current >> >> beta) >> >> >> next week. So I won't touch it any longer. >> >> >> >> >> >> Looking forward to your feedback, >> >> >> Rainer >> >> >> >> >> >>> >> >> >>> Thank you very much for your support. >> >> >>> >> >> >>> Scott >> >> >>> _______________________________________________ >> >> >>> rsyslog mailing list >> >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> >> >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Fri Jul 4 12:44:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 4 Jul 2008 12:44:24 +0200 Subject: [rsyslog] struct alignment problem - who can help? ;) In-Reply-To: <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA309336@grfint2.intern.adiscon.com><17945960807040328o4841e26dv37828465534ab49d@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30933B@grfint2.intern.adiscon.com> <17945960807040338u1ebc94cfh1a5754696cc2a2a@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30933D@grfint2.intern.adiscon.com> ROFL... you got it! I have forgotten to #include "config.h"! I knew that it must have been something simple. Thanks a lot, you saved me from pulling out the rest of my hair ;) Rainer PS: and, yes, it looks like I need to reevaluate mode-specific structure members now that we aim at a real runtime. Doesn't look so smart any longer... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Phuong > Sent: Friday, July 04, 2008 12:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > > Hi Rainer, > > But in release mode, NDEBUG is intentionally defined in the config.h. > This is what I recalled from looking at 3.16.1 and 3.17.5. So from the > perspective of rsyslog, NDEBUG is defined and the opposite for the > test-parse.c (because config.h is not included). > > Scott > > On Fri, Jul 4, 2008 at 3:32 AM, Rainer Gerhards > wrote: > > Hi Scott, > > > > thanks for the help, I'll further investigate. BUT: both the runtime > AND > > the tool are compiled in non-debug mode. So in this case, neither of > > them should have the object cookie, thus the alignment should be the > > same. But, as I said, I am grateful for any clue and will check if > there > > is some issue along these lines :) In any case, it seems to be > > inconsistent and should be fixed :) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Scott Phuong > >> Sent: Friday, July 04, 2008 12:29 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] struct alignment problem - who can help? ;) > >> > >> Here is my analysis: > >> In ctok_teken_t, there is a BEGINobjInstance. BEGINobjInstance is a > >> #define and is defined to obj_t objData which is the same for both > >> NDEBUG defined and not defined. But obj_t is a typedef of struct > >> obj_s. In the structure definition for obj_s in obj-types.h are the > >> following lines: > >> #ifndef NDEBUG > >> unsigned int iObjCooCKiE; > >> #endif > >> > >> NDEBUG is defined in config.h which was included in the runtime for > >> rsyslog but not for test-parse.c > >> > >> I didn't compile the code to test my theory. I just did a quick > check. > >> > >> Hope that solved your problem. > >> > >> Scott > >> > >> > >> I think this may be why the you are getting this structure > >> misalignment? > >> > >> > >> On Fri, Jul 4, 2008 at 2:24 AM, Rainer Gerhards > >> wrote: > >> > Hi list, > >> > > >> > I am posting a question here in the hope that some of the > > subscribers > >> > may be able to lend me a helping hand. > >> > > >> > Recently, I have begun to add a testbench to rsyslog. The idea is > >> that > >> > over time the project should have canned tests which are easy to > run > >> on > >> > each version (as part of make distcheck at latest), increasing the > >> > overall code quality. > >> > > >> > "All" (so far two ;)) tests are located in the tests subdirectory. > >> One > >> > test fails if compiled in release mode, that is rscript-parse.c. I > >> have > >> > tracked down the failure cause and it is different struct member > >> > alignment in the ctok_token_t structure. The alignment is > different > >> in > >> > the rsyslog runtime and this test tool (I have compared the > offsets > >> > computed for pToken->tok and they are different). So far, so good. > >> But I > >> > do not find the cause of this misalignment. I checked the make > >> output, > >> > and as it looks both the runtime as well as the tool are compiled > >> with > >> > the same compiler options. Also, in debug mode it works OK, but in > >> > release mode (--disable-debug) it fails. > >> > > >> > I am sure the problem is something quite simple, but I have run > out > >> of > >> > ideas of what it may be (or, more probable, I simply overlook > >> > something). > >> > > >> > Any help would be deeply appreciated. > >> > > >> > Thanks, > >> > Rainer > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From friedl at hq.adiscon.com Mon Jul 7 16:02:21 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 7 Jul 2008 16:02:21 +0200 Subject: [rsyslog] rsyslog 3.19.9 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED69@grfint2.intern.adiscon.com> Hi all, Rsyslog 3.19.9, a member of the development branch, has been released. Documentation on how to setup TLS syslog has been greatly improved. Also, omusrmsg has been rewritten to utilize rsyslog's threading instead of forking of children to send the messages. This is expected to also solve some issues with zombie processes. Finally, an issue with TLS anonymous mode, which still required client certificates, and a problem with the RainerScript parser, which did not detect every syntax error, was solved. This is a recommended update for all development branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-117.phtml Changelog: http://www.rsyslog.com/Article250.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rfujita at redhat.com Wed Jul 9 10:34:27 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 17:34:27 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. Message-ID: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> Hi List, $subject And I guess that documents are managed with Wiki. Do you have a plan to do i10n/i18n with rsyslog? Best Rio. ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 10:46:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 9 Jul 2008 10:46:34 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> Hi Rio-san, many thanks for your help! Much appreciated. As far as the documentation goes, there are actually two places. One is the wiki ( http://wiki.rsyslog.com ) , but then there is also a set (called "the core set") of html documentation. The reasoning is as follows: The wiki is a great resource and easy to edit. However, it does not provide the versioning we need. For example, there are changes in the way things can be configured between v2 and v3. In the wiki, version-specifc pages are a bit hard to find. Also, the wiki is only available online. To solve this, we have the core set: html files which document rsyslog's config directives as well as generic use cases. This is contained in git and under version control. An exactly matching version of the documentation is distributed with each source tarball. I think it would be very useful to have a Japanese version of (at least some) documents of the core set. If you are interested in that, I would add your translations to git and the distribution set. > Do you have a plan to do i10n/i18n with rsyslog? Could you elaborate a little bit on this? Unfortunately, I do neither speak Japanse nor any other language with a non-western script. I asked around several times and some folks from Japan told me that rsyslog works well with Japanese characters. But I am always interested in enhancing this support - I just need some guidance what is needed. Looking forward to your relpy, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Wednesday, July 09, 2008 10:34 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] I'd like to translate documents into Japanese. > > Hi List, > > $subject > And I guess that documents are managed with Wiki. > Do you have a plan to do i10n/i18n with rsyslog? > > Best Rio. > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Wed Jul 9 11:33:29 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 18:33:29 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> Message-ID: <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> Hi Rainer-san, Thank you for your quick response! Too quick, I was surprised. I understand the status of the project and what I should do. #The reason why I asked how the contributors collaborate for rsyslog was that I cloned the repository with git and I couldn't find any docbook or .po files. As soon as possible, I'll start to translate docs into Japanese. BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. But I can't find the source file of this version in the git tree and Wiki. Why? Best Rio. On 2008/07/09, at 17:46, Rainer Gerhards wrote: > Hi Rio-san, > > many thanks for your help! Much appreciated. > > As far as the documentation goes, there are actually two places. One > is > the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > (called "the core set") of html documentation. The reasoning is as > follows: The wiki is a great resource and easy to edit. However, it > does > not provide the versioning we need. For example, there are changes in > the way things can be configured between v2 and v3. In the wiki, > version-specifc pages are a bit hard to find. Also, the wiki is only > available online. > > To solve this, we have the core set: html files which document > rsyslog's > config directives as well as generic use cases. This is contained in > git > and under version control. An exactly matching version of the > documentation is distributed with each source tarball. > > I think it would be very useful to have a Japanese version of (at > least > some) documents of the core set. If you are interested in that, I > would > add your translations to git and the distribution set. > >> Do you have a plan to do i10n/i18n with rsyslog? > > Could you elaborate a little bit on this? Unfortunately, I do neither > speak Japanse nor any other language with a non-western script. I > asked > around several times and some folks from Japan told me that rsyslog > works well with Japanese characters. But I am always interested in > enhancing this support - I just need some guidance what is needed. > > Looking forward to your relpy, > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Wednesday, July 09, 2008 10:34 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] I'd like to translate documents into Japanese. >> >> Hi List, >> >> $subject >> And I guess that documents are managed with Wiki. >> Do you have a plan to do i10n/i18n with rsyslog? >> >> Best Rio. >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 11:43:17 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 11:43:17 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> Message-ID: <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: > Hi Rainer-san, > > Thank you for your quick response! > Too quick, I was surprised. I am trying my best - doesn't always work :) > > I understand the status of the project and what I should do. > > #The reason why I asked how the contributors collaborate for rsyslog > was that I cloned the repository with git and I couldn't find any > docbook or .po files. The doc set is in the ./doc subfolder, with .html files (obviously not what you looked for :)). I think it would be a good time now to create a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the files and I would integrate them as soon as I have them availalbe. > > As soon as possible, I'll start to translate docs into Japanese. > > BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. > But I can't find the source file of this version in the git tree and > Wiki. > Why? I restructured the directory structure a bit in 3.19. It is now in the ./tools directory, because it belongs to the rsyslogd tool. Here is a quick link: http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD All the best, Rainer > > Best Rio. > > On 2008/07/09, at 17:46, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > many thanks for your help! Much appreciated. > > > > As far as the documentation goes, there are actually two places. One > > is > > the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > > (called "the core set") of html documentation. The reasoning is as > > follows: The wiki is a great resource and easy to edit. However, it > > does > > not provide the versioning we need. For example, there are changes in > > the way things can be configured between v2 and v3. In the wiki, > > version-specifc pages are a bit hard to find. Also, the wiki is only > > available online. > > > > To solve this, we have the core set: html files which document > > rsyslog's > > config directives as well as generic use cases. This is contained in > > git > > and under version control. An exactly matching version of the > > documentation is distributed with each source tarball. > > > > I think it would be very useful to have a Japanese version of (at > > least > > some) documents of the core set. If you are interested in that, I > > would > > add your translations to git and the distribution set. > > > >> Do you have a plan to do i10n/i18n with rsyslog? > > > > Could you elaborate a little bit on this? Unfortunately, I do neither > > speak Japanse nor any other language with a non-western script. I > > asked > > around several times and some folks from Japan told me that rsyslog > > works well with Japanese characters. But I am always interested in > > enhancing this support - I just need some guidance what is needed. > > > > Looking forward to your relpy, > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Wednesday, July 09, 2008 10:34 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] I'd like to translate documents into Japanese. > >> > >> Hi List, > >> > >> $subject > >> And I guess that documents are managed with Wiki. > >> Do you have a plan to do i10n/i18n with rsyslog? > >> > >> Best Rio. > >> > >> > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Wed Jul 9 11:59:48 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Wed, 9 Jul 2008 18:59:48 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Message-ID: Hi Rainer-san, On 2008/07/09, at 18:43, Rainer Gerhards wrote: > I am trying my best - doesn't always work :) I'm working for Red Hat's Tokyo office. Do you live in Germany? If so, we can co-work in the daytime! > The doc set is in the ./doc subfolder, with .html files (obviously not > what you looked for :)). I think it would be a good time now to create > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > files and I would integrate them as soon as I have them availalbe. I agree with you totally. Because Fedora 9 has included rsyslog, lots Japanese and other non English/German speakers have started using it. Sooner or later, we'll need to create ./doc/other_lang directories :) > I restructured the directory structure a bit in 3.19. It is now in > the ./tools directory, because it belongs to the rsyslogd tool. > > Here is a quick link: > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD Oh, I find the file rsyslogd.8 in tools directory. I'll start translating it first. Best Rio. On 2008/07/09, at 18:43, Rainer Gerhards wrote: > Hi Rio-san, > > On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: >> Hi Rainer-san, >> >> Thank you for your quick response! >> Too quick, I was surprised. > > I am trying my best - doesn't always work :) > >> >> I understand the status of the project and what I should do. >> >> #The reason why I asked how the contributors collaborate for rsyslog >> was that I cloned the repository with git and I couldn't find any >> docbook or .po files. > > The doc set is in the ./doc subfolder, with .html files (obviously not > what you looked for :)). I think it would be a good time now to create > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > files and I would integrate them as soon as I have them availalbe. >> >> As soon as possible, I'll start to translate docs into Japanese. >> >> BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. >> But I can't find the source file of this version in the git tree and >> Wiki. >> Why? > > I restructured the directory structure a bit in 3.19. It is now in > the ./tools directory, because it belongs to the rsyslogd tool. > > Here is a quick link: > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > All the best, > Rainer >> >> Best Rio. >> >> On 2008/07/09, at 17:46, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> many thanks for your help! Much appreciated. >>> >>> As far as the documentation goes, there are actually two places. One >>> is >>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a set >>> (called "the core set") of html documentation. The reasoning is as >>> follows: The wiki is a great resource and easy to edit. However, it >>> does >>> not provide the versioning we need. For example, there are changes >>> in >>> the way things can be configured between v2 and v3. In the wiki, >>> version-specifc pages are a bit hard to find. Also, the wiki is only >>> available online. >>> >>> To solve this, we have the core set: html files which document >>> rsyslog's >>> config directives as well as generic use cases. This is contained in >>> git >>> and under version control. An exactly matching version of the >>> documentation is distributed with each source tarball. >>> >>> I think it would be very useful to have a Japanese version of (at >>> least >>> some) documents of the core set. If you are interested in that, I >>> would >>> add your translations to git and the distribution set. >>> >>>> Do you have a plan to do i10n/i18n with rsyslog? >>> >>> Could you elaborate a little bit on this? Unfortunately, I do >>> neither >>> speak Japanse nor any other language with a non-western script. I >>> asked >>> around several times and some folks from Japan told me that rsyslog >>> works well with Japanese characters. But I am always interested in >>> enhancing this support - I just need some guidance what is needed. >>> >>> Looking forward to your relpy, >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Wednesday, July 09, 2008 10:34 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] I'd like to translate documents into Japanese. >>>> >>>> Hi List, >>>> >>>> $subject >>>> And I guess that documents are managed with Wiki. >>>> Do you have a plan to do i10n/i18n with rsyslog? >>>> >>>> Best Rio. >>>> >>>> >>> ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> ######################################################################## >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> ######################################################################## >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 12:06:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 12:06:11 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> Message-ID: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Wed, 2008-07-09 at 18:59 +0900, Ryo Fujita wrote: > Hi Rainer-san, > > On 2008/07/09, at 18:43, Rainer Gerhards wrote: > > > I am trying my best - doesn't always work :) > > > I'm working for Red Hat's Tokyo office. > Do you live in Germany? > If so, we can co-work in the daytime! indeed, I do. So we can at least try. > > > The doc set is in the ./doc subfolder, with .html files (obviously not > > what you looked for :)). I think it would be a good time now to create > > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > > files and I would integrate them as soon as I have them availalbe. > > I agree with you totally. > Because Fedora 9 has included rsyslog, > lots Japanese and other non English/German speakers have started using > it. > Sooner or later, we'll need to create ./doc/other_lang directories :) I'll begin this when I start the next devel branch. I scheduled it for this week, but I got a bug report for the beta, which I would like to fix first. But in any case... soon! > > > I restructured the directory structure a bit in 3.19. It is now in > > the ./tools directory, because it belongs to the rsyslogd tool. > > > > Here is a quick link: > > > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > Oh, I find the file rsyslogd.8 in tools directory. > I'll start translating it first. Excellent. If you go through ./doc, you'll also notice that there is room for improvement. I think the structure is not very good today. Suggestions are welcome :) Rainer > > Best Rio. > > On 2008/07/09, at 18:43, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: > >> Hi Rainer-san, > >> > >> Thank you for your quick response! > >> Too quick, I was surprised. > > > > I am trying my best - doesn't always work :) > > > >> > >> I understand the status of the project and what I should do. > >> > >> #The reason why I asked how the contributors collaborate for rsyslog > >> was that I cloned the repository with git and I couldn't find any > >> docbook or .po files. > > > > The doc set is in the ./doc subfolder, with .html files (obviously not > > what you looked for :)). I think it would be a good time now to create > > a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over the > > files and I would integrate them as soon as I have them availalbe. > >> > >> As soon as possible, I'll start to translate docs into Japanese. > >> > >> BTW, Fedora 9 has the man page, rsyslogd(8) with the version 3.14.0. > >> But I can't find the source file of this version in the git tree and > >> Wiki. > >> Why? > > > > I restructured the directory structure a bit in 3.19. It is now in > > the ./tools directory, because it belongs to the rsyslogd tool. > > > > Here is a quick link: > > > > http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD > > > > > > All the best, > > Rainer > >> > >> Best Rio. > >> > >> On 2008/07/09, at 17:46, Rainer Gerhards wrote: > >> > >>> Hi Rio-san, > >>> > >>> many thanks for your help! Much appreciated. > >>> > >>> As far as the documentation goes, there are actually two places. One > >>> is > >>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a set > >>> (called "the core set") of html documentation. The reasoning is as > >>> follows: The wiki is a great resource and easy to edit. However, it > >>> does > >>> not provide the versioning we need. For example, there are changes > >>> in > >>> the way things can be configured between v2 and v3. In the wiki, > >>> version-specifc pages are a bit hard to find. Also, the wiki is only > >>> available online. > >>> > >>> To solve this, we have the core set: html files which document > >>> rsyslog's > >>> config directives as well as generic use cases. This is contained in > >>> git > >>> and under version control. An exactly matching version of the > >>> documentation is distributed with each source tarball. > >>> > >>> I think it would be very useful to have a Japanese version of (at > >>> least > >>> some) documents of the core set. If you are interested in that, I > >>> would > >>> add your translations to git and the distribution set. > >>> > >>>> Do you have a plan to do i10n/i18n with rsyslog? > >>> > >>> Could you elaborate a little bit on this? Unfortunately, I do > >>> neither > >>> speak Japanse nor any other language with a non-western script. I > >>> asked > >>> around several times and some folks from Japan told me that rsyslog > >>> works well with Japanese characters. But I am always interested in > >>> enhancing this support - I just need some guidance what is needed. > >>> > >>> Looking forward to your relpy, > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>>> Sent: Wednesday, July 09, 2008 10:34 AM > >>>> To: rsyslog at lists.adiscon.com > >>>> Subject: [rsyslog] I'd like to translate documents into Japanese. > >>>> > >>>> Hi List, > >>>> > >>>> $subject > >>>> And I guess that documents are managed with Wiki. > >>>> Do you have a plan to do i10n/i18n with rsyslog? > >>>> > >>>> Best Rio. > >>>> > >>>> > >>> ####################################################################### > >>>> # > >>>> Ryo Fujita > >>>> Senior Solution Architect, RHCE > >>>> Red Hat K.K. > >>>> TEL +81-3-5798-8500 > >>>> FAX +81-3-5798-8599 > >>>> Ebisu Neonato 8F > >>>> 4-1-18 Ebisu, Shibuya-ku, > >>>> Tokyo Japan 1500013 > >>>> > >>> ####################################################################### > >>>> # > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> ######################################################################## > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> ######################################################################## > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 9 12:32:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 09 Jul 2008 12:32:38 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Message-ID: <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> Hi Rio-san and others, one question comes up on my mind: > > Oh, I find the file rsyslogd.8 in tools directory. > > I'll start translating it first. The man file is inside the tools directory, but not at all language specific. Do you think it makes sense to create ./tools/mans// (with lan being the ISO 2-char language code) Or is there any other suggestion? Feedback appreciated, Rainer From rfujita at redhat.com Wed Jul 9 17:10:46 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 00:10:46 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> Message-ID: <4FD7940E-8E4F-41BB-AC31-578082706F55@redhat.com> Hi Rainer-san, > I'll begin this when I start the next devel branch. I scheduled it for > this week, but I got a bug report for the beta, which I would like to > fix first. But in any case... soon! It's a good timing. I need some time to investigate the best way to include translation stuff into the tree. > Excellent. If you go through ./doc, you'll also notice that there is > room for improvement. I think the structure is not very good today. > Suggestions are welcome :) My colleagues are the best to ask how to restructure the tree in order to translate docs. And one of my colleagues, Noriko Mizumoto-san is the editor of fedora- trans ja project. http://docs.fedoraproject.org/translation-quick-start-guide/ja_JP/index.html I'll ask her tomorrow. Best Rio. On 2008/07/09, at 19:06, Rainer Gerhards wrote: > Hi Rio-san, > > On Wed, 2008-07-09 at 18:59 +0900, Ryo Fujita wrote: >> Hi Rainer-san, >> >> On 2008/07/09, at 18:43, Rainer Gerhards wrote: >> >>> I am trying my best - doesn't always work :) >> >> >> I'm working for Red Hat's Tokyo office. >> Do you live in Germany? >> If so, we can co-work in the daytime! > > indeed, I do. So we can at least try. > >> >>> The doc set is in the ./doc subfolder, with .html files (obviously >>> not >>> what you looked for :)). I think it would be a good time now to >>> create >>> a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over >>> the >>> files and I would integrate them as soon as I have them availalbe. >> >> I agree with you totally. >> Because Fedora 9 has included rsyslog, >> lots Japanese and other non English/German speakers have started >> using >> it. >> Sooner or later, we'll need to create ./doc/other_lang directories :) > > I'll begin this when I start the next devel branch. I scheduled it for > this week, but I got a bug report for the beta, which I would like to > fix first. But in any case... soon! > >> >>> I restructured the directory structure a bit in 3.19. It is now in >>> the ./tools directory, because it belongs to the rsyslogd tool. >>> >>> Here is a quick link: >>> >>> http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD >> >> >> Oh, I find the file rsyslogd.8 in tools directory. >> I'll start translating it first. > > Excellent. If you go through ./doc, you'll also notice that there is > room for improvement. I think the structure is not very good today. > Suggestions are welcome :) > > Rainer >> >> Best Rio. >> >> On 2008/07/09, at 18:43, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> On Wed, 2008-07-09 at 18:33 +0900, Ryo Fujita wrote: >>>> Hi Rainer-san, >>>> >>>> Thank you for your quick response! >>>> Too quick, I was surprised. >>> >>> I am trying my best - doesn't always work :) >>> >>>> >>>> I understand the status of the project and what I should do. >>>> >>>> #The reason why I asked how the contributors collaborate for >>>> rsyslog >>>> was that I cloned the repository with git and I couldn't find any >>>> docbook or .po files. >>> >>> The doc set is in the ./doc subfolder, with .html files (obviously >>> not >>> what you looked for :)). I think it would be a good time now to >>> create >>> a ./doc/en and ./doc/jp. Do you agree? If so, you could copy over >>> the >>> files and I would integrate them as soon as I have them availalbe. >>>> >>>> As soon as possible, I'll start to translate docs into Japanese. >>>> >>>> BTW, Fedora 9 has the man page, rsyslogd(8) with the version >>>> 3.14.0. >>>> But I can't find the source file of this version in the git tree >>>> and >>>> Wiki. >>>> Why? >>> >>> I restructured the directory structure a bit in 3.19. It is now in >>> the ./tools directory, because it belongs to the rsyslogd tool. >>> >>> Here is a quick link: >>> >>> http://git.adiscon.com/?p=rsyslog.git;a=tree;f=tools;h=20be95b3073dc891603b489c68f0c4d0fc5b215b;hb=HEAD >>> >>> >>> All the best, >>> Rainer >>>> >>>> Best Rio. >>>> >>>> On 2008/07/09, at 17:46, Rainer Gerhards wrote: >>>> >>>>> Hi Rio-san, >>>>> >>>>> many thanks for your help! Much appreciated. >>>>> >>>>> As far as the documentation goes, there are actually two places. >>>>> One >>>>> is >>>>> the wiki ( http://wiki.rsyslog.com ) , but then there is also a >>>>> set >>>>> (called "the core set") of html documentation. The reasoning is as >>>>> follows: The wiki is a great resource and easy to edit. However, >>>>> it >>>>> does >>>>> not provide the versioning we need. For example, there are changes >>>>> in >>>>> the way things can be configured between v2 and v3. In the wiki, >>>>> version-specifc pages are a bit hard to find. Also, the wiki is >>>>> only >>>>> available online. >>>>> >>>>> To solve this, we have the core set: html files which document >>>>> rsyslog's >>>>> config directives as well as generic use cases. This is >>>>> contained in >>>>> git >>>>> and under version control. An exactly matching version of the >>>>> documentation is distributed with each source tarball. >>>>> >>>>> I think it would be very useful to have a Japanese version of (at >>>>> least >>>>> some) documents of the core set. If you are interested in that, I >>>>> would >>>>> add your translations to git and the distribution set. >>>>> >>>>>> Do you have a plan to do i10n/i18n with rsyslog? >>>>> >>>>> Could you elaborate a little bit on this? Unfortunately, I do >>>>> neither >>>>> speak Japanse nor any other language with a non-western script. I >>>>> asked >>>>> around several times and some folks from Japan told me that >>>>> rsyslog >>>>> works well with Japanese characters. But I am always interested in >>>>> enhancing this support - I just need some guidance what is needed. >>>>> >>>>> Looking forward to your relpy, >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>>> Sent: Wednesday, July 09, 2008 10:34 AM >>>>>> To: rsyslog at lists.adiscon.com >>>>>> Subject: [rsyslog] I'd like to translate documents into Japanese. >>>>>> >>>>>> Hi List, >>>>>> >>>>>> $subject >>>>>> And I guess that documents are managed with Wiki. >>>>>> Do you have a plan to do i10n/i18n with rsyslog? >>>>>> >>>>>> Best Rio. >>>>>> >>>>>> >>>>> ####################################################################### >>>>>> # >>>>>> Ryo Fujita >>>>>> Senior Solution Architect, RHCE >>>>>> Red Hat K.K. >>>>>> TEL +81-3-5798-8500 >>>>>> FAX +81-3-5798-8599 >>>>>> Ebisu Neonato 8F >>>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>>> Tokyo Japan 1500013 >>>>>> >>>>> ####################################################################### >>>>>> # >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> ######################################################################## >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> ######################################################################## >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> ######################################################################## >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> ######################################################################## >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rfujita at redhat.com Wed Jul 9 17:15:01 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 00:15:01 +0900 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com> <5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com> <1215596597.7184.29.camel@rgf9dev.intern.adiscon.com> <1215597971.7184.33.camel@rgf9dev.intern.adiscon.com> <1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> Message-ID: <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> Hi List, +1 But I'll ask it to my colleagues who contribute to translation matter tomorrow. I'm sure that they know the smartest way to do it. Best Rio. On 2008/07/09, at 19:32, Rainer Gerhards wrote: > Hi Rio-san and others, > > one question comes up on my mind: > >>> Oh, I find the file rsyslogd.8 in tools directory. >>> I'll start translating it first. > > The man file is inside the tools directory, but not at all language > specific. Do you think it makes sense to create > > ./tools/mans// (with lan being the ISO 2-char language code) > > Or is there any other suggestion? > > Feedback appreciated, > Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Wed Jul 9 17:16:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 9 Jul 2008 17:16:01 +0200 Subject: [rsyslog] I'd like to translate documents into Japanese. In-Reply-To: <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> References: <0FA4369F-E6CA-4183-842B-751A424A1A0D@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED84@grfint2.intern.adiscon.com><5582994A-7EDB-439F-ACCA-BA66996460F9@redhat.com><1215596597.7184.29.camel@rgf9dev.intern.adiscon.com><1215597971.7184.33.camel@rgf9dev.intern.adiscon.com><1215599558.7184.36.camel@rgf9dev.intern.adiscon.com> <60A7B816-0074-4A98-B5A6-2B68878ABC34@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED92@grfint2.intern.adiscon.com> Thanks, I am standing by for more votes and the insight! :) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Wednesday, July 09, 2008 5:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] I'd like to translate documents into Japanese. > > Hi List, > > +1 > > But I'll ask it to my colleagues who contribute to translation matter > tomorrow. > I'm sure that they know the smartest way to do it. > > Best Rio. > > On 2008/07/09, at 19:32, Rainer Gerhards wrote: > > > Hi Rio-san and others, > > > > one question comes up on my mind: > > > >>> Oh, I find the file rsyslogd.8 in tools directory. > >>> I'll start translating it first. > > > > The man file is inside the tools directory, but not at all language > > specific. Do you think it makes sense to create > > > > ./tools/mans// (with lan being the ISO 2-char language code) > > > > Or is there any other suggestion? > > > > Feedback appreciated, > > Rainer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Thu Jul 10 08:51:21 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 15:51:21 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure Message-ID: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> Hi List, I consulted with my colleagues about the smartest way to reconstruct the documents of rsyslog. 1.Work Flow matter The attached image is just an idea for rational flowchart to make translation easier. #If this ML forbids me to attach any files, you can see it on my site. http://rio.tc/2008/07/10-144007.php I'm worried that this flow will give the contributers much trouble. - HTML(from Wiki) to DocBook conversion 'tidy' will help us a little, but exporter of media Wiki is poor to export 'well-formed HTML'. We need to edit and check them manually. Of course, I willingly do this. But it may take a number of days. - Writers need to edit DocBook XML after the reconstruction. As you know, HTML format is not good for a source of multi-output. DocBook XML is a perfect one except for edit :) - Automake related matter There can be two streams to generate man and html from DocBook XML. One : When you edit DocBook, you commit DocBook, HTML and man files to git tree. Of course this method requires us to prepare an environment where you can use xsltproc/docbook2man. Two : By adding some sequences into Makefile, HTML and man files can be generated automatically. All persons who want to build rsyslog from tar ball need to prepare the environment. And we need to put some check routines for dependencies into Makefile. 2.git tree reconstruct only with 'doc' directory `-- doc |-- Makefile.am |-- conf | |-- en | | `-- rsyslog-example.conf | |-- OTHER_LANGUAGES(iso code) | `-- jp | `-- rsyslog-example.conf # annotations are translated. |-- html | |-- en | | |-- bugs.html | | |-- OTHER_HTMLS | | `-- version_naming.html | |-- OTHER_LANGUAGES(iso code) | `-- jp | |-- bugs.html | |-- OTHER_HTMLS | `-- version_naming.html |-- images | |-- gssapi.png | |-- OTHER_IMAGES | `-- tls_cert_ca.jpg |-- man | |-- en | | |-- man5 | | | `-- rsyslog.conf.5 | | `-- man8 | | `-- rsyslogd.8 | |-- OTHER_LANGUAGES(iso code) | `-- jp | |-- man5 | | `-- rsyslog.conf.5 | `-- man8 | `-- rsyslogd.8 `-- src |-- dias | |-- classes.dia | |-- OTHER_DIA_FILES | `-- tls_cert_ca.dia `-- docbook |-- en | |-- bugs.xml | |-- rsyslog.conf.5.xml | |-- rsyslogd.8.xml | |-- OTHER_XMLS | `-- version_naming.xml `-- ja |-- bugs.html |-- rsyslog.conf.5.xml |-- rsyslogd.8.xml |-- OTHER_XMLS `-- version_naming.html # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ directory. Your suggestions will be highly appreciated!! Best Rio. ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Thu Jul 10 09:03:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 10 Jul 2008 09:03:18 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> Hi Rio-san, I need to leave soon, but a quick feedback: the proposal looks *very* interesting. I need to digest it a bit. I have no experience with DocBook, so I probably need to check that out, first. That may take a short while (should you happen to know a good "getting started guide", I'd grateful if you send me a link ;)). From first thought, I would prefer to generate the html et al outputs from a single source (where only that source is in git). Feedback from other list members is also appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Thursday, July 10, 2008 8:51 AM > To: rsyslog-users > Subject: [rsyslog] Suggestion for Documentation Reconstructure > > Hi List, > > I consulted with my colleagues about the smartest way to reconstruct > the documents of rsyslog. > > 1.Work Flow matter > The attached image is just an idea for rational flowchart to make > translation easier. > #If this ML forbids me to attach any files, you can see it on my site. > http://rio.tc/2008/07/10-144007.php > > I'm worried that this flow will give the contributers much trouble. > > - HTML(from Wiki) to DocBook conversion > 'tidy' will help us a little, but exporter of media Wiki is poor to > export 'well-formed HTML'. > We need to edit and check them manually. > Of course, I willingly do this. But it may take a number of days. > > - Writers need to edit DocBook XML after the reconstruction. > As you know, HTML format is not good for a source of multi-output. > DocBook XML is a perfect one except for edit :) > > - Automake related matter > There can be two streams to generate man and html from DocBook XML. > One : When you edit DocBook, you commit DocBook, HTML and man files > to git tree. > Of course this method requires us to prepare an > environment where you can use xsltproc/docbook2man. > Two : By adding some sequences into Makefile, HTML and man files > can be generated automatically. > All persons who want to build rsyslog from tar ball need > to prepare the environment. > And we need to put some check routines for dependencies > into Makefile. > > 2.git tree reconstruct only with 'doc' directory > > `-- doc > |-- Makefile.am > |-- conf > | |-- en > | | `-- rsyslog-example.conf > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | `-- rsyslog-example.conf # annotations are translated. > |-- html > | |-- en > | | |-- bugs.html > | | |-- OTHER_HTMLS > | | `-- version_naming.html > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | |-- bugs.html > | |-- OTHER_HTMLS > | `-- version_naming.html > |-- images > | |-- gssapi.png > | |-- OTHER_IMAGES > | `-- tls_cert_ca.jpg > |-- man > | |-- en > | | |-- man5 > | | | `-- rsyslog.conf.5 > | | `-- man8 > | | `-- rsyslogd.8 > | |-- OTHER_LANGUAGES(iso code) > | `-- jp > | |-- man5 > | | `-- rsyslog.conf.5 > | `-- man8 > | `-- rsyslogd.8 > `-- src > |-- dias > | |-- classes.dia > | |-- OTHER_DIA_FILES > | `-- tls_cert_ca.dia > `-- docbook > |-- en > | |-- bugs.xml > | |-- rsyslog.conf.5.xml > | |-- rsyslogd.8.xml > | |-- OTHER_XMLS > | `-- version_naming.xml > `-- ja > |-- bugs.html > |-- rsyslog.conf.5.xml > |-- rsyslogd.8.xml > |-- OTHER_XMLS > `-- version_naming.html > > # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ > directory. > > Your suggestions will be highly appreciated!! > > Best Rio. > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # From rfujita at redhat.com Thu Jul 10 09:39:37 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 10 Jul 2008 16:39:37 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> Message-ID: <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Hi Rainer-san, DocBook.org is here. http://www.docbook.org/ And IBM has a good guide for beginners. http://www.ibm.com/developerworks/xml/library/xml-matters3.html Recently, Red Hat uses DocBook format to generate web pages ,PDFs and so on. For example, https://www.redhat.com/docs/manuals/enterprise/ A document having both html and PDF is generated from DocBook. gnome-doc-utils and kdesdk-utils include very convenient tools to handle DocBook and related formats. Best Rio. On 2008/07/10, at 16:03, Rainer Gerhards wrote: > Hi Rio-san, > > I need to leave soon, but a quick feedback: the proposal looks *very* > interesting. I need to digest it a bit. I have no experience with > DocBook, so I probably need to check that out, first. That may take a > short while (should you happen to know a good "getting started guide", > I'd grateful if you send me a link ;)). From first thought, I would > prefer to generate the html et al outputs from a single source (where > only that source is in git). > > Feedback from other list members is also appreciated. > > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Thursday, July 10, 2008 8:51 AM >> To: rsyslog-users >> Subject: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi List, >> >> I consulted with my colleagues about the smartest way to reconstruct >> the documents of rsyslog. >> >> 1.Work Flow matter >> The attached image is just an idea for rational flowchart to make >> translation easier. >> #If this ML forbids me to attach any files, you can see it on my >> site. >> http://rio.tc/2008/07/10-144007.php >> >> I'm worried that this flow will give the contributers much trouble. >> >> - HTML(from Wiki) to DocBook conversion >> 'tidy' will help us a little, but exporter of media Wiki is poor to >> export 'well-formed HTML'. >> We need to edit and check them manually. >> Of course, I willingly do this. But it may take a number of days. >> >> - Writers need to edit DocBook XML after the reconstruction. >> As you know, HTML format is not good for a source of multi-output. >> DocBook XML is a perfect one except for edit :) >> >> - Automake related matter >> There can be two streams to generate man and html from DocBook XML. >> One : When you edit DocBook, you commit DocBook, HTML and man files >> to git tree. >> Of course this method requires us to prepare an >> environment where you can use xsltproc/docbook2man. >> Two : By adding some sequences into Makefile, HTML and man files >> can be generated automatically. >> All persons who want to build rsyslog from tar ball need >> to prepare the environment. >> And we need to put some check routines for dependencies >> into Makefile. >> >> 2.git tree reconstruct only with 'doc' directory >> >> `-- doc >> |-- Makefile.am >> |-- conf >> | |-- en >> | | `-- rsyslog-example.conf >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | `-- rsyslog-example.conf # annotations are translated. >> |-- html >> | |-- en >> | | |-- bugs.html >> | | |-- OTHER_HTMLS >> | | `-- version_naming.html >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | |-- bugs.html >> | |-- OTHER_HTMLS >> | `-- version_naming.html >> |-- images >> | |-- gssapi.png >> | |-- OTHER_IMAGES >> | `-- tls_cert_ca.jpg >> |-- man >> | |-- en >> | | |-- man5 >> | | | `-- rsyslog.conf.5 >> | | `-- man8 >> | | `-- rsyslogd.8 >> | |-- OTHER_LANGUAGES(iso code) >> | `-- jp >> | |-- man5 >> | | `-- rsyslog.conf.5 >> | `-- man8 >> | `-- rsyslogd.8 >> `-- src >> |-- dias >> | |-- classes.dia >> | |-- OTHER_DIA_FILES >> | `-- tls_cert_ca.dia >> `-- docbook >> |-- en >> | |-- bugs.xml >> | |-- rsyslog.conf.5.xml >> | |-- rsyslogd.8.xml >> | |-- OTHER_XMLS >> | `-- version_naming.xml >> `-- ja >> |-- bugs.html >> |-- rsyslog.conf.5.xml >> |-- rsyslogd.8.xml >> |-- OTHER_XMLS >> `-- version_naming.html >> >> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ >> directory. >> >> Your suggestions will be highly appreciated!! >> >> Best Rio. >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 11 16:35:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:35:59 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. It includes all features and bug-fixes from the beta, among others the ability to natively send email. There is one minor bugfix for 3.17.5 included, where RainerScript did not always properly detect syntax errors. This is a recommended update for all v3-stable branch users. Change Log: http://www.rsyslog.com/Article254.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml Please note that I also created a 3.16.3 branch, immediately outdated, for those that prefer to keep the previous version for a couple of days. That version contains an important bugfix which prevents a big memory leak when running the queue in one of the disk modes. This patch is of course also included in 3.18.0. Find 3.16.3 here: http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml Just to make sure I am not misunderstood: 3.18.0 is the current v3-stable. Other v3-stables are not officially supported (except, of course, via a commercial support contract). 3.16.3 will not receive any patches in the future. So I suggest moving over to 3.18.0. It has been tested for over two month and looks quite ready for prime time. I would also like to express my sincere thanks to all of you who provided bug reports and patches. This is immensely helpful. I hope the release is useful. As always, feedback is appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Fri Jul 11 16:42:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:42:04 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> HI all, Michael Biebl just told me that some html files are missing in the 3.18.0 tarball. Please hold download, I am updating it right now... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, July 11, 2008 4:36 PM > To: rsyslog-users > Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > Hi all, > > we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. > It includes all features and bug-fixes from the beta, among others the > ability to natively send email. There is one minor bugfix for 3.17.5 > included, where RainerScript did not always properly detect syntax > errors. This is a recommended update for all v3-stable branch users. > > Change Log: > > http://www.rsyslog.com/Article254.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > Please note that I also created a 3.16.3 branch, immediately outdated, > for those that prefer to keep the previous version for a couple of > days. > That version contains an important bugfix which prevents a big memory > leak when running the queue in one of the disk modes. This patch is of > course also included in 3.18.0. Find 3.16.3 here: > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > Just to make sure I am not misunderstood: 3.18.0 is the current > v3-stable. Other v3-stables are not officially supported (except, of > course, via a commercial support contract). 3.16.3 will not receive any > patches in the future. So I suggest moving over to 3.18.0. It has been > tested for over two month and looks quite ready for prime time. > > I would also like to express my sincere thanks to all of you who > provided bug reports and patches. This is immensely helpful. > > I hope the release is useful. As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 11 16:57:27 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 11 Jul 2008 16:57:27 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EDBA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDBC@grfint2.intern.adiscon.com> ...and now this issue is fixed. The new tarball is online. If in doubt which one you loaded, please check the md5sum. The issue was that some documentation files were not properly included inside the tarball. The automatic checks did not detect that. The problem seems to have lurked for quite a while. Many thanks to Michael Biebl for pointing out the issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, July 11, 2008 4:42 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > HI all, > > Michael Biebl just told me that some html files are missing in the > 3.18.0 tarball. Please hold download, I am updating it right now... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Friday, July 11, 2008 4:36 PM > > To: rsyslog-users > > Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > > > Hi all, > > > > we have just released rsyslog 3.18.0, a new v3-stable version. > Rsyslog > > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta > branch. > > It includes all features and bug-fixes from the beta, among others > the > > ability to natively send email. There is one minor bugfix for 3.17.5 > > included, where RainerScript did not always properly detect syntax > > errors. This is a recommended update for all v3-stable branch users. > > > > Change Log: > > > > http://www.rsyslog.com/Article254.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > > > Please note that I also created a 3.16.3 branch, immediately > outdated, > > for those that prefer to keep the previous version for a couple of > > days. > > That version contains an important bugfix which prevents a big memory > > leak when running the queue in one of the disk modes. This patch is > of > > course also included in 3.18.0. Find 3.16.3 here: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > > > Just to make sure I am not misunderstood: 3.18.0 is the current > > v3-stable. Other v3-stables are not officially supported (except, of > > course, via a commercial support contract). 3.16.3 will not receive > any > > patches in the future. So I suggest moving over to 3.18.0. It has > been > > tested for over two month and looks quite ready for prime time. > > > > I would also like to express my sincere thanks to all of you who > > provided bug reports and patches. This is immensely helpful. > > > > I hope the release is useful. As always, feedback is appreciated. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Jul 14 12:06:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 14 Jul 2008 12:06:00 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Hi all, Michael Biebl noticed a small inconsistency in kernel log format: 3.16.x (and before): Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. 3.18.0: Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg started. Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. As you can see, there is a space after kernel: in the pre-3.18.0 messages. This also brings us down to a general inconsistency: Unfortunately, RFC 3164 defines the after TAG not as a delimiter but as part of the CONTENT of the message. This leads to some inconsistency in practice. Depending on who generated the message, there may be a space after the TAG or not. If it is, that space must be submitted as part of the message itself. In versions up to 3.16.x, kernel log messages were generated by old code, which did not know about tag and simply generated a non-structured message. With 3.17.x and above, the klog code is rewritten and correctly fills in all properties. This leads to the "missing" space, because the space is neither part of the tag nor part of the message. I have now three options: 1. leave as is (without a space) In that case, some log parsers may run into troubles 2. add a at the end of the TAG That will probably lead to even more confusion, as matches against TAG would need to include that space. 3. add a in front of each kernel message This comes close to the previous mode, but is "a bit" clumpsy and hackish. It will also make all database tables etc have messages starting with . Similar as 2, MSG matching rules are affected (but I consider this less severe, as matches inside the MSG are usually driven by searches and not absolute positions). I think that real solutions are numbers 1 and 3, with 3 probably having the best arguments. However, I have to admit I do not like this option very much at all. A fourth option would be to add two new property replacer argument "add at end if none is already there" and "drop first if there is one". I could then modify the default templates to use the first one with Tag and the later with MSG (instead of the regular ones). That would be a good fix, probably my favorite, BUT it would also cause format change, now in other messages. Plus, it is a little bit to code and I prefer to do as few code changes to a stable as possible. This issue has unfortunately been lurking ever since the beginning of rsyslog (as it is rooted in rfc 3164) and I should probably have it addressed earlier. But as you see: it is ugly... ;) So question now: what do the list members think? Feedback is highly appreciated. Thanks, Rainer From pascal at impressionet.ch Mon Jul 14 13:31:57 2008 From: pascal at impressionet.ch (Pascal Mainini) Date: Mon, 14 Jul 2008 13:31:57 +0200 Subject: [rsyslog] MARK frequency, too Message-ID: <487B392D.3090408@impressionet.ch> Hi all I'm refering to this mail: http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html I'm having the same issue as described in the Mail above, $MarkMessagePeriod gets completly ignored. I tried putting it at various places in the config-file, with tabs or whitespaces etc., without any result. I'm using the package from debian etch (on debian etch, of course). Find below my configfile (with hostnames anonymized). /etc/rsyslog.d is empty, so no further config should get included... Also, options for rsyslogd are "-c3 -s ". Also, I was wondering: I have a selector *.* which directs anything into a logfile - but not the mark-messages. Why is it that way? And are there maybe other messages which are ignored by a *.* selector? Help would be very much appreciated! Thanks a lot in advance, kind regards, Pascal ---- # /etc/rsyslog.conf Configuration file for rsyslog v3. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) $ModLoad immark # provides --MARK-- message capability $MarkMessagePeriod 300 # provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use default timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # # Set the default permissions for all log files. # $FileOwner root $FileGroup adm $FileCreateMode 0640 # # Include all config files in /etc/rsyslog.d/ # $IncludeConfig /etc/rsyslog.d/*.conf ############### #### RULES #### ############### +hostA *.* -/var/log/remote/hostA.log & -/var/log/full -hostA +hostB *.* -/var/log/remote/hostB.log & -/var/log/full -hostB +hostC *.* -/var/log/remote/hostC.log & -/var/log/full -hostC ####################### #### LOCAL LOGGING #### ####################### +localhostname *.* -/var/log/full # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #*.* -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # #daemon.*;mail.*;\ # news.err;\ # *.=debug;*.=info;\ # *.=notice;*.=warn |/dev/xconsole # From rgerhards at hq.adiscon.com Mon Jul 14 13:43:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 14 Jul 2008 13:43:42 +0200 Subject: [rsyslog] MARK frequency, too In-Reply-To: <487B392D.3090408@impressionet.ch> References: <487B392D.3090408@impressionet.ch> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> Hi, looks like I overlooked hte other post you mention. I'll have a look into the issue, but need to finish some other things first. I'd appreciate if you could file a bug report, so that it will not be forgotten. You can do that at http://bugzilla.adiscon.com Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Pascal Mainini > Sent: Monday, July 14, 2008 1:32 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] MARK frequency, too > > Hi all > > I'm refering to this mail: > > http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html > > I'm having the same issue as described in the Mail above, > $MarkMessagePeriod gets completly ignored. I tried putting it at > various places in the config-file, with tabs or whitespaces etc., > without any result. I'm using the package from debian etch (on > debian etch, of course). > > Find below my configfile (with hostnames anonymized). > /etc/rsyslog.d is empty, so no further config should get included... > Also, options for rsyslogd are "-c3 -s ". > > Also, I was wondering: I have a selector *.* which directs anything > into a logfile - but not the mark-messages. Why is it that way? > And are there maybe other messages which are ignored by a *.* selector? > > Help would be very much appreciated! > > Thanks a lot in advance, kind regards, > > Pascal > > ---- > > # /etc/rsyslog.conf Configuration file for rsyslog v3. > # > # For more information see > # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html > > > ################# > #### MODULES #### > ################# > > $ModLoad imuxsock # provides support for local system logging > $ModLoad imklog # provides kernel logging support (previously done by > rklogd) > > $ModLoad immark # provides --MARK-- message capability > $MarkMessagePeriod 300 > > # provides UDP syslog reception > $ModLoad imudp > $UDPServerRun 514 > > # provides TCP syslog reception > #$ModLoad imtcp > #$InputTCPServerRun 514 > > > ########################### > #### GLOBAL DIRECTIVES #### > ########################### > > # > # Use default timestamp format. > # To enable high precision timestamps, comment out the following line. > # > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # > # Set the default permissions for all log files. > # > $FileOwner root > $FileGroup adm > $FileCreateMode 0640 > > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/*.conf > > > ############### > #### RULES #### > ############### > > +hostA > *.* -/var/log/remote/hostA.log > & -/var/log/full > -hostA > > +hostB > *.* -/var/log/remote/hostB.log > & -/var/log/full > -hostB > > +hostC > *.* -/var/log/remote/hostC.log > & -/var/log/full > -hostC > > ####################### > #### LOCAL LOGGING #### > ####################### > > +localhostname > > *.* -/var/log/full > # > # First some standard log files. Log by facility. > # > auth,authpriv.* /var/log/auth.log > *.*;auth,authpriv.none -/var/log/syslog > #*.* -/var/log/syslog > #cron.* /var/log/cron.log > daemon.* -/var/log/daemon.log > kern.* -/var/log/kern.log > lpr.* -/var/log/lpr.log > mail.* -/var/log/mail.log > user.* -/var/log/user.log > > # > # Logging for the mail system. Split it up so that > # it is easy to write scripts to parse these files. > # > mail.info -/var/log/mail.info > mail.warn -/var/log/mail.warn > mail.err /var/log/mail.err > > # > # Logging for INN news system. > # > news.crit /var/log/news/news.crit > news.err /var/log/news/news.err > news.notice -/var/log/news/news.notice > > # > # Some "catch-all" log files. > # > *.=debug;\ > auth,authpriv.none;\ > news.none;mail.none -/var/log/debug > *.=info;*.=notice;*.=warn;\ > auth,authpriv.none;\ > cron,daemon.none;\ > mail,news.none -/var/log/messages > > # > # Emergencies are sent to everybody logged in. > # > *.emerg * > > # > # I like to have messages displayed on the console, but only on a > virtual > # console I usually leave idle. > # > #daemon,mail.*;\ > # news.=crit;news.=err;news.=notice;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn /dev/tty8 > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use > it, > # you must invoke `xconsole' with the `-file' option: > # > # $ xconsole -file /dev/xconsole [...] > # > # NOTE: adjust the list below, or you'll go crazy if you have a > reasonably > # busy site.. > # > #daemon.*;mail.*;\ > # news.err;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn |/dev/xconsole > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From aoz.syn at gmail.com Mon Jul 14 16:17:31 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 14 Jul 2008 08:17:31 -0600 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: <4255c2570807140717h1a118d86n6dccc31777a98111@mail.gmail.com> > So question now: what do the list members think? I think I may be partially re-stating #4 by saying this, but I think the most pragmatic approach would be to implement strict RFC compliance, then add non-default options to break that in one way or another for backwards compatibility. Make it explicit (in the option name and/or documentation) that by doing so they're breaking RFC 3164; you could even request a courtesy contact to you or the list so you could roughly track its use and thereby when you can drop it (if ever). Of course, that makes more code and more complexity; I'd also support leaving people with inflexible/broken parsers out in the cold, but I'm pretty heartless like that. From mbiebl at gmail.com Tue Jul 15 14:54:48 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Tue, 15 Jul 2008 14:54:48 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: 2008/7/14 Rainer Gerhards : > Hi all, > > Michael Biebl noticed a small inconsistency in kernel log format: > > 3.16.x (and before): > Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): > wlan0: link becomes ready > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > 3.18.0: > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > started. > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. another example: Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down See the <6> after the kernel: tag. It's also new. Is this some kind of log level, set by the kernel? I have never seen, that other syslogger (like syslog-ng, sysklogd set this). > > As you can see, there is a space after kernel: in the pre-3.18.0 > messages. This also brings us down to a general inconsistency: > > Unfortunately, RFC 3164 defines the after TAG not as a delimiter > but as part of the CONTENT of the message. This leads to some > inconsistency in practice. Depending on who generated the message, there > may be a space after the TAG or not. If it is, that space must be > submitted as part of the message itself. > > In versions up to 3.16.x, kernel log messages were generated by old > code, which did not know about tag and simply generated a non-structured > message. With 3.17.x and above, the klog code is rewritten and correctly > fills in all properties. This leads to the "missing" space, because the > space is neither part of the tag nor part of the message. > > I have now three options: > > 1. leave as is (without a space) > In that case, some log parsers may run into troubles This was a rather vague concern of mine. I don't actually know, if other software is affected by this change. > 2. add a at the end of the TAG > That will probably lead to even more confusion, as matches against TAG > would need to include that space. > > 3. add a in front of each kernel message > This comes close to the previous mode, but is "a bit" clumpsy and > hackish. It will also make all database tables etc have messages > starting with . Similar as 2, MSG matching rules are affected (but I > consider this less severe, as matches inside the MSG are usually driven > by searches and not absolute positions). Just curious: For other log messages, that are not generated by the kernel, you have a space after the programname: tag, e.g. Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 Is the space after dhclient: part of the message or part of the :programname tag? Imho, if there is no clear definition in the RFC, how the message should be written, it's best to stay backwards compatible. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Tue Jul 15 15:03:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 15:03:42 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2 and 3.18.0 In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> Message-ID: <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > 2008/7/14 Rainer Gerhards : > > Hi all, > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > 3.16.x (and before): > > Jul 11 17:33:28 pluto kernel: [14177.432349] ADDRCONF(NETDEV_CHANGE): > > wlan0: link becomes ready > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > 3.18.0: > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > > started. > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > another example: > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > See the <6> after the kernel: tag. It's also new. > Is this some kind of log level, set by the kernel? This does not look correct. Under FreeBSD, it is typical to have a PRI inside kernel messages. But for linux kernels I tested with, I did not see it and (I think) so I did not support it. But if you see it on Debian, it seems to happen. A cure is to use the same logic that is used for BSD. > I have never seen, that other syslogger (like syslog-ng, sysklogd set this). > > > > > As you can see, there is a space after kernel: in the pre-3.18.0 > > messages. This also brings us down to a general inconsistency: > > > > Unfortunately, RFC 3164 defines the after TAG not as a delimiter > > but as part of the CONTENT of the message. This leads to some > > inconsistency in practice. Depending on who generated the message, there > > may be a space after the TAG or not. If it is, that space must be > > submitted as part of the message itself. > > > > In versions up to 3.16.x, kernel log messages were generated by old > > code, which did not know about tag and simply generated a non-structured > > message. With 3.17.x and above, the klog code is rewritten and correctly > > fills in all properties. This leads to the "missing" space, because the > > space is neither part of the tag nor part of the message. > > > > I have now three options: > > > > 1. leave as is (without a space) > > In that case, some log parsers may run into troubles > > This was a rather vague concern of mine. I don't actually know, if > other software is affected by this change. I share this concern, but also have no additional information. I wanted to point out the options. I have to admit I am still undecided. But that nobody violently objected so far makes this look like it is not so important... > > > 2. add a at the end of the TAG > > That will probably lead to even more confusion, as matches against TAG > > would need to include that space. > > > > 3. add a in front of each kernel message > > This comes close to the previous mode, but is "a bit" clumpsy and > > hackish. It will also make all database tables etc have messages > > starting with . Similar as 2, MSG matching rules are affected (but I > > consider this less severe, as matches inside the MSG are usually driven > > by searches and not absolute positions). > > Just curious: For other log messages, that are not generated by the > kernel, you have a space after the programname: tag, e.g. > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > Is the space after dhclient: part of the message or part of the > :programname tag? In our private communication, I thought it is part of TAG. But that is wrong. It actually is part of MSG - at least for legacy/RFC3164 syslog. For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is well-defined and part of neither. There, it is a delimiter (as one would expect). I you use rsyslog only with syslog-protocol messages (and the syslog-protocol templates), this is more or less no issue. But who does today...? ;) The real problem is that legacy syslog has many different interpretations and we break one or the other in either way... > > Imho, if there is no clear definition in the RFC, how the message > should be written, it's best to stay backwards compatible. Is that a vote for option 3, stick a space in front of each kernel message? ;) > > > Cheers, > Michael > > From rgerhards at hq.adiscon.com Tue Jul 15 16:10:20 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:10:20 +0200 Subject: [rsyslog] slight kernel log format inconsistency between 3.16.2and 3.18.0 In-Reply-To: <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com> <1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> Michael, I have reviewed some of the code in question. It looks like a bug. It's actually not a klog driver issue, the PRI parsing is done at an upper klog layer and thus should work for linux as well. I would like to generate a version with some additional instrumentation. Could you run it and report the results back (maybe via private mail to keep the list free of noise). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 15, 2008 3:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] slight kernel log format inconsistency between > 3.16.2and 3.18.0 > > On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > > 2008/7/14 Rainer Gerhards : > > > Hi all, > > > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > > > 3.16.x (and before): > > > Jul 11 17:33:28 pluto kernel: [14177.432349] > ADDRCONF(NETDEV_CHANGE): > > > wlan0: link becomes ready > > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > > > > 3.18.0: > > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = /proc/kmsg > > > started. > > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > > > another example: > > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > > See the <6> after the kernel: tag. It's also new. > > Is this some kind of log level, set by the kernel? > > This does not look correct. Under FreeBSD, it is typical to have a PRI > inside kernel messages. But for linux kernels I tested with, I did not > see it and (I think) so I did not support it. But if you see it on > Debian, it seems to happen. A cure is to use the same logic that is > used > for BSD. > > > I have never seen, that other syslogger (like syslog-ng, sysklogd set > this). > > > > > > > > As you can see, there is a space after kernel: in the pre- > 3.18.0 > > > messages. This also brings us down to a general inconsistency: > > > > > > Unfortunately, RFC 3164 defines the after TAG not as a > delimiter > > > but as part of the CONTENT of the message. This leads to some > > > inconsistency in practice. Depending on who generated the message, > there > > > may be a space after the TAG or not. If it is, that space must be > > > submitted as part of the message itself. > > > > > > In versions up to 3.16.x, kernel log messages were generated by old > > > code, which did not know about tag and simply generated a non- > structured > > > message. With 3.17.x and above, the klog code is rewritten and > correctly > > > fills in all properties. This leads to the "missing" space, because > the > > > space is neither part of the tag nor part of the message. > > > > > > I have now three options: > > > > > > 1. leave as is (without a space) > > > In that case, some log parsers may run into troubles > > > > This was a rather vague concern of mine. I don't actually know, if > > other software is affected by this change. > > I share this concern, but also have no additional information. I wanted > to point out the options. I have to admit I am still undecided. But > that > nobody violently objected so far makes this look like it is not so > important... > > > > > > 2. add a at the end of the TAG > > > That will probably lead to even more confusion, as matches against > TAG > > > would need to include that space. > > > > > > 3. add a in front of each kernel message > > > This comes close to the previous mode, but is "a bit" clumpsy and > > > hackish. It will also make all database tables etc have messages > > > starting with . Similar as 2, MSG matching rules are affected > (but I > > > consider this less severe, as matches inside the MSG are usually > driven > > > by searches and not absolute positions). > > > > Just curious: For other log messages, that are not generated by the > > kernel, you have a space after the programname: tag, e.g. > > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > > > Is the space after dhclient: part of the message or part of the > > :programname tag? > > In our private communication, I thought it is part of TAG. But that is > wrong. It actually is part of MSG - at least for legacy/RFC3164 syslog. > > For the new IETF syslog RFC series (syslog-protocol, -tls et al) it is > well-defined and part of neither. There, it is a delimiter (as one > would > expect). I you use rsyslog only with syslog-protocol messages (and the > syslog-protocol templates), this is more or less no issue. But who does > today...? ;) > > The real problem is that legacy syslog has many different > interpretations and we break one or the other in either way... > > > > > Imho, if there is no clear definition in the RFC, how the message > > should be written, it's best to stay backwards compatible. > > Is that a vote for option 3, stick a space in front of each kernel > message? ;) > > > > > > > Cheers, > > Michael > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 15 16:22:07 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:22:07 +0200 Subject: [rsyslog] MARK frequency, too In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> References: <487B392D.3090408@impressionet.ch> <577465F99B41C842AAFBE9ED71E70ABA44EDCA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDC@grfint2.intern.adiscon.com> FYI: I have found and fixed this bug: http://bugzilla.adiscon.com/process_bug.cgi Thanks for reporting :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, July 14, 2008 1:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] MARK frequency, too > > Hi, > > looks like I overlooked hte other post you mention. I'll have a look > into the issue, but need to finish some other things first. I'd > appreciate if you could file a bug report, so that it will not be > forgotten. You can do that at > > http://bugzilla.adiscon.com > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Pascal Mainini > > Sent: Monday, July 14, 2008 1:32 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] MARK frequency, too > > > > Hi all > > > > I'm refering to this mail: > > > > http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg00673.html > > > > I'm having the same issue as described in the Mail above, > > $MarkMessagePeriod gets completly ignored. I tried putting it at > > various places in the config-file, with tabs or whitespaces etc., > > without any result. I'm using the package from debian etch (on > > debian etch, of course). > > > > Find below my configfile (with hostnames anonymized). > > /etc/rsyslog.d is empty, so no further config should get included... > > Also, options for rsyslogd are "-c3 -s ". > > > > Also, I was wondering: I have a selector *.* which directs anything > > into a logfile - but not the mark-messages. Why is it that way? > > And are there maybe other messages which are ignored by a *.* > selector? > > > > Help would be very much appreciated! > > > > Thanks a lot in advance, kind regards, > > > > Pascal > > > > ---- > > > > # /etc/rsyslog.conf Configuration file for rsyslog v3. > > # > > # For more information see > > # > /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html > > > > > > ################# > > #### MODULES #### > > ################# > > > > $ModLoad imuxsock # provides support for local system logging > > $ModLoad imklog # provides kernel logging support (previously done > by > > rklogd) > > > > $ModLoad immark # provides --MARK-- message capability > > $MarkMessagePeriod 300 > > > > # provides UDP syslog reception > > $ModLoad imudp > > $UDPServerRun 514 > > > > # provides TCP syslog reception > > #$ModLoad imtcp > > #$InputTCPServerRun 514 > > > > > > ########################### > > #### GLOBAL DIRECTIVES #### > > ########################### > > > > # > > # Use default timestamp format. > > # To enable high precision timestamps, comment out the following > line. > > # > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > > > # > > # Set the default permissions for all log files. > > # > > $FileOwner root > > $FileGroup adm > > $FileCreateMode 0640 > > > > # > > # Include all config files in /etc/rsyslog.d/ > > # > > $IncludeConfig /etc/rsyslog.d/*.conf > > > > > > ############### > > #### RULES #### > > ############### > > > > +hostA > > *.* -/var/log/remote/hostA.log > > & -/var/log/full > > -hostA > > > > +hostB > > *.* -/var/log/remote/hostB.log > > & -/var/log/full > > -hostB > > > > +hostC > > *.* -/var/log/remote/hostC.log > > & -/var/log/full > > -hostC > > > > ####################### > > #### LOCAL LOGGING #### > > ####################### > > > > +localhostname > > > > *.* -/var/log/full > > # > > # First some standard log files. Log by facility. > > # > > auth,authpriv.* /var/log/auth.log > > *.*;auth,authpriv.none -/var/log/syslog > > #*.* -/var/log/syslog > > #cron.* /var/log/cron.log > > daemon.* -/var/log/daemon.log > > kern.* -/var/log/kern.log > > lpr.* -/var/log/lpr.log > > mail.* -/var/log/mail.log > > user.* -/var/log/user.log > > > > # > > # Logging for the mail system. Split it up so that > > # it is easy to write scripts to parse these files. > > # > > mail.info -/var/log/mail.info > > mail.warn -/var/log/mail.warn > > mail.err /var/log/mail.err > > > > # > > # Logging for INN news system. > > # > > news.crit /var/log/news/news.crit > > news.err /var/log/news/news.err > > news.notice -/var/log/news/news.notice > > > > # > > # Some "catch-all" log files. > > # > > *.=debug;\ > > auth,authpriv.none;\ > > news.none;mail.none -/var/log/debug > > *.=info;*.=notice;*.=warn;\ > > auth,authpriv.none;\ > > cron,daemon.none;\ > > mail,news.none -/var/log/messages > > > > # > > # Emergencies are sent to everybody logged in. > > # > > *.emerg * > > > > # > > # I like to have messages displayed on the console, but only on a > > virtual > > # console I usually leave idle. > > # > > #daemon,mail.*;\ > > # news.=crit;news.=err;news.=notice;\ > > # *.=debug;*.=info;\ > > # *.=notice;*.=warn /dev/tty8 > > > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use > > it, > > # you must invoke `xconsole' with the `-file' option: > > # > > # $ xconsole -file /dev/xconsole [...] > > # > > # NOTE: adjust the list below, or you'll go crazy if you have a > > reasonably > > # busy site.. > > # > > #daemon.*;mail.*;\ > > # news.err;\ > > # *.=debug;*.=info;\ > > # *.=notice;*.=warn |/dev/xconsole > > # > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 15 16:38:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 15 Jul 2008 16:38:46 +0200 Subject: [rsyslog] rsyslog 3.19.10 (beta) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDDF@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.10. This starts a new iteration of the beta branch, bringing all the great features of the previous development version to it. It contains all features from 3.19.9 and also a number of bug fixes. The most important one is for a bad memory leak that occurred if queues were running in one of the disk-queuing modes. Also, a number of cross-platform problems on FreeBSD were sorted out. This is a suggested update for all beta branch users. Just in case you wonder: at this time there is NO development branch version available. One will be announced in the not so distant future. ChangeLog: http://www.rsyslog.com/Article256.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-120.phtml As always, feedback is appreciated. I hope this release is useful. Rainer Gerhards From hks.private at gmail.com Tue Jul 15 20:14:39 2008 From: hks.private at gmail.com ((private) HKS) Date: Tue, 15 Jul 2008 14:14:39 -0400 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: FYI, the links at the rsyslog status page are a bit funky: http://www.rsyslog.com/doc-status.html Beta 3.19.10's download link takes you to a page describing 3.17.3 beta with a download link for 3.19.10. v3 stable 3.18.0's download link takes you to a page describing 3.19.10 beta with a download link for the same. v2 stable 2.0.5's download link takes you to a page mostly describing 2.0.5 but with the rsyslog version misreported (and perhaps md5 checksum?). -HKS On Fri, Jul 11, 2008 at 10:35 AM, Rainer Gerhards wrote: > Hi all, > > we have just released rsyslog 3.18.0, a new v3-stable version. Rsyslog > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta branch. > It includes all features and bug-fixes from the beta, among others the > ability to natively send email. There is one minor bugfix for 3.17.5 > included, where RainerScript did not always properly detect syntax > errors. This is a recommended update for all v3-stable branch users. > > Change Log: > > http://www.rsyslog.com/Article254.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > Please note that I also created a 3.16.3 branch, immediately outdated, > for those that prefer to keep the previous version for a couple of days. > That version contains an important bugfix which prevents a big memory > leak when running the queue in one of the disk modes. This patch is of > course also included in 3.18.0. Find 3.16.3 here: > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > Just to make sure I am not misunderstood: 3.18.0 is the current > v3-stable. Other v3-stables are not officially supported (except, of > course, via a commercial support contract). 3.16.3 will not receive any > patches in the future. So I suggest moving over to 3.18.0. It has been > tested for over two month and looks quite ready for prime time. > > I would also like to express my sincere thanks to all of you who > provided bug reports and patches. This is immensely helpful. > > I hope the release is useful. As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Wed Jul 16 08:30:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 16 Jul 2008 08:30:56 +0200 Subject: [rsyslog] rsyslog 3.18.0 (v3-stable) released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA44EDB9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EDE3@grfint2.intern.adiscon.com> Hi HKS, Thanks, looks like I had a bad time with the links... Corrected now. I have also checked 2.0.5, the version was incorrect (copy&paste error), but the md5sum is correct. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Tuesday, July 15, 2008 8:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.18.0 (v3-stable) released > > FYI, the links at the rsyslog status page are a bit funky: > http://www.rsyslog.com/doc-status.html > > Beta 3.19.10's download link takes you to a page describing 3.17.3 > beta with a download link for 3.19.10. > > v3 stable 3.18.0's download link takes you to a page describing > 3.19.10 beta with a download link for the same. > > v2 stable 2.0.5's download link takes you to a page mostly describing > 2.0.5 but with the rsyslog version misreported (and perhaps md5 > checksum?). > > -HKS > > On Fri, Jul 11, 2008 at 10:35 AM, Rainer Gerhards > wrote: > > Hi all, > > > > we have just released rsyslog 3.18.0, a new v3-stable version. > Rsyslog > > 3.18.0 begins a new v3-stable based on the previous 3.17.5 beta > branch. > > It includes all features and bug-fixes from the beta, among others > the > > ability to natively send email. There is one minor bugfix for 3.17.5 > > included, where RainerScript did not always properly detect syntax > > errors. This is a recommended update for all v3-stable branch users. > > > > Change Log: > > > > http://www.rsyslog.com/Article254.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-119.phtml > > > > Please note that I also created a 3.16.3 branch, immediately > outdated, > > for those that prefer to keep the previous version for a couple of > days. > > That version contains an important bugfix which prevents a big memory > > leak when running the queue in one of the disk modes. This patch is > of > > course also included in 3.18.0. Find 3.16.3 here: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-118.phtml > > > > Just to make sure I am not misunderstood: 3.18.0 is the current > > v3-stable. Other v3-stables are not officially supported (except, of > > course, via a commercial support contract). 3.16.3 will not receive > any > > patches in the future. So I suggest moving over to 3.18.0. It has > been > > tested for over two month and looks quite ready for prime time. > > > > I would also like to express my sincere thanks to all of you who > > provided bug reports and patches. This is immensely helpful. > > > > I hope the release is useful. As always, feedback is appreciated. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 17 12:43:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 12:43:54 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Hi all, I just got an idea for a feature inspired by the ramlog project. I have blogged about it: http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h tml If you have some minutes to spare, please review and provide feedback. Thanks, Rainer From mic at npgx.com.au Thu Jul 17 13:09:27 2008 From: mic at npgx.com.au (Michael Mansour) Date: Thu, 17 Jul 2008 22:09:27 +1100 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <20080717110306.M99550@npgx.com.au> Hi Rainer, > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. I've read and honestly have no opinion on it :) The last time I tried delayed writes with rsyslog the machines I set them up on basically died (load averages above 30, so they were complete unresponsive). I log rsyslog to a MySQL DB (for maillogs only) and it took me a while to realise it was the rsyslog changes I made that killed the mail servers. They even hung on boot ie. would not complete a boot process. When I got into single user mode and didn't start up any services, I was able to change the rsyslog.conf file back to my older one, and everything was fine again. This was some versions back (but still the 3.x stable series) but because these were production servers, I didn't bother analysing what went wrong or why it happened, I was just glad I had my mail servers back again and working. I might give this a second attempt at some stage this year, because I like the feature for queuing entries if the DB is down, but until then, I really don't have much to say about the delayed writes stuff :) Michael. > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From rgerhards at hq.adiscon.com Thu Jul 17 13:45:49 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 13:45:49 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <20080717110306.M99550@npgx.com.au> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <20080717110306.M99550@npgx.com.au> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE12@grfint2.intern.adiscon.com> Hi Michael, I think you were hit by this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=86 interestingly, nobody who experienced it bothered to report, until recently. Thus it was for quite a while inside the core engine ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Mansour > Sent: Thursday, July 17, 2008 1:09 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback request: in-memory buffering messages > > Hi Rainer, > > > Hi all, > > > > I just got an idea for a feature inspired by the ramlog project. I > have > > blogged about it: > > > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram- > buffer.h > > tml > > > > If you have some minutes to spare, please review and provide > feedback. > > I've read and honestly have no opinion on it :) > > The last time I tried delayed writes with rsyslog the machines I set > them up > on basically died (load averages above 30, so they were complete > unresponsive). > > I log rsyslog to a MySQL DB (for maillogs only) and it took me a while > to > realise it was the rsyslog changes I made that killed the mail servers. > They > even hung on boot ie. would not complete a boot process. When I got > into > single user mode and didn't start up any services, I was able to change > the > rsyslog.conf file back to my older one, and everything was fine again. > > This was some versions back (but still the 3.x stable series) but > because > these were production servers, I didn't bother analysing what went > wrong or > why it happened, I was just glad I had my mail servers back again and > working. > > I might give this a second attempt at some stage this year, because I > like the > feature for queuing entries if the DB is down, but until then, I really > don't > have much to say about the delayed writes stuff :) > > Michael. > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > ------- End of Original Message ------- > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Thu Jul 17 16:13:10 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 17 Jul 2008 23:13:10 +0900 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <5646AC09-7597-4549-8193-BD6C187DEA7E@redhat.com> Hi Rainer-san, +1 How to use logging facilities varies according to circumstances. For example, RHEL is used for from edge to mission critical. Our customers want robust logging functions, like writing to MySQL or transporting via TCP/RELP. But some users want 'greener' machines. It sounds interesting to combine the feature to write log messages to RAM and the powertop project. http://www.lesswatts.org/projects/powertop/ Best Rio. On 2008/07/17, at 19:43, Rainer Gerhards wrote: > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I > have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From hks.private at gmail.com Thu Jul 17 16:19:11 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 17 Jul 2008 10:19:11 -0400 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: This is certainly something I would use. I've got a number of mission critical devices running off of flash drives (OpenBSD firewalls), and the less they have to write to disk the better. This would also be handy on a high-use database machine, where excessive disk writes could have a significant impact on service delivery. I doubt it would see very widely used, but it would make rsyslog appealing to another niche. -HKS On Thu, Jul 17, 2008 at 6:43 AM, Rainer Gerhards wrote: > Hi all, > > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml > > If you have some minutes to spare, please review and provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From aoz.syn at gmail.com Thu Jul 17 16:20:07 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 08:20:07 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> Message-ID: <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> > I just got an idea for a feature inspired by the ramlog project. I have > blogged about it: > > http://blog.gerhards.net/2008/07/writing-syslog-messages-to-ram-buffer.h > tml Prior art exists and I think it's well worth the effort, particularly for busy servers on reliable power. If I read your post right, you're considering something rather similar to the syslog-ng options 'flush_lines' and 'flush_timeout'? From rgerhards at hq.adiscon.com Thu Jul 17 16:22:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 16:22:43 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> > Prior art exists and I think it's well worth the effort, particularly > for busy servers on reliable power. If I read your post right, you're > considering something rather similar to the syslog-ng options > 'flush_lines' and 'flush_timeout'? I actually (really ;)) do not know syslog-ng and keep myself somewhat away from it to prevent accidental steal of "whatever" from there. But now that you raise the point: could you quickly describe how it works. >From your mail, it sounds like what I am thinking about... Rainer From mbiebl at gmail.com Thu Jul 17 16:32:08 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Thu, 17 Jul 2008 16:32:08 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Message-ID: Hi, I just wanted to say, that I like the idea of using docbook/xml to generate the (html) documentation (and possibly the man pages). Imo this would make the documentation more consistent (consistent font sizes, headers, footers etc.), more usable (You'd get something like a TOC and possibly better cross-linking) and perhaps easier keeping up-to-date. Cheers, Michael 2008/7/10 Ryo Fujita : > Hi Rainer-san, > > DocBook.org is here. > http://www.docbook.org/ > > And IBM has a good guide for beginners. > http://www.ibm.com/developerworks/xml/library/xml-matters3.html > > Recently, Red Hat uses DocBook format to generate web pages ,PDFs and > so on. > For example, > https://www.redhat.com/docs/manuals/enterprise/ > A document having both html and PDF is generated from DocBook. > > gnome-doc-utils and kdesdk-utils include very convenient tools to > handle DocBook and related formats. > > Best Rio. > > On 2008/07/10, at 16:03, Rainer Gerhards wrote: > >> Hi Rio-san, >> >> I need to leave soon, but a quick feedback: the proposal looks *very* >> interesting. I need to digest it a bit. I have no experience with >> DocBook, so I probably need to check that out, first. That may take a >> short while (should you happen to know a good "getting started guide", >> I'd grateful if you send me a link ;)). From first thought, I would >> prefer to generate the html et al outputs from a single source (where >> only that source is in git). >> >> Feedback from other list members is also appreciated. >> >> Rainer >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>> Sent: Thursday, July 10, 2008 8:51 AM >>> To: rsyslog-users >>> Subject: [rsyslog] Suggestion for Documentation Reconstructure >>> >>> Hi List, >>> >>> I consulted with my colleagues about the smartest way to reconstruct >>> the documents of rsyslog. >>> >>> 1.Work Flow matter >>> The attached image is just an idea for rational flowchart to make >>> translation easier. >>> #If this ML forbids me to attach any files, you can see it on my >>> site. >>> http://rio.tc/2008/07/10-144007.php >>> >>> I'm worried that this flow will give the contributers much trouble. >>> >>> - HTML(from Wiki) to DocBook conversion >>> 'tidy' will help us a little, but exporter of media Wiki is poor to >>> export 'well-formed HTML'. >>> We need to edit and check them manually. >>> Of course, I willingly do this. But it may take a number of days. >>> >>> - Writers need to edit DocBook XML after the reconstruction. >>> As you know, HTML format is not good for a source of multi-output. >>> DocBook XML is a perfect one except for edit :) >>> >>> - Automake related matter >>> There can be two streams to generate man and html from DocBook XML. >>> One : When you edit DocBook, you commit DocBook, HTML and man files >>> to git tree. >>> Of course this method requires us to prepare an >>> environment where you can use xsltproc/docbook2man. >>> Two : By adding some sequences into Makefile, HTML and man files >>> can be generated automatically. >>> All persons who want to build rsyslog from tar ball need >>> to prepare the environment. >>> And we need to put some check routines for dependencies >>> into Makefile. >>> >>> 2.git tree reconstruct only with 'doc' directory >>> >>> `-- doc >>> |-- Makefile.am >>> |-- conf >>> | |-- en >>> | | `-- rsyslog-example.conf >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | `-- rsyslog-example.conf # annotations are translated. >>> |-- html >>> | |-- en >>> | | |-- bugs.html >>> | | |-- OTHER_HTMLS >>> | | `-- version_naming.html >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | |-- bugs.html >>> | |-- OTHER_HTMLS >>> | `-- version_naming.html >>> |-- images >>> | |-- gssapi.png >>> | |-- OTHER_IMAGES >>> | `-- tls_cert_ca.jpg >>> |-- man >>> | |-- en >>> | | |-- man5 >>> | | | `-- rsyslog.conf.5 >>> | | `-- man8 >>> | | `-- rsyslogd.8 >>> | |-- OTHER_LANGUAGES(iso code) >>> | `-- jp >>> | |-- man5 >>> | | `-- rsyslog.conf.5 >>> | `-- man8 >>> | `-- rsyslogd.8 >>> `-- src >>> |-- dias >>> | |-- classes.dia >>> | |-- OTHER_DIA_FILES >>> | `-- tls_cert_ca.dia >>> `-- docbook >>> |-- en >>> | |-- bugs.xml >>> | |-- rsyslog.conf.5.xml >>> | |-- rsyslogd.8.xml >>> | |-- OTHER_XMLS >>> | `-- version_naming.xml >>> `-- ja >>> |-- bugs.html >>> |-- rsyslog.conf.5.xml >>> |-- rsyslogd.8.xml >>> |-- OTHER_XMLS >>> `-- version_naming.html >>> >>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ >>> directory. >>> >>> Your suggestions will be highly appreciated!! >>> >>> Best Rio. >>> >>> >> ####################################################################### >>> # >>> Ryo Fujita >>> Senior Solution Architect, RHCE >>> Red Hat K.K. >>> TEL +81-3-5798-8500 >>> FAX +81-3-5798-8599 >>> Ebisu Neonato 8F >>> 4-1-18 Ebisu, Shibuya-ku, >>> Tokyo Japan 1500013 >>> >> ####################################################################### >>> # >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From aoz.syn at gmail.com Thu Jul 17 17:08:22 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 09:08:22 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> Message-ID: <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> > I actually (really ;)) do not know syslog-ng and keep myself somewhat > away from it to prevent accidental steal of "whatever" from there. But > now that you raise the point: could you quickly describe how it works. > From your mail, it sounds like what I am thinking about... I 100% agree with and applaud that separation. As you've stated before, they make a good product and don't warrant any interference. Not touching the docs myself (since I know too much of it by heart anyway), flush_lines allows the admin to configure a particular number of log entries to buffer before forcing a flush to disk, whereas flush_timeout configures the maximum interval between disk flushes. Unlike ramlog, it seems to do this with internal allocations and doesn't rely on a ramdisk. I usually set it rather conservatively (20 and 600 respectively), but I can definitely see being more aggressive on a dedicated log collector with critical power or a laptop in ultra-low power mode. This has been a feature of the public version of syslog-ng for as long as I can remember (or four years, whichever is sooner ;). Combined with disk queues I can see a very nice tiered approach to handling extremely high volumes of log data in a rather reliable manner. From rgerhards at hq.adiscon.com Thu Jul 17 18:29:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 18:29:30 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> Message-ID: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> On Thu, 2008-07-17 at 09:08 -0600, RB wrote: > > I actually (really ;)) do not know syslog-ng and keep myself somewhat > > away from it to prevent accidental steal of "whatever" from there. But > > now that you raise the point: could you quickly describe how it works. > > From your mail, it sounds like what I am thinking about... > > I 100% agree with and applaud that separation. As you've stated > before, they make a good product and don't warrant any interference. Definitely. > > Not touching the docs myself (since I know too much of it by heart > anyway), flush_lines allows the admin to configure a particular number > of log entries to buffer before forcing a flush to disk, whereas > flush_timeout configures the maximum interval between disk flushes. > Unlike ramlog, it seems to do this with internal allocations and > doesn't rely on a ramdisk. I usually set it rather conservatively (20 > and 600 respectively), but I can definitely see being more aggressive > on a dedicated log collector with critical power or a laptop in > ultra-low power mode. This begins to make more and more sense. Just one thing to make sure we are talking along the same lines. The ramdisk approach provides a way to save the IO while still being able to look at the log lines as fast as they come in. If, in rsyslog, I create a memory buffer, I can persist lines to disk only after it is "time to write them to disk" (whatever that triggers). So you will not be able to observe the messages while they stick inside rsyslog's queue. I guess for many cases this is not really relevant. And of course, it is something that must be configurable on an action basis, so different files may use different settings. Thinking more about a potential algorithm, I tend to think it would probably useful to write log files in chunks of n bytes instead of n lines. I am thinking along the lines of matching up the output buffer with the disk sector size (or any multiple of it) so that partial writes of the same sector are avoided. Of course, that involves some stating of the file to obtain the size of the first buffer to be written (filesize mod sector [allocation unit] size). I also requires me to find out the allocation unit size for a given file system. It obviously involves writing incomplete lines. It requires additional code. So the best solution may actually be to permit partial writes (as initially thought) but recommend large buffers. The overall effect could justify the performance impact from doing non-optimal writes. But I am in too much detail. The actual questions are: a) is it OK that log data is visible only after the (write) delay? b) does it sound useful to buffer based on allocation unit sizes? Thanks, Rainer > > This has been a feature of the public version of syslog-ng for as long > as I can remember (or four years, whichever is sooner ;). Combined > with disk queues I can see a very nice tiered approach to handling > extremely high volumes of log data in a rather reliable manner. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 17 18:33:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 17 Jul 2008 18:33:38 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> Message-ID: <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> I still haven't found the time to have an in-depth look at docbook, but I strongly tend to agree to Michael's conclusion. It looks like a problem solver for some ugly problems I am facing. I will probably re-evaluate the release goal for 3.21.x. It may happen that I push back the script engine evolution to after summer to do some cleanup, performance improvement, doc work and maybe a rewrite of the file output handler. I find it a bit hard to think what is more important, but I think we can stay with the non-scriptable config for two more month. Maybe I can add custom functions as a smaller improvement sometime while doing the other things. In any case, I am happy to have Rio-san over here and as such someone to ask for solutions to the doc problem. Thanks a lot! Rainer On Thu, 2008-07-17 at 16:32 +0200, Michael Biebl wrote: > Hi, > > I just wanted to say, that I like the idea of using docbook/xml to > generate the (html) documentation (and possibly the man pages). > Imo this would make the documentation more consistent (consistent font > sizes, headers, footers etc.), more usable (You'd get something like a > TOC and possibly better cross-linking) and perhaps easier keeping > up-to-date. > > Cheers, > Michael > > > 2008/7/10 Ryo Fujita : > > Hi Rainer-san, > > > > DocBook.org is here. > > http://www.docbook.org/ > > > > And IBM has a good guide for beginners. > > http://www.ibm.com/developerworks/xml/library/xml-matters3.html > > > > Recently, Red Hat uses DocBook format to generate web pages ,PDFs and > > so on. > > For example, > > https://www.redhat.com/docs/manuals/enterprise/ > > A document having both html and PDF is generated from DocBook. > > > > gnome-doc-utils and kdesdk-utils include very convenient tools to > > handle DocBook and related formats. > > > > Best Rio. > > > > On 2008/07/10, at 16:03, Rainer Gerhards wrote: > > > >> Hi Rio-san, > >> > >> I need to leave soon, but a quick feedback: the proposal looks *very* > >> interesting. I need to digest it a bit. I have no experience with > >> DocBook, so I probably need to check that out, first. That may take a > >> short while (should you happen to know a good "getting started guide", > >> I'd grateful if you send me a link ;)). From first thought, I would > >> prefer to generate the html et al outputs from a single source (where > >> only that source is in git). > >> > >> Feedback from other list members is also appreciated. > >> > >> Rainer > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>> Sent: Thursday, July 10, 2008 8:51 AM > >>> To: rsyslog-users > >>> Subject: [rsyslog] Suggestion for Documentation Reconstructure > >>> > >>> Hi List, > >>> > >>> I consulted with my colleagues about the smartest way to reconstruct > >>> the documents of rsyslog. > >>> > >>> 1.Work Flow matter > >>> The attached image is just an idea for rational flowchart to make > >>> translation easier. > >>> #If this ML forbids me to attach any files, you can see it on my > >>> site. > >>> http://rio.tc/2008/07/10-144007.php > >>> > >>> I'm worried that this flow will give the contributers much trouble. > >>> > >>> - HTML(from Wiki) to DocBook conversion > >>> 'tidy' will help us a little, but exporter of media Wiki is poor to > >>> export 'well-formed HTML'. > >>> We need to edit and check them manually. > >>> Of course, I willingly do this. But it may take a number of days. > >>> > >>> - Writers need to edit DocBook XML after the reconstruction. > >>> As you know, HTML format is not good for a source of multi-output. > >>> DocBook XML is a perfect one except for edit :) > >>> > >>> - Automake related matter > >>> There can be two streams to generate man and html from DocBook XML. > >>> One : When you edit DocBook, you commit DocBook, HTML and man files > >>> to git tree. > >>> Of course this method requires us to prepare an > >>> environment where you can use xsltproc/docbook2man. > >>> Two : By adding some sequences into Makefile, HTML and man files > >>> can be generated automatically. > >>> All persons who want to build rsyslog from tar ball need > >>> to prepare the environment. > >>> And we need to put some check routines for dependencies > >>> into Makefile. > >>> > >>> 2.git tree reconstruct only with 'doc' directory > >>> > >>> `-- doc > >>> |-- Makefile.am > >>> |-- conf > >>> | |-- en > >>> | | `-- rsyslog-example.conf > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | `-- rsyslog-example.conf # annotations are translated. > >>> |-- html > >>> | |-- en > >>> | | |-- bugs.html > >>> | | |-- OTHER_HTMLS > >>> | | `-- version_naming.html > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | |-- bugs.html > >>> | |-- OTHER_HTMLS > >>> | `-- version_naming.html > >>> |-- images > >>> | |-- gssapi.png > >>> | |-- OTHER_IMAGES > >>> | `-- tls_cert_ca.jpg > >>> |-- man > >>> | |-- en > >>> | | |-- man5 > >>> | | | `-- rsyslog.conf.5 > >>> | | `-- man8 > >>> | | `-- rsyslogd.8 > >>> | |-- OTHER_LANGUAGES(iso code) > >>> | `-- jp > >>> | |-- man5 > >>> | | `-- rsyslog.conf.5 > >>> | `-- man8 > >>> | `-- rsyslogd.8 > >>> `-- src > >>> |-- dias > >>> | |-- classes.dia > >>> | |-- OTHER_DIA_FILES > >>> | `-- tls_cert_ca.dia > >>> `-- docbook > >>> |-- en > >>> | |-- bugs.xml > >>> | |-- rsyslog.conf.5.xml > >>> | |-- rsyslogd.8.xml > >>> | |-- OTHER_XMLS > >>> | `-- version_naming.xml > >>> `-- ja > >>> |-- bugs.html > >>> |-- rsyslog.conf.5.xml > >>> |-- rsyslogd.8.xml > >>> |-- OTHER_XMLS > >>> `-- version_naming.html > >>> > >>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./tools/ > >>> directory. > >>> > >>> Your suggestions will be highly appreciated!! > >>> > >>> Best Rio. > >>> > >>> > >> ####################################################################### > >>> # > >>> Ryo Fujita > >>> Senior Solution Architect, RHCE > >>> Red Hat K.K. > >>> TEL +81-3-5798-8500 > >>> FAX +81-3-5798-8599 > >>> Ebisu Neonato 8F > >>> 4-1-18 Ebisu, Shibuya-ku, > >>> Tokyo Japan 1500013 > >>> > >> ####################################################################### > >>> # > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > ######################################################################## > > Ryo Fujita > > Senior Solution Architect, RHCE > > Red Hat K.K. > > TEL +81-3-5798-8500 > > FAX +81-3-5798-8599 > > Ebisu Neonato 8F > > 4-1-18 Ebisu, Shibuya-ku, > > Tokyo Japan 1500013 > > ######################################################################## > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > From martinmie at PartyGaming.com Thu Jul 17 18:41:19 2008 From: martinmie at PartyGaming.com (Martin Mielke) Date: Thu, 17 Jul 2008 18:41:19 +0200 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com><4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com><4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: <0B1B9163138571439EAF804646F3F6AA050C3078@GIBSVWIN004X.partygaming.local> Hi all, [ big snip ] > > The actual questions are: > > a) is it OK that log data is visible only after the (write) delay? Hmmm.. no. This would eliminate the "magic" of watching system events in real-time as in the commonly used 'tail -n0 -f /var/log/messages' which is often used to troubleshoot problems... Unless you provide some tools to implement such functionality by watching the log events in memory... huh... My 2 cents... Martin > b) does it sound useful to buffer based on allocation unit sizes? > > Thanks, > Rainer > > > > This has been a feature of the public version of syslog-ng for as long > > as I can remember (or four years, whichever is sooner ;). Combined > > with disk queues I can see a very nice tiered approach to handling > > extremely high volumes of log data in a rather reliable manner. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Thu Jul 17 20:16:03 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 17 Jul 2008 14:16:03 -0400 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: > The actual questions are: > > a) is it OK that log data is visible only after the (write) delay? > b) does it sound useful to buffer based on allocation unit sizes? a) It depends on the situation. If you're using this to conserve power or flash-disk writes, then it's going to be a pain. If you're using this to tune performance in a high log-volume environment, then this is an acceptable tradeoff. Perhaps a secondary script that would send a signal to rsyslogd and print the lines to STDOUT would be an acceptable compromise? b) It's more granular than many (most?) users will need, including myself, but I'm sure some people will find this very useful. -HKS From aoz.syn at gmail.com Thu Jul 17 21:11:24 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 17 Jul 2008 13:11:24 -0600 Subject: [rsyslog] Feedback request: in-memory buffering messages In-Reply-To: <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE0E@grfint2.intern.adiscon.com> <4255c2570807170720v1a36d1d9kb699a853fa8ed282@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44EE19@grfint2.intern.adiscon.com> <4255c2570807170808i2c8027b0i74ede09b94919719@mail.gmail.com> <1216312170.7184.83.camel@rgf9dev.intern.adiscon.com> Message-ID: <4255c2570807171211q76742249o6ae3917feec70a62@mail.gmail.com> > a) is it OK that log data is visible only after the (write) delay? For my purposes it is; I know some of the other responses have been that it's not, but perhaps setting up a signal (maybe USR2) for triggering the flush mechanism would help those situations. Of course, people wanting "real-time" data would probably either leave off the configurations (knowing they're doing so at the expense of performance) or change it at runtime and HUP. > b) does it sound useful to buffer based on allocation unit sizes? I had actually considered suggesting this as a third option (differentiating from other implementations), but abandoned it since I was unsure of the value and felt the implementation may be overly complex. If you implement byte/allocation-aligned flushes, my preference would be to have it in addition to but mutually (configuration) exclusive with record-aligned ones. Partial-sector writes are less of a concern to me in most cases than getting the whole message to disk, but I can again see the usefulness of allocation alignment in a very high-volume environment or one where disk performance is constrained. Regardless, it's a fascinating development and one that will come in very handy. From rfujita at redhat.com Fri Jul 18 04:19:53 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 11:19:53 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> Message-ID: <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Hi, As I found an excellent template in the SRPM of docbook2X package, I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. First of all, let me finish the work related to man pages. I hope Rainer-san and other contributors devote yourself to make rsyslog better :) Best Rio. On 2008/07/18, at 1:33, Rainer Gerhards wrote: > I still haven't found the time to have an in-depth look at docbook, > but > I strongly tend to agree to Michael's conclusion. It looks like a > problem solver for some ugly problems I am facing. > > I will probably re-evaluate the release goal for 3.21.x. It may happen > that I push back the script engine evolution to after summer to do > some > cleanup, performance improvement, doc work and maybe a rewrite of the > file output handler. I find it a bit hard to think what is more > important, but I think we can stay with the non-scriptable config for > two more month. Maybe I can add custom functions as a smaller > improvement sometime while doing the other things. > > In any case, I am happy to have Rio-san over here and as such > someone to > ask for solutions to the doc problem. Thanks a lot! > > Rainer > > On Thu, 2008-07-17 at 16:32 +0200, Michael Biebl wrote: >> Hi, >> >> I just wanted to say, that I like the idea of using docbook/xml to >> generate the (html) documentation (and possibly the man pages). >> Imo this would make the documentation more consistent (consistent >> font >> sizes, headers, footers etc.), more usable (You'd get something >> like a >> TOC and possibly better cross-linking) and perhaps easier keeping >> up-to-date. >> >> Cheers, >> Michael >> >> >> 2008/7/10 Ryo Fujita : >>> Hi Rainer-san, >>> >>> DocBook.org is here. >>> http://www.docbook.org/ >>> >>> And IBM has a good guide for beginners. >>> http://www.ibm.com/developerworks/xml/library/xml-matters3.html >>> >>> Recently, Red Hat uses DocBook format to generate web pages ,PDFs >>> and >>> so on. >>> For example, >>> https://www.redhat.com/docs/manuals/enterprise/ >>> A document having both html and PDF is generated from DocBook. >>> >>> gnome-doc-utils and kdesdk-utils include very convenient tools to >>> handle DocBook and related formats. >>> >>> Best Rio. >>> >>> On 2008/07/10, at 16:03, Rainer Gerhards wrote: >>> >>>> Hi Rio-san, >>>> >>>> I need to leave soon, but a quick feedback: the proposal looks >>>> *very* >>>> interesting. I need to digest it a bit. I have no experience with >>>> DocBook, so I probably need to check that out, first. That may >>>> take a >>>> short while (should you happen to know a good "getting started >>>> guide", >>>> I'd grateful if you send me a link ;)). From first thought, I would >>>> prefer to generate the html et al outputs from a single source >>>> (where >>>> only that source is in git). >>>> >>>> Feedback from other list members is also appreciated. >>>> >>>> Rainer >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>> Sent: Thursday, July 10, 2008 8:51 AM >>>>> To: rsyslog-users >>>>> Subject: [rsyslog] Suggestion for Documentation Reconstructure >>>>> >>>>> Hi List, >>>>> >>>>> I consulted with my colleagues about the smartest way to >>>>> reconstruct >>>>> the documents of rsyslog. >>>>> >>>>> 1.Work Flow matter >>>>> The attached image is just an idea for rational flowchart to make >>>>> translation easier. >>>>> #If this ML forbids me to attach any files, you can see it on my >>>>> site. >>>>> http://rio.tc/2008/07/10-144007.php >>>>> >>>>> I'm worried that this flow will give the contributers much >>>>> trouble. >>>>> >>>>> - HTML(from Wiki) to DocBook conversion >>>>> 'tidy' will help us a little, but exporter of media Wiki is >>>>> poor to >>>>> export 'well-formed HTML'. >>>>> We need to edit and check them manually. >>>>> Of course, I willingly do this. But it may take a number of days. >>>>> >>>>> - Writers need to edit DocBook XML after the reconstruction. >>>>> As you know, HTML format is not good for a source of multi- >>>>> output. >>>>> DocBook XML is a perfect one except for edit :) >>>>> >>>>> - Automake related matter >>>>> There can be two streams to generate man and html from DocBook >>>>> XML. >>>>> One : When you edit DocBook, you commit DocBook, HTML and man >>>>> files >>>>> to git tree. >>>>> Of course this method requires us to prepare an >>>>> environment where you can use xsltproc/docbook2man. >>>>> Two : By adding some sequences into Makefile, HTML and man files >>>>> can be generated automatically. >>>>> All persons who want to build rsyslog from tar ball >>>>> need >>>>> to prepare the environment. >>>>> And we need to put some check routines for >>>>> dependencies >>>>> into Makefile. >>>>> >>>>> 2.git tree reconstruct only with 'doc' directory >>>>> >>>>> `-- doc >>>>> |-- Makefile.am >>>>> |-- conf >>>>> | |-- en >>>>> | | `-- rsyslog-example.conf >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | `-- rsyslog-example.conf # annotations are translated. >>>>> |-- html >>>>> | |-- en >>>>> | | |-- bugs.html >>>>> | | |-- OTHER_HTMLS >>>>> | | `-- version_naming.html >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | |-- bugs.html >>>>> | |-- OTHER_HTMLS >>>>> | `-- version_naming.html >>>>> |-- images >>>>> | |-- gssapi.png >>>>> | |-- OTHER_IMAGES >>>>> | `-- tls_cert_ca.jpg >>>>> |-- man >>>>> | |-- en >>>>> | | |-- man5 >>>>> | | | `-- rsyslog.conf.5 >>>>> | | `-- man8 >>>>> | | `-- rsyslogd.8 >>>>> | |-- OTHER_LANGUAGES(iso code) >>>>> | `-- jp >>>>> | |-- man5 >>>>> | | `-- rsyslog.conf.5 >>>>> | `-- man8 >>>>> | `-- rsyslogd.8 >>>>> `-- src >>>>> |-- dias >>>>> | |-- classes.dia >>>>> | |-- OTHER_DIA_FILES >>>>> | `-- tls_cert_ca.dia >>>>> `-- docbook >>>>> |-- en >>>>> | |-- bugs.xml >>>>> | |-- rsyslog.conf.5.xml >>>>> | |-- rsyslogd.8.xml >>>>> | |-- OTHER_XMLS >>>>> | `-- version_naming.xml >>>>> `-- ja >>>>> |-- bugs.html >>>>> |-- rsyslog.conf.5.xml >>>>> |-- rsyslogd.8.xml >>>>> |-- OTHER_XMLS >>>>> `-- version_naming.html >>>>> >>>>> # rsyslog.conf.5 and rsyslogd.8 files will be removed from ./ >>>>> tools/ >>>>> directory. >>>>> >>>>> Your suggestions will be highly appreciated!! >>>>> >>>>> Best Rio. >>>>> >>>>> >>>> ####################################################################### >>>>> # >>>>> Ryo Fujita >>>>> Senior Solution Architect, RHCE >>>>> Red Hat K.K. >>>>> TEL +81-3-5798-8500 >>>>> FAX +81-3-5798-8599 >>>>> Ebisu Neonato 8F >>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>> Tokyo Japan 1500013 >>>>> >>>> ####################################################################### >>>>> # >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> ######################################################################## >>> Ryo Fujita >>> Senior Solution Architect, RHCE >>> Red Hat K.K. >>> TEL +81-3-5798-8500 >>> FAX +81-3-5798-8599 >>> Ebisu Neonato 8F >>> 4-1-18 Ebisu, Shibuya-ku, >>> Tokyo Japan 1500013 >>> ######################################################################## >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From mbiebl at gmail.com Fri Jul 18 04:38:31 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 18 Jul 2008 04:38:31 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Message-ID: 2008/7/18 Ryo Fujita : > Hi, > > As I found an excellent template in the SRPM of docbook2X package, > I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. Do you have a public git repo somewhere and a separate branch, where you make these changes? It would be interesting to track the progress. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rfujita at redhat.com Fri Jul 18 05:17:36 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 12:17:36 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com> <89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com> <1216312418.7184.87.camel@rgf9dev.intern.adiscon.com> <2C40F387-51F1-4107-950F-C5D407352139@redhat.com> Message-ID: <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> Hi Mike-san, I have no git repo now. So, I sent the files I converted to Rainer-san directly. But I recently feel a necessity to build a public repo. If I have a time to do it this weekend, I'll try it :) Best Rio. On 2008/07/18, at 11:38, Michael Biebl wrote: > 2008/7/18 Ryo Fujita : >> Hi, >> >> As I found an excellent template in the SRPM of docbook2X package, >> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > > Do you have a public git repo somewhere and a separate branch, where > you make these changes? > > It would be interesting to track the progress. > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 08:14:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 08:14:26 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com> <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Hi Rio-san, I would really appreciate if you could build a git repro. I think that would be a very valuable resource and help streamline the integration process :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 5:18 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi Mike-san, > > I have no git repo now. > So, I sent the files I converted to Rainer-san directly. > But I recently feel a necessity to build a public repo. > If I have a time to do it this weekend, I'll try it :) > > Best Rio. > > On 2008/07/18, at 11:38, Michael Biebl wrote: > > > 2008/7/18 Ryo Fujita : > >> Hi, > >> > >> As I found an excellent template in the SRPM of docbook2X package, > >> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > > > > Do you have a public git repo somewhere and a separate branch, where > > you make these changes? > > > > It would be interesting to track the progress. > > > > Cheers, > > Michael > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 08:43:33 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 15:43:33 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com> <547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Message-ID: Hi all, Please check the following git repo. http://rio.tc/git/rsyslog Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? # It's my first time to build my own git repo :) On 2008/07/18, at 15:14, Rainer Gerhards wrote: > Hi Rio-san, > > I would really appreciate if you could build a git repro. I think that > would be a very valuable resource and help streamline the integration > process :) > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 5:18 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi Mike-san, >> >> I have no git repo now. >> So, I sent the files I converted to Rainer-san directly. >> But I recently feel a necessity to build a public repo. >> If I have a time to do it this weekend, I'll try it :) >> >> Best Rio. >> >> On 2008/07/18, at 11:38, Michael Biebl wrote: >> >>> 2008/7/18 Ryo Fujita : >>>> Hi, >>>> >>>> As I found an excellent template in the SRPM of docbook2X package, >>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>> >>> Do you have a public git repo somewhere and a separate branch, where >>> you make these changes? >>> >>> It would be interesting to track the progress. >>> >>> Cheers, >>> Michael >>> >>> -- >>> Why is it that all of the instruments seeking intelligent life in > the >>> universe are pointed away from Earth? >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 09:24:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 09:24:19 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> Hi Rio-san, I get a "file not found" error. Unfortunately, I am far from being a git-wizard, so I can not provide much advise :( Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 8:44 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi all, > > Please check the following git repo. > http://rio.tc/git/rsyslog > Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? > > # It's my first time to build my own git repo :) > > On 2008/07/18, at 15:14, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > I would really appreciate if you could build a git repro. I think > that > > would be a very valuable resource and help streamline the integration > > process :) > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Friday, July 18, 2008 5:18 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >> > >> Hi Mike-san, > >> > >> I have no git repo now. > >> So, I sent the files I converted to Rainer-san directly. > >> But I recently feel a necessity to build a public repo. > >> If I have a time to do it this weekend, I'll try it :) > >> > >> Best Rio. > >> > >> On 2008/07/18, at 11:38, Michael Biebl wrote: > >> > >>> 2008/7/18 Ryo Fujita : > >>>> Hi, > >>>> > >>>> As I found an excellent template in the SRPM of docbook2X package, > >>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > >>> > >>> Do you have a public git repo somewhere and a separate branch, > where > >>> you make these changes? > >>> > >>> It would be interesting to track the progress. > >>> > >>> Cheers, > >>> Michael > >>> > >>> -- > >>> Why is it that all of the instruments seeking intelligent life in > > the > >>> universe are pointed away from Earth? > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 09:32:59 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 16:32:59 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> Message-ID: <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> Hi Rainer-san, Did you access with git? Access with web browser may fail or get 404 not found. I succeeded as follows. $ git clone http://rio.tc/git/rsyslog Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ Getting alternates list for http://rio.tc/git/rsyslog Getting pack list for http://rio.tc/git/rsyslog Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 $ cd rsyslog $ git branch -r origin/HEAD origin/beta origin/master origin/queuememleak origin/rfujita origin/rscript origin/tests origin/v1-stable origin/v2-stable origin/v3-stable origin/v3stable-klogbug origin/wall $ git-pull http://rio.tc/git/rsyslog rfujita Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http Updating 37bac26..9d841de Fast forward man/Makefile.am | 101 ++++ man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ man/en/man8/rsyslogd.8 | 303 +++++++++++ man/entities/xinclude.dtd | 31 + man/ja/man8/rsyslogd.8 | 319 +++++++++++ man/xml-source/en/rsyslog.conf.5.xml | 994 +++++++++++++++++++++++++ ++++++++ man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ man/xml-source/ja/rsyslog.conf.5.xml | 997 +++++++++++++++++++++++++ +++++++++ man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ man/xslt/docbook.xsl | 127 +++++ man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ man/xslt/man.xsl | 70 +++ man/xslt/settings.xsl | 11 + 13 files changed, 5464 insertions(+), 0 deletions(-) create mode 100644 man/Makefile.am create mode 100644 man/en/man5/rsyslog.conf.5 create mode 100644 man/en/man8/rsyslogd.8 create mode 100644 man/entities/xinclude.dtd create mode 100644 man/ja/man8/rsyslogd.8 create mode 100644 man/xml-source/en/rsyslog.conf.5.xml create mode 100644 man/xml-source/en/rsyslogd.8.xml create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml create mode 100644 man/xml-source/ja/rsyslogd.8.xml create mode 100644 man/xslt/docbook.xsl create mode 100644 man/xslt/expand-sambadoc.xsl create mode 100644 man/xslt/man.xsl create mode 100644 man/xslt/settings.xsl On 2008/07/18, at 16:24, Rainer Gerhards wrote: > Hi Rio-san, > > I get a "file not found" error. Unfortunately, I am far from being a > git-wizard, so I can not provide much advise :( > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 8:44 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi all, >> >> Please check the following git repo. >> http://rio.tc/git/rsyslog >> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? >> >> # It's my first time to build my own git repo :) >> >> On 2008/07/18, at 15:14, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> I would really appreciate if you could build a git repro. I think >> that >>> would be a very valuable resource and help streamline the > integration >>> process :) >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Friday, July 18, 2008 5:18 AM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >>>> >>>> Hi Mike-san, >>>> >>>> I have no git repo now. >>>> So, I sent the files I converted to Rainer-san directly. >>>> But I recently feel a necessity to build a public repo. >>>> If I have a time to do it this weekend, I'll try it :) >>>> >>>> Best Rio. >>>> >>>> On 2008/07/18, at 11:38, Michael Biebl wrote: >>>> >>>>> 2008/7/18 Ryo Fujita : >>>>>> Hi, >>>>>> >>>>>> As I found an excellent template in the SRPM of docbook2X > package, >>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>>>> >>>>> Do you have a public git repo somewhere and a separate branch, >> where >>>>> you make these changes? >>>>> >>>>> It would be interesting to track the progress. >>>>> >>>>> Cheers, >>>>> Michael >>>>> >>>>> -- >>>>> Why is it that all of the instruments seeking intelligent life in >>> the >>>>> universe are pointed away from Earth? >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> >> > ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> >> > ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 09:46:33 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 09:46:33 +0200 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> Hi Rio-san, doh ;) Yes, I was after the web interface. I now pulled the repo with git and everything works fine :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > Sent: Friday, July 18, 2008 9:33 AM > To: rsyslog-users > Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > > Hi Rainer-san, > > Did you access with git? > Access with web browser may fail or get 404 not found. > > I succeeded as follows. > > $ git clone http://rio.tc/git/rsyslog > Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ > Getting alternates list for http://rio.tc/git/rsyslog > Getting pack list for http://rio.tc/git/rsyslog > Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b > Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b > which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 > > > $ cd rsyslog > $ git branch -r > origin/HEAD > origin/beta > origin/master > origin/queuememleak > origin/rfujita > origin/rscript > origin/tests > origin/v1-stable > origin/v2-stable > origin/v3-stable > origin/v3stable-klogbug > origin/wall > > $ git-pull http://rio.tc/git/rsyslog rfujita > Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http > Updating 37bac26..9d841de > > Fast forward > man/Makefile.am | 101 ++++ > man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ > man/en/man8/rsyslogd.8 | 303 +++++++++++ > man/entities/xinclude.dtd | 31 + > man/ja/man8/rsyslogd.8 | 319 +++++++++++ > man/xml-source/en/rsyslog.conf.5.xml | 994 +++++++++++++++++++++++++ > ++++++++ > man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ > man/xml-source/ja/rsyslog.conf.5.xml | 997 +++++++++++++++++++++++++ > +++++++++ > man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ > man/xslt/docbook.xsl | 127 +++++ > man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ > man/xslt/man.xsl | 70 +++ > man/xslt/settings.xsl | 11 + > 13 files changed, 5464 insertions(+), 0 deletions(-) > create mode 100644 man/Makefile.am > create mode 100644 man/en/man5/rsyslog.conf.5 > create mode 100644 man/en/man8/rsyslogd.8 > create mode 100644 man/entities/xinclude.dtd > create mode 100644 man/ja/man8/rsyslogd.8 > create mode 100644 man/xml-source/en/rsyslog.conf.5.xml > create mode 100644 man/xml-source/en/rsyslogd.8.xml > create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml > create mode 100644 man/xml-source/ja/rsyslogd.8.xml > create mode 100644 man/xslt/docbook.xsl > create mode 100644 man/xslt/expand-sambadoc.xsl > create mode 100644 man/xslt/man.xsl > create mode 100644 man/xslt/settings.xsl > > On 2008/07/18, at 16:24, Rainer Gerhards wrote: > > > Hi Rio-san, > > > > I get a "file not found" error. Unfortunately, I am far from being a > > git-wizard, so I can not provide much advise :( > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >> Sent: Friday, July 18, 2008 8:44 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >> > >> Hi all, > >> > >> Please check the following git repo. > >> http://rio.tc/git/rsyslog > >> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? > >> > >> # It's my first time to build my own git repo :) > >> > >> On 2008/07/18, at 15:14, Rainer Gerhards wrote: > >> > >>> Hi Rio-san, > >>> > >>> I would really appreciate if you could build a git repro. I think > >> that > >>> would be a very valuable resource and help streamline the > > integration > >>> process :) > >>> > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita > >>>> Sent: Friday, July 18, 2008 5:18 AM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure > >>>> > >>>> Hi Mike-san, > >>>> > >>>> I have no git repo now. > >>>> So, I sent the files I converted to Rainer-san directly. > >>>> But I recently feel a necessity to build a public repo. > >>>> If I have a time to do it this weekend, I'll try it :) > >>>> > >>>> Best Rio. > >>>> > >>>> On 2008/07/18, at 11:38, Michael Biebl wrote: > >>>> > >>>>> 2008/7/18 Ryo Fujita : > >>>>>> Hi, > >>>>>> > >>>>>> As I found an excellent template in the SRPM of docbook2X > > package, > >>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. > >>>>> > >>>>> Do you have a public git repo somewhere and a separate branch, > >> where > >>>>> you make these changes? > >>>>> > >>>>> It would be interesting to track the progress. > >>>>> > >>>>> Cheers, > >>>>> Michael > >>>>> > >>>>> -- > >>>>> Why is it that all of the instruments seeking intelligent life in > >>> the > >>>>> universe are pointed away from Earth? > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>> > >> > > > ####################################################################### > >>>> # > >>>> Ryo Fujita > >>>> Senior Solution Architect, RHCE > >>>> Red Hat K.K. > >>>> TEL +81-3-5798-8500 > >>>> FAX +81-3-5798-8599 > >>>> Ebisu Neonato 8F > >>>> 4-1-18 Ebisu, Shibuya-ku, > >>>> Tokyo Japan 1500013 > >>>> > >>> > >> > > > ####################################################################### > >>>> # > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > > > ####################################################################### > >> # > >> Ryo Fujita > >> Senior Solution Architect, RHCE > >> Red Hat K.K. > >> TEL +81-3-5798-8500 > >> FAX +81-3-5798-8599 > >> Ebisu Neonato 8F > >> 4-1-18 Ebisu, Shibuya-ku, > >> Tokyo Japan 1500013 > >> > > > ####################################################################### > >> # > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ####################################################################### > # > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ####################################################################### > # > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rfujita at redhat.com Fri Jul 18 09:50:28 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 18 Jul 2008 16:50:28 +0900 Subject: [rsyslog] Suggestion for Documentation Reconstructure In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> References: <69DEFD0F-7957-4E9C-BFF7-85E46E01365C@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44ED94@grfint2.intern.adiscon.com><89CD2AEC-6157-4ADE-831D-62C22526F04C@redhat.com><1216312418.7184.87.camel@rgf9dev.intern.adiscon.com><2C40F387-51F1-4107-950F-C5D407352139@redhat.com><547E3E1B-059B-47B3-A179-5FF3EF33B4B6@redhat.com><577465F99B41C842AAFBE9ED71E70ABA44EE1E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EE21@grfint2.intern.adiscon.com> <7E6012D3-DAD5-4A82-9EE4-644B3C63D4B6@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA44EE22@grfint2.intern.adiscon.com> Message-ID: Fine! I don't know how to publish git directory for web browser. If I find the way to do, all you can access with browser :) Best Rio. On 2008/07/18, at 16:46, Rainer Gerhards wrote: > Hi Rio-san, > > doh ;) Yes, I was after the web interface. I now pulled the repo with > git and everything works fine :) > > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >> Sent: Friday, July 18, 2008 9:33 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >> >> Hi Rainer-san, >> >> Did you access with git? >> Access with web browser may fail or get 404 not found. >> >> I succeeded as follows. >> >> $ git clone http://rio.tc/git/rsyslog >> Initialized empty Git repository in /Users/rio/Desktop/rsyslog/.git/ >> Getting alternates list for http://rio.tc/git/rsyslog >> Getting pack list for http://rio.tc/git/rsyslog >> Getting index for pack 949f41e2e385b1cea225439a0353e07b12ce5b8b >> Getting pack 949f41e2e385b1cea225439a0353e07b12ce5b8b >> which contains 1fd0d48dee4d10012a35a6005a5c9b9f973180b0 >> >> >> $ cd rsyslog >> $ git branch -r >> origin/HEAD >> origin/beta >> origin/master >> origin/queuememleak >> origin/rfujita >> origin/rscript >> origin/tests >> origin/v1-stable >> origin/v2-stable >> origin/v3-stable >> origin/v3stable-klogbug >> origin/wall >> >> $ git-pull http://rio.tc/git/rsyslog rfujita >> Fetching refs/heads/rfujita from http://rio.tc/git/rsyslog using http >> Updating 37bac26..9d841de >> >> Fast forward >> man/Makefile.am | 101 ++++ >> man/en/man5/rsyslog.conf.5 | 686 +++++++++++++++++++++++ >> man/en/man8/rsyslogd.8 | 303 +++++++++++ >> man/entities/xinclude.dtd | 31 + >> man/ja/man8/rsyslogd.8 | 319 +++++++++++ >> man/xml-source/en/rsyslog.conf.5.xml | 994 > +++++++++++++++++++++++++ >> ++++++++ >> man/xml-source/en/rsyslogd.8.xml | 685 +++++++++++++++++++++++ >> man/xml-source/ja/rsyslog.conf.5.xml | 997 > +++++++++++++++++++++++++ >> +++++++++ >> man/xml-source/ja/rsyslogd.8.xml | 633 +++++++++++++++++++++ >> man/xslt/docbook.xsl | 127 +++++ >> man/xslt/expand-sambadoc.xsl | 507 +++++++++++++++++ >> man/xslt/man.xsl | 70 +++ >> man/xslt/settings.xsl | 11 + >> 13 files changed, 5464 insertions(+), 0 deletions(-) >> create mode 100644 man/Makefile.am >> create mode 100644 man/en/man5/rsyslog.conf.5 >> create mode 100644 man/en/man8/rsyslogd.8 >> create mode 100644 man/entities/xinclude.dtd >> create mode 100644 man/ja/man8/rsyslogd.8 >> create mode 100644 man/xml-source/en/rsyslog.conf.5.xml >> create mode 100644 man/xml-source/en/rsyslogd.8.xml >> create mode 100644 man/xml-source/ja/rsyslog.conf.5.xml >> create mode 100644 man/xml-source/ja/rsyslogd.8.xml >> create mode 100644 man/xslt/docbook.xsl >> create mode 100644 man/xslt/expand-sambadoc.xsl >> create mode 100644 man/xslt/man.xsl >> create mode 100644 man/xslt/settings.xsl >> >> On 2008/07/18, at 16:24, Rainer Gerhards wrote: >> >>> Hi Rio-san, >>> >>> I get a "file not found" error. Unfortunately, I am far from being a >>> git-wizard, so I can not provide much advise :( >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>> Sent: Friday, July 18, 2008 8:44 AM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Suggestion for Documentation Reconstructure >>>> >>>> Hi all, >>>> >>>> Please check the following git repo. >>>> http://rio.tc/git/rsyslog >>>> Can you see the repo named 'rfujita' and 'rsyslog/man/' directory? >>>> >>>> # It's my first time to build my own git repo :) >>>> >>>> On 2008/07/18, at 15:14, Rainer Gerhards wrote: >>>> >>>>> Hi Rio-san, >>>>> >>>>> I would really appreciate if you could build a git repro. I think >>>> that >>>>> would be a very valuable resource and help streamline the >>> integration >>>>> process :) >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Ryo Fujita >>>>>> Sent: Friday, July 18, 2008 5:18 AM >>>>>> To: rsyslog-users >>>>>> Subject: Re: [rsyslog] Suggestion for Documentation > Reconstructure >>>>>> >>>>>> Hi Mike-san, >>>>>> >>>>>> I have no git repo now. >>>>>> So, I sent the files I converted to Rainer-san directly. >>>>>> But I recently feel a necessity to build a public repo. >>>>>> If I have a time to do it this weekend, I'll try it :) >>>>>> >>>>>> Best Rio. >>>>>> >>>>>> On 2008/07/18, at 11:38, Michael Biebl wrote: >>>>>> >>>>>>> 2008/7/18 Ryo Fujita : >>>>>>>> Hi, >>>>>>>> >>>>>>>> As I found an excellent template in the SRPM of docbook2X >>> package, >>>>>>>> I'm rewriting the two man pages, rsyslogd.8 and rsyslog.conf.5. >>>>>>> >>>>>>> Do you have a public git repo somewhere and a separate branch, >>>> where >>>>>>> you make these changes? >>>>>>> >>>>>>> It would be interesting to track the progress. >>>>>>> >>>>>>> Cheers, >>>>>>> Michael >>>>>>> >>>>>>> -- >>>>>>> Why is it that all of the instruments seeking intelligent life > in >>>>> the >>>>>>> universe are pointed away from Earth? >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> >>>>>> >>>>> >>>> >>> >> > ####################################################################### >>>>>> # >>>>>> Ryo Fujita >>>>>> Senior Solution Architect, RHCE >>>>>> Red Hat K.K. >>>>>> TEL +81-3-5798-8500 >>>>>> FAX +81-3-5798-8599 >>>>>> Ebisu Neonato 8F >>>>>> 4-1-18 Ebisu, Shibuya-ku, >>>>>> Tokyo Japan 1500013 >>>>>> >>>>> >>>> >>> >> > ####################################################################### >>>>>> # >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> >> > ####################################################################### >>>> # >>>> Ryo Fujita >>>> Senior Solution Architect, RHCE >>>> Red Hat K.K. >>>> TEL +81-3-5798-8500 >>>> FAX +81-3-5798-8599 >>>> Ebisu Neonato 8F >>>> 4-1-18 Ebisu, Shibuya-ku, >>>> Tokyo Japan 1500013 >>>> >>> >> > ####################################################################### >>>> # >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > ####################################################################### >> # >> Ryo Fujita >> Senior Solution Architect, RHCE >> Red Hat K.K. >> TEL +81-3-5798-8500 >> FAX +81-3-5798-8599 >> Ebisu Neonato 8F >> 4-1-18 Ebisu, Shibuya-ku, >> Tokyo Japan 1500013 >> > ####################################################################### >> # >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 18 14:34:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 14:34:41 +0200 Subject: [rsyslog] slight kernel log format inconsistencybetween 3.16.2and 3.18.0 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EDC9@grfint2.intern.adiscon.com><1216127022.7184.74.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EDDB@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE2F@grfint2.intern.adiscon.com> FYI: I have now taken action against the format problem. I think the changelog entries tells it all: - added a new property replacer option "sp-if-no-1st-sp" to cover a problem with RFC 3164 based interpretation of tag separation. While it is a generic approach, it fixes a format problem introduced in 3.18.0, where kernel messages no longer had a space after the tag. This is done by a modification of the default templates. Please note that this may affect some messages where there intentionally is no space between the tag and the first character of the message content. If so, this needs to be worked around via a specific template. However, we consider this scenario to be quite remote and, even if it exists, it is not expected that it will actually cause problems with log parsers (instead, we assume the new default template behavior may fix previous problems with log parsers due to the missing space). It is not a perfect solution, but I hope a pragmatic and good-enough one. Other than format issues, I had also some performance concerns. What I have now implemented affects performance very little. Most importantly, it enables us to stay away from copying large strings in memory just to prepend a space. If some has a concern, please voice it. This patch will be part of 3.21.0 (as it looks to be released today) and, if no hard objection is received, 3.18.1. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 15, 2008 4:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] slight kernel log format inconsistencybetween > 3.16.2and 3.18.0 > > Michael, > > I have reviewed some of the code in question. It looks like a bug. It's > actually not a klog driver issue, the PRI parsing is done at an upper > klog layer and thus should work for linux as well. I would like to > generate a version with some additional instrumentation. Could you run > it and report the results back (maybe via private mail to keep the list > free of noise). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Tuesday, July 15, 2008 3:04 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] slight kernel log format inconsistency between > > 3.16.2and 3.18.0 > > > > On Tue, 2008-07-15 at 14:54 +0200, Michael Biebl wrote: > > > 2008/7/14 Rainer Gerhards : > > > > Hi all, > > > > > > > > Michael Biebl noticed a small inconsistency in kernel log format: > > > > > > > > 3.16.x (and before): > > > > Jul 11 17:33:28 pluto kernel: [14177.432349] > > ADDRCONF(NETDEV_CHANGE): > > > > wlan0: link becomes ready > > > > Jul 11 17:53:17 pluto kernel: Kernel logging (proc) stopped. > > > > > > > > > > > > 3.18.0: > > > > Jul 11 17:53:17 pluto kernel:imklog 3.18.0, log source = > /proc/kmsg > > > > started. > > > > Jul 11 17:55:56 pluto kernel:Kernel logging (proc) stopped. > > > > > > another example: > > > Jul 15 13:43:42 pluto kernel:<6>[47539.265140] eth0: link down > > > See the <6> after the kernel: tag. It's also new. > > > Is this some kind of log level, set by the kernel? > > > > This does not look correct. Under FreeBSD, it is typical to have a > PRI > > inside kernel messages. But for linux kernels I tested with, I did > not > > see it and (I think) so I did not support it. But if you see it on > > Debian, it seems to happen. A cure is to use the same logic that is > > used > > for BSD. > > > > > I have never seen, that other syslogger (like syslog-ng, sysklogd > set > > this). > > > > > > > > > > > As you can see, there is a space after kernel: in the pre- > > 3.18.0 > > > > messages. This also brings us down to a general inconsistency: > > > > > > > > Unfortunately, RFC 3164 defines the after TAG not as a > > delimiter > > > > but as part of the CONTENT of the message. This leads to some > > > > inconsistency in practice. Depending on who generated the > message, > > there > > > > may be a space after the TAG or not. If it is, that space must be > > > > submitted as part of the message itself. > > > > > > > > In versions up to 3.16.x, kernel log messages were generated by > old > > > > code, which did not know about tag and simply generated a non- > > structured > > > > message. With 3.17.x and above, the klog code is rewritten and > > correctly > > > > fills in all properties. This leads to the "missing" space, > because > > the > > > > space is neither part of the tag nor part of the message. > > > > > > > > I have now three options: > > > > > > > > 1. leave as is (without a space) > > > > In that case, some log parsers may run into troubles > > > > > > This was a rather vague concern of mine. I don't actually know, if > > > other software is affected by this change. > > > > I share this concern, but also have no additional information. I > wanted > > to point out the options. I have to admit I am still undecided. But > > that > > nobody violently objected so far makes this look like it is not so > > important... > > > > > > > > > 2. add a at the end of the TAG > > > > That will probably lead to even more confusion, as matches > against > > TAG > > > > would need to include that space. > > > > > > > > 3. add a in front of each kernel message > > > > This comes close to the previous mode, but is "a bit" clumpsy and > > > > hackish. It will also make all database tables etc have messages > > > > starting with . Similar as 2, MSG matching rules are affected > > (but I > > > > consider this less severe, as matches inside the MSG are usually > > driven > > > > by searches and not absolute positions). > > > > > > Just curious: For other log messages, that are not generated by the > > > kernel, you have a space after the programname: tag, e.g. > > > Jul 15 13:43:50 pluto dhclient: DHCPACK from 192.168.2.1 > > > > > > Is the space after dhclient: part of the message or part of the > > > :programname tag? > > > > In our private communication, I thought it is part of TAG. But that > is > > wrong. It actually is part of MSG - at least for legacy/RFC3164 > syslog. > > > > For the new IETF syslog RFC series (syslog-protocol, -tls et al) it > is > > well-defined and part of neither. There, it is a delimiter (as one > > would > > expect). I you use rsyslog only with syslog-protocol messages (and > the > > syslog-protocol templates), this is more or less no issue. But who > does > > today...? ;) > > > > The real problem is that legacy syslog has many different > > interpretations and we break one or the other in either way... > > > > > > > > Imho, if there is no clear definition in the RFC, how the message > > > should be written, it's best to stay backwards compatible. > > > > Is that a vote for option 3, stick a space in front of each kernel > > message? ;) > > > > > > > > > > > Cheers, > > > Michael > > > > > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 18 14:42:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 18 Jul 2008 14:42:12 +0200 Subject: [rsyslog] preview of 3.18.1 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE30@grfint2.intern.adiscon.com> Hi all, as you are probably aware, 3.18.1 contains a "fix" for a format issue with kernel messages. This fix has a somewhat broader scope. In an ideal world, I would have preferred to run it through the beta phase and not apply it directly to stable. But this is counter-productive, as the format issue may affect users of v3-stable *right now*. So I am doing the second best thing: I have prepared a preview tarball and hope that some of you will implement it and tell me if it works OK. I do not expect any problems, but, hey, there is always a difference between lab and practice. I have pushed back 3.18.1 in the hopes to get some feedback. I need to release 3.18.1 soon, preferably by next Monday or Tuesday. So I would appreciate any feedback you can offer (including "no problem experienced" feedback). The tarball is available at: http://download.rsyslog.com/rsyslog/rsyslog-3.18.1-Test1.tar.gz This most probably is the official 3.18.1, except for some changes to cover the new release number (except, of course, feedback requires changes). Thanks, Rainer From friedl at hq.adiscon.com Fri Jul 18 17:25:23 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 18 Jul 2008 17:25:23 +0200 Subject: [rsyslog] rsyslog 3.21.0 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE36@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.21.0, the first release of the new 3.21.x devel branch. It includes all fixes and additions of the current v3-stable and beta plus an improved testbench. Please note that this includes some vital bugfixes from earlier releases. 3.21.0 lays the foundation for the next round of rsyslog enhancements. It is a recommended update for all devel branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-121.phtml Changelog: http://www.rsyslog.com/Article258.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Mon Jul 21 18:16:24 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 21 Jul 2008 18:16:24 +0200 Subject: [rsyslog] rsyslog 3.18.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE53@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.18.1. It is a v3-stable branch version. It offers a number of bug fixes and solves portability issues on FreeBSD. Most importantly, a format difference in kernel logs between 3.16.x and 3.18.0 has been corrected. This is a recommended update for all v3-stable branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-122.phtml Changelog: http://www.rsyslog.com/Article260.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From martinmie at PartyGaming.com Tue Jul 22 10:38:32 2008 From: martinmie at PartyGaming.com (Martin Mielke) Date: Tue, 22 Jul 2008 10:38:32 +0200 Subject: [rsyslog] Whitepapers on perfomance Message-ID: <0B1B9163138571439EAF804646F3F6AA051AD910@GIBSVWIN004X.partygaming.local> Hi list, Can someone point me to a official document covering rsyslog's performance under heavy load? TIA, Martin From rgerhards at hq.adiscon.com Tue Jul 22 12:16:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 22 Jul 2008 12:16:28 +0200 Subject: [rsyslog] rsyslog scheduled as new default syslogd on Debian Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> Hi folks, I thought I share the good news. I have just blogged about the details: http://blog.gerhards.net/2008/07/rsyslog-will-become-debians-default.htm l Special thanks go to Michael Biebl, who is the driving force behind this development. Rainer From rfujita at redhat.com Wed Jul 23 19:10:56 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Thu, 24 Jul 2008 02:10:56 +0900 Subject: [rsyslog] rsyslog scheduled as new default syslogd on Debian In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44EE5B@grfint2.intern.adiscon.com> Message-ID: Hi Rainer-san, Congrats! And the news cheer me up, too! Best Rio. On 2008/07/22, at 19:16, Rainer Gerhards wrote: > Hi folks, > > I thought I share the good news. I have just blogged about the > details: > > http://blog.gerhards.net/2008/07/rsyslog-will-become-debians-default.htm > l > > Special thanks go to Michael Biebl, who is the driving force behind > this > development. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From mmcgrath at redhat.com Wed Jul 23 21:21:46 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 14:21:46 -0500 (CDT) Subject: [rsyslog] rsyslog dropping logs Message-ID: I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central logging server. When running tcpdump I'm seeing all the udp packets coming in but many of them are not getting logged. And we're talking like 10% or so getting logged (maybe less) and the rest are just lost. I've attached my config file. (side note, if I'm doing something stupid in the config please correct me) -Mike -------------- next part -------------- #rsyslog v3 config file #### GLOBAL DIRECTIVES #### # Use default timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # File syncing capability is disabled by default. This feature is usually not required, # not useful and an extreme performance hit #$ActionFileEnableSync on # An "In-Memory Queue" is created for remote logging. # $WorkDirectory /var/spool/rsyslog # where to place spool files # $ActionQueueFileName queue # unique name prefix for spool files # $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) # $ActionQueueSaveOnShutdown on # save messages to disk on shutdown # $ActionQueueType LinkedList # run asynchronously # $ActionResumeRetryCount -1 # infinety retries if host is down #### MODULES #### $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # provides kernel logging support (previously done by rklogd) #$ModLoad immark.so # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp.so $UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp.so #$InputTCPServerRun 514 $FileGroup sysadmin $FileCreateMode 0640 $DirGroup sysadmin $DirCreateMode 0750 #### RULES #### # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;local1.none;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log $template RawMessage,"%msg%\n" $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" if $app-name contains 'access.log' then -?HttpAccessTemplate;RawMessage $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" if $app-name contains 'error.log' then -?HttpErrorTemplate;RawMessage $template DynaFile,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/messages.log" *.*;local1.none -?DynaFile From Samuel.Kielek at marriott.com Wed Jul 23 21:44:25 2008 From: Samuel.Kielek at marriott.com (Kielek, Samuel) Date: Wed, 23 Jul 2008 15:44:25 -0400 Subject: [rsyslog] rsyslog dropping logs In-Reply-To: References: Message-ID: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> Mike, You're using some v3 features that will not work with v2 such as the conditional filtering. I'll email you off list with a copy of my config (also using RHEL 5.2 here). -Sam -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Mike McGrath Sent: Wednesday, July 23, 2008 3:22 PM To: rsyslog at lists.adiscon.com Subject: [rsyslog] rsyslog dropping logs I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central logging server. When running tcpdump I'm seeing all the udp packets coming in but many of them are not getting logged. And we're talking like 10% or so getting logged (maybe less) and the rest are just lost. I've attached my config file. (side note, if I'm doing something stupid in the config please correct me) -Mike From mmcgrath at redhat.com Wed Jul 23 23:04:24 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 16:04:24 -0500 (CDT) Subject: [rsyslog] Leading whitespace Message-ID: Right now I'm doing live logging of my http servers to my rsyslog host. I've got this code on the rsyslog server: $template RawMessage,"%msg%\n" $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" On the http servers I've got: CustomLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-access.log" combined ErrorLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-error.log" It works great except for one thing. The logs have an additional whitespace in front of each line. for example: 111.111.111.11 - - [23/Jul/2008:21:01:32 +0000] "GET /static/images/fedora-logo.png HTTP/1.1" 200 5354 "http://fedoraproject.org/static/css/fedora.css" "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.15) Gecko/20080702Fedora/2.0.0.15-1.fc8 Firefox/2.0.0.15" Any ideas? -Mike From mmcgrath at redhat.com Wed Jul 23 23:06:24 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 23 Jul 2008 16:06:24 -0500 (CDT) Subject: [rsyslog] rsyslog dropping logs In-Reply-To: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> References: <140D865F4BA13C4B9D3AFEFEAD1EA53206631767@HDQNCEXCL1V2.mihdq.marrcorp.marriott.com> Message-ID: On Wed, 23 Jul 2008, Kielek, Samuel wrote: > Mike, > > You're using some v3 features that will not work with v2 such as the > conditional filtering. I'll email you off list with a copy of my config > (also using RHEL 5.2 here). > Thanks, I upgraded to a v3 version with my old config, same issue. Used the config provided off list and it works great. even in v3. -Mike From rgerhards at hq.adiscon.com Thu Jul 24 12:27:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 24 Jul 2008 12:27:05 +0200 Subject: [rsyslog] Leading whitespace In-Reply-To: References: Message-ID: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> This is rooted in RFC 3164. A description can be found here: http://lists.adiscon.net/pipermail/rsyslog/2008-July/000893.html If you want to consistently get rid of that SP, you can remove it via the property replacer. I think this works: ?$template RawMessage,"%msg:2%\n" if not, then this ;) ??$template RawMessage,"%msg:2:2048%\n" Let us know if that cured the problem. Rainer On Wed, 2008-07-23 at 16:04 -0500, Mike McGrath wrote: > Right now I'm doing live logging of my http servers to my rsyslog host. > I've got this code on the rsyslog server: > > $template RawMessage,"%msg%\n" > $template HttpAccessTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" > $template HttpErrorTemplate,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/http/%APP-NAME%" > > > On the http servers I've got: > > CustomLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-access.log" combined > ErrorLog "| /usr/bin/logger -p local1.info -t join.fedoraproject.org-error.log" > > > It works great except for one thing. The logs have an additional > whitespace in front of each line. for example: > > 111.111.111.11 - - [23/Jul/2008:21:01:32 +0000] "GET /static/images/fedora-logo.png HTTP/1.1" 200 5354 "http://fedoraproject.org/static/css/fedora.css" "Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.15) Gecko/20080702Fedora/2.0.0.15-1.fc8 Firefox/2.0.0.15" > > Any ideas? > > -Mike > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 24 12:40:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 24 Jul 2008 12:40:24 +0200 Subject: [rsyslog] rsyslog dropping logs In-Reply-To: References: Message-ID: <1216896024.7184.189.camel@rgf9dev.intern.adiscon.com> (I am not commenting on v2 vs. v3 as this is already done) First of all, we need to keep in mind that UDP is inherently lossy. Even when a frame is seen received by the local stack, it does not mean that it will eventually be forwarded to the application. If message bursts come in very quickly and the OS scheduler does not schedule the app fast enough to receive this messages (or the app is too slow in itself! ;)) new frames may overwrite frames inside the stack's receive buffers. So it is always a good idea to avoid UDP if that's possible. HOWEVER, I, too, find it somewhat unusual that around 90% of all incoming frames are discarded before the rsyslog receiver could process them. One explanation I have is that you have bursts (or volume in general) that outperforms the configured actions. Having seen the config file, and seeing it does not include any database writer, it is hard to imagine this should happen, assuming reasonable hardware sizing is used. A cause could be excessive synchronous writes. Many rules do not put a dash in front of the file name and without it (in v2), every write is immediately synced. This is very costly. But still, I have never seen that this alone outperforms a system. To dig deeper into what is happening, a debug log would be most useful, together with the information which frames have been seen in tcpdump but NOT in one of the log files. You can enable debug mode via -dn command line switch and is recommended to run rsyslog interactively while doing so. Then, you can simply capture its output via stdout redirection. Please note that debug mode generates considerable output, and requires considerable additional processing time. In any case, though, it should show us where the bottleneck is. Please note that I need a consistent excerpt from the debug log that shows how things began and how it worked during the fault conditions. Usually, this means I need everything ;) Debug logs may also reveal sensitive information, even passwords, so you should be careful in what you do. I am used to log files around the size of 1GB. With reasonable compression, the transfer is usually not a problem (but I suggest you place them on a server for me to download). Download links and/or smaller logs you can email me privately at rgerhards at gmail.com (please NOT at my primary, adiscon, email address). I hope this helps and I am looking forward for the additional information. Rainer On Wed, 2008-07-23 at 14:21 -0500, Mike McGrath wrote: > I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central > logging server. When running tcpdump I'm seeing all the udp packets > coming in but many of them are not getting logged. And we're talking > like 10% or so getting logged (maybe less) and the rest are just lost. > I've attached my config file. > > (side note, if I'm doing something stupid in the config please correct me) > > -Mike > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From mmcgrath at redhat.com Thu Jul 24 15:41:59 2008 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 24 Jul 2008 08:41:59 -0500 (CDT) Subject: [rsyslog] Leading whitespace In-Reply-To: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> References: <1216895225.7184.177.camel@rgf9dev.intern.adiscon.com> Message-ID: On Thu, 24 Jul 2008, Rainer Gerhards wrote: > This is rooted in RFC 3164. A description can be found here: > > http://lists.adiscon.net/pipermail/rsyslog/2008-July/000893.html > > If you want to consistently get rid of that SP, you can remove it via > the property replacer. I think this works: > > ?$template RawMessage,"%msg:2%\n" > > if not, then this ;) > > ??$template RawMessage,"%msg:2:2048%\n" > > Let us know if that cured the problem. > %msg:2:2048% seems to work perfectly, thank you. -Mike From Kevin.Goutos at budget.state.ny.us Thu Jul 24 21:50:04 2008 From: Kevin.Goutos at budget.state.ny.us (Goutos, Kevin (DOB)) Date: Thu, 24 Jul 2008 15:50:04 -0400 Subject: [rsyslog] Help with Ommail Configuration Message-ID: Hello all, First off, I am not very Linux savvy. I don't have a lot of experience other then a basic course. This is probably way past my knowledge, but I really need to get it done. Appreciate any help you guys have to offer. I am working on a Red Hat Enterprise 4 box and I am running the latest edition of rsyslog. I currently have rsyslog configured to receive messages remotely via UDP. I am trying to set it up so that it will send out E-mail messages to the system Admin's based on the severity level of the log files I am receiving. I would like it so that any device that sends a log with a critical, alert, or emergency level facility will send out an e-mail to a specific address. Here is my rsyslog.conf file. I used the sample code from Rainer Gerhards configuration page. I tried sending a test syslog with 'hard disk fatal failure' in it, but it is not sending out mail. Also tried without the templates below thinking it would just send me an email for every syslog that I received, but it doesn't appear to be sending mail. Any thoughts on what I am doing wrong. I'm sure there is a lot I need to do, so please let me know. Thanks! $template mailSubject,"disk problem on %hostname%" $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' I edited out the 3 lines below in the code for security reasons.. $ActionMailSMTPServer $ActionMailFrom $ActionMailTo Here is my code from rsyslog.conf below. # if you experience problems, check # http://www.rsyslog.com/troubleshoot for assistance # rsyslog v3: load input modules # If you do not load inputs, nothing happens! # You may need to set the module load path if modules are not found. $ModLoad immark.so # provides --MARK-- message capability $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # kernel logging (formerly provided by rklogd) $ModLoad ommail $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none -/var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* -/var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit -/var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log #Catch all incoming syslog messages *.* /var/log/catchall;TraditionalFormatWithPRI # Remote Logging (we use TCP for reliable delivery) # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. $WorkDirectory /rsyslog/spool # where to place spool files $ActionQueueFileName uniqName # unique name prefix for spool files $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueType LinkedList # run asynchronously $ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional *.* @10.57.106.140:514 $ModLoad ommail $ActionMailSMTPServer $ActionMailFrom $ActionMailTo $template mailSubject,"disk problem on %hostname%" $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" $ActionMailSubject mailSubject # make sure we receive a mail only once in six # hours (21,600 seconds ;)) $ActionExecOnlyOnceEveryInterval 21600 # the if ... then ... mailBody mus be on one line! if $msg contains 'hard disk fatal failure' then :ommail:;mailBody # ######### Receiving Messages from Remote Hosts ########## # TCP Syslog Server: # provides TCP syslog reception and GSS-API (if compiled to support it) $ModLoad imtcp.so # load module $InputTCPServerRun 514 # start up TCP listener at port 514 # UDP Syslog Server: $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port -------------------------------------------------------- This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. From hks.private at gmail.com Thu Jul 24 23:01:42 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 24 Jul 2008 17:01:42 -0400 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: A few things to try: - You load ommail.so twice, once at the top and once right above your $ActionMail... lines. I don't think this will break it, but it's unnecessary - delete the second one. - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going to attempt sending a message once every 6 hours. For testing purposes, this will be obnoxious. Until you're ready for production, change it to: $ActionExecOnlyOnceEveryInterval 30 - Send everything to make sure you're not missing it based on something in the property-replacer/templates/whatever. Replace "if $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: *.* :ommail:;mailBody Try again. Try logging a few messages from the localhost first (with RHEL you can just run "logger test" to log a user.notice message with content "test") and see if you get them. If you don't, check the mail logs on your mail server to see if it ever received the message. If not, it's time to break out tcpdump and see if the packets are ever being generated. Hope that helps. -HKS On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) wrote: > Hello all, > > First off, I am not very Linux savvy. I don't have a lot of experience > other then a basic course. This is probably way past my knowledge, but I > really need to get it done. Appreciate any help you guys have to offer. > > I am working on a Red Hat Enterprise 4 box and I am running the latest > edition of rsyslog. I currently have rsyslog configured to receive > messages remotely via UDP. I am trying to set it up so that it will send > out E-mail messages to the system Admin's based on the severity level of > the log files I am receiving. I would like it so that any device that > sends a log with a critical, alert, or emergency level facility will > send out an e-mail to a specific address. > > Here is my rsyslog.conf file. I used the sample code from Rainer > Gerhards configuration page. I tried sending a test syslog with 'hard > disk fatal failure' in it, but it is not sending out mail. Also tried > without the templates below thinking it would just send me an email for > every syslog that I received, but it doesn't appear to be sending mail. > Any thoughts on what I am doing wrong. I'm sure there is a lot I need to > do, so please let me know. Thanks! > > $template mailSubject,"disk problem on %hostname%" > $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > > I edited out the 3 lines below in the code for security reasons.. > $ActionMailSMTPServer > $ActionMailFrom > $ActionMailTo > > > > Here is my code from rsyslog.conf below. > > > > # if you experience problems, check > # http://www.rsyslog.com/troubleshoot for assistance > > # rsyslog v3: load input modules > # If you do not load inputs, nothing happens! > # You may need to set the module load path if modules are not found. > > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging (e.g. > via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > $ModLoad ommail > > $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > > # Log all kernel messages to the console. > # Logging much else clutters up the screen. > #kern.* /dev/console > > # Log anything (except mail) of level info or higher. > # Don't log private authentication messages! > *.info;mail.none;authpriv.none;cron.none > -/var/log/messages > > # The authpriv file has restricted access. > authpriv.* /var/log/secure > > # Log all the mail messages in one place. > mail.* > -/var/log/maillog > > # Log cron stuff > cron.* -/var/log/cron > > # Everybody gets emergency messages > *.emerg * > > # Save news errors of level crit and higher in a special file. > uucp,news.crit > -/var/log/spooler > > # Save boot messages also to boot.log > local7.* > /var/log/boot.log > > #Catch all incoming syslog messages > *.* > /var/log/catchall;TraditionalFormatWithPRI > > # Remote Logging (we use TCP for reliable delivery) > # An on-disk queue is created for this action. If the remote host is > # down, messages are spooled to disk and sent when it is up again. > $WorkDirectory /rsyslog/spool # where to place spool files > $ActionQueueFileName uniqName # unique name prefix for spool files > $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > possible) > $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > $ActionQueueType LinkedList # run asynchronously > $ActionResumeRetryCount -1 # infinite retries if host is down > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > *.* @10.57.106.140:514 > > $ModLoad ommail > $ActionMailSMTPServer > $ActionMailFrom > $ActionMailTo > $template mailSubject,"disk problem on %hostname%" > $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > $ActionMailSubject mailSubject > # make sure we receive a mail only once in six > # hours (21,600 seconds ;)) > $ActionExecOnlyOnceEveryInterval 21600 > # the if ... then ... mailBody mus be on one line! > if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > > > # ######### Receiving Messages from Remote Hosts ########## > # TCP Syslog Server: > # provides TCP syslog reception and GSS-API (if compiled to support it) > $ModLoad imtcp.so # load module > $InputTCPServerRun 514 # start up TCP listener at port 514 > > # UDP Syslog Server: > $ModLoad imudp.so # provides UDP syslog reception > $UDPServerRun 514 # start a UDP syslog server at standard port > -------------------------------------------------------- > This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From hks.private at gmail.com Thu Jul 24 23:35:39 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 24 Jul 2008 17:35:39 -0400 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: Oh - one other possibility. rsyslogd has to have mail enabled at compile time to work with it at runtime. check your catchall logfile - if it has messages like this, you didn't compile it in: rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object file: No such file or directory To fix this, you'll need to reinstall. This time, when you run ./configure be sure to include --enable-mail. make install clean and you should be good to go... -HKS On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS wrote: > A few things to try: > > - You load ommail.so twice, once at the top and once right above your > $ActionMail... lines. I don't think this will break it, but it's > unnecessary - delete the second one. > > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > to attempt sending a message once every 6 hours. For testing purposes, > this will be obnoxious. Until you're ready for production, change it > to: > $ActionExecOnlyOnceEveryInterval 30 > > - Send everything to make sure you're not missing it based on > something in the property-replacer/templates/whatever. Replace "if > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: > *.* :ommail:;mailBody > > Try again. Try logging a few messages from the localhost first (with > RHEL you can just run "logger test" to log a user.notice message with > content "test") and see if you get them. > > If you don't, check the mail logs on your mail server to see if it > ever received the message. If not, it's time to break out tcpdump and > see if the packets are ever being generated. > > Hope that helps. > > -HKS > > > > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > wrote: >> Hello all, >> >> First off, I am not very Linux savvy. I don't have a lot of experience >> other then a basic course. This is probably way past my knowledge, but I >> really need to get it done. Appreciate any help you guys have to offer. >> >> I am working on a Red Hat Enterprise 4 box and I am running the latest >> edition of rsyslog. I currently have rsyslog configured to receive >> messages remotely via UDP. I am trying to set it up so that it will send >> out E-mail messages to the system Admin's based on the severity level of >> the log files I am receiving. I would like it so that any device that >> sends a log with a critical, alert, or emergency level facility will >> send out an e-mail to a specific address. >> >> Here is my rsyslog.conf file. I used the sample code from Rainer >> Gerhards configuration page. I tried sending a test syslog with 'hard >> disk fatal failure' in it, but it is not sending out mail. Also tried >> without the templates below thinking it would just send me an email for >> every syslog that I received, but it doesn't appear to be sending mail. >> Any thoughts on what I am doing wrong. I'm sure there is a lot I need to >> do, so please let me know. Thanks! >> >> $template mailSubject,"disk problem on %hostname%" >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >> >> I edited out the 3 lines below in the code for security reasons.. >> $ActionMailSMTPServer >> $ActionMailFrom >> $ActionMailTo >> >> >> >> Here is my code from rsyslog.conf below. >> >> >> >> # if you experience problems, check >> # http://www.rsyslog.com/troubleshoot for assistance >> >> # rsyslog v3: load input modules >> # If you do not load inputs, nothing happens! >> # You may need to set the module load path if modules are not found. >> >> $ModLoad immark.so # provides --MARK-- message capability >> $ModLoad imuxsock.so # provides support for local system logging (e.g. >> via logger command) >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >> $ModLoad ommail >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >> >> # Log all kernel messages to the console. >> # Logging much else clutters up the screen. >> #kern.* /dev/console >> >> # Log anything (except mail) of level info or higher. >> # Don't log private authentication messages! >> *.info;mail.none;authpriv.none;cron.none >> -/var/log/messages >> >> # The authpriv file has restricted access. >> authpriv.* /var/log/secure >> >> # Log all the mail messages in one place. >> mail.* >> -/var/log/maillog >> >> # Log cron stuff >> cron.* -/var/log/cron >> >> # Everybody gets emergency messages >> *.emerg * >> >> # Save news errors of level crit and higher in a special file. >> uucp,news.crit >> -/var/log/spooler >> >> # Save boot messages also to boot.log >> local7.* >> /var/log/boot.log >> >> #Catch all incoming syslog messages >> *.* >> /var/log/catchall;TraditionalFormatWithPRI >> >> # Remote Logging (we use TCP for reliable delivery) >> # An on-disk queue is created for this action. If the remote host is >> # down, messages are spooled to disk and sent when it is up again. >> $WorkDirectory /rsyslog/spool # where to place spool files >> $ActionQueueFileName uniqName # unique name prefix for spool files >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >> possible) >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> $ActionQueueType LinkedList # run asynchronously >> $ActionResumeRetryCount -1 # infinite retries if host is down >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> *.* @10.57.106.140:514 >> >> $ModLoad ommail >> $ActionMailSMTPServer >> $ActionMailFrom >> $ActionMailTo >> $template mailSubject,"disk problem on %hostname%" >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >> $ActionMailSubject mailSubject >> # make sure we receive a mail only once in six >> # hours (21,600 seconds ;)) >> $ActionExecOnlyOnceEveryInterval 21600 >> # the if ... then ... mailBody mus be on one line! >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >> >> >> # ######### Receiving Messages from Remote Hosts ########## >> # TCP Syslog Server: >> # provides TCP syslog reception and GSS-API (if compiled to support it) >> $ModLoad imtcp.so # load module >> $InputTCPServerRun 514 # start up TCP listener at port 514 >> >> # UDP Syslog Server: >> $ModLoad imudp.so # provides UDP syslog reception >> $UDPServerRun 514 # start a UDP syslog server at standard port >> -------------------------------------------------------- >> This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. If you have received this e-mail in error, or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately if you have received this e-mail by mistake, and delete it from your system. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > From rgerhards at hq.adiscon.com Fri Jul 25 11:41:50 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 11:41:50 +0200 Subject: [rsyslog] the philosophy behind rsyslog, phplogcon et al... Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEA0@grfint2.intern.adiscon.com> Hi all, This page is for those of you interested in the philosophy behind rsyslog and its sister projects: http://blog.gerhards.net/2008/07/what-is-event-and-what-event-log.html Be warned, it is a bit technical, but I think quite useful to look behind the curtain. Feedback is welcome. Thanks, Rainer From rgerhards at hq.adiscon.com Fri Jul 25 12:31:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 12:31:51 +0200 Subject: [rsyslog] Help with Ommail Configuration In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEA4@grfint2.intern.adiscon.com> Thanks, HKS, for the good answers. There is one thing I would like to bring to your attention: I often see that people do not check for rsyslog error messages themselves. That complicates setup. I do not blame anyone, but would like to make you aware of the problem. I have just blogged about a potential (partial) solution and will try to implement it: http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht ml Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Thursday, July 24, 2008 11:36 PM > To: rsyslog-users > Subject: Re: [rsyslog] Help with Ommail Configuration > > Oh - one other possibility. rsyslogd has to have mail enabled at > compile time to work with it at runtime. check your catchall logfile - > if it has messages like this, you didn't compile it in: > > rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > file: No such file or directory > > To fix this, you'll need to reinstall. This time, when you run > ./configure be sure to include --enable-mail. make install clean and > you should be good to go... > > -HKS > > On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS > wrote: > > A few things to try: > > > > - You load ommail.so twice, once at the top and once right above > your > > $ActionMail... lines. I don't think this will break it, but it's > > unnecessary - delete the second one. > > > > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > > to attempt sending a message once every 6 hours. For testing > purposes, > > this will be obnoxious. Until you're ready for production, change it > > to: > > $ActionExecOnlyOnceEveryInterval 30 > > > > - Send everything to make sure you're not missing it based on > > something in the property-replacer/templates/whatever. Replace "if > > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" with: > > *.* :ommail:;mailBody > > > > Try again. Try logging a few messages from the localhost first (with > > RHEL you can just run "logger test" to log a user.notice message with > > content "test") and see if you get them. > > > > If you don't, check the mail logs on your mail server to see if it > > ever received the message. If not, it's time to break out tcpdump and > > see if the packets are ever being generated. > > > > Hope that helps. > > > > -HKS > > > > > > > > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > > wrote: > >> Hello all, > >> > >> First off, I am not very Linux savvy. I don't have a lot of > experience > >> other then a basic course. This is probably way past my knowledge, > but I > >> really need to get it done. Appreciate any help you guys have to > offer. > >> > >> I am working on a Red Hat Enterprise 4 box and I am running the > latest > >> edition of rsyslog. I currently have rsyslog configured to receive > >> messages remotely via UDP. I am trying to set it up so that it will > send > >> out E-mail messages to the system Admin's based on the severity > level of > >> the log files I am receiving. I would like it so that any device > that > >> sends a log with a critical, alert, or emergency level facility will > >> send out an e-mail to a specific address. > >> > >> Here is my rsyslog.conf file. I used the sample code from Rainer > >> Gerhards configuration page. I tried sending a test syslog with > 'hard > >> disk fatal failure' in it, but it is not sending out mail. Also > tried > >> without the templates below thinking it would just send me an email > for > >> every syslog that I received, but it doesn't appear to be sending > mail. > >> Any thoughts on what I am doing wrong. I'm sure there is a lot I > need to > >> do, so please let me know. Thanks! > >> > >> $template mailSubject,"disk problem on %hostname%" > >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >> > >> I edited out the 3 lines below in the code for security reasons.. > >> $ActionMailSMTPServer > >> $ActionMailFrom > >> $ActionMailTo > >> > >> > >> > >> Here is my code from rsyslog.conf below. > >> > >> > >> > >> # if you experience problems, check > >> # http://www.rsyslog.com/troubleshoot for assistance > >> > >> # rsyslog v3: load input modules > >> # If you do not load inputs, nothing happens! > >> # You may need to set the module load path if modules are not found. > >> > >> $ModLoad immark.so # provides --MARK-- message capability > >> $ModLoad imuxsock.so # provides support for local system logging > (e.g. > >> via logger command) > >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >> $ModLoad ommail > >> > >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >> > >> # Log all kernel messages to the console. > >> # Logging much else clutters up the screen. > >> #kern.* /dev/console > >> > >> # Log anything (except mail) of level info or higher. > >> # Don't log private authentication messages! > >> *.info;mail.none;authpriv.none;cron.none > >> -/var/log/messages > >> > >> # The authpriv file has restricted access. > >> authpriv.* > /var/log/secure > >> > >> # Log all the mail messages in one place. > >> mail.* > >> -/var/log/maillog > >> > >> # Log cron stuff > >> cron.* - > /var/log/cron > >> > >> # Everybody gets emergency messages > >> *.emerg * > >> > >> # Save news errors of level crit and higher in a special file. > >> uucp,news.crit > >> -/var/log/spooler > >> > >> # Save boot messages also to boot.log > >> local7.* > >> /var/log/boot.log > >> > >> #Catch all incoming syslog messages > >> *.* > >> /var/log/catchall;TraditionalFormatWithPRI > >> > >> # Remote Logging (we use TCP for reliable delivery) > >> # An on-disk queue is created for this action. If the remote host is > >> # down, messages are spooled to disk and sent when it is up again. > >> $WorkDirectory /rsyslog/spool # where to place spool files > >> $ActionQueueFileName uniqName # unique name prefix for spool files > >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >> possible) > >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >> $ActionQueueType LinkedList # run asynchronously > >> $ActionResumeRetryCount -1 # infinite retries if host is down > >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> *.* @10.57.106.140:514 > >> > >> $ModLoad ommail > >> $ActionMailSMTPServer > >> $ActionMailFrom > >> $ActionMailTo > >> $template mailSubject,"disk problem on %hostname%" > >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >> $ActionMailSubject mailSubject > >> # make sure we receive a mail only once in six > >> # hours (21,600 seconds ;)) > >> $ActionExecOnlyOnceEveryInterval 21600 > >> # the if ... then ... mailBody mus be on one line! > >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >> > >> > >> # ######### Receiving Messages from Remote Hosts ########## > >> # TCP Syslog Server: > >> # provides TCP syslog reception and GSS-API (if compiled to support > it) > >> $ModLoad imtcp.so # load module > >> $InputTCPServerRun 514 # start up TCP listener at port 514 > >> > >> # UDP Syslog Server: > >> $ModLoad imudp.so # provides UDP syslog reception > >> $UDPServerRun 514 # start a UDP syslog server at standard port > >> -------------------------------------------------------- > >> This e-mail, including any attachments, may be confidential, > privileged or otherwise legally protected. If you have received this e- > mail in error, or from someone who was not authorized to send it to > you, do not disseminate, copy or otherwise use this e-mail or its > attachments. Please notify the sender immediately if you have received > this e-mail by mistake, and delete it from your system. > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Fri Jul 25 16:33:26 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 25 Jul 2008 10:33:26 -0400 Subject: [rsyslog] rsyslog error messages Message-ID: Hi Rainer, Interesting post. The tendency to ignore errors/diagnostics is not limited to rsyslog, it's something that plagues pretty much every piece of software ever created anywhere. Some users are ignorant of basic diagnostic tools like syslog, --help flags, tcpdump, and man pages. Others just need a bit of prodding before they slap themselves and realize that they forgot some basic troubleshooting. Some are lazy slobs. However, rsyslog lends itself to error reports that lack troubleshooting for one huge reason: errors are not reliably reported in the default configuration. It's happened a few times that I start rsyslogd at the command line, and yet there's nothing in the process table, and nothing dumped to my logs. I've been able to track down the problem with -dn switches, but that involves weeding through a lot of irrelevant information. There are three modifications I'd love to see that would help resolve this: 1 - Configuration errors should be fatal 2 - Configuration and other runtime errors should be printed to STDERR 3 - By default, rsyslogd should log all of its own errors (fatal or not) to /var/log/rsyslog.log. This should also be user-configurable My reasoning: 1 - If rsyslog doesn't understand your config file, obviously it won't be doing what you intend it to. There's little benefit in running with a horked configuration, but if you don't test thoroughly, the cost could be devastating. Some users won't know they're not logging things until they need them - then, if they keep their jobs, they probably won't keep rsyslog ("This software doesn't even log ssh attempts!"). 2 - This will avoid a lot of silly questions without any supporting information. At least when you get a complaint like "Rsyslogd won't start!" it will generally include a "It just says 'configuration file invalid: broken syntax at line 135.'" For most users, this kind of error reporting will be enough to lead them to a solution. If nothing else, it removes the need to explain how to find the errors. 3 - For those more subtle problems, and things you might miss upon bootup. This also gives you a simple, easily accessible debugging source with minimal security implications and additional code and no required user configuration. I'm not sure how the code is written, but I'd prefer to see this happen even if the configuration file can't be read. Perhaps an $RsyslogErrors directive that, by default, references a different module that just dumps directly to a file? Users can configure it to put rsyslog errors through the actual logging engine and then manage it with typical syslog selector/action lines. The HTTP server is an interesting idea, but not something I would make use of. The potential security problems are enormous, and it would introduce too many additional factors to the debugging process: is the firewall right? Was there already a process listening on that port? Is the user's configuration file correct? These are the kinds of things you already have to deal with on rsyslog itself - why force the debugging process to go through them again? Hope that helps, and sorry for the long email. -HKS On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards wrote: > Thanks, HKS, for the good answers. There is one thing I would like to > bring to your attention: I often see that people do not check for > rsyslog error messages themselves. That complicates setup. I do not > blame anyone, but would like to make you aware of the problem. > > I have just blogged about a potential (partial) solution and will try to > implement it: > > http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > ml > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of (private) HKS >> Sent: Thursday, July 24, 2008 11:36 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Help with Ommail Configuration >> >> Oh - one other possibility. rsyslogd has to have mail enabled at >> compile time to work with it at runtime. check your catchall logfile - >> if it has messages like this, you didn't compile it in: >> >> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', >> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object >> file: No such file or directory >> >> To fix this, you'll need to reinstall. This time, when you run >> ./configure be sure to include --enable-mail. make install clean and >> you should be good to go... >> >> -HKS >> >> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >> wrote: >> > A few things to try: >> > >> > - You load ommail.so twice, once at the top and once right above >> your >> > $ActionMail... lines. I don't think this will break it, but it's >> > unnecessary - delete the second one. >> > >> > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going >> > to attempt sending a message once every 6 hours. For testing >> purposes, >> > this will be obnoxious. Until you're ready for production, change it >> > to: >> > $ActionExecOnlyOnceEveryInterval 30 >> > >> > - Send everything to make sure you're not missing it based on >> > something in the property-replacer/templates/whatever. Replace "if >> > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > with: >> > *.* :ommail:;mailBody >> > >> > Try again. Try logging a few messages from the localhost first (with >> > RHEL you can just run "logger test" to log a user.notice message > with >> > content "test") and see if you get them. >> > >> > If you don't, check the mail logs on your mail server to see if it >> > ever received the message. If not, it's time to break out tcpdump > and >> > see if the packets are ever being generated. >> > >> > Hope that helps. >> > >> > -HKS >> > >> > >> > >> > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) >> > wrote: >> >> Hello all, >> >> >> >> First off, I am not very Linux savvy. I don't have a lot of >> experience >> >> other then a basic course. This is probably way past my knowledge, >> but I >> >> really need to get it done. Appreciate any help you guys have to >> offer. >> >> >> >> I am working on a Red Hat Enterprise 4 box and I am running the >> latest >> >> edition of rsyslog. I currently have rsyslog configured to receive >> >> messages remotely via UDP. I am trying to set it up so that it will >> send >> >> out E-mail messages to the system Admin's based on the severity >> level of >> >> the log files I am receiving. I would like it so that any device >> that >> >> sends a log with a critical, alert, or emergency level facility > will >> >> send out an e-mail to a specific address. >> >> >> >> Here is my rsyslog.conf file. I used the sample code from Rainer >> >> Gerhards configuration page. I tried sending a test syslog with >> 'hard >> >> disk fatal failure' in it, but it is not sending out mail. Also >> tried >> >> without the templates below thinking it would just send me an email >> for >> >> every syslog that I received, but it doesn't appear to be sending >> mail. >> >> Any thoughts on what I am doing wrong. I'm sure there is a lot I >> need to >> >> do, so please let me know. Thanks! >> >> >> >> $template mailSubject,"disk problem on %hostname%" >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >> >> >> >> I edited out the 3 lines below in the code for security reasons.. >> >> $ActionMailSMTPServer >> >> $ActionMailFrom >> >> $ActionMailTo >> >> >> >> >> >> >> >> Here is my code from rsyslog.conf below. >> >> >> >> >> >> >> >> # if you experience problems, check >> >> # http://www.rsyslog.com/troubleshoot for assistance >> >> >> >> # rsyslog v3: load input modules >> >> # If you do not load inputs, nothing happens! >> >> # You may need to set the module load path if modules are not > found. >> >> >> >> $ModLoad immark.so # provides --MARK-- message capability >> >> $ModLoad imuxsock.so # provides support for local system logging >> (e.g. >> >> via logger command) >> >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >> >> $ModLoad ommail >> >> >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >> >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >> >> >> >> # Log all kernel messages to the console. >> >> # Logging much else clutters up the screen. >> >> #kern.* > /dev/console >> >> >> >> # Log anything (except mail) of level info or higher. >> >> # Don't log private authentication messages! >> >> *.info;mail.none;authpriv.none;cron.none >> >> -/var/log/messages >> >> >> >> # The authpriv file has restricted access. >> >> authpriv.* >> /var/log/secure >> >> >> >> # Log all the mail messages in one place. >> >> mail.* >> >> -/var/log/maillog >> >> >> >> # Log cron stuff >> >> cron.* - >> /var/log/cron >> >> >> >> # Everybody gets emergency messages >> >> *.emerg * >> >> >> >> # Save news errors of level crit and higher in a special file. >> >> uucp,news.crit >> >> -/var/log/spooler >> >> >> >> # Save boot messages also to boot.log >> >> local7.* >> >> /var/log/boot.log >> >> >> >> #Catch all incoming syslog messages >> >> *.* >> >> /var/log/catchall;TraditionalFormatWithPRI >> >> >> >> # Remote Logging (we use TCP for reliable delivery) >> >> # An on-disk queue is created for this action. If the remote host > is >> >> # down, messages are spooled to disk and sent when it is up again. >> >> $WorkDirectory /rsyslog/spool # where to place spool files >> >> $ActionQueueFileName uniqName # unique name prefix for spool files >> >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >> >> possible) >> >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >> >> $ActionQueueType LinkedList # run asynchronously >> >> $ActionResumeRetryCount -1 # infinite retries if host is down >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> >> *.* @10.57.106.140:514 >> >> >> >> $ModLoad ommail >> >> $ActionMailSMTPServer >> >> $ActionMailFrom >> >> $ActionMailTo >> >> $template mailSubject,"disk problem on %hostname%" >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >> >> $ActionMailSubject mailSubject >> >> # make sure we receive a mail only once in six >> >> # hours (21,600 seconds ;)) >> >> $ActionExecOnlyOnceEveryInterval 21600 >> >> # the if ... then ... mailBody mus be on one line! >> >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >> >> >> >> >> >> # ######### Receiving Messages from Remote Hosts ########## >> >> # TCP Syslog Server: >> >> # provides TCP syslog reception and GSS-API (if compiled to support >> it) >> >> $ModLoad imtcp.so # load module >> >> $InputTCPServerRun 514 # start up TCP listener at port 514 >> >> >> >> # UDP Syslog Server: >> >> $ModLoad imudp.so # provides UDP syslog reception >> >> $UDPServerRun 514 # start a UDP syslog server at standard port >> >> -------------------------------------------------------- >> >> This e-mail, including any attachments, may be confidential, >> privileged or otherwise legally protected. If you have received this > e- >> mail in error, or from someone who was not authorized to send it to >> you, do not disseminate, copy or otherwise use this e-mail or its >> attachments. Please notify the sender immediately if you have received >> this e-mail by mistake, and delete it from your system. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rfujita at redhat.com Fri Jul 25 16:56:04 2008 From: rfujita at redhat.com (Ryo Fujita) Date: Fri, 25 Jul 2008 23:56:04 +0900 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: Hi HKS-san, +1 > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.' In addition, '... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.' Any ideas? Best Rio. On 2008/07/25, at 23:33, (private) HKS wrote: > Hi Rainer, > > Interesting post. The tendency to ignore errors/diagnostics is not > limited to rsyslog, it's something that plagues pretty much every > piece of software ever created anywhere. Some users are ignorant of > basic diagnostic tools like syslog, --help flags, tcpdump, and man > pages. Others just need a bit of prodding before they slap themselves > and realize that they forgot some basic troubleshooting. Some are lazy > slobs. > > However, rsyslog lends itself to error reports that lack > troubleshooting for one huge reason: errors are not reliably reported > in the default configuration. It's happened a few times that I start > rsyslogd at the command line, and yet there's nothing in the process > table, and nothing dumped to my logs. I've been able to track down the > problem with -dn switches, but that involves weeding through a lot of > irrelevant information. > > There are three modifications I'd love to see that would help > resolve this: > > 1 - Configuration errors should be fatal > 2 - Configuration and other runtime errors should be printed to STDERR > 3 - By default, rsyslogd should log all of its own errors (fatal or > not) to /var/log/rsyslog.log. This should also be user-configurable > > My reasoning: > > 1 - If rsyslog doesn't understand your config file, obviously it won't > be doing what you intend it to. There's little benefit in running with > a horked configuration, but if you don't test thoroughly, the cost > could be devastating. Some users won't know they're not logging things > until they need them - then, if they keep their jobs, they probably > won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.'" For most users, this kind of > error reporting will be enough to lead them to a solution. If nothing > else, it removes the need to explain how to find the errors. > > 3 - For those more subtle problems, and things you might miss upon > bootup. This also gives you a simple, easily accessible debugging > source with minimal security implications and additional code and no > required user configuration. I'm not sure how the code is written, but > I'd prefer to see this happen even if the configuration file can't be > read. Perhaps an $RsyslogErrors directive that, by default, references > a different module that just dumps directly to a file? Users can > configure it to put rsyslog errors through the actual logging engine > and then manage it with typical syslog selector/action lines. > > The HTTP server is an interesting idea, but not something I would make > use of. The potential security problems are enormous, and it would > introduce too many additional factors to the debugging process: is the > firewall right? Was there already a process listening on that port? Is > the user's configuration file correct? These are the kinds of things > you already have to deal with on rsyslog itself - why force the > debugging process to go through them again? > > Hope that helps, and sorry for the long email. > -HKS > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > wrote: >> Thanks, HKS, for the good answers. There is one thing I would like to >> bring to your attention: I often see that people do not check for >> rsyslog error messages themselves. That complicates setup. I do not >> blame anyone, but would like to make you aware of the problem. >> >> I have just blogged about a potential (partial) solution and will >> try to >> implement it: >> >> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht >> ml >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of (private) HKS >>> Sent: Thursday, July 24, 2008 11:36 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] Help with Ommail Configuration >>> >>> Oh - one other possibility. rsyslogd has to have mail enabled at >>> compile time to work with it at runtime. check your catchall >>> logfile - >>> if it has messages like this, you didn't compile it in: >>> >>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', >>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object >>> file: No such file or directory >>> >>> To fix this, you'll need to reinstall. This time, when you run >>> ./configure be sure to include --enable-mail. make install clean and >>> you should be good to go... >>> >>> -HKS >>> >>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >> > >>> wrote: >>>> A few things to try: >>>> >>>> - You load ommail.so twice, once at the top and once right above >>> your >>>> $ActionMail... lines. I don't think this will break it, but it's >>>> unnecessary - delete the second one. >>>> >>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going >>>> to attempt sending a message once every 6 hours. For testing >>> purposes, >>>> this will be obnoxious. Until you're ready for production, change >>>> it >>>> to: >>>> $ActionExecOnlyOnceEveryInterval 30 >>>> >>>> - Send everything to make sure you're not missing it based on >>>> something in the property-replacer/templates/whatever. Replace "if >>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody" >> with: >>>> *.* :ommail:;mailBody >>>> >>>> Try again. Try logging a few messages from the localhost first >>>> (with >>>> RHEL you can just run "logger test" to log a user.notice message >> with >>>> content "test") and see if you get them. >>>> >>>> If you don't, check the mail logs on your mail server to see if it >>>> ever received the message. If not, it's time to break out tcpdump >> and >>>> see if the packets are ever being generated. >>>> >>>> Hope that helps. >>>> >>>> -HKS >>>> >>>> >>>> >>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) >>>> wrote: >>>>> Hello all, >>>>> >>>>> First off, I am not very Linux savvy. I don't have a lot of >>> experience >>>>> other then a basic course. This is probably way past my knowledge, >>> but I >>>>> really need to get it done. Appreciate any help you guys have to >>> offer. >>>>> >>>>> I am working on a Red Hat Enterprise 4 box and I am running the >>> latest >>>>> edition of rsyslog. I currently have rsyslog configured to receive >>>>> messages remotely via UDP. I am trying to set it up so that it >>>>> will >>> send >>>>> out E-mail messages to the system Admin's based on the severity >>> level of >>>>> the log files I am receiving. I would like it so that any device >>> that >>>>> sends a log with a critical, alert, or emergency level facility >> will >>>>> send out an e-mail to a specific address. >>>>> >>>>> Here is my rsyslog.conf file. I used the sample code from Rainer >>>>> Gerhards configuration page. I tried sending a test syslog with >>> 'hard >>>>> disk fatal failure' in it, but it is not sending out mail. Also >>> tried >>>>> without the templates below thinking it would just send me an >>>>> email >>> for >>>>> every syslog that I received, but it doesn't appear to be sending >>> mail. >>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I >>> need to >>>>> do, so please let me know. Thanks! >>>>> >>>>> $template mailSubject,"disk problem on %hostname%" >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' >>>>> >>>>> I edited out the 3 lines below in the code for security reasons.. >>>>> $ActionMailSMTPServer >>>>> $ActionMailFrom >>>>> $ActionMailTo >>>>> >>>>> >>>>> >>>>> Here is my code from rsyslog.conf below. >>>>> >>>>> >>>>> >>>>> # if you experience problems, check >>>>> # http://www.rsyslog.com/troubleshoot for assistance >>>>> >>>>> # rsyslog v3: load input modules >>>>> # If you do not load inputs, nothing happens! >>>>> # You may need to set the module load path if modules are not >> found. >>>>> >>>>> $ModLoad immark.so # provides --MARK-- message capability >>>>> $ModLoad imuxsock.so # provides support for local system logging >>> (e.g. >>>>> via logger command) >>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) >>>>> $ModLoad ommail >>>>> >>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% >>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" >>>>> >>>>> # Log all kernel messages to the console. >>>>> # Logging much else clutters up the screen. >>>>> #kern.* >> /dev/console >>>>> >>>>> # Log anything (except mail) of level info or higher. >>>>> # Don't log private authentication messages! >>>>> *.info;mail.none;authpriv.none;cron.none >>>>> -/var/log/messages >>>>> >>>>> # The authpriv file has restricted access. >>>>> authpriv.* >>> /var/log/secure >>>>> >>>>> # Log all the mail messages in one place. >>>>> mail.* >>>>> -/var/log/maillog >>>>> >>>>> # Log cron stuff >>>>> cron.* - >>> /var/log/cron >>>>> >>>>> # Everybody gets emergency messages >>>>> *.emerg * >>>>> >>>>> # Save news errors of level crit and higher in a special file. >>>>> uucp,news.crit >>>>> -/var/log/spooler >>>>> >>>>> # Save boot messages also to boot.log >>>>> local7.* >>>>> /var/log/boot.log >>>>> >>>>> #Catch all incoming syslog messages >>>>> *.* >>>>> /var/log/catchall;TraditionalFormatWithPRI >>>>> >>>>> # Remote Logging (we use TCP for reliable delivery) >>>>> # An on-disk queue is created for this action. If the remote host >> is >>>>> # down, messages are spooled to disk and sent when it is up again. >>>>> $WorkDirectory /rsyslog/spool # where to place spool files >>>>> $ActionQueueFileName uniqName # unique name prefix for spool files >>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as >>>>> possible) >>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown >>>>> $ActionQueueType LinkedList # run asynchronously >>>>> $ActionResumeRetryCount -1 # infinite retries if host is down >>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port >>>>> optional >>>>> *.* @10.57.106.140:514 >>>>> >>>>> $ModLoad ommail >>>>> $ActionMailSMTPServer >>>>> $ActionMailFrom >>>>> $ActionMailTo >>>>> $template mailSubject,"disk problem on %hostname%" >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" >>>>> $ActionMailSubject mailSubject >>>>> # make sure we receive a mail only once in six >>>>> # hours (21,600 seconds ;)) >>>>> $ActionExecOnlyOnceEveryInterval 21600 >>>>> # the if ... then ... mailBody mus be on one line! >>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody >>>>> >>>>> >>>>> # ######### Receiving Messages from Remote Hosts ########## >>>>> # TCP Syslog Server: >>>>> # provides TCP syslog reception and GSS-API (if compiled to >>>>> support >>> it) >>>>> $ModLoad imtcp.so # load module >>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 >>>>> >>>>> # UDP Syslog Server: >>>>> $ModLoad imudp.so # provides UDP syslog reception >>>>> $UDPServerRun 514 # start a UDP syslog server at standard port >>>>> -------------------------------------------------------- >>>>> This e-mail, including any attachments, may be confidential, >>> privileged or otherwise legally protected. If you have received this >> e- >>> mail in error, or from someone who was not authorized to send it to >>> you, do not disseminate, copy or otherwise use this e-mail or its >>> attachments. Please notify the sender immediately if you have >>> received >>> this e-mail by mistake, and delete it from your system. >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ######################################################################## Ryo Fujita Senior Solution Architect, RHCE Red Hat K.K. TEL +81-3-5798-8500 FAX +81-3-5798-8599 Ebisu Neonato 8F 4-1-18 Ebisu, Shibuya-ku, Tokyo Japan 1500013 ######################################################################## From rgerhards at hq.adiscon.com Fri Jul 25 17:16:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 17:16:12 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> On Fri, 2008-07-25 at 10:33 -0400, (private) HKS wrote: > Hi Rainer, > > Interesting post. The tendency to ignore errors/diagnostics is not > limited to rsyslog, it's something that plagues pretty much every > piece of software ever created anywhere. I totally agree, but I think the situation is even worse for a syslogd. Because it *is* the usual error reporting system. I think we are a bit in the same situation (if you allow that analogy) like HIV which is attacking the human immune system. We have a problem in the troubleshooting system itself, and it may even prevent us from conveying troubleshooting information :( > Some users are ignorant of > basic diagnostic tools like syslog, --help flags, tcpdump, and man > pages. Others just need a bit of prodding before they slap themselves > and realize that they forgot some basic troubleshooting. Some are lazy > slobs. > > However, rsyslog lends itself to error reports that lack > troubleshooting for one huge reason: errors are not reliably reported > in the default configuration. > It's happened a few times that I start > rsyslogd at the command line, and yet there's nothing in the process > table, and nothing dumped to my logs. Could you elaborate a bit about above sentence - I have to admit I do not fully get you... > I've been able to track down the > problem with -dn switches, but that involves weeding through a lot of > irrelevant information. > > There are three modifications I'd love to see that would help resolve this: > > 1 - Configuration errors should be fatal > 2 - Configuration and other runtime errors should be printed to STDERR Does that really help? If so, that would be extremely easy to add. But does somebody see them? > 3 - By default, rsyslogd should log all of its own errors (fatal or > not) to /var/log/rsyslog.log. This should also be user-configurable This default comes into the way of some distros: they use different pathes. I really don't like the idea of embedding any information about the file system into the binary. Just think about embedded systems... > > My reasoning: > > 1 - If rsyslog doesn't understand your config file, obviously it won't > be doing what you intend it to. There's little benefit in running with > a horked configuration, but if you don't test thoroughly, the cost > could be devastating. Some users won't know they're not logging things > until they need them - then, if they keep their jobs, they probably > won't keep rsyslog ("This software doesn't even log ssh attempts!"). That's quite controversal and and does not match rsyslog's root philosophy. Let me explain: The syslogd is an extremely important piece of software. If it is not functioning, a) vital data can be lost b) the system as whole may hang (due to the fact that the log socket gets filled up and so other processes block on it) As such, rsyslogd is written in a sense that it tries to continue to work under almost all circumstances. It even tries to survive a system-wide low memory condition, though I am not 100% convinced it will manage to do that in all cases. At least it is coded careful to survive as much as possible. Now let's assume that a configuration error happens that prevents rsyslogd from executing an otherwise perfect configuration. Should we actually stop it from running? And live with the results? Or should we do whatever we can do and do that as good as possible? What, if it is not a configuration issue but a hard disk failure or an admin error that make part of the config unavailable (e.g. not able to load module, not able to write file)? Assuming we can carry out other actions - should we really abort? This may even come at the price that we can NOT warn users (via wall) of a serious problem. And what with warnings? Things that look wrong, but may be OK. Reason to abort? Or continue run? Tough decision. So I still think the right thing to do is to run as long as there is limited sense in it, because everything else would actually be worse. Please note that rsyslogd even comes with a hardcoded emergency configuration which is used in case the real configuration file can not be obtained. What I could add, however, is an option to make it stop on any config error (not warning). However, I fear that, if turned on, the results can be fatal in some cases... > > 2 - This will avoid a lot of silly questions without any supporting > information. At least when you get a complaint like "Rsyslogd won't > start!" it will generally include a "It just says 'configuration file > invalid: broken syntax at line 135.'" For most users, this kind of > error reporting will be enough to lead them to a solution. If nothing > else, it removes the need to explain how to find the errors. > > 3 - For those more subtle problems, and things you might miss upon > bootup. This also gives you a simple, easily accessible debugging > source with minimal security implications and additional code and no > required user configuration. I'm not sure how the code is written, but > I'd prefer to see this happen even if the configuration file can't be > read. Perhaps an $RsyslogErrors directive that, by default, references > a different module that just dumps directly to a file? Users can > configure it to put rsyslog errors through the actual logging engine > and then manage it with typical syslog selector/action lines. All internally-generated messages are run through a specific interface. For the "diag plugin", I will define a hook, where a diag server can register to receive these messages. Maybe a good compromise would be to add this $RsyslogErrors directive and make it point to a file. So, still, we do not necessarily have it, but package maintainers would hopefully include it and point it to a specific file (what still means I do not necessarily know where to look). A problem is source tarball install, in which case the directive may not be set. But with a proper sample rsyslog.conf... > The HTTP server is an interesting idea, but not something I would make > use of. The potential security problems are enormous, and it would > introduce too many additional factors to the debugging process: is the > firewall right? Was there already a process listening on that port? Is > the user's configuration file correct? These are the kinds of things > you already have to deal with on rsyslog itself - why force the > debugging process to go through them again? I see (and share) much of your argument. This is why I stayed away for such a long time. But if you think a bit more about it, it has *very interesting* capabilities. Eg. I may also use it to gather some real-time view of internal state, emit commands like HUP and do a number of other nice things. Again, security is problematic. In any case, I'll give it a try. I have started to write a small plugin which will dump the error messages within 10 minutes after startup. It will not be a real http server but rather a listener that response to anyone who connects to it (so that nc can be used as a client ;)). This is very rough, but it has two side-effects: a) it provides something to actually experiment with (and to extend if we decide to keep that route) b) it helps create the plumbing inside the internal interface that could also be used by any other method, so the potential waste of time is limited So while I object some suggestions, I am very interested in a discussion. The rsyslog "keep running" philosophy *is* a discussion topic, just expect me to argue strongly in favor of it (pls think about the HIV analogy to see why I think we have a unique situation in case of a syslogd). Thanks for the good post, and I guess I just wrote another long mail ;) Rainer > > Hope that helps, and sorry for the long email. > -HKS > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > wrote: > > Thanks, HKS, for the good answers. There is one thing I would like to > > bring to your attention: I often see that people do not check for > > rsyslog error messages themselves. That complicates setup. I do not > > blame anyone, but would like to make you aware of the problem. > > > > I have just blogged about a potential (partial) solution and will try to > > implement it: > > > > http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > > ml > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of (private) HKS > >> Sent: Thursday, July 24, 2008 11:36 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Help with Ommail Configuration > >> > >> Oh - one other possibility. rsyslogd has to have mail enabled at > >> compile time to work with it at runtime. check your catchall logfile - > >> if it has messages like this, you didn't compile it in: > >> > >> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > >> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > >> file: No such file or directory > >> > >> To fix this, you'll need to reinstall. This time, when you run > >> ./configure be sure to include --enable-mail. make install clean and > >> you should be good to go... > >> > >> -HKS > >> > >> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS > >> wrote: > >> > A few things to try: > >> > > >> > - You load ommail.so twice, once at the top and once right above > >> your > >> > $ActionMail... lines. I don't think this will break it, but it's > >> > unnecessary - delete the second one. > >> > > >> > - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > >> > to attempt sending a message once every 6 hours. For testing > >> purposes, > >> > this will be obnoxious. Until you're ready for production, change it > >> > to: > >> > $ActionExecOnlyOnceEveryInterval 30 > >> > > >> > - Send everything to make sure you're not missing it based on > >> > something in the property-replacer/templates/whatever. Replace "if > >> > $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > > with: > >> > *.* :ommail:;mailBody > >> > > >> > Try again. Try logging a few messages from the localhost first (with > >> > RHEL you can just run "logger test" to log a user.notice message > > with > >> > content "test") and see if you get them. > >> > > >> > If you don't, check the mail logs on your mail server to see if it > >> > ever received the message. If not, it's time to break out tcpdump > > and > >> > see if the packets are ever being generated. > >> > > >> > Hope that helps. > >> > > >> > -HKS > >> > > >> > > >> > > >> > On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > >> > wrote: > >> >> Hello all, > >> >> > >> >> First off, I am not very Linux savvy. I don't have a lot of > >> experience > >> >> other then a basic course. This is probably way past my knowledge, > >> but I > >> >> really need to get it done. Appreciate any help you guys have to > >> offer. > >> >> > >> >> I am working on a Red Hat Enterprise 4 box and I am running the > >> latest > >> >> edition of rsyslog. I currently have rsyslog configured to receive > >> >> messages remotely via UDP. I am trying to set it up so that it will > >> send > >> >> out E-mail messages to the system Admin's based on the severity > >> level of > >> >> the log files I am receiving. I would like it so that any device > >> that > >> >> sends a log with a critical, alert, or emergency level facility > > will > >> >> send out an e-mail to a specific address. > >> >> > >> >> Here is my rsyslog.conf file. I used the sample code from Rainer > >> >> Gerhards configuration page. I tried sending a test syslog with > >> 'hard > >> >> disk fatal failure' in it, but it is not sending out mail. Also > >> tried > >> >> without the templates below thinking it would just send me an email > >> for > >> >> every syslog that I received, but it doesn't appear to be sending > >> mail. > >> >> Any thoughts on what I am doing wrong. I'm sure there is a lot I > >> need to > >> >> do, so please let me know. Thanks! > >> >> > >> >> $template mailSubject,"disk problem on %hostname%" > >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >> >> > >> >> I edited out the 3 lines below in the code for security reasons.. > >> >> $ActionMailSMTPServer > >> >> $ActionMailFrom > >> >> $ActionMailTo > >> >> > >> >> > >> >> > >> >> Here is my code from rsyslog.conf below. > >> >> > >> >> > >> >> > >> >> # if you experience problems, check > >> >> # http://www.rsyslog.com/troubleshoot for assistance > >> >> > >> >> # rsyslog v3: load input modules > >> >> # If you do not load inputs, nothing happens! > >> >> # You may need to set the module load path if modules are not > > found. > >> >> > >> >> $ModLoad immark.so # provides --MARK-- message capability > >> >> $ModLoad imuxsock.so # provides support for local system logging > >> (e.g. > >> >> via logger command) > >> >> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >> >> $ModLoad ommail > >> >> > >> >> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >> >> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >> >> > >> >> # Log all kernel messages to the console. > >> >> # Logging much else clutters up the screen. > >> >> #kern.* > > /dev/console > >> >> > >> >> # Log anything (except mail) of level info or higher. > >> >> # Don't log private authentication messages! > >> >> *.info;mail.none;authpriv.none;cron.none > >> >> -/var/log/messages > >> >> > >> >> # The authpriv file has restricted access. > >> >> authpriv.* > >> /var/log/secure > >> >> > >> >> # Log all the mail messages in one place. > >> >> mail.* > >> >> -/var/log/maillog > >> >> > >> >> # Log cron stuff > >> >> cron.* - > >> /var/log/cron > >> >> > >> >> # Everybody gets emergency messages > >> >> *.emerg * > >> >> > >> >> # Save news errors of level crit and higher in a special file. > >> >> uucp,news.crit > >> >> -/var/log/spooler > >> >> > >> >> # Save boot messages also to boot.log > >> >> local7.* > >> >> /var/log/boot.log > >> >> > >> >> #Catch all incoming syslog messages > >> >> *.* > >> >> /var/log/catchall;TraditionalFormatWithPRI > >> >> > >> >> # Remote Logging (we use TCP for reliable delivery) > >> >> # An on-disk queue is created for this action. If the remote host > > is > >> >> # down, messages are spooled to disk and sent when it is up again. > >> >> $WorkDirectory /rsyslog/spool # where to place spool files > >> >> $ActionQueueFileName uniqName # unique name prefix for spool files > >> >> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >> >> possible) > >> >> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >> >> $ActionQueueType LinkedList # run asynchronously > >> >> $ActionResumeRetryCount -1 # infinite retries if host is down > >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> >> *.* @10.57.106.140:514 > >> >> > >> >> $ModLoad ommail > >> >> $ActionMailSMTPServer > >> >> $ActionMailFrom > >> >> $ActionMailTo > >> >> $template mailSubject,"disk problem on %hostname%" > >> >> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >> >> $ActionMailSubject mailSubject > >> >> # make sure we receive a mail only once in six > >> >> # hours (21,600 seconds ;)) > >> >> $ActionExecOnlyOnceEveryInterval 21600 > >> >> # the if ... then ... mailBody mus be on one line! > >> >> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >> >> > >> >> > >> >> # ######### Receiving Messages from Remote Hosts ########## > >> >> # TCP Syslog Server: > >> >> # provides TCP syslog reception and GSS-API (if compiled to support > >> it) > >> >> $ModLoad imtcp.so # load module > >> >> $InputTCPServerRun 514 # start up TCP listener at port 514 > >> >> > >> >> # UDP Syslog Server: > >> >> $ModLoad imudp.so # provides UDP syslog reception > >> >> $UDPServerRun 514 # start a UDP syslog server at standard port > >> >> -------------------------------------------------------- > >> >> This e-mail, including any attachments, may be confidential, > >> privileged or otherwise legally protected. If you have received this > > e- > >> mail in error, or from someone who was not authorized to send it to > >> you, do not disseminate, copy or otherwise use this e-mail or its > >> attachments. Please notify the sender immediately if you have received > >> this e-mail by mistake, and delete it from your system. > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Jul 25 17:25:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 17:25:31 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: Message-ID: <1216999531.7184.219.camel@rgf9dev.intern.adiscon.com> Hi Rio-san, On Fri, 2008-07-25 at 23:56 +0900, Ryo Fujita wrote: > Hi HKS-san, > > +1 > > > 2 - This will avoid a lot of silly questions without any supporting > > information. At least when you get a complaint like "Rsyslogd won't > > start!" it will generally include a "It just says 'configuration file > > invalid: broken syntax at line 135.' > > In addition, > '... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.' Same thought over here :) The recent build even emits live links to troubleshooting information. Log lines look like this: 2008-07-25T17:22:18.523900+02:00 rgf9dev rsyslogd-3003: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] Note the part in the bracket at the end of the message. That repository is far from being complete, but I hope more and more information will be added. I hope this is a useful addition. Together with the error number, we can hopefully most often pinpoint the source of the problem (though the 3003 above is a good example of where it may not be possible to tell the exact cause, but for sure give a good hint). BTW: this is also one thing that makes me think about the scripting engine. It shall provide better error reporting capabilties, but that's harder than it initially sounds ;) Rainer > > Any ideas? > > Best Rio. > > On 2008/07/25, at 23:33, (private) HKS wrote: > > > Hi Rainer, > > > > Interesting post. The tendency to ignore errors/diagnostics is not > > limited to rsyslog, it's something that plagues pretty much every > > piece of software ever created anywhere. Some users are ignorant of > > basic diagnostic tools like syslog, --help flags, tcpdump, and man > > pages. Others just need a bit of prodding before they slap themselves > > and realize that they forgot some basic troubleshooting. Some are lazy > > slobs. > > > > However, rsyslog lends itself to error reports that lack > > troubleshooting for one huge reason: errors are not reliably reported > > in the default configuration. It's happened a few times that I start > > rsyslogd at the command line, and yet there's nothing in the process > > table, and nothing dumped to my logs. I've been able to track down the > > problem with -dn switches, but that involves weeding through a lot of > > irrelevant information. > > > > There are three modifications I'd love to see that would help > > resolve this: > > > > 1 - Configuration errors should be fatal > > 2 - Configuration and other runtime errors should be printed to STDERR > > 3 - By default, rsyslogd should log all of its own errors (fatal or > > not) to /var/log/rsyslog.log. This should also be user-configurable > > > > My reasoning: > > > > 1 - If rsyslog doesn't understand your config file, obviously it won't > > be doing what you intend it to. There's little benefit in running with > > a horked configuration, but if you don't test thoroughly, the cost > > could be devastating. Some users won't know they're not logging things > > until they need them - then, if they keep their jobs, they probably > > won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > > > 2 - This will avoid a lot of silly questions without any supporting > > information. At least when you get a complaint like "Rsyslogd won't > > start!" it will generally include a "It just says 'configuration file > > invalid: broken syntax at line 135.'" For most users, this kind of > > error reporting will be enough to lead them to a solution. If nothing > > else, it removes the need to explain how to find the errors. > > > > 3 - For those more subtle problems, and things you might miss upon > > bootup. This also gives you a simple, easily accessible debugging > > source with minimal security implications and additional code and no > > required user configuration. I'm not sure how the code is written, but > > I'd prefer to see this happen even if the configuration file can't be > > read. Perhaps an $RsyslogErrors directive that, by default, references > > a different module that just dumps directly to a file? Users can > > configure it to put rsyslog errors through the actual logging engine > > and then manage it with typical syslog selector/action lines. > > > > The HTTP server is an interesting idea, but not something I would make > > use of. The potential security problems are enormous, and it would > > introduce too many additional factors to the debugging process: is the > > firewall right? Was there already a process listening on that port? Is > > the user's configuration file correct? These are the kinds of things > > you already have to deal with on rsyslog itself - why force the > > debugging process to go through them again? > > > > Hope that helps, and sorry for the long email. > > -HKS > > > > > > > > > > On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards > > wrote: > >> Thanks, HKS, for the good answers. There is one thing I would like to > >> bring to your attention: I often see that people do not check for > >> rsyslog error messages themselves. That complicates setup. I do not > >> blame anyone, but would like to make you aware of the problem. > >> > >> I have just blogged about a potential (partial) solution and will > >> try to > >> implement it: > >> > >> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht > >> ml > >> > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of (private) HKS > >>> Sent: Thursday, July 24, 2008 11:36 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] Help with Ommail Configuration > >>> > >>> Oh - one other possibility. rsyslogd has to have mail enabled at > >>> compile time to work with it at runtime. check your catchall > >>> logfile - > >>> if it has messages like this, you didn't compile it in: > >>> > >>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so', > >>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object > >>> file: No such file or directory > >>> > >>> To fix this, you'll need to reinstall. This time, when you run > >>> ./configure be sure to include --enable-mail. make install clean and > >>> you should be good to go... > >>> > >>> -HKS > >>> > >>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS >>> > > >>> wrote: > >>>> A few things to try: > >>>> > >>>> - You load ommail.so twice, once at the top and once right above > >>> your > >>>> $ActionMail... lines. I don't think this will break it, but it's > >>>> unnecessary - delete the second one. > >>>> > >>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going > >>>> to attempt sending a message once every 6 hours. For testing > >>> purposes, > >>>> this will be obnoxious. Until you're ready for production, change > >>>> it > >>>> to: > >>>> $ActionExecOnlyOnceEveryInterval 30 > >>>> > >>>> - Send everything to make sure you're not missing it based on > >>>> something in the property-replacer/templates/whatever. Replace "if > >>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody" > >> with: > >>>> *.* :ommail:;mailBody > >>>> > >>>> Try again. Try logging a few messages from the localhost first > >>>> (with > >>>> RHEL you can just run "logger test" to log a user.notice message > >> with > >>>> content "test") and see if you get them. > >>>> > >>>> If you don't, check the mail logs on your mail server to see if it > >>>> ever received the message. If not, it's time to break out tcpdump > >> and > >>>> see if the packets are ever being generated. > >>>> > >>>> Hope that helps. > >>>> > >>>> -HKS > >>>> > >>>> > >>>> > >>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB) > >>>> wrote: > >>>>> Hello all, > >>>>> > >>>>> First off, I am not very Linux savvy. I don't have a lot of > >>> experience > >>>>> other then a basic course. This is probably way past my knowledge, > >>> but I > >>>>> really need to get it done. Appreciate any help you guys have to > >>> offer. > >>>>> > >>>>> I am working on a Red Hat Enterprise 4 box and I am running the > >>> latest > >>>>> edition of rsyslog. I currently have rsyslog configured to receive > >>>>> messages remotely via UDP. I am trying to set it up so that it > >>>>> will > >>> send > >>>>> out E-mail messages to the system Admin's based on the severity > >>> level of > >>>>> the log files I am receiving. I would like it so that any device > >>> that > >>>>> sends a log with a critical, alert, or emergency level facility > >> will > >>>>> send out an e-mail to a specific address. > >>>>> > >>>>> Here is my rsyslog.conf file. I used the sample code from Rainer > >>>>> Gerhards configuration page. I tried sending a test syslog with > >>> 'hard > >>>>> disk fatal failure' in it, but it is not sending out mail. Also > >>> tried > >>>>> without the templates below thinking it would just send me an > >>>>> email > >>> for > >>>>> every syslog that I received, but it doesn't appear to be sending > >>> mail. > >>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I > >>> need to > >>>>> do, so please let me know. Thanks! > >>>>> > >>>>> $template mailSubject,"disk problem on %hostname%" > >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"' > >>>>> > >>>>> I edited out the 3 lines below in the code for security reasons.. > >>>>> $ActionMailSMTPServer > >>>>> $ActionMailFrom > >>>>> $ActionMailTo > >>>>> > >>>>> > >>>>> > >>>>> Here is my code from rsyslog.conf below. > >>>>> > >>>>> > >>>>> > >>>>> # if you experience problems, check > >>>>> # http://www.rsyslog.com/troubleshoot for assistance > >>>>> > >>>>> # rsyslog v3: load input modules > >>>>> # If you do not load inputs, nothing happens! > >>>>> # You may need to set the module load path if modules are not > >> found. > >>>>> > >>>>> $ModLoad immark.so # provides --MARK-- message capability > >>>>> $ModLoad imuxsock.so # provides support for local system logging > >>> (e.g. > >>>>> via logger command) > >>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > >>>>> $ModLoad ommail > >>>>> > >>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated% > >>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" > >>>>> > >>>>> # Log all kernel messages to the console. > >>>>> # Logging much else clutters up the screen. > >>>>> #kern.* > >> /dev/console > >>>>> > >>>>> # Log anything (except mail) of level info or higher. > >>>>> # Don't log private authentication messages! > >>>>> *.info;mail.none;authpriv.none;cron.none > >>>>> -/var/log/messages > >>>>> > >>>>> # The authpriv file has restricted access. > >>>>> authpriv.* > >>> /var/log/secure > >>>>> > >>>>> # Log all the mail messages in one place. > >>>>> mail.* > >>>>> -/var/log/maillog > >>>>> > >>>>> # Log cron stuff > >>>>> cron.* - > >>> /var/log/cron > >>>>> > >>>>> # Everybody gets emergency messages > >>>>> *.emerg * > >>>>> > >>>>> # Save news errors of level crit and higher in a special file. > >>>>> uucp,news.crit > >>>>> -/var/log/spooler > >>>>> > >>>>> # Save boot messages also to boot.log > >>>>> local7.* > >>>>> /var/log/boot.log > >>>>> > >>>>> #Catch all incoming syslog messages > >>>>> *.* > >>>>> /var/log/catchall;TraditionalFormatWithPRI > >>>>> > >>>>> # Remote Logging (we use TCP for reliable delivery) > >>>>> # An on-disk queue is created for this action. If the remote host > >> is > >>>>> # down, messages are spooled to disk and sent when it is up again. > >>>>> $WorkDirectory /rsyslog/spool # where to place spool files > >>>>> $ActionQueueFileName uniqName # unique name prefix for spool files > >>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as > >>>>> possible) > >>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown > >>>>> $ActionQueueType LinkedList # run asynchronously > >>>>> $ActionResumeRetryCount -1 # infinite retries if host is down > >>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port > >>>>> optional > >>>>> *.* @10.57.106.140:514 > >>>>> > >>>>> $ModLoad ommail > >>>>> $ActionMailSMTPServer > >>>>> $ActionMailFrom > >>>>> $ActionMailTo > >>>>> $template mailSubject,"disk problem on %hostname%" > >>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" > >>>>> $ActionMailSubject mailSubject > >>>>> # make sure we receive a mail only once in six > >>>>> # hours (21,600 seconds ;)) > >>>>> $ActionExecOnlyOnceEveryInterval 21600 > >>>>> # the if ... then ... mailBody mus be on one line! > >>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody > >>>>> > >>>>> > >>>>> # ######### Receiving Messages from Remote Hosts ########## > >>>>> # TCP Syslog Server: > >>>>> # provides TCP syslog reception and GSS-API (if compiled to > >>>>> support > >>> it) > >>>>> $ModLoad imtcp.so # load module > >>>>> $InputTCPServerRun 514 # start up TCP listener at port 514 > >>>>> > >>>>> # UDP Syslog Server: > >>>>> $ModLoad imudp.so # provides UDP syslog reception > >>>>> $UDPServerRun 514 # start a UDP syslog server at standard port > >>>>> -------------------------------------------------------- > >>>>> This e-mail, including any attachments, may be confidential, > >>> privileged or otherwise legally protected. If you have received this > >> e- > >>> mail in error, or from someone who was not authorized to send it to > >>> you, do not disseminate, copy or otherwise use this e-mail or its > >>> attachments. Please notify the sender immediately if you have > >>> received > >>> this e-mail by mistake, and delete it from your system. > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > ######################################################################## > Ryo Fujita > Senior Solution Architect, RHCE > Red Hat K.K. > TEL +81-3-5798-8500 > FAX +81-3-5798-8599 > Ebisu Neonato 8F > 4-1-18 Ebisu, Shibuya-ku, > Tokyo Japan 1500013 > ######################################################################## > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Fri Jul 25 17:44:18 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 25 Jul 2008 11:44:18 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> Message-ID: Responses inline. > I totally agree, but I think the situation is even worse for a syslogd. > Because it *is* the usual error reporting system. I think we are a bit > in the same situation (if you allow that analogy) like HIV which is > attacking the human immune system. We have a problem in the > troubleshooting system itself, and it may even prevent us from conveying > troubleshooting information :( That's a good way of putting it. This is one of the reasons I think it's so important to print errors to STDERR - if the logging mechanisms are broken for some reason, then at least they're going somewhere. And I sincerely hope that users have the sense to start it from the command line the first time rather than sticking it into their system startup scripts and rebooting. >> It's happened a few times that I start >> rsyslogd at the command line, and yet there's nothing in the process >> table, and nothing dumped to my logs. > > Could you elaborate a bit about above sentence - I have to admit I do > not fully get you... I've messed up my configs bad enough a couple times that when I start rsyslogd, nothing apparently happens. No errors to STDERR, no errors in my logs, and the process isn't running. Rewriting the config or running with -dn is the solution. >> 2 - Configuration and other runtime errors should be printed to STDERR > > Does that really help? If so, that would be extremely easy to add. But > does somebody see them? Absolutely. The first time I configure any daemon, I do it in a test environment and run it by hand until I have the configuration acting exactly as I want. Most users take the "run-by-hand" precaution even if they don't have the luxury of a test setup. These will also pop up when a user tries to restart the daemon after making a config change. I'm not sure if they'll be visible when a HUP signal is sent, but that would make me awfully happy. >> 3 - By default, rsyslogd should log all of its own errors (fatal or >> not) to /var/log/rsyslog.log. This should also be user-configurable > > This default comes into the way of some distros: they use different > pathes. I really don't like the idea of embedding any information about > the file system into the binary. Just think about embedded systems... Good point. >> 1 - If rsyslog doesn't understand your config file, obviously it won't >> be doing what you intend it to. There's little benefit in running with >> a horked configuration, but if you don't test thoroughly, the cost >> could be devastating. Some users won't know they're not logging things >> until they need them - then, if they keep their jobs, they probably >> won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > That's quite controversal and and does not match rsyslog's root > philosophy. Let me explain: > > The syslogd is an extremely important piece of software. If it is not > functioning, > > a) vital data can be lost > b) the system as whole may hang (due to the fact that the log > socket gets filled up and so other processes block on it) > > As such, rsyslogd is written in a sense that it tries to continue to > work under almost all circumstances. It even tries to survive a > system-wide low memory condition, though I am not 100% convinced it will > manage to do that in all cases. At least it is coded careful to survive > as much as possible. > > Now let's assume that a configuration error happens that prevents > rsyslogd from executing an otherwise perfect configuration. Should we > actually stop it from running? And live with the results? Or should we > do whatever we can do and do that as good as possible? What, if it is > not a configuration issue but a hard disk failure or an admin error that > make part of the config unavailable (e.g. not able to load module, not > able to write file)? Assuming we can carry out other actions - should we > really abort? This may even come at the price that we can NOT warn users > (via wall) of a serious problem. And what with warnings? Things that > look wrong, but may be OK. Reason to abort? Or continue run? Tough > decision. So I still think the right thing to do is to run as long as > there is limited sense in it, because everything else would actually be > worse. Please note that rsyslogd even comes with a hardcoded emergency > configuration which is used in case the real configuration file can not > be obtained. > > What I could add, however, is an option to make it stop on any config > error (not warning). However, I fear that, if turned on, the results can > be fatal in some cases... You make a lot of good points here. Does the hardcoded emergency configuration that rsyslogd uses actually log? I only ask because it's easy (if you fail to read the documentation) to write a config file that doesn't do anything. Would it make sense for rsyslog to determine whether the passed configuration actually asks it to do anything, and complaining if not? Printing errors to STDERR would help with this. Another possibility might be adding a flag (-n is typical for a lot of systems, but that's already in use) that will tell rsyslog to parse the configuration, spit out any problems, and quit. That way, administrators can modify a config file on a live system and verify it's syntax without hosing the actual rsyslogd process. >> 3 - For those more subtle problems, and things you might miss upon >> bootup. This also gives you a simple, easily accessible debugging >> source with minimal security implications and additional code and no >> required user configuration. I'm not sure how the code is written, but >> I'd prefer to see this happen even if the configuration file can't be >> read. Perhaps an $RsyslogErrors directive that, by default, references >> a different module that just dumps directly to a file? Users can >> configure it to put rsyslog errors through the actual logging engine >> and then manage it with typical syslog selector/action lines. > > All internally-generated messages are run through a specific interface. > For the "diag plugin", I will define a hook, where a diag server can > register to receive these messages. Maybe a good compromise would be to > add this $RsyslogErrors directive and make it point to a file. So, > still, we do not necessarily have it, but package maintainers would > hopefully include it and point it to a specific file (what still means I > do not necessarily know where to look). A problem is source tarball > install, in which case the directive may not be set. But with a proper > sample rsyslog.conf... I think a locally installed sample rsyslog.conf is a solid workaround for the "different distros, different locations" problem. > >> The HTTP server is an interesting idea, but not something I would make >> use of. The potential security problems are enormous, and it would >> introduce too many additional factors to the debugging process: is the >> firewall right? Was there already a process listening on that port? Is >> the user's configuration file correct? These are the kinds of things >> you already have to deal with on rsyslog itself - why force the >> debugging process to go through them again? > > I see (and share) much of your argument. This is why I stayed away for > such a long time. But if you think a bit more about it, it has *very > interesting* capabilities. Eg. I may also use it to gather some > real-time view of internal state, emit commands like HUP and do a number > of other nice things. Again, security is problematic. > > In any case, I'll give it a try. I have started to write a small plugin > which will dump the error messages within 10 minutes after startup. It > will not be a real http server but rather a listener that response to > anyone who connects to it (so that nc can be used as a client ;)). This > is very rough, but it has two side-effects: > > a) it provides something to actually experiment with (and to extend if > we decide to keep that route) > b) it helps create the plumbing inside the internal interface that could > also be used by any other method, so the potential waste of time is > limited It sounds interesting, and I agree that there's a lot of potential there. Among them, the possibility of adding an XML-RPC API to allow for greater interface flexibility...but frankly, that scares the hell out of me. ;) I'll wait to see what you come up with. -HKS From rgerhards at hq.adiscon.com Fri Jul 25 18:07:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 25 Jul 2008 18:07:51 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> Message-ID: <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> inline... On Fri, 2008-07-25 at 11:44 -0400, (private) HKS wrote: > Responses inline. > > > I totally agree, but I think the situation is even worse for a syslogd. > > Because it *is* the usual error reporting system. I think we are a bit > > in the same situation (if you allow that analogy) like HIV which is > > attacking the human immune system. We have a problem in the > > troubleshooting system itself, and it may even prevent us from conveying > > troubleshooting information :( > > That's a good way of putting it. This is one of the reasons I think > it's so important to print errors to STDERR - if the logging > mechanisms are broken for some reason, then at least they're going > somewhere. And I sincerely hope that users have the sense to start it > from the command line the first time rather than sticking it into > their system startup scripts and rebooting. If I look at some of the support calls I see, I doubt that ;) The problem is that there are a lot of novices who have never ever done any real system administration. But as the rsyslog homepage says: "Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user." Look at the end of the sentence. In any case, I see the utility of what you say, if not in the novice case, then with more experienced users. > >> It's happened a few times that I start > >> rsyslogd at the command line, and yet there's nothing in the process > >> table, and nothing dumped to my logs. > > > > Could you elaborate a bit about above sentence - I have to admit I do > > not fully get you... > > I've messed up my configs bad enough a couple times that when I start > rsyslogd, nothing apparently happens. No errors to STDERR, no errors > in my logs, and the process isn't running. Rewriting the config or > running with -dn is the solution. Ah, ok, now I understand. Thanks. > > > > >> 2 - Configuration and other runtime errors should be printed to STDERR > > > > Does that really help? If so, that would be extremely easy to add. But > > does somebody see them? > > Absolutely. The first time I configure any daemon, I do it in a test > environment and run it by hand until I have the configuration acting > exactly as I want. I understand. That makes a lot of sense. As I said, it should be easy to implement. I'll once again shuffle priorities a bit and make that the top one. Should not take too long to implement. I'll probably have a switch to disable it, but being enabled by default. > Most users take the "run-by-hand" precaution even > if they don't have the luxury of a test setup. These will also pop up > when a user tries to restart the daemon after making a config change. > I'm not sure if they'll be visible when a HUP signal is sent, but that > would make me awfully happy. So expect to be happy some time next week ;) > > > >> 3 - By default, rsyslogd should log all of its own errors (fatal or > >> not) to /var/log/rsyslog.log. This should also be user-configurable > > > > This default comes into the way of some distros: they use different > > pathes. I really don't like the idea of embedding any information about > > the file system into the binary. Just think about embedded systems... > > Good point. > > > >> 1 - If rsyslog doesn't understand your config file, obviously it won't > >> be doing what you intend it to. There's little benefit in running with > >> a horked configuration, but if you don't test thoroughly, the cost > >> could be devastating. Some users won't know they're not logging things > >> until they need them - then, if they keep their jobs, they probably > >> won't keep rsyslog ("This software doesn't even log ssh attempts!"). > > > > That's quite controversal and and does not match rsyslog's root > > philosophy. Let me explain: > > > > The syslogd is an extremely important piece of software. If it is not > > functioning, > > > > a) vital data can be lost > > b) the system as whole may hang (due to the fact that the log > > socket gets filled up and so other processes block on it) > > > > As such, rsyslogd is written in a sense that it tries to continue to > > work under almost all circumstances. It even tries to survive a > > system-wide low memory condition, though I am not 100% convinced it will > > manage to do that in all cases. At least it is coded careful to survive > > as much as possible. > > > > Now let's assume that a configuration error happens that prevents > > rsyslogd from executing an otherwise perfect configuration. Should we > > actually stop it from running? And live with the results? Or should we > > do whatever we can do and do that as good as possible? What, if it is > > not a configuration issue but a hard disk failure or an admin error that > > make part of the config unavailable (e.g. not able to load module, not > > able to write file)? Assuming we can carry out other actions - should we > > really abort? This may even come at the price that we can NOT warn users > > (via wall) of a serious problem. And what with warnings? Things that > > look wrong, but may be OK. Reason to abort? Or continue run? Tough > > decision. So I still think the right thing to do is to run as long as > > there is limited sense in it, because everything else would actually be > > worse. Please note that rsyslogd even comes with a hardcoded emergency > > configuration which is used in case the real configuration file can not > > be obtained. > > > > What I could add, however, is an option to make it stop on any config > > error (not warning). However, I fear that, if turned on, the results can > > be fatal in some cases... > > You make a lot of good points here. Does the hardcoded emergency > configuration that rsyslogd uses actually log? Yes, but in a very limited sense. It puts *.err onto the console, does a wall for *.panic and writes everything else to the controlling tty. I have to admit that this code isn't awfully well tested, usually only when something is done at the config code (and sometimes not even then...). > I only ask because it's > easy (if you fail to read the documentation) to write a config file > that doesn't do anything. Would it make sense for rsyslog to determine > whether the passed configuration actually asks it to do anything, and > complaining if not? That's an interesting idea. I am not sure if this is already checked. I can at least check if there is no action at all, which means there is no useful work to be done. The question remains, as usual, where to complain to. But that could be another situation where the hardcoded config could be activated. I have also thought about an implicit write user, with the user being root by default (and being overwritable via command line and config file). > > Printing errors to STDERR would help with this. Another possibility > might be adding a flag (-n is typical for a lot of systems, but that's > already in use) that will tell rsyslog to parse the configuration, > spit out any problems, and quit. That way, administrators can modify a > config file on a live system and verify it's syntax without hosing the > actual rsyslogd process. That's also a good idea. Rsyslog is quite modular (not yet as much as I would like it to be, but well enough). So I could even generate a dedicated config check tool. That would not touch any running configuration at all. But maybe the switch is easier ;) I need to think on how this affects the testbed I have begun to work on. Looks like this could also drive some automated tests. > > > > >> 3 - For those more subtle problems, and things you might miss upon > >> bootup. This also gives you a simple, easily accessible debugging > >> source with minimal security implications and additional code and no > >> required user configuration. I'm not sure how the code is written, but > >> I'd prefer to see this happen even if the configuration file can't be > >> read. Perhaps an $RsyslogErrors directive that, by default, references > >> a different module that just dumps directly to a file? Users can > >> configure it to put rsyslog errors through the actual logging engine > >> and then manage it with typical syslog selector/action lines. > > > > All internally-generated messages are run through a specific interface. > > For the "diag plugin", I will define a hook, where a diag server can > > register to receive these messages. Maybe a good compromise would be to > > add this $RsyslogErrors directive and make it point to a file. So, > > still, we do not necessarily have it, but package maintainers would > > hopefully include it and point it to a specific file (what still means I > > do not necessarily know where to look). A problem is source tarball > > install, in which case the directive may not be set. But with a proper > > sample rsyslog.conf... > > I think a locally installed sample rsyslog.conf is a solid workaround > for the "different distros, different locations" problem. > > > > >> The HTTP server is an interesting idea, but not something I would make > >> use of. The potential security problems are enormous, and it would > >> introduce too many additional factors to the debugging process: is the > >> firewall right? Was there already a process listening on that port? Is > >> the user's configuration file correct? These are the kinds of things > >> you already have to deal with on rsyslog itself - why force the > >> debugging process to go through them again? > > > > I see (and share) much of your argument. This is why I stayed away for > > such a long time. But if you think a bit more about it, it has *very > > interesting* capabilities. Eg. I may also use it to gather some > > real-time view of internal state, emit commands like HUP and do a number > > of other nice things. Again, security is problematic. > > > > In any case, I'll give it a try. I have started to write a small plugin > > which will dump the error messages within 10 minutes after startup. It > > will not be a real http server but rather a listener that response to > > anyone who connects to it (so that nc can be used as a client ;)). This > > is very rough, but it has two side-effects: > > > > a) it provides something to actually experiment with (and to extend if > > we decide to keep that route) > > b) it helps create the plumbing inside the internal interface that could > > also be used by any other method, so the potential waste of time is > > limited > > It sounds interesting, and I agree that there's a lot of potential > there. Among them, the possibility of adding an XML-RPC API to allow > for greater interface flexibility...but frankly, that scares the hell > out of me. ;) I'll wait to see what you come up with. hehe... We'll see, but I'd say its faaar down the road. One interesting aspect, though, could be a realtime view of syslog data as it is flowing through the system. That could even be integrated into phpLogCon. In any case, I do NOT want to add much (http) application logic / presentation to such a system. But a few html (or xml ;)) pages generated at appropriate times could be really useful. Just think about things like a "granular HUP", like being able to tell to close some files specifically etc etc. But that's too far advanced. For me personally, it would be quite useful to get access to state variables (like mem utilization, object instances, error count, frame rate) of the running system - not under a debugger but in a real live system. From the developers (and troubleshooter's point of view), that would be very interesting. I could even hold the last 2000 or so messages of the debug log in memory, plus the initial first 1000. Together, I guess, they can solve over 90% of all things I have used debug logs for. But this can only be done if I have dynamic runtime access to the live instance. And this obviously also means there is a big security issue ;) Please keep the thoughts flowing, we make good progess :) Rainer > > -HKS > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Mon Jul 28 17:54:04 2008 From: hks.private at gmail.com ((private) HKS) Date: Mon, 28 Jul 2008 11:54:04 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> Message-ID: I don't have much more to add to this discussion - I think we're pretty much on the same page. I'm looking forward to seeing the results of your work. -HKS On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards wrote: > > > Please keep the thoughts flowing, we make good progess :) > Rainer From rory at ooma.com Tue Jul 29 01:11:52 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 16:11:52 -0700 Subject: [rsyslog] error with phplogcon Message-ID: <488E5238.9070606@ooma.com> The installation instructions say I need php, apache and mysql. When I run the installation for phplogcon, it finishes and then I get an error: Errordetails: Error, MYSQL Extensions are not enabled! Function 'mysql_connect' does not exist. There are several things I could install here... I installed the mysql-connector-odbc rpm, but this may or may not be the right thing. It still doesn't work, and docs on this are pretty scarce. So, what am I missing? Which rpm should I be using to get the mysql_connect function? From samuel at dragonboricua.net Tue Jul 29 02:06:53 2008 From: samuel at dragonboricua.net (Elisamuel Resto) Date: Mon, 28 Jul 2008 20:06:53 -0400 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E5238.9070606@ooma.com> References: <488E5238.9070606@ooma.com> Message-ID: <20080728200653.38ae2bed@hades.dragonboricua.com> On Mon, 28 Jul 2008 16:11:52 -0700, Rory Toma wrote: > The installation instructions say I need php, apache and mysql. > > When I run the installation for phplogcon, it finishes and then I get an > error: > > Errordetails: > Error, MYSQL Extensions are not enabled! Function 'mysql_connect' does > not exist. > > There are several things I could install here... I installed the > mysql-connector-odbc rpm, but this may or may not be the right thing. It > still doesn't work, and docs on this are pretty scarce. So, what am I > missing? Which rpm should I be using to get the mysql_connect function? I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, depending on your installed php version). ODBC connector is quite a different thing. -- Elisamuel Resto Source Mage General Guru / http://sourcemage.org GPG ID: 18615F19/1024D / http://simplysam.us From rory at ooma.com Tue Jul 29 02:48:52 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 17:48:52 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080728200653.38ae2bed@hades.dragonboricua.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> Message-ID: <488E68F4.20903@ooma.com> Yes, turns out after random google searching that I needed php-mysql. I am past this stage now. However... I now get at error "No syslog records found. - Error Details Unknown or unhandled error occurred" I can connect manually and do a select * from the SystemEvents db and see records, using the user/passwd combo. One thing I couldn't find... What is supposed to go into the field "Table type"? I put in "rsyslog" but I'm not sure what is supposed to go there. thx > > I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, > depending on your installed php version). ODBC connector is quite a different > thing. > > From mic at npgx.com.au Tue Jul 29 02:56:11 2008 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 29 Jul 2008 11:56:11 +1100 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E68F4.20903@ooma.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> Message-ID: <20080729005501.M56815@npgx.com.au> Hi, > Yes, turns out after random google searching that I needed php- > mysql. I am past this stage now. > > However... > > I now get at error > > "No syslog records found. - Error Details > Unknown or unhandled error occurred" I get this error all the time when I install phplogcon. It's just the case of the "systemevents" name in the config file, I change it to "SystemEvents" and it works. Maybe it's an idea to post your config to the list (just the last bit at the bottom) so we can see what you're doing. Regards, Michael. > I can connect manually and do a select * from the SystemEvents db > and see records, using the user/passwd combo. > > One thing I couldn't find... What is supposed to go into the field > "Table type"? I put in "rsyslog" but I'm not sure what is supposed > to go there. > > thx > > > > I believe it would be php-mysql or php#-mysql (where # is either 4 or 5, > > depending on your installed php version). ODBC connector is quite a different > > thing. > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From rory at ooma.com Tue Jul 29 03:02:19 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 18:02:19 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080729005501.M56815@npgx.com.au> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> Message-ID: <488E6C1B.8000208@ooma.com> Here ya go, thx. $CFG['Sources']['Source1']['ID'] = 'Source1'; $CFG['Sources']['Source1']['Name'] = 'rsyslog'; $CFG['Sources']['Source1']['SourceType'] = 2; $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; $CFG['Sources']['Source1']['DBType'] = '0'; $CFG['Sources']['Source1']['DBServer'] = 'localhost'; $CFG['Sources']['Source1']['DBName'] = 'Syslog'; $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; From mic at npgx.com.au Tue Jul 29 03:58:11 2008 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 29 Jul 2008 12:58:11 +1100 Subject: [rsyslog] error with phplogcon In-Reply-To: <488E6C1B.8000208@ooma.com> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> <488E6C1B.8000208@ooma.com> Message-ID: <20080729015636.M15076@npgx.com.au> Hi, > Here ya go, thx. > > $CFG['Sources']['Source1']['ID'] = 'Source1'; > $CFG['Sources']['Source1']['Name'] = 'rsyslog'; > $CFG['Sources']['Source1']['SourceType'] = 2; > $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; > $CFG['Sources']['Source1']['DBType'] = '0'; > $CFG['Sources']['Source1']['DBServer'] = 'localhost'; > $CFG['Sources']['Source1']['DBName'] = 'Syslog'; > $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; > $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; > $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; > $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; This is mine: $CFG['DefaultSourceID'] = 'Source1'; $CFG['Sources']['Source1']['ID'] = 'Source1'; $CFG['Sources']['Source1']['Name'] = 'somename'; $CFG['Sources']['Source1']['ViewID'] = 'SYSLOG'; $CFG['Sources']['Source1']['SourceType'] = SOURCE_DB; $CFG['Sources']['Source1']['DBTableType'] = 'monitorware'; $CFG['Sources']['Source1']['DBServer'] = 'somedbserver'; $CFG['Sources']['Source1']['DBName'] = 'somedbname'; $CFG['Sources']['Source1']['DBUser'] = 'someuser'; $CFG['Sources']['Source1']['DBPassword'] = 'longstringofcharacters'; $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; Regards, Michael. From rory at ooma.com Tue Jul 29 04:02:54 2008 From: rory at ooma.com (Rory Toma) Date: Mon, 28 Jul 2008 19:02:54 -0700 Subject: [rsyslog] error with phplogcon In-Reply-To: <20080729015636.M15076@npgx.com.au> References: <488E5238.9070606@ooma.com> <20080728200653.38ae2bed@hades.dragonboricua.com> <488E68F4.20903@ooma.com> <20080729005501.M56815@npgx.com.au> <488E6C1B.8000208@ooma.com> <20080729015636.M15076@npgx.com.au> Message-ID: <488E7A4E.3030103@ooma.com> Thx, that fixed it. I didn't iterate over each difference, but... thx again. Michael Mansour wrote: > Hi, > > >> Here ya go, thx. >> >> $CFG['Sources']['Source1']['ID'] = 'Source1'; >> $CFG['Sources']['Source1']['Name'] = 'rsyslog'; >> $CFG['Sources']['Source1']['SourceType'] = 2; >> $CFG['Sources']['Source1']['DBTableType'] = 'rsyslog'; >> $CFG['Sources']['Source1']['DBType'] = '0'; >> $CFG['Sources']['Source1']['DBServer'] = 'localhost'; >> $CFG['Sources']['Source1']['DBName'] = 'Syslog'; >> $CFG['Sources']['Source1']['DBUser'] = 'syslogreader'; >> $CFG['Sources']['Source1']['DBPassword'] = 'somecleverpasswd'; >> $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; >> $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; >> > > This is mine: > > $CFG['DefaultSourceID'] = 'Source1'; > > $CFG['Sources']['Source1']['ID'] = 'Source1'; > $CFG['Sources']['Source1']['Name'] = 'somename'; > $CFG['Sources']['Source1']['ViewID'] = 'SYSLOG'; > $CFG['Sources']['Source1']['SourceType'] = SOURCE_DB; > $CFG['Sources']['Source1']['DBTableType'] = 'monitorware'; > $CFG['Sources']['Source1']['DBServer'] = 'somedbserver'; > $CFG['Sources']['Source1']['DBName'] = 'somedbname'; > $CFG['Sources']['Source1']['DBUser'] = 'someuser'; > $CFG['Sources']['Source1']['DBPassword'] = 'longstringofcharacters'; > $CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents'; > $CFG['Sources']['Source1']['DBEnableRowCounting'] = false; > > Regards, > > Michael. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Tue Jul 29 10:10:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 10:10:45 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> I have implemented much yesterday. It is available via git right now: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05b 76862ea2e211cc09b30fe I will create a testbed for it today and hopefully be able to release it some time this afternoon. Unfortunately, it's a busy day... Feedback is appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Monday, July 28, 2008 5:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > I don't have much more to add to this discussion - I think we're > pretty much on the same page. I'm looking forward to seeing the > results of your work. > > -HKS > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > wrote: > > > > > > Please keep the thoughts flowing, we make good progess :) > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 29 15:57:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 15:57:31 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> OK, I have now a preview tarball available. I'd appreciate if you could give it a try. Use -N1 on the command line to make it do a config check, only. It is available here: http://download.rsyslog.com/download/rsyslog-3.21.1.tar.gz md5sum: I will replace that file once the "official" 3.21.1 is out. I have one problem with autotools: a "make distcheck" fails, as far as I can see because rsyslogd is not built inside the tools subdirectory. If someone has a suggestion for a fix (or what could cause the problem), I would appreciate that. Actually this "make distcheck" problem is what made me hold back the official release. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 29, 2008 10:11 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > I have implemented much yesterday. It is available via git right now: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05 > b > 76862ea2e211cc09b30fe > > I will create a testbed for it today and hopefully be able to release > it > some time this afternoon. Unfortunately, it's a busy day... > > Feedback is appreciated. > Rainer > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of (private) HKS > > Sent: Monday, July 28, 2008 5:54 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog error messages > > > > I don't have much more to add to this discussion - I think we're > > pretty much on the same page. I'm looking forward to seeing the > > results of your work. > > > > -HKS > > > > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > > wrote: > > > > > > > > > Please keep the thoughts flowing, we make good progess :) > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Jul 29 15:58:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 29 Jul 2008 15:58:36 +0200 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com><1217002071.7184.237.camel@rgf9dev.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EED9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEDA@grfint2.intern.adiscon.com> It would have been better if I had actually included the md5sum ;). It is b9ca041c00d093981e6c6e35d360e6e9 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, July 29, 2008 3:58 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog error messages > > OK, I have now a preview tarball available. I'd appreciate if you could > give it a try. > > Use -N1 on the command line to make it do a config check, only. > > It is available here: > > http://download.rsyslog.com/download/rsyslog-3.21.1.tar.gz > md5sum: > > I will replace that file once the "official" 3.21.1 is out. > > I have one problem with autotools: a "make distcheck" fails, as far as > I > can see because rsyslogd is not built inside the tools subdirectory. If > someone has a suggestion for a fix (or what could cause the problem), I > would appreciate that. Actually this "make distcheck" problem is what > made me hold back the official release. > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Tuesday, July 29, 2008 10:11 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog error messages > > > > I have implemented much yesterday. It is available via git right now: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05 > > b > > 76862ea2e211cc09b30fe > > > > I will create a testbed for it today and hopefully be able to release > > it > > some time this afternoon. Unfortunately, it's a busy day... > > > > Feedback is appreciated. > > Rainer > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of (private) HKS > > > Sent: Monday, July 28, 2008 5:54 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog error messages > > > > > > I don't have much more to add to this discussion - I think we're > > > pretty much on the same page. I'm looking forward to seeing the > > > results of your work. > > > > > > -HKS > > > > > > > > > On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards > > > wrote: > > > > > > > > > > > > Please keep the thoughts flowing, we make good progess :) > > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hks.private at gmail.com Tue Jul 29 16:14:08 2008 From: hks.private at gmail.com ((private) HKS) Date: Tue, 29 Jul 2008 10:14:08 -0400 Subject: [rsyslog] rsyslog error messages In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> References: <1216998972.7184.213.camel@rgf9dev.intern.adiscon.com> <1217002071.7184.237.camel@rgf9dev.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44EEBE@grfint2.intern.adiscon.com> Message-ID: I'll test this as soon as I can, but it may be a couple days... -HKS On Tue, Jul 29, 2008 at 4:10 AM, Rainer Gerhards wrote: > I have implemented much yesterday. It is available via git right now: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d2feb7063e73938c05b > 76862ea2e211cc09b30fe > > I will create a testbed for it today and hopefully be able to release it > some time this afternoon. Unfortunately, it's a busy day... > > Feedback is appreciated. > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of (private) HKS >> Sent: Monday, July 28, 2008 5:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog error messages >> >> I don't have much more to add to this discussion - I think we're >> pretty much on the same page. I'm looking forward to seeing the >> results of your work. >> >> -HKS >> >> >> On Fri, Jul 25, 2008 at 12:07 PM, Rainer Gerhards >> wrote: >> > >> > >> > Please keep the thoughts flowing, we make good progess :) >> > Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From mattjhell at gmail.com Wed Jul 30 00:28:43 2008 From: mattjhell at gmail.com (Matt Hellman) Date: Tue, 29 Jul 2008 17:28:43 -0500 Subject: [rsyslog] Thinking about syslog forwarding infrastructure Message-ID: In the interest of brevity, I'm leaving out details...if you need more, just ask. We have a security event monitoring system that processes probably in the neighborhood of 100 million syslog messages per day (I know precisely how many events it has processed, but it doesn't break them down by protocol). In some of our WAN sites, I would like to implement a local system that will receive all the local syslog messages and ship some/all back to the main collector on our LAN. The main collector on the LAN would receive the bulk of its message from other LAN systems. Then the main collect would ship some/all to the SEM environment. I'm leaning towards using rsyslog for this task and have a few questions: 1) What kind of system [rough estimate] would I need for the main collector if assume 200 million syslog messages per day and peak that is triple that average rate (~7000 eps)? 2) Can I enable rate limiting in a way that will: A1) start dropping messages beyond a given threshold A2) start intelligently dropping messages beyond a given threshold (i.e. start dropping events matching this regex) B) allow me to alert someone that this is occurring (is written to log file, etc) From rory at ooma.com Wed Jul 30 03:14:52 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 18:14:52 -0700 Subject: [rsyslog] mysql inserts and message queues Message-ID: <488FC08C.4080201@ooma.com> I have enabled mysql and disk message queues as per below: $ModLoad immark.so # provides --MARK-- message capability $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $ModLoad imklog.so # kernel logging (formerly provided by rklogd) $ModLoad ommysql.so $ModLoad imudp.so # provides UDP syslog reception $UDPServerRun 514 # start a UDP syslog server at standard port 514 $WorkDirectory /var/rsyslog $ActionQueueType LinkedList $ActionQueueFileName dbq $ActionResumeRetryCount -1 *.* :ommysql:127.0.0.1,Syslog,syslogwriter,writeme # *.* /var/log/messages The question I have is... does the disk queue drain w/o me having to intervene? I currently have 323 queue files (I was adding some keys and playing with tuning mysql... kind of slowed it down) and the number of queue files is not going down. thx From rgerhards at hq.adiscon.com Wed Jul 30 07:45:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 07:45:38 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <488FC08C.4080201@ooma.com> References: <488FC08C.4080201@ooma.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> I don't see any obvious error and, yes, the queue files should be deleted. Actually, you should only see any files at all if the inserts take too long. What you have may be an artifact of a previous failure. To get down to what it is, we need to check how things progress. Is your database populated? In any case, a debug log (rsyslogd with -dn additional options run interactively) will tell us what is there. Also, there should be a .qi file. Let us know its contents. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rory Toma > Sent: Wednesday, July 30, 2008 3:15 AM > To: rsyslog-users > Subject: [rsyslog] mysql inserts and message queues > > I have enabled mysql and disk message queues as per below: > > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging (e.g. > via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by rklogd) > $ModLoad ommysql.so > $ModLoad imudp.so # provides UDP syslog reception > $UDPServerRun 514 # start a UDP syslog server at standard port 514 > > $WorkDirectory /var/rsyslog > > $ActionQueueType LinkedList > $ActionQueueFileName dbq > $ActionResumeRetryCount -1 > > *.* :ommysql:127.0.0.1,Syslog,syslogwriter,writeme > # *.* /var/log/messages > > > The question I have is... does the disk queue drain w/o me having to > intervene? I currently have 323 queue files (I was adding some keys and > playing with tuning mysql... kind of slowed it down) and the number of > queue files is not going down. > > thx > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rory at ooma.com Wed Jul 30 07:58:32 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 22:58:32 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> Message-ID: <48900308.3070805@ooma.com> Rainer Gerhards wrote: > I don't see any obvious error and, yes, the queue files should be > deleted. Actually, you should only see any files at all if the inserts > take too long. What you have may be an artifact of a previous failure. > > To get down to what it is, we need to check how things progress. Is your > database populated? In any case, a debug log (rsyslogd with -dn > additional options run interactively) will tell us what is there. Also, > there should be a .qi file. Let us know its contents. > > Thanks, > Rainer > > the db is getting populated and there is no .qi file. If I run in with "-dn" there is a ton of output, so I need to know what to look for. I've been running for about a day, and I'm at 20+ million records. I do get an error -2040 when accessing on-disk files. Should I just nuke these? From rory at ooma.com Wed Jul 30 08:07:59 2008 From: rory at ooma.com (Rory Toma) Date: Tue, 29 Jul 2008 23:07:59 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <48900308.3070805@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> Message-ID: <4890053F.40606@ooma.com> As a side note, I stopped mysql, saw the number of queue files increase, restarted mysql and they went back down to the original 300 or so. So, maybe I just have some horked queue files. Rory Toma wrote: > Rainer Gerhards wrote: > >> I don't see any obvious error and, yes, the queue files should be >> deleted. Actually, you should only see any files at all if the inserts >> take too long. What you have may be an artifact of a previous failure. >> >> To get down to what it is, we need to check how things progress. Is your >> database populated? In any case, a debug log (rsyslogd with -dn >> additional options run interactively) will tell us what is there. Also, >> there should be a .qi file. Let us know its contents. >> >> Thanks, >> Rainer >> >> >> > the db is getting populated and there is no .qi file. If I run in with > "-dn" there is a ton of output, so I need to know what to look for. I've > been running for about a day, and I'm at 20+ million records. > > I do get an error -2040 when accessing on-disk files. Should I just nuke > these? > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Wed Jul 30 08:30:28 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 08:30:28 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <48900308.3070805@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> Message-ID: <1217399428.7184.260.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-29 at 22:58 -0700, Rory Toma wrote: > Rainer Gerhards wrote: > > I don't see any obvious error and, yes, the queue files should be > > deleted. Actually, you should only see any files at all if the inserts > > take too long. What you have may be an artifact of a previous failure. > > > > To get down to what it is, we need to check how things progress. Is your > > database populated? In any case, a debug log (rsyslogd with -dn > > additional options run interactively) will tell us what is there. Also, > > there should be a .qi file. Let us know its contents. > > > > Thanks, > > Rainer > > > > > the db is getting populated and there is no .qi file. No .qi file is a good indication (though not sufficient) that the queue files are artifacts of a previous failure. > If I run in with > "-dn" there is a ton of output, so I need to know what to look for. I've > been running for about a day, and I'm at 20+ million records. I usually know only when I see. Based on this case, I'd be interested in the first 2000 lines and another 1000 lines while it is processing records. That should at least provide enough of a clue to ask for something more specific. > > I do get an error -2040 when accessing on-disk files. Should I just nuke > these? This can be OK. I think you see the -2040 (file not found) when it is looking for the .qi files. Rainer > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 30 08:31:08 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 08:31:08 +0200 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <4890053F.40606@ooma.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> <4890053F.40606@ooma.com> Message-ID: <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> On Tue, 2008-07-29 at 23:07 -0700, Rory Toma wrote: > As a side note, I stopped mysql, saw the number of queue files increase, > restarted mysql and they went back down to the original 300 or so. > > So, maybe I just have some horked queue files. that's now even more probable. But let's verify with the debug log. Rainer > > > Rory Toma wrote: > > Rainer Gerhards wrote: > > > >> I don't see any obvious error and, yes, the queue files should be > >> deleted. Actually, you should only see any files at all if the inserts > >> take too long. What you have may be an artifact of a previous failure. > >> > >> To get down to what it is, we need to check how things progress. Is your > >> database populated? In any case, a debug log (rsyslogd with -dn > >> additional options run interactively) will tell us what is there. Also, > >> there should be a .qi file. Let us know its contents. > >> > >> Thanks, > >> Rainer > >> > >> > >> > > the db is getting populated and there is no .qi file. If I run in with > > "-dn" there is a ton of output, so I need to know what to look for. I've > > been running for about a day, and I'm at 20+ million records. > > > > I do get an error -2040 when accessing on-disk files. Should I just nuke > > these? > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From friedl at hq.adiscon.com Wed Jul 30 17:07:22 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 30 Jul 2008 17:07:22 +0200 Subject: [rsyslog] rsyslog 3.21.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEE8@grfint2.intern.adiscon.com> Hi all, we have just released rsyslog 3.21.1, a development branch version. It contains some bug fixes and support for improved config file error checking. Most importantly, rsyslogd now has a "config file verification option", which makes it verify the config file but not start up or interfere with a running configuration. That should make troubleshooting config problems much easier. The version now also writes all internally-generated messages to stderr (if not disabled). Thanks to HKS for suggesting the new features. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-123.phtml Changelog: http://www.rsyslog.com/Article262.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Wed Jul 30 17:13:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 17:13:35 +0200 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: References: Message-ID: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Hi Matt, On Tue, 2008-07-29 at 17:28 -0500, Matt Hellman wrote: > In the interest of brevity, I'm leaving out details...if you need > more, just ask. > > We have a security event monitoring system that processes probably in > the neighborhood of 100 million syslog messages per day (I know > precisely how many events it has processed, but it doesn't break them > down by protocol). In some of our WAN sites, I would like to > implement a local system that will receive all the local syslog > messages and ship some/all back to the main collector on our LAN. The > main collector on the LAN would receive the bulk of its message from > other LAN systems. Then the main collect would ship some/all to the > SEM environment. I'm leaning towards using rsyslog for this task and > have a few questions: > > 1) What kind of system [rough estimate] would I need for the main > collector if assume 200 million syslog messages per day and peak that > is triple that average rate (~7000 eps)? Quite honestly: I don't know. Which rules you carry out has a big effect. But I have no real good big deployment numbers. The old game: everyone is interested in them, no-one conveys them (hint: let me know if you have some ;)). > > 2) Can I enable rate limiting in a way that will: > A1) start dropping messages beyond a given threshold you can do that > > A2) start intelligently dropping messages beyond a given threshold > (i.e. start dropping events matching this regex) not yet, but an interesting idea > > B) allow me to alert someone that this is occurring (is written to > log file, etc) mmhhh... not really. That's another interesting idea, and it should be simple to enable. It conveys that to the debug log, but does not emit a user message. In any case, I think there are a couple of docs you need to read and *understand* for this scale of deployment. Ask if you do not understand them - I have written them and may have left too much out just out of habit ;) First and foremost, you must understand rsyslog queues. They handle all queueing and rate limiting. The relevant doc is: http://www.rsyslog.com/doc-queues.html Then, I suggest to have a quick look at some use cases. I probably is a good idea to read the queue doc once, go over the use cases and the re-read the queue doc with the use cases on your mind: http://www.rsyslog.com/doc-queues.html http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html http://wiki.rsyslog.com/index.php/OffPeakHours Finally, IMHO you need to think about syslog reliability and the service level you can expect. This is not only true for rsyslog, but the other folks don't tell you about the reliability issues. Read this: http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html I hope this gets you started. Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Jul 30 17:15:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 30 Jul 2008 17:15:32 +0200 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> References: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEE9@grfint2.intern.adiscon.com> I was too quick. A small yet important thing to add: > > A2) start intelligently dropping messages beyond a given > threshold > > (i.e. start dropping events matching this regex) > > not yet, but an interesting idea Well, it is semi-intelligent. Messages are discarded based on severity level. But you can not use any other property. The details are in the queue doc. Rainer From mattjhell at gmail.com Wed Jul 30 19:33:14 2008 From: mattjhell at gmail.com (Matt Hellman) Date: Wed, 30 Jul 2008 12:33:14 -0500 Subject: [rsyslog] Thinking about syslog forwarding infrastructure In-Reply-To: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> References: <1217430815.7184.306.camel@rgf9dev.intern.adiscon.com> Message-ID: Thanks for the reply Rainer. >> 1) What kind of system [rough estimate] would I need for the main >> collector if assume 200 million syslog messages per day and peak that >> is triple that average rate (~7000 eps)? > > Quite honestly: I don't know. Which rules you carry out has a big > effect. But I have no real good big deployment numbers. The old game: > everyone is interested in them, no-one conveys them (hint: let me know > if you have some ;)). crap. well, I can probably test this easily enough myself. I just feel better knowing that someone has already done it. >> A2) start intelligently dropping messages beyond a given threshold >> (i.e. start dropping events matching this regex) > > not yet, but an interesting idea well, regex wouldn't be the only "intelligent" way to drop messages. I suppose anything that isn't arbitrary might be considered intelligent. Currently this is done based on priority, which won't work well for us because we use a product (Snare) that converts windows events into syslog that all have the same priority. FWIW, this is a common way for SEM products to collect Windows events. >> B) allow me to alert someone that this is occurring (is written to >> log file, etc) > > mmhhh... not really. That's another interesting idea, and it should be > simple to enable. It conveys that to the debug log, but does not emit a > user message. I was thinking about this and I don't necessarily need the product to emit something directly to a user, if that's what you mean. I plan to buffer to disk. Can I create a process to monitor the queue files or something --warning: I have printed but at best skimmed many of the docs you reference;-) > In any case, I think there are a couple of docs you need to read and > *understand* for this scale of deployment. Ask if you do not understand > them - I have written them and may have left too much out just out of > habit ;) re: doc links. Thanks. I was being lazy and trying to avoid having to read them prematurely;-) I think I'm too the point where I believe rsyslog can theoretically deliver on my requirements though. It's time to dig in. From rory at ooma.com Wed Jul 30 20:30:43 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 11:30:43 -0700 Subject: [rsyslog] mysql inserts and message queues In-Reply-To: <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> References: <488FC08C.4080201@ooma.com> <577465F99B41C842AAFBE9ED71E70ABA44EEDC@grfint2.intern.adiscon.com> <48900308.3070805@ooma.com> <4890053F.40606@ooma.com> <1217399468.7184.262.camel@rgf9dev.intern.adiscon.com> Message-ID: <4890B353.6000607@ooma.com> Rainer Gerhards wrote: > On Tue, 2008-07-29 at 23:07 -0700, Rory Toma wrote: > >> As a side note, I stopped mysql, saw the number of queue files increase, >> restarted mysql and they went back down to the original 300 or so. >> >> So, maybe I just have some horked queue files. >> > > that's now even more probable. But let's verify with the debug log. > > OK, here's the log file, about 20-30 seconds of runtime... http://www.colinburns.com/rsyslog/rsyslog.txt.gz thx From julianokyap at gmail.com Thu Jul 31 00:18:09 2008 From: julianokyap at gmail.com (Julian Yap) Date: Wed, 30 Jul 2008 12:18:09 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: Is there a way to set an Alert when multiple repeated lines are found in a log? I want to spawn an email Alert if a message is received 3 times. Example log lines: Jul 30 04:19:29 localhost program: Error detected Jul 30 05:19:29 localhost program: Error detected Jul 30 06:19:29 localhost program: Error detected Thanks, Julian From rory at ooma.com Thu Jul 31 02:15:41 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 17:15:41 -0700 Subject: [rsyslog] tips for managing data Message-ID: <4891042D.9040805@ooma.com> So, my current mysql rsyslog drops about 20 million rows of data per day. Over time, this gets slow as tables grow. I'm not a dba, so I was wondering if anyone had some suggestions for keeping performance still on the order of seconds, and not minutes or hours. thx I did add a key for EventSource, as that is commonly searched. However, using PhpLogCon, it seems that if I search using the web interface (i.e. I click on a host entry and hit the available searches) it is relatively quick. However, changing the text field that is generated and hitting the "search" button is slow. Do these two methods use the same query, or is something else going on? thx From rory at ooma.com Thu Jul 31 04:09:52 2008 From: rory at ooma.com (Rory Toma) Date: Wed, 30 Jul 2008 19:09:52 -0700 Subject: [rsyslog] tips for managing data In-Reply-To: <4891042D.9040805@ooma.com> References: <4891042D.9040805@ooma.com> Message-ID: <48911EF0.1010004@ooma.com> OK, so it seems that doing a query from the query line does a LIKE, which can take significantly longer (sample query 8 seconds vs. 50 msecs...) So, replacing the LIKE % in logstreamdb.class.db with an = speeds things up quite a but, but I lose some flexibility. Is there some kind of search syntax where I can differentiate between LIKE and =? If not, I'm thinking something like: source:foo.bar.com # would be using = ~source:foo # would be using LIKE Rory Toma wrote: > So, my current mysql rsyslog drops about 20 million rows of data per day. > > Over time, this gets slow as tables grow. > > I'm not a dba, so I was wondering if anyone had some suggestions for > keeping performance still on the order of seconds, and not minutes or hours. > > thx > > I did add a key for EventSource, as that is commonly searched. However, > using PhpLogCon, it seems that if I search using the web interface (i.e. > I click on a host entry and hit the available searches) it is relatively > quick. However, changing the text field that is generated and hitting > the "search" button is slow. Do these two methods use the same query, or > is something else going on? > > thx > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From alorbach at ro1.adiscon.com Thu Jul 31 10:15:23 2008 From: alorbach at ro1.adiscon.com (Andre Lorbach) Date: Thu, 31 Jul 2008 10:15:23 +0200 Subject: [rsyslog] tips for managing data In-Reply-To: <48911EF0.1010004@ooma.com> References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com> Message-ID: Hi, the like query can indeed have quiet an impact on performance when doing queries on large databases. But I think we can expand the syntax, so you can either search by part of a string (LIKE '%search%') or the whole string (= 'search'). This should be rather easy to implement. I will put this on my todolist, if it is as easy as I think, the next minor update of the devel branch will contain this new feature. Best regards, Andre Lorbach > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rory Toma > Sent: Thursday, July 31, 2008 4:10 AM > To: rsyslog-users > Subject: Re: [rsyslog] tips for managing data > > OK, so it seems that doing a query from the query line does a LIKE, > which can take significantly longer (sample query 8 seconds vs. 50 msecs...) > > So, replacing the LIKE % in logstreamdb.class.db with an = speeds things > up quite a but, but I lose some flexibility. Is there some kind of > search syntax where I can differentiate between LIKE and =? > > If not, I'm thinking something like: > > source:foo.bar.com # would be using = > > ~source:foo # would be using LIKE > > > > Rory Toma wrote: > > So, my current mysql rsyslog drops about 20 million rows of data per day. > > > > Over time, this gets slow as tables grow. > > > > I'm not a dba, so I was wondering if anyone had some suggestions for > > keeping performance still on the order of seconds, and not minutes or hours. > > > > thx > > > > I did add a key for EventSource, as that is commonly searched. However, > > using PhpLogCon, it seems that if I search using the web interface (i.e. > > I click on a host entry and hit the available searches) it is relatively > > quick. However, changing the text field that is generated and hitting > > the "search" button is slow. Do these two methods use the same query, or > > is something else going on? > > > > thx > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From ml at darville.vm.bytemark.co.uk Thu Jul 31 13:11:22 2008 From: ml at darville.vm.bytemark.co.uk (David Darville) Date: Thu, 31 Jul 2008 12:11:22 +0100 Subject: [rsyslog] Changing hostname field Message-ID: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Hello everyone I am trying to configure rsyslog to service a number of chroot jails in addition to the host itself. But I need to change the hostname field of the syslog messages from the different jails, so that I place them in the right log file on the central logging host. My current rsyslog.conf is as follows: $ModLoad imuxsock $ModLoad imklog $ModLoad immark $ModLoad omrelp $AddUnixListenSocket /jail/1/dev/log $AddUnixListenSocket /jail/2/dev/log *.* :omrelp:10.0.0.4:2514 Can anyone please advice me on how to do that? --- David Darville From hks.private at gmail.com Thu Jul 31 15:48:37 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 31 Jul 2008 09:48:37 -0400 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: References: Message-ID: Not in rsyslogd itself, but you could do this with Swatch, Nagios, or some other monitoring-type software. -HKS On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: > Is there a way to set an Alert when multiple repeated lines are found in a log? > > I want to spawn an email Alert if a message is received 3 times. > > Example log lines: > Jul 30 04:19:29 localhost program: Error detected > Jul 30 05:19:29 localhost program: Error detected > Jul 30 06:19:29 localhost program: Error detected > > Thanks, > Julian > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From hks.private at gmail.com Thu Jul 31 16:00:09 2008 From: hks.private at gmail.com ((private) HKS) Date: Thu, 31 Jul 2008 10:00:09 -0400 Subject: [rsyslog] Changing hostname field In-Reply-To: <20080731111122.GA24000@darville.vm.bytemark.co.uk> References: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Message-ID: Do the jails all share the same hostname and IP? If not, you should be able to use the %hostname% or %fromhost% properties. If so, are they each running their own instance of (r)syslogd? -HKS On Thu, Jul 31, 2008 at 7:11 AM, David Darville wrote: > Hello everyone > > I am trying to configure rsyslog to service a number of chroot jails in > addition to the host itself. > > But I need to change the hostname field of the syslog messages from the > different jails, so that I place them in the right log file on the central > logging host. > > My current rsyslog.conf is as follows: > > $ModLoad imuxsock > $ModLoad imklog > $ModLoad immark > $ModLoad omrelp > > $AddUnixListenSocket /jail/1/dev/log > $AddUnixListenSocket /jail/2/dev/log > > *.* :omrelp:10.0.0.4:2514 > > > Can anyone please advice me on how to do that? > > > --- > > David Darville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rory at ooma.com Thu Jul 31 16:32:11 2008 From: rory at ooma.com (Rory Toma) Date: Thu, 31 Jul 2008 07:32:11 -0700 Subject: [rsyslog] tips for managing data In-Reply-To: References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com> Message-ID: <4891CCEB.50905@ooma.com> Cool. For me, it seems that using LIKE is most useful when searching the message text. So, something like: source:foo ~bar would produce where fromhost = 'foo' and message LIKE '%bar%' thx Andre Lorbach wrote: > Hi, > > the like query can indeed have quiet an impact on performance when doing > queries on large databases. > But I think we can expand the syntax, so you can either search by part > of a string (LIKE '%search%') or the whole string (= 'search'). This > should be rather easy to implement. I will put this on my todolist, if > it is as easy as I think, the next minor update of the devel branch will > contain this new feature. > > Best regards, > Andre Lorbach > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rory Toma >> Sent: Thursday, July 31, 2008 4:10 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] tips for managing data >> >> OK, so it seems that doing a query from the query line does a LIKE, >> which can take significantly longer (sample query 8 seconds vs. 50 >> > msecs...) > >> So, replacing the LIKE % in logstreamdb.class.db with an = speeds >> > things > >> up quite a but, but I lose some flexibility. Is there some kind of >> search syntax where I can differentiate between LIKE and =? >> >> If not, I'm thinking something like: >> >> source:foo.bar.com # would be using = >> >> ~source:foo # would be using LIKE >> >> >> >> Rory Toma wrote: >> >>> So, my current mysql rsyslog drops about 20 million rows of data per >>> > day. > >>> Over time, this gets slow as tables grow. >>> >>> I'm not a dba, so I was wondering if anyone had some suggestions for >>> keeping performance still on the order of seconds, and not minutes >>> > or hours. > >>> thx >>> >>> I did add a key for EventSource, as that is commonly searched. >>> > However, > >>> using PhpLogCon, it seems that if I search using the web interface >>> > (i.e. > >>> I click on a host entry and hit the available searches) it is >>> > relatively > >>> quick. However, changing the text field that is generated and >>> > hitting > >>> the "search" button is slow. Do these two methods use the same >>> > query, or > >>> is something else going on? >>> >>> thx >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From ml at darville.vm.bytemark.co.uk Thu Jul 31 16:46:49 2008 From: ml at darville.vm.bytemark.co.uk (David Darville) Date: Thu, 31 Jul 2008 15:46:49 +0100 Subject: [rsyslog] Changing hostname field In-Reply-To: References: <20080731111122.GA24000@darville.vm.bytemark.co.uk> Message-ID: <20080731144649.GA24505@darville.vm.bytemark.co.uk> The jails all have their own unique hostname (and IP), but all share an rsyslogd instance running on the main host, and the %hostname% and %fromhost% in all the log messages from the jails are set to the hostname of the main host. And that is what I want to change. On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote: > Do the jails all share the same hostname and IP? If not, you should be > able to use the %hostname% or %fromhost% properties. > > If so, are they each running their own instance of (r)syslogd? > > -HKS > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville > wrote: > > Hello everyone > > > > I am trying to configure rsyslog to service a number of chroot jails in > > addition to the host itself. > > > > But I need to change the hostname field of the syslog messages from the > > different jails, so that I place them in the right log file on the central > > logging host. > > > > My current rsyslog.conf is as follows: > > > > $ModLoad imuxsock > > $ModLoad imklog > > $ModLoad immark > > $ModLoad omrelp > > > > $AddUnixListenSocket /jail/1/dev/log > > $AddUnixListenSocket /jail/2/dev/log > > > > *.* :omrelp:10.0.0.4:2514 > > > > > > Can anyone please advice me on how to do that? > > > > > > --- > > > > David Darville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Thu Jul 31 17:04:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 17:04:18 +0200 Subject: [rsyslog] Changing hostname field Message-ID: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com> Use a template with fixed name. --- Urspr?ngliche Nachricht --- Von: "David Darville" Betreff: Re: [rsyslog] Changing hostname field Datum: 31. Juli 2008 Uhrzeit: 16:46:59 The jails all have their own unique hostname (and IP), but all share an rsyslogd instance running on the main host, and the %hostname% and %fromhost% in all the log messages from the jails are set to the hostname of the main host. And that is what I want to change. On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote: > Do the jails all share the same hostname and IP? If not, you should be > able to use the %hostname% or %fromhost% properties. > > If so, are they each running their own instance of (r)syslogd? > > -HKS > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville > wrote: > > Hello everyone > > > > I am trying to configure rsyslog to service a number of chroot jails in > > addition to the host itself. > > > > But I need to change the hostname field of the syslog messages from the > > different jails, so that I place them in the right log file on the central > > logging host. > > > > My current rsyslog.conf is as follows: > > > > $ModLoad imuxsock > > $ModLoad imklog > > $ModLoad immark > > $ModLoad omrelp > > > > $AddUnixListenSocket /jail/1/dev/log > > $AddUnixListenSocket /jail/2/dev/log > > > > *.* :omrelp:10.0.0.4:2514 > > > > > > Can anyone please advice me on how to do that? > > > > > > --- > > > > David Darville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From julianokyap at gmail.com Thu Jul 31 20:27:14 2008 From: julianokyap at gmail.com (Julian Yap) Date: Thu, 31 Jul 2008 08:27:14 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: References: Message-ID: Hmm, Nagios is a pain to set up. Looking for something more light weight... Was hoping that I could have consolidated lots of Alerts under Rsyslog. Any other suggestions besides Swatch? On 7/31/08, (private) HKS wrote: > Not in rsyslogd itself, but you could do this with Swatch, Nagios, or > some other monitoring-type software. > > -HKS > > On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >> Is there a way to set an Alert when multiple repeated lines are found in a >> log? >> >> I want to spawn an email Alert if a message is received 3 times. >> >> Example log lines: >> Jul 30 04:19:29 localhost program: Error detected >> Jul 30 05:19:29 localhost program: Error detected >> Jul 30 06:19:29 localhost program: Error detected >> >> Thanks, >> Julian >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 20:37:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 20:37:05 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> What exactly do you need to do except the "three in a row" alert? ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Gesendet: 31.07.08 20:27 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found Hmm, Nagios is a pain to set up. Looking for something more light weight... Was hoping that I could have consolidated lots of Alerts under Rsyslog. Any other suggestions besides Swatch? On 7/31/08, (private) HKS wrote: > Not in rsyslogd itself, but you could do this with Swatch, Nagios, or > some other monitoring-type software. > > -HKS > > On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >> Is there a way to set an Alert when multiple repeated lines are found in a >> log? >> >> I want to spawn an email Alert if a message is received 3 times. >> >> Example log lines: >> Jul 30 04:19:29 localhost program: Error detected >> Jul 30 05:19:29 localhost program: Error detected >> Jul 30 06:19:29 localhost program: Error detected >> >> Thanks, >> Julian >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From julianokyap at gmail.com Thu Jul 31 21:59:51 2008 From: julianokyap at gmail.com (Julian Yap) Date: Thu, 31 Jul 2008 09:59:51 -1000 Subject: [rsyslog] Alert when multiple repeated lines are found In-Reply-To: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> References: <001401c8f33c$82dcb5e1$060013ac@intern.adiscon.com> Message-ID: That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 22:38:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 22:38:38 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com> To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence: 1. a - a - a - b 2. a - a - b - a Result: in case 1 an alert is triggered, in case 2 not. Is this understanding correct? rainer ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Cc: "rgerhards at hq.adiscon.com" ; "hks.private at gmail.com" Gesendet: 31.07.08 21:59 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Thu Jul 31 23:19:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 31 Jul 2008 23:19:35 +0200 Subject: [rsyslog] Alert when multiple repeated lines are found Message-ID: <001601c8f353$363e638d$060013ac@intern.adiscon.com> Oh, and one thing i forgot: what makes an event identical? Same message except timestamp - or what (eg same host, same tag, ...) rainer ----- Urspr?ngliche Nachricht ----- Von: "Rainer Gerhards" An: "Julian Yap" Cc: "rsyslog at lists.adiscon.com" Gesendet: 31.07.08 22:39 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence: 1. a - a - a - b 2. a - a - b - a Result: in case 1 an alert is triggered, in case 2 not. Is this understanding correct? rainer ----- Urspr?ngliche Nachricht ----- Von: "Julian Yap" An: "rsyslog-users" Cc: "rgerhards at hq.adiscon.com" ; "hks.private at gmail.com" Gesendet: 31.07.08 21:59 Betreff: Re: [rsyslog] Alert when multiple repeated lines are found That's pretty much it for now. I've written Alerts for single line events. But for one particular event, it's only really a factor if it happens tree times in a row. On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards wrote: > What exactly do you need to do except the "three in a row" alert? > > ----- Urspr?ngliche Nachricht ----- > Von: "Julian Yap" > An: "rsyslog-users" > Gesendet: 31.07.08 20:27 > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found > > Hmm, Nagios is a pain to set up. Looking for something more light > weight... Was hoping that I could have consolidated lots of Alerts > under Rsyslog. > > Any other suggestions besides Swatch? > > > > On 7/31/08, (private) HKS wrote: >> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or >> some other monitoring-type software. >> >> -HKS >> >> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap wrote: >>> Is there a way to set an Alert when multiple repeated lines are found in a >>> log? >>> >>> I want to spawn an email Alert if a message is received 3 times. >>> >>> Example log lines: >>> Jul 30 04:19:29 localhost program: Error detected >>> Jul 30 05:19:29 localhost program: Error detected >>> Jul 30 06:19:29 localhost program: Error detected >>> >>> Thanks, >>> Julian >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog