From rgerhards at hq.adiscon.com Tue Nov 2 11:24:59 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Nov 2010 11:24:59 +0100 Subject: [rsyslog] epoll() for RELP? References: <4CCD70A6.7080105@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD606@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Sunday, October 31, 2010 2:36 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] epoll() for RELP? > > Greetings, > > I stumbled across the --enable-unlimited-select configure option [1] > and > an older conversation with Rainer on the mailing list [2] indicating > that epoll() is only supported for imudp and the plain tcp netstream > driver. > > What is the current status for stable version 5.6.0 and epoll() for all > other inputs? No yet done, I am unfortunately busy with more currently more important things. > I'm especially interested in epoll() support for RELP. > > FYI, I'm unable to compile 5.6.0 with --enable-unlimited-select: > > > CCLD lmnsd_ptcp.la > > .libs/lmnsd_ptcp_la-nsdpoll_ptcp.o: In function `freeFdSet': > > /home/john/Projects/src/rsyslog/rsyslog- > 5.6.0/runtime/unlimited_select.h:39: multiple definition of `freeFdSet' > > .libs/lmnsd_ptcp_la- > nsdsel_ptcp.o:/home/john/Projects/src/rsyslog/rsyslog- > 5.6.0/runtime/unlimited_select.h:39: first defined here > > collect2: ld returned 1 exit status > > make[2]: *** [lmnsd_ptcp.la] Error 1 thanks, will check out that! Rainer > > Best regards, > John > > [1] http://www.gossamer-threads.com/lists/rsyslog/users/3029 > [2] http://www.mail-archive.com/rsyslog at lists.adiscon.com/msg02950.html > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From sivan at omniqueue.com Wed Nov 3 15:17:42 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 3 Nov 2010 16:17:42 +0200 Subject: [rsyslog] EventId field Message-ID: Hi all (Rainer, et al) ! I would like to know what's the use of the EventId in the creatDB.sql schema for logging to mysql? Can it be used for custom data passing by a program using it for message passing? Oh, and rsyslog rocks. Thanks! -Sivan From rgerhards at hq.adiscon.com Wed Nov 3 15:19:22 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 15:19:22 +0100 Subject: [rsyslog] EventId field References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> Hi Sivan, that field is normally not used by rsyslog ;) The schema supports all of Adiscon's MonitorWare products. EventID is usually populated by the (commercial) Windows products, which store the Windows Event ID there. Having a common schema simplifies things, so I pulled the complete schema over. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sivan Greenberg > Sent: Wednesday, November 03, 2010 3:18 PM > To: rsyslog-users > Subject: [rsyslog] EventId field > > Hi all (Rainer, et al) ! > > I would like to know what's the use of the EventId in the creatDB.sql > schema for logging to mysql? Can it be used for custom data passing by > a program using it for message passing? > > Oh, and rsyslog rocks. Thanks! > > -Sivan > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From linuxmail at 4lin.net Wed Nov 3 14:08:52 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Wed, 03 Nov 2010 14:08:52 +0100 Subject: [rsyslog] host exclude from *.info Message-ID: <1288789732.23247.13.camel@pcdenny.rbg.informatik.tu-darmstadt.de> hi, we use rsyslog 4.7.2 as central logserver and we logging in two files: /var/log/remote//messages and in /var/adm/messages and I don't want the second one for remote logging hosts (like info, notice etc.). Our config is: ========================== $ModLoad immark $ModLoad imuxsock $ModLoad imtcp.so $InputTCPServerRun 514 $ModLoad imudp.so $UDPServerRun 514 $InputTCPServerRun 10514 $FileOwner root $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $template DynAuth, "/var/log/remote/%HOSTNAME%/secure.log" $template DynMSG, "/var/log/remote/%HOSTNAME%/messages" $template Dynmail, "/var/log/remote/%HOSTNAME%/maillog" $template Dyncron,"/var/log/remote/%HOSTNAME%/cron" $template Dynspool, "/var/log/remote/%HOSTNAME%/spooler" $template Dynboot, "/var/log/remote/%HOSTNAME%/boot.log" $template Dynlocal3, "/var/log/remote/%HOSTNAME%/local3.log" $template Dynlocal4, "/var/log/remote/%HOSTNAME%/local4.log" $template Dynlocal5, "/var/log/remote/%HOSTNAME%/local5.log" $template Dynlocal6, "/var/log/remote/%HOSTNAME%/local6.log" $template Dynlocal7, "/var/log/remote/%HOSTNAME%/local7.log" authpriv.* ?DynAuth *.info,mail.none,authpriv.none,cron.none ?DynMSG mail.* -?Dynmail cron.* ?Dyncron news.crit ?Dynspool local3.* ?Dynlocal3 local4.* ?Dynlocal4 local5.* ?Dynlocal5 local6.* ?Dynlocal6 local7.* ?Dynlocal7 *.emerg * *.alert root *.alert /dev/sysmsg auth.info /var/adm/authlog *.err /var/adm/syslog *.notice /var/adm/messages mail.debug @mail local0.info /var/adm/login local4.info /var/log/openldap/ldap.log daemon.debug /var/adm/messages daemon.info /var/adm/messages ============================== So how can I keep remote hosts away from my local logfiles and fill them up? cu denny From rgerhards at hq.adiscon.com Wed Nov 3 15:38:29 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 15:38:29 +0100 Subject: [rsyslog] Mail Server Troubles Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD623@GRFEXC.intern.adiscon.com> Hi all, it looks like we had (at least I hope past is right ;)) some mail server troubles. This also affected the rsyslog mailing list. Mail was deferred from some while in a seemingly indeterministic way ;) I also see some other mails coming in in batches now. If you have mailed me and I need a bit longer to answer, please bear with me. If in doubt, please re-send mails. Thanks, Rainer From linuxmail at 4lin.net Wed Nov 3 13:01:10 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Wed, 03 Nov 2010 13:01:10 +0100 Subject: [rsyslog] host exclude from *.info Message-ID: <1288785670.23247.11.camel@pcdenny.rbg.informatik.tu-darmstadt.de> hi, we use rsyslog as central logserver and we logging in two files: /var/log/remote//messages and in /var/adm/messages and I don't want the second one. Our config is: ========================== $ModLoad immark $ModLoad imuxsock $ModLoad imtcp.so $InputTCPServerRun 514 $ModLoad imudp.so $UDPServerRun 514 $InputTCPServerRun 10514 $FileOwner root $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $template DynAuth, "/var/log/remote/%HOSTNAME%/secure.log" $template DynMSG, "/var/log/remote/%HOSTNAME%/messages" $template Dynmail, "/var/log/remote/%HOSTNAME%/maillog" $template Dyncron,"/var/log/remote/%HOSTNAME%/cron" $template Dynspool, "/var/log/remote/%HOSTNAME%/spooler" $template Dynboot, "/var/log/remote/%HOSTNAME%/boot.log" $template Dynlocal3, "/var/log/remote/%HOSTNAME%/local3.log" $template Dynlocal4, "/var/log/remote/%HOSTNAME%/local4.log" $template Dynlocal5, "/var/log/remote/%HOSTNAME%/local5.log" $template Dynlocal6, "/var/log/remote/%HOSTNAME%/local6.log" $template Dynlocal7, "/var/log/remote/%HOSTNAME%/local7.log" authpriv.* ?DynAuth *.info,mail.none,authpriv.none,cron.none ?DynMSG mail.* -?Dynmail cron.* ?Dyncron news.crit ?Dynspool local3.* ?Dynlocal3 local4.* ?Dynlocal4 local5.* ?Dynlocal5 local6.* ?Dynlocal6 local7.* ?Dynlocal7 *.emerg * *.alert root *.alert /dev/sysmsg auth.info /var/adm/authlog *.err /var/adm/syslog *.notice /var/adm/messages mail.debug @mail local0.info /var/adm/login local4.info /var/log/openldap/ldap.log daemon.debug /var/adm/messages daemon.info /var/adm/messages ============================== So how can I keep remote hosts away from my local logfiles and fill them up? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From sivan at omniqueue.com Wed Nov 3 16:01:00 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 3 Nov 2010 17:01:00 +0200 Subject: [rsyslog] EventId field In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> Message-ID: Hey! On Wed, Nov 3, 2010 at 4:19 PM, Rainer Gerhards wrote: > Hi Sivan, > > that field is normally not used by rsyslog ;) The schema supports all of > Adiscon's MonitorWare products. EventID is usually populated by the > (commercial) Windows products, which store the Windows Event ID there. Having > a common schema simplifies things, so I pulled the complete schema over. Indeed, thanks for the note. That makes a lot of sense. -Sivan From rgerhards at hq.adiscon.com Wed Nov 3 11:08:03 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 11:08:03 +0100 Subject: [rsyslog] Feedback Request: NULs in strings? Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD617@GRFEXC.intern.adiscon.com> Hi all, once again (this time with libee/liblognorm and CEE) I ran into the problem of C strings and NUL characters inside log data. I have written up a relatively small blog post here: http://blog.gerhards.net/2010/11/logging-and-c-nul-problem.html I would appreciate any feedback you have on the issue. I would also appreciate if you could point me to other places where I could ask for feedback. I am totally undecided, but think this is a very important issue (even though on a low detail level). Feedback very much appreciated. Rainer From sean at conman.org Wed Nov 3 16:52:16 2010 From: sean at conman.org (Sean Conner) Date: Wed, 3 Nov 2010 11:52:16 -0400 Subject: [rsyslog] Feedback Request: NULs in strings? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD617@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD617@GRFEXC.intern.adiscon.com> Message-ID: <20101103155216.GB21709@brevard.conman.org> It was thus said that the Great Rainer Gerhards once stated: > Hi all, > > once again (this time with libee/liblognorm and CEE) I ran into the problem > of C strings and NUL characters inside log data. > > I have written up a relatively small blog post here: > > http://blog.gerhards.net/2010/11/logging-and-c-nul-problem.html > > I would appreciate any feedback you have on the issue. I would also > appreciate if you could point me to other places where I could ask for > feedback. I am totally undecided, but think this is a very important issue > (even though on a low detail level). What is the semantic meaning of a NUL? Because there are other problematic characters (say, CTRL-Z under Windows) that can also cause issues with the tool stream. Knowing how an ASCII NUL is used (or meant to be used) could help decide how to handle it. -spc From rgerhards at hq.adiscon.com Wed Nov 3 17:15:13 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 17:15:13 +0100 Subject: [rsyslog] Feedback Request: NULs in strings? References: <9B6E2A8877C38245BFB15CC491A11DA71DD617@GRFEXC.intern.adiscon.com> <20101103155216.GB21709@brevard.conman.org> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD62D@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sean Conner > Sent: Wednesday, November 03, 2010 4:52 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback Request: NULs in strings? > > It was thus said that the Great Rainer Gerhards once stated: > > Hi all, > > > > once again (this time with libee/liblognorm and CEE) I ran into the > problem > > of C strings and NUL characters inside log data. > > > > I have written up a relatively small blog post here: > > > > http://blog.gerhards.net/2010/11/logging-and-c-nul-problem.html > > > > I would appreciate any feedback you have on the issue. I would also > > appreciate if you could point me to other places where I could ask > for > > feedback. I am totally undecided, but think this is a very important > issue > > (even though on a low detail level). > > What is the semantic meaning of a NUL? Because there are other > problematic characters (say, CTRL-Z under Windows) that can also cause > issues with the tool stream. Knowing how an ASCII NUL is used (or > meant to > be used) could help decide how to handle it. The original US-ASCII semantic is a NOP (no-operation). NULs were used to stuff in extra bytes to give terminal devices (VT 100 and similar ones) some time to execute "more demanding" escape sequences (like clear screen). During processing these sequences, some input characters could be lost (especially at higher rates like 2,400 baud or above ;)). So NUL were sent as loss victims. However, this is not the problem. The semantic in the C context is that NUL is an indicator for "end of string". Rainer From sean at conman.org Wed Nov 3 17:22:29 2010 From: sean at conman.org (Sean Conner) Date: Wed, 3 Nov 2010 12:22:29 -0400 Subject: [rsyslog] Feedback Request: NULs in strings? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD62D@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD617@GRFEXC.intern.adiscon.com> <20101103155216.GB21709@brevard.conman.org> <9B6E2A8877C38245BFB15CC491A11DA71DD62D@GRFEXC.intern.adiscon.com> Message-ID: <20101103162229.GC21709@brevard.conman.org> It was thus said that the Great Rainer Gerhards once stated: > > -----Original Message----- > > > > What is the semantic meaning of a NUL? Because there are other > > problematic characters (say, CTRL-Z under Windows) that can also cause > > issues with the tool stream. Knowing how an ASCII NUL is used (or > > meant to > > be used) could help decide how to handle it. > > The original US-ASCII semantic is a NOP (no-operation). NULs were used to > stuff in extra bytes to give terminal devices (VT 100 and similar ones) some > time to execute "more demanding" escape sequences (like clear screen). During > processing these sequences, some input characters could be lost (especially > at higher rates like 2,400 baud or above ;)). So NUL were sent as loss > victims. > > However, this is not the problem. The semantic in the C context is that NUL > is an indicator for "end of string". No, I meant the semantic meaning of NUL is syslog messages. Why would you want to log a message with such a character? (or any of the control characters really) -spc From rgerhards at hq.adiscon.com Wed Nov 3 17:25:29 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 17:25:29 +0100 Subject: [rsyslog] Feedback Request: NULs in strings? References: <9B6E2A8877C38245BFB15CC491A11DA71DD617@GRFEXC.intern.adiscon.com><20101103155216.GB21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62D@GRFEXC.intern.adiscon.com> <20101103162229.GC21709@brevard.conman.org> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com> > No, I meant the semantic meaning of NUL is syslog messages. Why > would you > want to log a message with such a character? (or any of the control > characters really) I really don't want it, but an attacker may induce it. The need, as far as the discussion goes, stems back to the fact that we cannot reliably forbid its use. But you are right at the heart of the discussion: Should we try to forbid it (knowing that we can't 100% ensure that) or should we somehow handle it. Rainer From sean at conman.org Wed Nov 3 17:49:49 2010 From: sean at conman.org (Sean Conner) Date: Wed, 3 Nov 2010 12:49:49 -0400 Subject: [rsyslog] Feedback Request: NULs in strings? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com> References: <20101103162229.GC21709@brevard.conman.org> <9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com> Message-ID: <20101103164949.GD21709@brevard.conman.org> It was thus said that the Great Rainer Gerhards once stated: > > No, I meant the semantic meaning of NUL is syslog messages. Why > > would you > > want to log a message with such a character? (or any of the control > > characters really) > > I really don't want it, but an attacker may induce it. The need, as far as > the discussion goes, stems back to the fact that we cannot reliably forbid > its use. But you are right at the heart of the discussion: > > Should we try to forbid it (knowing that we can't 100% ensure that) or should > we somehow handle it. I checked my syslogd [1] and I convert any control characters to spaces upon receipt of a message. The actual code I have: memset(&remote,0,sizeof(remote)); remsize = sizeof(remote); bytes = recvfrom(sock->sock,buffer,sizeof(buffer),0,(struct sockaddr *)&remote,&remsize); if (bytes == -1) { if (errno == EINTR) return; internal_log(LOG_DEBUG,"recvfrom() = %s",strerror(errno)); return; } buffer[bytes] = '\0'; for (size_t i = 0 ; buffer[i] != '\0'; i++) if (iscntrl(buffer[i])) buffer[i] = ' '; syslog_interp(&sock->local,&remote,buffer,&buffer[bytes]); To me, it seemed the most logical thing to do, given that RFC-3164 disallows control characters (if I recall it correctly). I can't control what I receive, but I can control what I process and send. -spc (Interesting bit o' trivia: Lua can handle NULs in strings) [1] A custom syslogd with Lua embedded http://www.conman.org/software/syslogintr/ From rgerhards at hq.adiscon.com Wed Nov 3 17:57:48 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 17:57:48 +0100 Subject: [rsyslog] Feedback Request: NULs in strings? References: <20101103162229.GC21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com> <20101103164949.GD21709@brevard.conman.org> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sean Conner > Sent: Wednesday, November 03, 2010 5:50 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback Request: NULs in strings? > > It was thus said that the Great Rainer Gerhards once stated: > > > No, I meant the semantic meaning of NUL is syslog messages. Why > > > would you > > > want to log a message with such a character? (or any of the > control > > > characters really) > > > > I really don't want it, but an attacker may induce it. The need, as > far as > > the discussion goes, stems back to the fact that we cannot reliably > forbid > > its use. But you are right at the heart of the discussion: > > > > Should we try to forbid it (knowing that we can't 100% ensure that) > or should > > we somehow handle it. > > I checked my syslogd [1] and I convert any control characters to > spaces > upon receipt of a message. The actual code I have: That's nice to know, but it unfortunately is not an answer to the question I asked. That question was not how a syslogd should handle NULs, but how a library should handle NULs that a library user (app) potentially pushes to it. My question was how to deal with that, and most importantly: is adoption more important than technical soundness? This is within the context of log normalization and the CEE effort. For syslog and rsyslog, the answer is already clear, you can find it in RFC5424 (NULs permitted, but receiver is permitted to change them to a different sequence). I am sorry if the context was not 100% clear, I had thought the blog post provided everything (and I was obviously wrong...). Rainer From heinbockel at mitre.org Wed Nov 3 18:28:29 2010 From: heinbockel at mitre.org (Heinbockel, Bill) Date: Wed, 3 Nov 2010 13:28:29 -0400 Subject: [rsyslog] Feedback Request: NULs in strings? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com> References: <20101103162229.GC21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com> <20101103164949.GD21709@brevard.conman.org> <9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com> Message-ID: <93ED0A84F9A1D74FA65021D940AA5884053E75C160@IMCMBX3.MITRE.ORG> > >That's nice to know, but it unfortunately is not an answer to the question I >asked. That question was not how a syslogd should handle NULs, but how a >library should handle NULs that a library user (app) potentially pushes to >it. My question was how to deal with that, and most importantly: is adoption >more important than technical soundness? > >This is within the context of log normalization and the CEE effort. For >syslog and rsyslog, the answer is already clear, you can find it in RFC5424 >(NULs permitted, but receiver is permitted to change them to a different >sequence). > For technical soundness, all data should be handled as an octet stream. For usability, some libraries will want the raw binary, some will want the higher level types with encoded values. If we are worried about adoption, why not support both in the API? Handle the data as native octets under the hood (assuming a C like language) Allow the user to set a flag if they want the returned value to be encoded Though this approach only works for values... For things like the RFC5424 structured data, the name in name=value pairs cannot contain characters such as NUL. In my option, what to do if you receive a Syslog message with a NUL in the name, is the bigger issue From rgerhards at hq.adiscon.com Wed Nov 3 18:35:16 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 18:35:16 +0100 Subject: [rsyslog] Feedback Request: NULs in strings? References: <20101103162229.GC21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com><20101103164949.GD21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com> <93ED0A84F9A1D74FA65021D940AA5884053E75C160@IMCMBX3.MITRE.ORG> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD635@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Heinbockel, Bill > Sent: Wednesday, November 03, 2010 6:28 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback Request: NULs in strings? > > > > >That's nice to know, but it unfortunately is not an answer to the > question I > >asked. That question was not how a syslogd should handle NULs, but how > a > >library should handle NULs that a library user (app) potentially > pushes to > >it. My question was how to deal with that, and most importantly: is > adoption > >more important than technical soundness? > > > >This is within the context of log normalization and the CEE effort. > For > >syslog and rsyslog, the answer is already clear, you can find it in > RFC5424 > >(NULs permitted, but receiver is permitted to change them to a > different > >sequence). > > > > For technical soundness, all data should be handled as an octet stream. > For usability, some libraries will want the raw binary, some will want > the higher level types with encoded values. > > If we are worried about adoption, why not support both in the API? That's a *very* interesting approach -- and probably quite doable. So we would leave it to the library user what to do. Sound much like the solution! > Handle the data as native octets under the hood (assuming a C like > language) > Allow the user to set a flag if they want the returned value to be > encoded > > > Though this approach only works for values... > For things like the RFC5424 structured data, the name in name=value > pairs cannot contain characters such as NUL. > In my option, what to do if you receive a Syslog message with a NUL in > the name, is the bigger issue Fortunately not. In RFC5424, full UTF-8 is permitted in values. However, the name alphabet is much more restricted. Out of my head, I think it is US-ASCII minus the control character set. During syslog standardization, we did not see need to support a larger alphabet (but I admit that it is debatable if national characters should be supported in names -- after long discussion we said "no"). Rainer From heinbockel at mitre.org Wed Nov 3 18:49:08 2010 From: heinbockel at mitre.org (Heinbockel, Bill) Date: Wed, 3 Nov 2010 13:49:08 -0400 Subject: [rsyslog] Feedback Request: NULs in strings? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD635@GRFEXC.intern.adiscon.com> References: <20101103162229.GC21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com><20101103164949.GD21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com> <93ED0A84F9A1D74FA65021D940AA5884053E75C160@IMCMBX3.MITRE.ORG> <9B6E2A8877C38245BFB15CC491A11DA71DD635@GRFEXC.intern.adiscon.com> Message-ID: <93ED0A84F9A1D74FA65021D940AA5884053E75C186@IMCMBX3.MITRE.ORG> >> For things like the RFC5424 structured data, the name in name=value >> pairs cannot contain characters such as NUL. >> In my option, what to do if you receive a Syslog message with a NUL in >> the name, is the bigger issue > >Fortunately not. In RFC5424, full UTF-8 is permitted in values. However, the >name alphabet is much more restricted. Out of my head, I think it is US- >ASCII >minus the control character set. During syslog standardization, we did not >see need to support a larger alphabet (but I admit that it is debatable if >national characters should be supported in names -- after long discussion we >said "no"). While it is against the specification, how do you handle the case were an invalid character is included in the name? This is an issue where most programs have issues; they assume that the message will according to the specification. The approach of what to do if an illegal character is encounter should be standardized as well... (though this applies more generally to "what do you do if you receive an invalid name in structured data) Off the top of my head, you have only 1 of 3 choices: (1) strip the offending characters (2) strip the whole name=value pair (3) drop the entire message From rgerhards at hq.adiscon.com Wed Nov 3 19:02:59 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Nov 2010 19:02:59 +0100 Subject: [rsyslog] Feedback Request: NULs in strings? References: <20101103162229.GC21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com><20101103164949.GD21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com><93ED0A84F9A1D74FA65021D940AA5884053E75C160@IMCMBX3.MITRE.ORG><9B6E2A8877C38245BFB15CC491A11DA71DD635@GRFEXC.intern.adiscon.com> <93ED0A84F9A1D74FA65021D940AA5884053E75C186@IMCMBX3.MITRE.ORG> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD638@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Heinbockel, Bill > Sent: Wednesday, November 03, 2010 6:49 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback Request: NULs in strings? > > While it is against the specification, how do you handle the case were > an invalid character is included in the name? This is an issue where > most programs have issues; they assume that the message will according > to the specification. The approach of what to do if an illegal > character is encounter should be standardized as well... (though this > applies more generally to "what do you do if you receive an invalid > name in structured data) > > Off the top of my head, you have only 1 of 3 choices: > (1) strip the offending characters > (2) strip the whole name=value pair > (3) drop the entire message You are absolutely right, with a special choice (4) drop the structured data element in which the invalid field exists But that's only an additional option because we have this additional layer ;) In RFC5424, we left this somewhat open, because some users insist on seeing even malformed data (if you think about it, there is some logic in this, otherwise you may have another attack factor). The overall spirit of RFC5424 is to either drop things that are malformed, change them into some safe encoding for local storage (like NUL -> "#00" -or so- for further processing) but keep them untouched when forwarding (because the next receiver may make sense of them). All of this is very debatable, and indeed we had very lengthy debates in the syslog wg ;) Rainer From sivan at omniqueue.com Wed Nov 3 19:12:10 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 3 Nov 2010 20:12:10 +0200 Subject: [rsyslog] EventId field In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, again! Now, actually to the real question ;) I would like to customize the fields rsyslog sends, or rather stores into MySQL. So for example, I would like to have a special ID that I could identify a group of related messages. Is that possible? Here[0] is is exemplified how to arrange the fields from rsyslog into mysql, so I would like to have something a long these lines, but to be able to add my own field so I can group by it instead of while executing an aftermath to analyze the mysql data for an id inside the message text. [0]: http://www.rsyslog.com/doc//ommysql.html Cheers, and thanks in advance! -Sivan On Wed, Nov 3, 2010 at 5:01 PM, Sivan Greenberg wrote: > Hey! > > On Wed, Nov 3, 2010 at 4:19 PM, Rainer Gerhards > wrote: >> Hi Sivan, >> >> that field is normally not used by rsyslog ;) The schema supports all of >> Adiscon's MonitorWare products. EventID is usually populated by the >> (commercial) Windows products, which store the Windows Event ID there. Having >> a common schema simplifies things, so I pulled the complete schema over. > > Indeed, thanks for the note. That makes a lot of sense. > > -Sivan > From david at lang.hm Wed Nov 3 21:22:46 2010 From: david at lang.hm (david at lang.hm) Date: Wed, 3 Nov 2010 13:22:46 -0700 (PDT) Subject: [rsyslog] Feedback Request: NULs in strings? In-Reply-To: <93ED0A84F9A1D74FA65021D940AA5884053E75C186@IMCMBX3.MITRE.ORG> References: <20101103162229.GC21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD62F@GRFEXC.intern.adiscon.com><20101103164949.GD21709@brevard.conman.org><9B6E2A8877C38245BFB15CC491A11DA71DD631@GRFEXC.intern.adiscon.com> <93ED0A84F9A1D74FA65021D940AA5884053E75C160@IMCMBX3.MITRE.ORG> <9B6E2A8877C38245BFB15CC491A11DA71DD635@GRFEXC.intern.adiscon.com> <93ED0A84F9A1D74FA65021D940AA5884053E75C186@IMCMBX3.MITRE.ORG> Message-ID: On Wed, 3 Nov 2010, Heinbockel, Bill wrote: >>> For things like the RFC5424 structured data, the name in name=value >>> pairs cannot contain characters such as NUL. >>> In my option, what to do if you receive a Syslog message with a NUL in >>> the name, is the bigger issue >> >> Fortunately not. In RFC5424, full UTF-8 is permitted in values. However, the >> name alphabet is much more restricted. Out of my head, I think it is US- >> ASCII >> minus the control character set. During syslog standardization, we did not >> see need to support a larger alphabet (but I admit that it is debatable if >> national characters should be supported in names -- after long discussion we >> said "no"). > > While it is against the specification, how do you handle the case were an invalid character is included in the name? This is an issue where most programs have issues; they assume that the message will according to the specification. The approach of what to do if an illegal character is encounter should be standardized as well... (though this applies more generally to "what do you do if you receive an invalid name in structured data) > > Off the top of my head, you have only 1 of 3 choices: > (1) strip the offending characters > (2) strip the whole name=value pair > (3) drop the entire message (4) change the offending characters in the messaage I've seen some software that just replaces all invalid characters with 'X' ryslog has a couple of ways to escape such characters (replacing them with #xxx for example) dropping the message, dropping the element, or dropping the characters all loose significant data. replacing the character with something else at least lets someone looking at it figure out that there was some garbage in the message (and if it's an ongong issue, they can sniff the network to see the raw data if nothing else) going back to the discussion on UTF data that we had several months ago, most of the time you can treat the data as an octet stream, but when you parse the data or display the data you need to worry about this sort of thing. when parsing a message, I think it's probably sanest to consider a NULL to be an end-of-message character in the datastream (unless the user explicitly configures the receiving library to allow nulls and escape them) NULL is a special case since it is used to represent the end of a string in C, and as a result in many other languages that are written in C. As a result the probability of it causing problems in other tools is _extremely_ high, much more so than any escape sequence or control characters. As a result, everything downstream should be protected UNLESS the user explicitly says to pass it through (and even then I would want to have the documentation strongly recommend against doing so) David Lang From david at lang.hm Wed Nov 3 21:25:35 2010 From: david at lang.hm (david at lang.hm) Date: Wed, 3 Nov 2010 13:25:35 -0700 (PDT) Subject: [rsyslog] EventId field In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 3 Nov 2010, Sivan Greenberg wrote: > Hi Rainer, again! > > Now, actually to the real question ;) > > I would like to customize the fields rsyslog sends, or rather stores > into MySQL. So for example, I would like to have a special ID that I > could identify a group of related messages. > > Is that possible? Here[0] is is exemplified how to arrange the fields > from rsyslog into mysql, so I would like to have something a long > these lines, but to be able to add my own field so I can group by it > instead of while executing an aftermath to analyze the mysql data for > an id inside the message text. > > [0]: http://www.rsyslog.com/doc//ommysql.html as this page shows, you can setup your own template to put whatever properties you want in whatever database columns (or tables) you want. just craft the SQL statement that you want to run and use that template. David Lang > Cheers, and thanks in advance! > > -Sivan > On Wed, Nov 3, 2010 at 5:01 PM, Sivan Greenberg wrote: >> Hey! >> >> On Wed, Nov 3, 2010 at 4:19 PM, Rainer Gerhards >> wrote: >>> Hi Sivan, >>> >>> that field is normally not used by rsyslog ;) The schema supports all of >>> Adiscon's MonitorWare products. EventID is usually populated by the >>> (commercial) Windows products, which store the Windows Event ID there. Having >>> a common schema simplifies things, so I pulled the complete schema over. >> >> Indeed, thanks for the note. That makes a lot of sense. >> >> -Sivan >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jason at jasonantman.com Wed Nov 3 21:48:27 2010 From: jason at jasonantman.com (Jason Antman) Date: Wed, 03 Nov 2010 16:48:27 -0400 Subject: [rsyslog] Rsyslog evaluation questions Message-ID: <4CD1CA9B.3090301@jasonantman.com> Hello, At $UNIVERSITY, we're planning to overhaul our centralized syslog infrastructure. At the moment, we're running syslog-ng on an aged Solaris/Sparc box and planning on replacing it with either syslog-ng or rsyslog on CentOS 5. We're currently handling approximately 3 million lines per day from 25 hosts. We can expect that to increase to approx. 5M+ lines from 40 hosts within 6 months. We're beginning an evaluation of rsyslog, and I have a few preliminary questions I can't find clear answers to in the docs, and was hoping perhaps someone with rsyslog in production could provide some input. 1) We do a lot of post-processing of logs to pull out relevant information. Specifically, we have quite a few scripts (PHP) that rely on preg_match pulling out capture groups and putting them in different columns in MySQL. Does rsyslog support any regexes in templates that would allow something like this? 2) I last visited rsyslog.com before the site redesign. Where has the documentation gone on the new site? I can't even seem to find a config file statement reference... 3) Assuming an even distribution over time (not quite accurate), any thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) with a single 2.8GHz CPU and 512MB RAM would go? Thanks for any advice, Jason Antman From marc.schiffbauer at mightycare.de Mon Nov 1 16:29:55 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Mon, 1 Nov 2010 16:29:55 +0100 (CET) Subject: [rsyslog] $OMFileZipLevel In-Reply-To: <1774970529.592.1288624821021.JavaMail.root@zimbra> Message-ID: <1355694908.596.1288625395245.JavaMail.root@zimbra> Hi all, is there a way to set $OMFileZipLevel for just one output file, not all? (rsyslog 4.6.4) TIA -Marc From sivan at omniqueue.com Wed Nov 3 22:18:08 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 3 Nov 2010 23:18:08 +0200 Subject: [rsyslog] EventId field In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> Message-ID: On Wed, Nov 3, 2010 at 10:25 PM, wrote: > On Wed, 3 Nov 2010, Sivan Greenberg wrote: > >> Hi Rainer, again! >> >> Now, actually to the real question ;) >> >> I would like to customize the fields rsyslog sends, or rather stores >> into MySQL. So for example, I would like to have a special ID that I >> could identify a group of related messages. >> >> Is that possible? Here[0] is is exemplified how to arrange the fields >> from rsyslog into mysql, so I would like to have something a long >> these lines, but to be able to add my own field so I can group by it >> instead of while executing an aftermath to analyze the mysql data for >> an id inside the message text. >> >> [0]: ?http://www.rsyslog.com/doc//ommysql.html > > as this page shows, you can setup your own template to put whatever > properties you want in whatever database columns (or tables) you want. > Yes, but how do I control which parts of a message will be treated as which fields? You see, I would like to have a %request_id% which would be the last field before the actual message text for example. Cheers, -Sivan From rgerhards at hq.adiscon.com Thu Nov 4 07:53:55 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 07:53:55 +0100 Subject: [rsyslog] $OMFileZipLevel References: <1355694908.596.1288625395245.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD63A@GRFEXC.intern.adiscon.com> The directive applies to the next output file. So you can set it in front of each output action to a different value. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Monday, November 01, 2010 4:30 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] $OMFileZipLevel > > Hi all, > > is there a way to set $OMFileZipLevel for just one output file, not > all? > (rsyslog 4.6.4) > > TIA > -Marc > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Nov 4 07:56:24 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 07:56:24 +0100 Subject: [rsyslog] EventId field References: <9B6E2A8877C38245BFB15CC491A11DA71DD61D@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD63B@GRFEXC.intern.adiscon.com> > Yes, but how do I control which parts of a message will be treated as > which fields? You see, I would like to have a %request_id% which would > be the last field before the actual message text for example. You need to look into the rsyslog property replacer, which can be used to extract parts of a fieled (e.g. msg or rawmsg). Given what you said, you probably need to work with regular expressions, but may get along with a cheaper thing, like delimited fields. To help crafting regexes, there is an online check tool available at: http://www.rsyslog.com/regex/ Rainer From linuxmail at 4lin.net Thu Nov 4 07:58:54 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Thu, 04 Nov 2010 07:58:54 +0100 Subject: [rsyslog] 4.7: discard seems to not working Message-ID: <1288853934.23247.23.camel@pcdenny.rbg.informatik.tu-darmstadt.de> hi, i want to delete some messages: :msg, startswith, "seahorse-agent:" ~ :msg, startswith, "dbus-daemon:" ~ :msg, startswith, "gnome-keyring-daemon:" ~ :msg, startswith, "quota_ufs:" ~ & ~ but it doesn't work: ufs: [ID 708496 kern.notice] quota_ufs: over hard disk limit (pid 578,... seahorse-agent[3085]: file seahorse-ssh-source.c: line ... is there any wrong with that? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From rgerhards at hq.adiscon.com Thu Nov 4 08:00:23 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 08:00:23 +0100 Subject: [rsyslog] 4.7: discard seems to not working References: <1288853934.23247.23.camel@pcdenny.rbg.informatik.tu-darmstadt.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD63E@GRFEXC.intern.adiscon.com> Yup, you have only one discard, at the last action. I guess you meant: :msg, startswith, "seahorse-agent:" ~ & ~ :msg, startswith, "dbus-daemon:" ~ & ~ :msg, startswith, "gnome-keyring-daemon:" ~ & ~ :msg, startswith, "quota_ufs:" ~ & ~ Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Denny Schierz > Sent: Thursday, November 04, 2010 7:59 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] 4.7: discard seems to not working > > hi, > > i want to delete some messages: > > :msg, startswith, "seahorse-agent:" ~ > :msg, startswith, "dbus-daemon:" ~ > :msg, startswith, "gnome-keyring-daemon:" ~ > :msg, startswith, "quota_ufs:" ~ > > & ~ > > but it doesn't work: > ufs: [ID 708496 kern.notice] quota_ufs: over hard disk limit > (pid 578,... > > seahorse-agent[3085]: file seahorse-ssh-source.c: line > ... > > is there any wrong with that? > > cu denny > From rgerhards at hq.adiscon.com Thu Nov 4 08:01:15 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 08:01:15 +0100 Subject: [rsyslog] 4.7: discard seems to not working References: <1288853934.23247.23.camel@pcdenny.rbg.informatik.tu-darmstadt.de> <9B6E2A8877C38245BFB15CC491A11DA71DD63E@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD63F@GRFEXC.intern.adiscon.com> Lol, sorry, totally scrwed myself. This is such a frequent error that I did not notice what you really did. Besides the last & ~ it looks good. I think a startup debug log would be useful. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, November 04, 2010 8:00 AM > To: rsyslog-users > Subject: Re: [rsyslog] 4.7: discard seems to not working > > Yup, you have only one discard, at the last action. I guess you meant: > > :msg, startswith, "seahorse-agent:" ~ > & ~ > :msg, startswith, "dbus-daemon:" ~ > & ~ > :msg, startswith, "gnome-keyring-daemon:" ~ > & ~ > :msg, startswith, "quota_ufs:" ~ > & ~ > > Rainer > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Denny Schierz > > Sent: Thursday, November 04, 2010 7:59 AM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] 4.7: discard seems to not working > > > > hi, > > > > i want to delete some messages: > > > > :msg, startswith, "seahorse-agent:" ~ > > :msg, startswith, "dbus-daemon:" ~ > > :msg, startswith, "gnome-keyring-daemon:" ~ > > :msg, startswith, "quota_ufs:" ~ > > > > & ~ > > > > but it doesn't work: > > ufs: [ID 708496 kern.notice] quota_ufs: over hard disk limit > > (pid 578,... > > > > seahorse-agent[3085]: file seahorse-ssh-source.c: line > > ... > > > > is there any wrong with that? > > > > cu denny > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marc.schiffbauer at mightycare.de Thu Nov 4 10:33:56 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 10:33:56 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <998847569.1127.1288862742968.JavaMail.root@zimbra> Message-ID: <166441385.1129.1288863236907.JavaMail.root@zimbra> Hi, I have a rather complex config which puts logs into a database by using several Templates with different SQL-Statements and so I use several Actions which match different log-lines. And I want rsyslog to do disk assisted queuing in case of DB-failure or lag. Question: Is is possible in 4.6.4 to have ONE queue with MULTIPLE actions so that queued SQL-Statements will be processed in correct order like it would be done without queueing? Or do I have to use the MainQueue for that which will affect log-files in case of DB failure? TIA -Marc From linuxmail at 4lin.net Thu Nov 4 10:38:53 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Thu, 04 Nov 2010 10:38:53 +0100 Subject: [rsyslog] 4.7: discard seems to not working In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD63E@GRFEXC.intern.adiscon.com> References: <1288853934.23247.23.camel@pcdenny.rbg.informatik.tu-darmstadt.de> <9B6E2A8877C38245BFB15CC491A11DA71DD63E@GRFEXC.intern.adiscon.com> Message-ID: <1288863533.23247.42.camel@pcdenny.rbg.informatik.tu-darmstadt.de> hi, it seems, that was it: Am Donnerstag, den 04.11.2010, 08:00 +0100 schrieb Rainer Gerhards: > :msg, startswith, "seahorse-agent:" ~ > & ~ > :msg, startswith, "dbus-daemon:" ~ > & ~ > :msg, startswith, "gnome-keyring-daemon:" ~ > & ~ > :msg, startswith, "quota_ufs:" ~ > & ~ now it works, only I did not understand, why for example: :msg, startswith, "dbus-daemon:" ~ or: :msg, contains, "dbus-daemon:" ~ doesn't work. I had to search for something in the text. So, if there some more messages from dbus, the first or second example aren't successful: a sample message: Nov 4 10:03:42 client79 dbus-daemon: Rejected send message, 3 matched rules; type="method_call", sender=":1.16" (uid=14326 pid=3104 comm="nautilus --no-default-window --sm-client-id defaul") interface="org.freedesktop.SystemToolsBackends" member="get" error name="(unset)" requested_reply=0 destination="org.freedesktop.SystemToolsBackends" (uid=0 pid=2859 comm="/usr/bin/system-tools-backends ")) any suggestions? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From rgerhards at hq.adiscon.com Thu Nov 4 10:41:15 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 10:41:15 +0100 Subject: [rsyslog] 4.7: discard seems to not working References: <1288853934.23247.23.camel@pcdenny.rbg.informatik.tu-darmstadt.de><9B6E2A8877C38245BFB15CC491A11DA71DD63E@GRFEXC.intern.adiscon.com> <1288863533.23247.42.camel@pcdenny.rbg.informatik.tu-darmstadt.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD644@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Denny Schierz > Sent: Thursday, November 04, 2010 10:39 AM > To: rsyslog-users > Subject: Re: [rsyslog] 4.7: discard seems to not working > > hi, > > it seems, that was it: > > Am Donnerstag, den 04.11.2010, 08:00 +0100 schrieb Rainer Gerhards: > > > :msg, startswith, "seahorse-agent:" ~ > > & ~ > > :msg, startswith, "dbus-daemon:" ~ > > & ~ > > :msg, startswith, "gnome-keyring-daemon:" ~ > > & ~ > > :msg, startswith, "quota_ufs:" ~ > > & ~ > > now it works, only I did not understand, why for example: > > :msg, startswith, "dbus-daemon:" ~ > > or: > > :msg, contains, "dbus-daemon:" ~ > > doesn't work. Sorry, I guess you didn't get my second mail. I screwed up, this should not make any difference at all! > I had to search for something in the text. So, if there > some more messages from dbus, the first or second example aren't > successful: > > a sample message: > > Nov 4 10:03:42 client79 dbus-daemon: Rejected send message, 3 matched > rules; type="method_call", sender=":1.16" (uid=14326 pid=3104 > comm="nautilus --no-default-window --sm-client-id defaul") > interface="org.freedesktop.SystemToolsBackends" member="get" error > name="(unset)" requested_reply=0 > destination="org.freedesktop.SystemToolsBackends" (uid=0 pid=2859 > comm="/usr/bin/system-tools-backends ")) > > any suggestions? Can you add *.* /paht/to/debug.log;RSYSLOG_DebugFormat and let us know what the debug format says for those messages that do not work. Rainer From linuxmail at 4lin.net Thu Nov 4 15:23:09 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Thu, 04 Nov 2010 15:23:09 +0100 Subject: [rsyslog] 4.7: discard seems to not working In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD644@GRFEXC.intern.adiscon.com> References: <1288853934.23247.23.camel@pcdenny.rbg.informatik.tu-darmstadt.de> <9B6E2A8877C38245BFB15CC491A11DA71DD63E@GRFEXC.intern.adiscon.com> <1288863533.23247.42.camel@pcdenny.rbg.informatik.tu-darmstadt.de> <9B6E2A8877C38245BFB15CC491A11DA71DD644@GRFEXC.intern.adiscon.com> Message-ID: <1288880589.490.7.camel@pcdenny.rbg.informatik.tu-darmstadt.de> Hi Rainer, > Sorry, I guess you didn't get my second mail. I screwed up, this should not > make any difference at all! changed the config and removed the unnecessarily & ~ ... it works too, but the other one not: the startup log is attached. cu denny -------------- next part -------------- 9959.814194850:1: rsyslogd 4.7.2 startup, compatibility mode 4, module path '', cwd:/usr/local/rsyslog-4.7.2 9959.814512050:1: caller requested object 'net', not found (iRet -3003) 9959.814543250:1: Requested to load module 'lmnet' 9959.814576100:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/lmnet.so' 9959.816398500:1: module of type 2 being loaded. 9959.816436300:1: source file conf.c requested reference for module 'lmnet', reference count now 1 9959.816466800:1: rsyslog runtime initialized, version 4.7.2, current users 1 9959.816514250:1: source file syslogd.c requested reference for module 'lmnet', reference count now 2 9959.817606150:1: module of type 1 being loaded. 9959.817649700:1: module of type 1 being loaded. 9959.817683850:1: source file omfwd.c requested reference for module 'lmnet', reference count now 3 9959.817737550:1: module of type 1 being loaded. 9959.817759550:1: entry point 'doHUP' not present in module 9959.817782650:1: module of type 1 being loaded. 9959.817803900:1: entry point 'doHUP' not present in module 9959.826947450:1: module of type 1 being loaded. 9959.826983750:1: entry point 'doHUP' not present in module 9959.827008900:1: module of type 1 being loaded. 9959.827030000:1: entry point 'doHUP' not present in module 9959.827556950:1: deque option f, optarg '/usr/local/rsyslog/etc/rsyslog.conf' 9959.827582500:1: deque option n, optarg '' 9959.827625450:1: Writing pidfile /var/run/rsyslogd.pid. 9959.828252600:1: rsyslog 4.7.2 - called init() 9959.828282150:1: Unloading non-static modules. 9959.828305450:1: module lmnet NOT unloaded because it still has a refcount of 3 9959.828324150:1: Clearing templates. 9959.828498400:1: cfline: ':msg, contains, "seahorse_ssh_source_init" ~' 9959.828526550:1: - property-based filter 9959.828561900:1: tried selector action for builtin-file: -2001 9959.828582950:1: tried selector action for builtin-pipe: -2001 9959.828603850:1: tried selector action for builtin-fwd: -2001 9959.828624050:1: tried selector action for builtin-shell: -2001 9959.828643950:1: discard 9959.828679250:1: tried selector action for builtin-discard: 0 9959.828700650:1: Module builtin-discard processed this config line. 9959.828731400:1: module is incompatible with RepeatedMsgReduction - turned off 9959.828771350:1: action 1 queue: save on shutdown 1, max disk space allowed 0 9959.828811050:1: action 1 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.828836600:1: Action 7c588: queue 7c620 created 9959.828856500:1: module is incompatible with RepeatedMsgReduction - turned off 9959.828878050:1: cfline: ':msg, contains, "monitor the GPG home directory" ~' 9959.828897900:1: selector line successfully processed 9959.828916450:1: - property-based filter 9959.828942450:1: tried selector action for builtin-file: -2001 9959.828961700:1: tried selector action for builtin-pipe: -2001 9959.828981350:1: tried selector action for builtin-fwd: -2001 9959.829001350:1: tried selector action for builtin-shell: -2001 9959.829027150:1: discard 9959.829046750:1: tried selector action for builtin-discard: 0 9959.829064650:1: Module builtin-discard processed this config line. 9959.829085450:1: module is incompatible with RepeatedMsgReduction - turned off 9959.829143400:1: action 2 queue: save on shutdown 1, max disk space allowed 0 9959.829174950:1: action 2 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.829199350:1: Action 7c908: queue 7c9a0 created 9959.829217700:1: module is incompatible with RepeatedMsgReduction - turned off 9959.829238200:1: cfline: ':msg, contains, "dbus-daemon" ~' 9959.829257000:1: selector line successfully processed 9959.829275500:1: - property-based filter 9959.829300650:1: tried selector action for builtin-file: -2001 9959.829320050:1: tried selector action for builtin-pipe: -2001 9959.829339350:1: tried selector action for builtin-fwd: -2001 9959.829358850:1: tried selector action for builtin-shell: -2001 9959.829377200:1: discard 9959.829403750:1: tried selector action for builtin-discard: 0 9959.829421900:1: Module builtin-discard processed this config line. 9959.829442150:1: module is incompatible with RepeatedMsgReduction - turned off 9959.829464750:1: action 3 queue: save on shutdown 1, max disk space allowed 0 9959.829491450:1: action 3 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.829515050:1: Action 7cc08: queue 7cca0 created 9959.829532800:1: module is incompatible with RepeatedMsgReduction - turned off 9959.829552850:1: cfline: ':msg, contains, "freedesktop" ~' 9959.829571300:1: selector line successfully processed 9959.829589400:1: - property-based filter 9959.829613900:1: tried selector action for builtin-file: -2001 9959.829633300:1: tried selector action for builtin-pipe: -2001 9959.829652700:1: tried selector action for builtin-fwd: -2001 9959.829671800:1: tried selector action for builtin-shell: -2001 9959.829689850:1: discard 9959.829714600:1: tried selector action for builtin-discard: 0 9959.829733850:1: Module builtin-discard processed this config line. 9959.829754100:1: module is incompatible with RepeatedMsgReduction - turned off 9959.829776550:1: action 4 queue: save on shutdown 1, max disk space allowed 0 9959.829803750:1: action 4 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.829827450:1: Action 7cf08: queue 7cfa0 created 9959.829845650:1: module is incompatible with RepeatedMsgReduction - turned off 9959.829866500:1: cfline: ':msg, contains, "gnome-keyring-daemon" ~' 9959.829885250:1: selector line successfully processed 9959.829903550:1: - property-based filter 9959.829928700:1: tried selector action for builtin-file: -2001 9959.829948050:1: tried selector action for builtin-pipe: -2001 9959.829967250:1: tried selector action for builtin-fwd: -2001 9959.829986800:1: tried selector action for builtin-shell: -2001 9959.830004950:1: discard 9959.830029550:1: tried selector action for builtin-discard: 0 9959.830048550:1: Module builtin-discard processed this config line. 9959.830068800:1: module is incompatible with RepeatedMsgReduction - turned off 9959.830091250:1: action 5 queue: save on shutdown 1, max disk space allowed 0 9959.830118600:1: action 5 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.830142200:1: Action 7d208: queue 7d2a0 created 9959.830160000:1: module is incompatible with RepeatedMsgReduction - turned off 9959.830179800:1: cfline: ':msg, contains, "quota_ufs" ~' 9959.830198200:1: selector line successfully processed 9959.830216650:1: - property-based filter 9959.830241200:1: tried selector action for builtin-file: -2001 9959.830260200:1: tried selector action for builtin-pipe: -2001 9959.830279300:1: tried selector action for builtin-fwd: -2001 9959.830298400:1: tried selector action for builtin-shell: -2001 9959.830316550:1: discard 9959.830334000:1: tried selector action for builtin-discard: 0 9959.830359250:1: Module builtin-discard processed this config line. 9959.830379500:1: module is incompatible with RepeatedMsgReduction - turned off 9959.830401750:1: action 6 queue: save on shutdown 1, max disk space allowed 0 9959.830429000:1: action 6 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.830452450:1: Action 7d508: queue 7d5a0 created 9959.830470300:1: module is incompatible with RepeatedMsgReduction - turned off 9959.830489950:1: cfline: ':msg, contains, "HAL" ~' 9959.830508400:1: selector line successfully processed 9959.830526850:1: - property-based filter 9959.830551400:1: tried selector action for builtin-file: -2001 9959.830570750:1: tried selector action for builtin-pipe: -2001 9959.830590150:1: tried selector action for builtin-fwd: -2001 9959.830609600:1: tried selector action for builtin-shell: -2001 9959.830627850:1: discard 9959.830645350:1: tried selector action for builtin-discard: 0 9959.830724650:1: Module builtin-discard processed this config line. 9959.830747450:1: module is incompatible with RepeatedMsgReduction - turned off 9959.830770350:1: action 7 queue: save on shutdown 1, max disk space allowed 0 9959.830797500:1: action 7 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.830820750:1: Action 7d808: queue 7d8a0 created 9959.830839100:1: module is incompatible with RepeatedMsgReduction - turned off 9959.830858700:1: cfline: '& ~' 9959.830877600:1: tried selector action for builtin-file: -2001 9959.830895500:1: tried selector action for builtin-pipe: -2001 9959.830914300:1: tried selector action for builtin-fwd: -2001 9959.830933750:1: tried selector action for builtin-shell: -2001 9959.830952000:1: discard 9959.830969800:1: tried selector action for builtin-discard: 0 9959.830987050:1: Module builtin-discard processed this config line. 9959.831006900:1: module is incompatible with RepeatedMsgReduction - turned off 9959.831061300:1: action 8 queue: save on shutdown 1, max disk space allowed 0 9959.831090350:1: action 8 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.831114550:1: Action 7da20: queue 7dab8 created 9959.831132650:1: module is incompatible with RepeatedMsgReduction - turned off 9959.831171450:1: cfline: '$ModLoad immark # provides --MARK-- message capability' 9959.831203700:1: Requested to load module 'immark' 9959.831234550:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/immark.so' 9959.831662950:1: module of type 0 being loaded. 9959.831699200:1: cfline: '$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)' 9959.831725000:1: Requested to load module 'imuxsock' 9959.831746200:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/imuxsock.so' 9959.832168000:1: imuxsock version 4.7.2 initializing 9959.832272750:1: module of type 0 being loaded. 9959.832313800:1: cfline: '$ModLoad imtcp.so # load module' 9959.832339750:1: Requested to load module 'imtcp.so' 9959.832361500:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/imtcp.so' 9959.832691850:1: source file imtcp.c requested reference for module 'lmnet', reference count now 4 9959.832725250:1: caller requested object 'netstrm', not found (iRet -3003) 9959.832794650:1: Requested to load module 'lmnetstrms' 9959.832816950:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/lmnetstrms.so' 9959.833234600:1: module of type 2 being loaded. 9959.833264500:1: source file imtcp.c requested reference for module 'lmnetstrms', reference count now 1 9959.833290300:1: caller requested object 'tcps_sess', not found (iRet -3003) 9959.833309100:1: Requested to load module 'lmtcpsrv' 9959.833329800:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/lmtcpsrv.so' 9959.833733850:1: source file tcps_sess.c requested reference for module 'lmnetstrms', reference count now 2 9959.833775200:1: source file tcpsrv.c requested reference for module 'lmnet', reference count now 5 9959.833808850:1: source file tcpsrv.c requested reference for module 'lmnetstrms', reference count now 3 9959.833846250:1: module of type 2 being loaded. 9959.833867750:1: source file imtcp.c requested reference for module 'lmtcpsrv', reference count now 1 9959.833890350:1: source file imtcp.c requested reference for module 'lmtcpsrv', reference count now 2 9959.834046600:1: module of type 0 being loaded. 9959.834082900:1: cfline: '$InputTCPServerRun 514 # start up TCP listener at port 514' 9959.834127050:1: cfline: '$ModLoad imudp.so # provides UDP syslog reception' 9959.834153500:1: Requested to load module 'imudp.so' 9959.834174750:1: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/imudp.so' 9959.834549500:1: source file imudp.c requested reference for module 'lmnet', reference count now 6 9959.834613400:1: module of type 0 being loaded. 9959.834643900:1: cfline: '$UDPServerRun 514 # start a UDP syslog server at standard port 514' 9959.834684350:1: Trying to open syslog UDP ports at *:514. 9959.834999900:1: cfline: '$InputTCPServerRun 10514' 9959.835049800:1: cfline: '$FileOwner root' 9959.835187100:1: uid 0 obtained for user 'root' 9959.835216100:1: cfline: '$FileGroup adm' 9959.835297000:1: gid 4 obtained for group 'adm' 9959.835326450:1: cfline: '$FileCreateMode 0640' 9959.835349750:1: cfline: '$DirCreateMode 0755' 9959.835381950:1: cfline: '$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat' 9959.835409000:1: cfline: '$template DynAuth, "/var/log/remote/%HOSTNAME%/secure.log"' 9959.835449300:1: cfline: '$template DynMSG, "/var/log/remote/%HOSTNAME%/messages"' 9959.835530850:1: cfline: '$template Dynmail, "/var/log/remote/%HOSTNAME%/maillog"' 9959.835569950:1: cfline: '$template Dyncron,"/var/log/remote/%HOSTNAME%/cron"' 9959.835604650:1: cfline: '$template Dynspool, "/var/log/remote/%HOSTNAME%/spooler"' 9959.835638650:1: cfline: '$template Dynboot, "/var/log/remote/%HOSTNAME%/boot.log"' 9959.835671750:1: cfline: '$template Dynlocal3, "/var/log/remote/%HOSTNAME%/local3.log"' 9959.835719100:1: cfline: '$template Dynlocal4, "/var/log/remote/%HOSTNAME%/local4.log"' 9959.835753950:1: cfline: '$template Dynlocal5, "/var/log/remote/%HOSTNAME%/local5.log"' 9959.835787300:1: cfline: '$template Dynlocal6, "/var/log/remote/%HOSTNAME%/local6.log"' 9959.835820200:1: cfline: '$template Dynlocal7, "/var/log/remote/%HOSTNAME%/local7.log"' 9959.835852400:1: cfline: 'authpriv.* ?DynAuth' 9959.835874100:1: selector line successfully processed 9959.835893650:1: - traditional PRI filter 9959.835914150:1: symbolic name: * ==> 255 9959.835950350:1: symbolic name: authpriv ==> 32 9959.835985500:1: tried selector action for builtin-file: 0 9959.836006150:1: Module builtin-file processed this config line. 9959.836029350:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.836048450:1: template: 'DynAuth' assigned 9959.836074950:1: action 9 queue: save on shutdown 1, max disk space allowed 0 9959.836104650:1: action 9 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.836136800:1: Action 81f68: queue 82000 created 9959.836161150:1: cfline: '*.info,mail.none,authpriv.none,cron.none ?DynMSG' 9959.836180650:1: selector line successfully processed 9959.836198850:1: - traditional PRI filter 9959.836217450:1: symbolic name: info ==> 6 9959.836243350:1: symbolic name: none ==> 16 9959.836268250:1: symbolic name: mail ==> 16 9959.836293050:1: symbolic name: none ==> 16 9959.836317300:1: symbolic name: authpriv ==> 32 9959.836341600:1: symbolic name: none ==> 16 9959.836365750:1: symbolic name: cron ==> 120 9959.836393900:1: tried selector action for builtin-file: 0 9959.836412350:1: Module builtin-file processed this config line. 9959.836433550:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.836451850:1: template: 'DynMSG' assigned 9959.836473550:1: action 10 queue: save on shutdown 1, max disk space allowed 0 9959.836500550:1: action 10 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.836531450:1: Action 82348: queue 823e0 created 9959.836553350:1: cfline: 'mail.* -?Dynmail' 9959.836572300:1: selector line successfully processed 9959.836590450:1: - traditional PRI filter 9959.836608600:1: symbolic name: * ==> 255 9959.836633200:1: symbolic name: mail ==> 16 9959.836661200:1: tried selector action for builtin-file: 0 9959.836679900:1: Module builtin-file processed this config line. 9959.836701300:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.836719750:1: template: 'Dynmail' assigned 9959.836741550:1: action 11 queue: save on shutdown 1, max disk space allowed 0 9959.836768500:1: action 11 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.836792000:1: Action 826f8: queue 82790 created 9959.836812150:1: cfline: 'cron.* ?Dyncron' 9959.836830500:1: selector line successfully processed 9959.836848850:1: - traditional PRI filter 9959.836928600:1: symbolic name: * ==> 255 9959.836956500:1: symbolic name: cron ==> 120 9959.837019950:1: tried selector action for builtin-file: 0 9959.837042650:1: Module builtin-file processed this config line. 9959.837064700:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.837083150:1: template: 'Dyncron' assigned 9959.837105300:1: action 12 queue: save on shutdown 1, max disk space allowed 0 9959.837132300:1: action 12 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.837155300:1: Action 82ad8: queue 82b70 created 9959.837175950:1: cfline: 'news.crit ?Dynspool' 9959.837194700:1: selector line successfully processed 9959.837212650:1: - traditional PRI filter 9959.837230650:1: symbolic name: crit ==> 2 9959.837254900:1: symbolic name: news ==> 56 9959.837282850:1: tried selector action for builtin-file: 0 9959.837301400:1: Module builtin-file processed this config line. 9959.837322300:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.837349100:1: template: 'Dynspool' assigned 9959.837372300:1: action 13 queue: save on shutdown 1, max disk space allowed 0 9959.837399550:1: action 13 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.837422750:1: Action 82eb8: queue 82f50 created 9959.837443000:1: cfline: 'local3.* ?Dynlocal3' 9959.837461550:1: selector line successfully processed 9959.837479700:1: - traditional PRI filter 9959.837497900:1: symbolic name: * ==> 255 9959.837522700:1: symbolic name: local3 ==> 152 9959.837551250:1: tried selector action for builtin-file: 0 9959.837569900:1: Module builtin-file processed this config line. 9959.837591000:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.837609550:1: template: 'Dynlocal3' assigned 9959.837630850:1: action 14 queue: save on shutdown 1, max disk space allowed 0 9959.837657550:1: action 14 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.837687600:1: Action 83298: queue 83330 created 9959.837709050:1: cfline: 'local4.* ?Dynlocal4' 9959.837727750:1: selector line successfully processed 9959.837745700:1: - traditional PRI filter 9959.837763650:1: symbolic name: * ==> 255 9959.837788300:1: symbolic name: local4 ==> 160 9959.837816700:1: tried selector action for builtin-file: 0 9959.837835050:1: Module builtin-file processed this config line. 9959.837855850:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.837874300:1: template: 'Dynlocal4' assigned 9959.837895400:1: action 15 queue: save on shutdown 1, max disk space allowed 0 9959.837921800:1: action 15 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.837945250:1: Action 83678: queue 83710 created 9959.837965050:1: cfline: 'local5.* ?Dynlocal5' 9959.837983400:1: selector line successfully processed 9959.838001700:1: - traditional PRI filter 9959.838019650:1: symbolic name: * ==> 255 9959.838052400:1: symbolic name: local5 ==> 168 9959.838081750:1: tried selector action for builtin-file: 0 9959.838100150:1: Module builtin-file processed this config line. 9959.838121400:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.838140150:1: template: 'Dynlocal5' assigned 9959.838161650:1: action 16 queue: save on shutdown 1, max disk space allowed 0 9959.838188400:1: action 16 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.838211700:1: Action 83a58: queue 83af0 created 9959.838231550:1: cfline: 'local6.* ?Dynlocal6' 9959.838250150:1: selector line successfully processed 9959.838267850:1: - traditional PRI filter 9959.838285550:1: symbolic name: * ==> 255 9959.838310150:1: symbolic name: local6 ==> 176 9959.838337850:1: tried selector action for builtin-file: 0 9959.838356350:1: Module builtin-file processed this config line. 9959.838377100:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.838402300:1: template: 'Dynlocal6' assigned 9959.838424850:1: action 17 queue: save on shutdown 1, max disk space allowed 0 9959.838451500:1: action 17 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.838474800:1: Action 83e38: queue 83ed0 created 9959.838494550:1: cfline: 'local7.* ?Dynlocal7' 9959.838513350:1: selector line successfully processed 9959.838531550:1: - traditional PRI filter 9959.838549350:1: symbolic name: * ==> 255 9959.838573950:1: symbolic name: local7 ==> 184 9959.838602200:1: tried selector action for builtin-file: 0 9959.838620500:1: Module builtin-file processed this config line. 9959.838641500:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.838659900:1: template: 'Dynlocal7' assigned 9959.838681150:1: action 18 queue: save on shutdown 1, max disk space allowed 0 9959.838708100:1: action 18 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.838738000:1: Action 84218: queue 842b0 created 9959.838761950:1: cfline: '*.emerg *' 9959.838780700:1: selector line successfully processed 9959.838798500:1: - traditional PRI filter 9959.838816800:1: symbolic name: emerg ==> 0 9959.838842450:1: tried selector action for builtin-file: -2001 9959.838860800:1: tried selector action for builtin-pipe: -2001 9959.838879950:1: tried selector action for builtin-fwd: -2001 9959.838899550:1: tried selector action for builtin-shell: -2001 9959.838919050:1: tried selector action for builtin-discard: -2001 9959.838938450:1: write-alltried selector action for builtin-usrmsg: 0 9959.838964500:1: Module builtin-usrmsg processed this config line. 9959.838985850:1: template: ' WallFmt' assigned 9959.839007950:1: action 19 queue: save on shutdown 1, max disk space allowed 0 9959.839034350:1: action 19 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.839057700:1: Action 84550: queue 845e8 created 9959.839087200:1: cfline: '*.alert root' 9959.839106200:1: selector line successfully processed 9959.839124350:1: - traditional PRI filter 9959.839142600:1: symbolic name: alert ==> 1 9959.839168000:1: tried selector action for builtin-file: -2001 9959.839185950:1: tried selector action for builtin-pipe: -2001 9959.839205050:1: tried selector action for builtin-fwd: -2001 9959.839224150:1: tried selector action for builtin-shell: -2001 9959.839243100:1: tried selector action for builtin-discard: -2001 9959.839261900:1: users: root 9959.839280850:1: tried selector action for builtin-usrmsg: 0 9959.839298850:1: Module builtin-usrmsg processed this config line. 9959.839319800:1: template: ' StdUsrMsgFmt' assigned 9959.839376500:1: action 20 queue: save on shutdown 1, max disk space allowed 0 9959.839407900:1: action 20 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.839431800:1: Action 84888: queue 84920 created 9959.839460200:1: cfline: '*.alert /dev/sysmsg' 9959.839480200:1: selector line successfully processed 9959.839498800:1: - traditional PRI filter 9959.839517300:1: symbolic name: alert ==> 1 9959.839606200:1: file stream sysmsg params: flush interval 0, async write 0 9959.839630500:1: tried selector action for builtin-file: 0 9959.839649550:1: Module builtin-file processed this config line. 9959.839671200:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.839693600:1: action 21 queue: save on shutdown 1, max disk space allowed 0 9959.839720250:1: action 21 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.839743700:1: Action 85148: queue 851e0 created 9959.839766050:1: cfline: 'auth.info /var/adm/authlog' 9959.839784800:1: selector line successfully processed 9959.839803000:1: - traditional PRI filter 9959.839821250:1: symbolic name: info ==> 6 9959.839845750:1: symbolic name: auth ==> 32 9959.839887200:1: file stream authlog params: flush interval 0, async write 0 9959.839964250:1: tried selector action for builtin-file: 0 9959.839983150:1: Module builtin-file processed this config line. 9959.840006700:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.840029250:1: action 22 queue: save on shutdown 1, max disk space allowed 0 9959.840056200:1: action 22 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.840079300:1: Action 85a08: queue 85ea8 created 9959.840099700:1: cfline: '*.err /var/adm/syslog' 9959.840118450:1: selector line successfully processed 9959.840136500:1: - traditional PRI filter 9959.840154550:1: symbolic name: err ==> 3 9959.840195050:1: file stream syslog params: flush interval 0, async write 0 9959.840216000:1: tried selector action for builtin-file: 0 9959.840234150:1: Module builtin-file processed this config line. 9959.840255700:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.840277550:1: action 23 queue: save on shutdown 1, max disk space allowed 0 9959.840313550:1: action 23 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.840336950:1: Action 866d0: queue 86768 created 9959.840359400:1: cfline: '*.notice /var/adm/messages' 9959.840378150:1: selector line successfully processed 9959.840395750:1: - traditional PRI filter 9959.840414100:1: symbolic name: notice ==> 5 9959.840489100:1: file stream messages params: flush interval 0, async write 0 9959.840512500:1: tried selector action for builtin-file: 0 9959.840530800:1: Module builtin-file processed this config line. 9959.840552350:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.840575250:1: action 24 queue: save on shutdown 1, max disk space allowed 0 9959.840602000:1: action 24 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.840625100:1: Action 86f90: queue 87028 created 9959.840655050:1: cfline: 'mail.debug @mail' 9959.840674700:1: selector line successfully processed 9959.840693000:1: - traditional PRI filter 9959.840711300:1: symbolic name: debug ==> 7 9959.840735750:1: symbolic name: mail ==> 16 9959.840760400:1: tried selector action for builtin-file: -2001 9959.840778300:1: tried selector action for builtin-pipe: -2001 9959.840799850:1: tried selector action for builtin-fwd: 0 9959.840817950:1: Module builtin-fwd processed this config line. 9959.840839500:1: template: 'RSYSLOG_TraditionalForwardFormat' assigned 9959.840862100:1: action 25 queue: save on shutdown 1, max disk space allowed 0 9959.840889100:1: action 25 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.840912650:1: Action 87288: queue 87320 created 9959.840934600:1: cfline: 'local0.info /var/adm/login' 9959.840953300:1: selector line successfully processed 9959.840971450:1: - traditional PRI filter 9959.840989350:1: symbolic name: info ==> 6 9959.841021850:1: symbolic name: local0 ==> 128 9959.841066300:1: file stream login params: flush interval 0, async write 0 9959.841087350:1: tried selector action for builtin-file: 0 9959.841105350:1: Module builtin-file processed this config line. 9959.841126600:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.841148950:1: action 26 queue: save on shutdown 1, max disk space allowed 0 9959.841175700:1: action 26 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.841199100:1: Action 88150: queue 881e8 created 9959.841219300:1: cfline: 'local7.info /var/adm/messages' 9959.841237900:1: selector line successfully processed 9959.841256000:1: - traditional PRI filter 9959.841274250:1: symbolic name: info ==> 6 9959.841298550:1: symbolic name: local7 ==> 184 9959.841336700:1: file stream messages params: flush interval 0, async write 0 9959.841389000:1: tried selector action for builtin-file: 0 9959.841411700:1: Module builtin-file processed this config line. 9959.841441800:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.841466050:1: action 27 queue: save on shutdown 1, max disk space allowed 0 9959.841493000:1: action 27 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.841515900:1: Action 88a10: queue 88aa8 created 9959.841537850:1: cfline: 'local4.info /var/log/openldap/ldap.log' 9959.841556500:1: selector line successfully processed 9959.841574400:1: - traditional PRI filter 9959.841592550:1: symbolic name: info ==> 6 9959.841617500:1: symbolic name: local4 ==> 160 9959.841660250:1: file stream ldap.log params: flush interval 0, async write 0 9959.841681350:1: tried selector action for builtin-file: 0 9959.841699250:1: Module builtin-file processed this config line. 9959.841720100:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.841742000:1: action 28 queue: save on shutdown 1, max disk space allowed 0 9959.841768550:1: action 28 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.841799750:1: Action 892d0: queue 89368 created 9959.841820850:1: cfline: 'daemon.debug /var/adm/messages' 9959.841839650:1: selector line successfully processed 9959.841857750:1: - traditional PRI filter 9959.841876050:1: symbolic name: debug ==> 7 9959.841900700:1: symbolic name: daemon ==> 24 9959.841938350:1: file stream messages params: flush interval 0, async write 0 9959.841959550:1: tried selector action for builtin-file: 0 9959.841977900:1: Module builtin-file processed this config line. 9959.841999200:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.842021650:1: action 29 queue: save on shutdown 1, max disk space allowed 0 9959.842048500:1: action 29 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.842071450:1: Action 89b90: queue 89c28 created 9959.842091400:1: cfline: 'daemon.info /var/adm/messages' 9959.842117050:1: selector line successfully processed 9959.842136800:1: - traditional PRI filter 9959.842155650:1: symbolic name: info ==> 6 9959.842180350:1: symbolic name: daemon ==> 24 9959.842217650:1: file stream messages params: flush interval 0, async write 0 9959.842238550:1: tried selector action for builtin-file: 0 9959.842256700:1: Module builtin-file processed this config line. 9959.842277750:1: template: 'RSYSLOG_TraditionalFileFormat' assigned 9959.842300350:1: action 30 queue: save on shutdown 1, max disk space allowed 0 9959.842327150:1: action 30 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 970, light delay 700 starting 9959.842350450:1: Action 8a450: queue 8a4e8 created 9959.842378600:1: selector line successfully processed 9959.842464400:1: main Q: is NOT disk-assisted 9959.842496400:1: main Q: type 0, enq-only 0, disk assisted 0, maxFileSz 1048576, qsize 0, child 0, full delay 9700, light delay 7000 starting 9959.842526850:1: main Q:Reg: finalizing construction of worker thread pool 9959.842559650:1: main Q:Reg/w0: finalizing construction of worker instance data 9959.842578800:1: main Q: queue starts up without (loading) any DA disk state (this is normal for the DA queue itself!) 9959.842604300:1: main Q:Reg: high activity - starting 1 additional worker thread(s). 9959.842623650:1: main Q:Reg/w0: receiving command 2 9959.842641950:1: wtiSetState PROBLEM, tCurrCmd 0 overwritten with 2, wanted to set 2 9959.843579450:1: main Q:Reg: started with state 0, num workers now 1 9959.843616800:1: Main processing queue is initialized and running 9959.843647850:1: Allowed TCP Senders: 9959.843667200:1: No restrictions set. 9959.843688600:1: Allowed UDP Senders: 9959.843742200:2: main Q:Reg/w0: receiving command 4 9959.843824850:2: wtiSetState PROBLEM, tCurrCmd 2 overwritten with 4, wanted to set 4 9959.843858500:1: No restrictions set. 9959.843898600:1: All Rulesets: 9959.843930500:2: main Q:Reg/w0: worker IDLE, waiting for work. 9959.844074500:1: ruleset 7a3f0: rsyslog ruleset RSYSLOG_DefaultRuleset: 9959.844111800:1: rule 7c468: rsyslog rule: 9959.844135850:1: PROPERTY-BASED Filter: 9959.844154500:1: Property.: 'msg' 9959.844171750:1: Operation: 'contains' 9959.844197250:1: Value....: 'seahorse_ssh_source_init' 9959.844214700:1: Action...: Actions: 9959.844241750:1: builtin-discard: Instance data: 0x0 9959.844266850:1: RepeatedMsgReduction: 0 9959.844284200:1: Resume Interval: 30 9959.844301300:1: Suspended: 0 9959.844324650:1: Disabled: 0 9959.844342150:1: Exec only when previous is suspended: 0 9959.844359100:1: 9959.844375950:1: 9959.844392800:1: 9959.844412050:1: rule 7c7e0: rsyslog rule: 9959.844434150:1: PROPERTY-BASED Filter: 9959.844451700:1: Property.: 'msg' 9959.844468600:1: Operation: 'contains' 9959.844492850:1: Value....: 'monitor the GPG home directory' 9959.844509750:1: Action...: Actions: 9959.844533450:1: builtin-discard: Instance data: 0x0 9959.844557150:1: RepeatedMsgReduction: 0 9959.844574500:1: Resume Interval: 30 9959.844609450:1: Suspended: 0 9959.844633300:1: Disabled: 0 9959.844650500:1: Exec only when previous is suspended: 0 9959.844667400:1: 9959.844684100:1: 9959.844700950:1: 9959.844719800:1: rule 7cb20: rsyslog rule: 9959.844742000:1: PROPERTY-BASED Filter: 9959.844759400:1: Property.: 'msg' 9959.844775950:1: Operation: 'contains' 9959.844800050:1: Value....: 'dbus-daemon' 9959.844817050:1: Action...: Actions: 9959.844840600:1: builtin-discard: Instance data: 0x0 9959.844864600:1: RepeatedMsgReduction: 0 9959.844881650:1: Resume Interval: 30 9959.844898600:1: Suspended: 0 9959.844921550:1: Disabled: 0 9959.844938650:1: Exec only when previous is suspended: 0 9959.844955600:1: 9959.844972250:1: 9959.844988900:1: 9959.845007800:1: rule 7ce20: rsyslog rule: 9959.845029850:1: PROPERTY-BASED Filter: 9959.845047400:1: Property.: 'msg' 9959.845064250:1: Operation: 'contains' 9959.845088500:1: Value....: 'freedesktop' 9959.845105500:1: Action...: Actions: 9959.845128750:1: builtin-discard: Instance data: 0x0 9959.845160700:1: RepeatedMsgReduction: 0 9959.845178350:1: Resume Interval: 30 9959.845195300:1: Suspended: 0 9959.845218350:1: Disabled: 0 9959.845235450:1: Exec only when previous is suspended: 0 9959.845252150:1: 9959.845268850:1: 9959.845285550:1: 9959.845304200:1: rule 7d120: rsyslog rule: 9959.845326600:1: PROPERTY-BASED Filter: 9959.845344050:1: Property.: 'msg' 9959.845360800:1: Operation: 'contains' 9959.845385250:1: Value....: 'gnome-keyring-daemon' 9959.845402300:1: Action...: Actions: 9959.845425500:1: builtin-discard: Instance data: 0x0 9959.845449300:1: RepeatedMsgReduction: 0 9959.845466500:1: Resume Interval: 30 9959.845483400:1: Suspended: 0 9959.845506350:1: Disabled: 0 9959.845523800:1: Exec only when previous is suspended: 0 9959.845540400:1: 9959.845557350:1: 9959.845573900:1: 9959.845592550:1: rule 7d420: rsyslog rule: 9959.845614950:1: PROPERTY-BASED Filter: 9959.845632600:1: Property.: 'msg' 9959.845649350:1: Operation: 'contains' 9959.845680150:1: Value....: 'quota_ufs' 9959.845697900:1: Action...: Actions: 9959.845721450:1: builtin-discard: Instance data: 0x0 9959.845745300:1: RepeatedMsgReduction: 0 9959.845762400:1: Resume Interval: 30 9959.845779350:1: Suspended: 0 9959.845802350:1: Disabled: 0 9959.845819300:1: Exec only when previous is suspended: 0 9959.845835950:1: 9959.845852600:1: 9959.845869250:1: 9959.845887950:1: rule 7d720: rsyslog rule: 9959.845910200:1: PROPERTY-BASED Filter: 9959.845927400:1: Property.: 'msg' 9959.845944100:1: Operation: 'contains' 9959.845968350:1: Value....: 'HAL' 9959.845985200:1: Action...: Actions: 9959.846008650:1: builtin-discard: Instance data: 0x0 9959.846032350:1: RepeatedMsgReduction: 0 9959.846049400:1: Resume Interval: 30 9959.846066300:1: Suspended: 0 9959.846089100:1: Disabled: 0 9959.846106250:1: Exec only when previous is suspended: 0 9959.846122850:1: 9959.846139600:1: 9959.846156850:1: builtin-discard: Instance data: 0x0 9959.846180750:1: RepeatedMsgReduction: 0 9959.846204800:1: Resume Interval: 30 9959.846222150:1: Suspended: 0 9959.846245050:1: Disabled: 0 9959.846262300:1: Exec only when previous is suspended: 0 9959.846278950:1: 9959.846295600:1: 9959.846312450:1: 9959.846331000:1: rule 81dd0: rsyslog rule: 9959.846353300:1: X X X X FF X X X X X X X X X X X X X X X X X X X X Actions: 9959.846521000:1: builtin-file: [dynamic] 9959.846546750:1: template='DynAuth' 9959.846564400:1: use async writer=0 9959.846581450:1: flush on TX end=1 9959.846598300:1: flush interval=1 9959.846615700:1: file cache size=10 9959.846633000:1: create directories: yes 9959.846650400:1: file owner 0, group 4 9959.846667850:1: force chown() for all files: no 9959.846685350:1: directory owner -1, group -1 9959.846703000:1: dir create mode 0755, file create mode 0640 9959.846720550:1: fail if owner/group can not be set: yes 9959.846738000:1: Instance data: 0x81e30 9959.846755100:1: RepeatedMsgReduction: 0 9959.846771950:1: Resume Interval: 30 9959.846795700:1: Suspended: 0 9959.846819550:1: Disabled: 0 9959.846836650:1: Exec only when previous is suspended: 0 9959.846853650:1: 9959.846870400:1: 9959.846887450:1: 9959.846906400:1: rule 82180: rsyslog rule: 9959.846928850:1: 7F 7F X 7F X 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F X 7F 7F 7F 7F 7F 7F 7F 7F 7F Actions: 9959.847100000:1: builtin-file: [dynamic] 9959.847123900:1: template='DynMSG' 9959.847141250:1: use async writer=0 9959.847158100:1: flush on TX end=1 9959.847175050:1: flush interval=1 9959.847192000:1: file cache size=10 9959.847208950:1: create directories: yes 9959.847225950:1: file owner 0, group 4 9959.847243200:1: force chown() for all files: no 9959.847260550:1: directory owner -1, group -1 9959.847277900:1: dir create mode 0755, file create mode 0640 9959.847295200:1: fail if owner/group can not be set: yes 9959.847312300:1: Instance data: 0x821e0 9959.847329200:1: RepeatedMsgReduction: 0 9959.847346450:1: Resume Interval: 30 9959.847363300:1: Suspended: 0 9959.847393500:1: Disabled: 0 9959.847410850:1: Exec only when previous is suspended: 0 9959.847427950:1: 9959.847444850:1: 9959.847461500:1: 9959.847480100:1: rule 82560: rsyslog rule: 9959.847502500:1: X X FF X X X X X X X X X X X X X X X X X X X X X X Actions: 9959.847669150:1: builtin-file: [dynamic] 9959.847693150:1: template='Dynmail' 9959.847710350:1: use async writer=0 9959.847727300:1: flush on TX end=1 9959.847744300:1: flush interval=1 9959.847761300:1: file cache size=10 9959.847778150:1: create directories: yes 9959.847795450:1: file owner 0, group 4 9959.847812650:1: force chown() for all files: no 9959.847829800:1: directory owner -1, group -1 9959.847847300:1: dir create mode 0755, file create mode 0640 9959.847864600:1: fail if owner/group can not be set: yes 9959.847881900:1: Instance data: 0x825c0 9959.847899100:1: RepeatedMsgReduction: 0 9959.847916150:1: Resume Interval: 30 9959.847933250:1: Suspended: 0 9959.847956200:1: Disabled: 0 9959.847980050:1: Exec only when previous is suspended: 0 9959.847997250:1: 9959.848014150:1: 9959.848030900:1: 9959.848049550:1: rule 82910: rsyslog rule: 9959.848071650:1: X X X X X X X X X X X X X X X FF X X X X X X X X X Actions: 9959.848238500:1: builtin-file: [dynamic] 9959.848262200:1: template='Dyncron' 9959.848279250:1: use async writer=0 9959.848296200:1: flush on TX end=1 9959.848313100:1: flush interval=1 9959.848330150:1: file cache size=10 9959.848347100:1: create directories: yes 9959.848364350:1: file owner 0, group 4 9959.848381400:1: force chown() for all files: no 9959.848398750:1: directory owner -1, group -1 9959.848416150:1: dir create mode 0755, file create mode 0640 9959.848433550:1: fail if owner/group can not be set: yes 9959.848450750:1: Instance data: 0x82970 9959.848467800:1: RepeatedMsgReduction: 0 9959.848484850:1: Resume Interval: 30 9959.848501650:1: Suspended: 0 9959.848524550:1: Disabled: 0 9959.848541450:1: Exec only when previous is suspended: 0 9959.848621650:1: 9959.848641200:1: 9959.848658600:1: 9959.848677800:1: rule 82cf0: rsyslog rule: 9959.848700150:1: X X X X X X X 7 X X X X X X X X X X X X X X X X X Actions: 9959.848868000:1: builtin-file: [dynamic] 9959.848892250:1: template='Dynspool' 9959.848909800:1: use async writer=0 9959.848926750:1: flush on TX end=1 9959.848943750:1: flush interval=1 9959.848960650:1: file cache size=10 9959.848977700:1: create directories: yes 9959.848994850:1: file owner 0, group 4 9959.849011950:1: force chown() for all files: no 9959.849029500:1: directory owner -1, group -1 9959.849046850:1: dir create mode 0755, file create mode 0640 9959.849064500:1: fail if owner/group can not be set: yes 9959.849081850:1: Instance data: 0x82d50 9959.849099050:1: RepeatedMsgReduction: 0 9959.849116200:1: Resume Interval: 30 9959.849133350:1: Suspended: 0 9959.849156150:1: Disabled: 0 9959.849173300:1: Exec only when previous is suspended: 0 9959.849190050:1: 9959.849214700:1: 9959.849232050:1: 9959.849250650:1: rule 830d0: rsyslog rule: 9959.849272650:1: X X X X X X X X X X X X X X X X X X X FF X X X X X Actions: 9959.849439200:1: builtin-file: [dynamic] 9959.849463200:1: template='Dynlocal3' 9959.849480900:1: use async writer=0 9959.849498100:1: flush on TX end=1 9959.849514950:1: flush interval=1 9959.849532200:1: file cache size=10 9959.849549150:1: create directories: yes 9959.849566450:1: file owner 0, group 4 9959.849583500:1: force chown() for all files: no 9959.849600650:1: directory owner -1, group -1 9959.849618350:1: dir create mode 0755, file create mode 0640 9959.849635500:1: fail if owner/group can not be set: yes 9959.849652750:1: Instance data: 0x83130 9959.849670050:1: RepeatedMsgReduction: 0 9959.849687100:1: Resume Interval: 30 9959.849704000:1: Suspended: 0 9959.849726900:1: Disabled: 0 9959.849744000:1: Exec only when previous is suspended: 0 9959.849760850:1: 9959.849777500:1: 9959.849794300:1: 9959.849820200:1: rule 834b0: rsyslog rule: 9959.849843150:1: X X X X X X X X X X X X X X X X X X X X FF X X X X Actions: 9959.850010100:1: builtin-file: [dynamic] 9959.850033800:1: template='Dynlocal4' 9959.850050850:1: use async writer=0 9959.850068150:1: flush on TX end=1 9959.850085250:1: flush interval=1 9959.850102150:1: file cache size=10 9959.850119100:1: create directories: yes 9959.850136300:1: file owner 0, group 4 9959.850153450:1: force chown() for all files: no 9959.850170900:1: directory owner -1, group -1 9959.850188350:1: dir create mode 0755, file create mode 0640 9959.850205550:1: fail if owner/group can not be set: yes 9959.850222650:1: Instance data: 0x83510 9959.850239850:1: RepeatedMsgReduction: 0 9959.850257050:1: Resume Interval: 30 9959.850274050:1: Suspended: 0 9959.850297000:1: Disabled: 0 9959.850314100:1: Exec only when previous is suspended: 0 9959.850330950:1: 9959.850347800:1: 9959.850364500:1: 9959.850383000:1: rule 83890: rsyslog rule: 9959.850411850:1: X X X X X X X X X X X X X X X X X X X X X FF X X X Actions: 9959.850579400:1: builtin-file: [dynamic] 9959.850602950:1: template='Dynlocal5' 9959.850620150:1: use async writer=0 9959.850637200:1: flush on TX end=1 9959.850653950:1: flush interval=1 9959.850671050:1: file cache size=10 9959.850687850:1: create directories: yes 9959.850705250:1: file owner 0, group 4 9959.850722250:1: force chown() for all files: no 9959.850739600:1: directory owner -1, group -1 9959.850757150:1: dir create mode 0755, file create mode 0640 9959.850774250:1: fail if owner/group can not be set: yes 9959.850791550:1: Instance data: 0x838f0 9959.850808550:1: RepeatedMsgReduction: 0 9959.850825600:1: Resume Interval: 30 9959.850842650:1: Suspended: 0 9959.850865350:1: Disabled: 0 9959.850882550:1: Exec only when previous is suspended: 0 9959.850899500:1: 9959.850916250:1: 9959.850933000:1: 9959.850951800:1: rule 83c70: rsyslog rule: 9959.850974100:1: X X X X X X X X X X X X X X X X X X X X X X FF X X Actions: 9959.851148800:1: builtin-file: [dynamic] 9959.851172850:1: template='Dynlocal6' 9959.851189950:1: use async writer=0 9959.851207100:1: flush on TX end=1 9959.851224050:1: flush interval=1 9959.851241150:1: file cache size=10 9959.851258250:1: create directories: yes 9959.851275350:1: file owner 0, group 4 9959.851292450:1: force chown() for all files: no 9959.851309850:1: directory owner -1, group -1 9959.851327150:1: dir create mode 0755, file create mode 0640 9959.851344650:1: fail if owner/group can not be set: yes 9959.851361750:1: Instance data: 0x83cd0 9959.851378900:1: RepeatedMsgReduction: 0 9959.851396100:1: Resume Interval: 30 9959.851413250:1: Suspended: 0 9959.851436300:1: Disabled: 0 9959.851453400:1: Exec only when previous is suspended: 0 9959.851470300:1: 9959.851487000:1: 9959.851504000:1: 9959.851522650:1: rule 84050: rsyslog rule: 9959.851545000:1: X X X X X X X X X X X X X X X X X X X X X X X FF X Actions: 9959.851718850:1: builtin-file: [dynamic] 9959.851743000:1: template='Dynlocal7' 9959.851760300:1: use async writer=0 9959.851777300:1: flush on TX end=1 9959.851794350:1: flush interval=1 9959.851811250:1: file cache size=10 9959.851828300:1: create directories: yes 9959.851845750:1: file owner 0, group 4 9959.851862650:1: force chown() for all files: no 9959.851880000:1: directory owner -1, group -1 9959.851897600:1: dir create mode 0755, file create mode 0640 9959.851914850:1: fail if owner/group can not be set: yes 9959.851932000:1: Instance data: 0x840b0 9959.851949100:1: RepeatedMsgReduction: 0 9959.851966350:1: Resume Interval: 30 9959.851983300:1: Suspended: 0 9959.852006000:1: Disabled: 0 9959.852023250:1: Exec only when previous is suspended: 0 9959.852040000:1: 9959.852056800:1: 9959.852073600:1: 9959.852092250:1: rule 84430: rsyslog rule: 9959.852114700:1: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Actions: 9959.852293450:1: builtin-usrmsg: Instance data: 0x84490 9959.852318050:1: RepeatedMsgReduction: 0 9959.852335300:1: Resume Interval: 30 9959.852352300:1: Suspended: 0 9959.852375200:1: Disabled: 0 9959.852392750:1: Exec only when previous is suspended: 0 9959.852409450:1: 9959.852425950:1: 9959.852442650:1: 9959.852461550:1: rule 84768: rsyslog rule: 9959.852483800:1: 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Actions: 9959.852655650:1: builtin-usrmsg: root, Instance data: 0x847c8 9959.852686450:1: RepeatedMsgReduction: 0 9959.852703800:1: Resume Interval: 30 9959.852720600:1: Suspended: 0 9959.852777950:1: Disabled: 0 9959.852795800:1: Exec only when previous is suspended: 0 9959.852812650:1: 9959.852829550:1: 9959.852846300:1: 9959.852865450:1: rule 84aa0: rsyslog rule: 9959.852888200:1: 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Actions: 9959.853059900:1: builtin-file: /dev/sysmsg 9959.853084500:1: template='/dev/sysmsg' 9959.853108600:1: use async writer=0 9959.853126950:1: flush on TX end=1 9959.853144100:1: flush interval=1 9959.853161400:1: file cache size=10 9959.853178150:1: create directories: yes 9959.853195400:1: file owner 0, group 4 9959.853212400:1: force chown() for all files: no 9959.853229900:1: directory owner -1, group -1 9959.853247500:1: dir create mode 0755, file create mode 0640 9959.853264500:1: fail if owner/group can not be set: yes 9959.853281650:1: Instance data: 0x84b00 9959.853298800:1: RepeatedMsgReduction: 0 9959.853315800:1: Resume Interval: 30 9959.853332800:1: Suspended: 0 9959.853355900:1: Disabled: 0 9959.853372950:1: Exec only when previous is suspended: 0 9959.853389600:1: 9959.853406450:1: 9959.853423200:1: 9959.853442100:1: rule 85360: rsyslog rule: 9959.853464550:1: X X X X 7F X X X X X X X X X X X X X X X X X X X X Actions: 9959.853631050:1: builtin-file: /var/adm/authlog 9959.853655250:1: template='/var/adm/authlog' 9959.853672050:1: use async writer=0 9959.853744050:1: flush on TX end=1 9959.853761950:1: flush interval=1 9959.853779100:1: file cache size=10 9959.853796350:1: create directories: yes 9959.853813350:1: file owner 0, group 4 9959.853830450:1: force chown() for all files: no 9959.853847850:1: directory owner -1, group -1 9959.853865150:1: dir create mode 0755, file create mode 0640 9959.853882150:1: fail if owner/group can not be set: yes 9959.853899100:1: Instance data: 0x853c0 9959.853916150:1: RepeatedMsgReduction: 0 9959.853933150:1: Resume Interval: 30 9959.853949850:1: Suspended: 0 9959.853972600:1: Disabled: 0 9959.853989500:1: Exec only when previous is suspended: 0 9959.854006100:1: 9959.854022800:1: 9959.854039550:1: 9959.854058450:1: rule 86028: rsyslog rule: 9959.854080900:1: F F F F F F F F F F F F F F F F F F F F F F F F F Actions: 9959.854250150:1: builtin-file: /var/adm/syslog 9959.854274200:1: template='/var/adm/syslog' 9959.854291250:1: use async writer=0 9959.854315250:1: flush on TX end=1 9959.854333750:1: flush interval=1 9959.854351350:1: file cache size=10 9959.854368450:1: create directories: yes 9959.854385600:1: file owner 0, group 4 9959.854402600:1: force chown() for all files: no 9959.854419750:1: directory owner -1, group -1 9959.854436950:1: dir create mode 0755, file create mode 0640 9959.854454000:1: fail if owner/group can not be set: yes 9959.854471050:1: Instance data: 0x86088 9959.854488150:1: RepeatedMsgReduction: 0 9959.854505050:1: Resume Interval: 30 9959.854521900:1: Suspended: 0 9959.854545000:1: Disabled: 0 9959.854561800:1: Exec only when previous is suspended: 0 9959.854578600:1: 9959.854595150:1: 9959.854612000:1: 9959.854630900:1: rule 868e8: rsyslog rule: 9959.854653550:1: 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F 3F Actions: 9959.854823550:1: builtin-file: /var/adm/messages 9959.854847850:1: template='/var/adm/messages' 9959.854864950:1: use async writer=0 9959.854881950:1: flush on TX end=1 9959.854905700:1: flush interval=1 9959.854923100:1: file cache size=10 9959.854940350:1: create directories: yes 9959.854957700:1: file owner 0, group 4 9959.854974550:1: force chown() for all files: no 9959.854992000:1: directory owner -1, group -1 9959.855009450:1: dir create mode 0755, file create mode 0640 9959.855026500:1: fail if owner/group can not be set: yes 9959.855043600:1: Instance data: 0x86948 9959.855060450:1: RepeatedMsgReduction: 0 9959.855077800:1: Resume Interval: 30 9959.855094750:1: Suspended: 0 9959.855117400:1: Disabled: 0 9959.855134400:1: Exec only when previous is suspended: 0 9959.855151200:1: 9959.855167850:1: 9959.855184600:1: 9959.855203400:1: rule 871a8: rsyslog rule: 9959.855225500:1: X X FF X X X X X X X X X X X X X X X X X X X X X X Actions: 9959.855391600:1: builtin-fwd: mail Instance data: 0x87208 9959.855422350:1: RepeatedMsgReduction: 0 9959.855439400:1: Resume Interval: 30 9959.855456250:1: Suspended: 0 9959.855485900:1: Disabled: 0 9959.855503400:1: Exec only when previous is suspended: 0 9959.855520000:1: 9959.855536700:1: 9959.855553450:1: 9959.855572450:1: rule 874a0: rsyslog rule: 9959.855594700:1: X X X X X X X X X X X X X X X X 7F X X X X X X X X Actions: 9959.855761250:1: builtin-file: /var/adm/login 9959.855785400:1: template='/var/adm/login' 9959.855802550:1: use async writer=0 9959.855819500:1: flush on TX end=1 9959.855836300:1: flush interval=1 9959.855853150:1: file cache size=10 9959.855870200:1: create directories: yes 9959.855887350:1: file owner 0, group 4 9959.855904300:1: force chown() for all files: no 9959.855921750:1: directory owner -1, group -1 9959.855939050:1: dir create mode 0755, file create mode 0640 9959.855956000:1: fail if owner/group can not be set: yes 9959.855973100:1: Instance data: 0x87500 9959.855990000:1: RepeatedMsgReduction: 0 9959.856007000:1: Resume Interval: 30 9959.856024050:1: Suspended: 0 9959.856047050:1: Disabled: 0 9959.856070150:1: Exec only when previous is suspended: 0 9959.856088050:1: 9959.856104850:1: 9959.856121500:1: 9959.856140000:1: rule 88368: rsyslog rule: 9959.856162100:1: X X X X X X X X X X X X X X X X X X X X X X X 7F X Actions: 9959.856328600:1: builtin-file: /var/adm/messages 9959.856352600:1: template='/var/adm/messages' 9959.856369850:1: use async writer=0 9959.856386800:1: flush on TX end=1 9959.856403600:1: flush interval=1 9959.856420500:1: file cache size=10 9959.856437450:1: create directories: yes 9959.856454450:1: file owner 0, group 4 9959.856471600:1: force chown() for all files: no 9959.856488850:1: directory owner -1, group -1 9959.856505950:1: dir create mode 0755, file create mode 0640 9959.856523000:1: fail if owner/group can not be set: yes 9959.856540200:1: Instance data: 0x883c8 9959.856557300:1: RepeatedMsgReduction: 0 9959.856574350:1: Resume Interval: 30 9959.856591100:1: Suspended: 0 9959.856613900:1: Disabled: 0 9959.856630950:1: Exec only when previous is suspended: 0 9959.856654100:1: 9959.856671400:1: 9959.856688400:1: 9959.856707400:1: rule 88c28: rsyslog rule: 9959.856730050:1: X X X X X X X X X X X X X X X X X X X X 7F X X X X Actions: 9959.856896200:1: builtin-file: /var/log/openldap/ldap.log 9959.856920100:1: template='/var/log/openldap/ldap.log' 9959.856937350:1: use async writer=0 9959.856954350:1: flush on TX end=1 9959.856971150:1: flush interval=1 9959.856988100:1: file cache size=10 9959.857005100:1: create directories: yes 9959.857022450:1: file owner 0, group 4 9959.857039400:1: force chown() for all files: no 9959.857056650:1: directory owner -1, group -1 9959.857074050:1: dir create mode 0755, file create mode 0640 9959.857091100:1: fail if owner/group can not be set: yes 9959.857108150:1: Instance data: 0x88c88 9959.857125150:1: RepeatedMsgReduction: 0 9959.857142150:1: Resume Interval: 30 9959.857159000:1: Suspended: 0 9959.857182000:1: Disabled: 0 9959.857199050:1: Exec only when previous is suspended: 0 9959.857222400:1: 9959.857239050:1: 9959.857255950:1: 9959.857274800:1: rule 894e8: rsyslog rule: 9959.857297200:1: X X X FF X X X X X X X X X X X X X X X X X X X X X Actions: 9959.857463000:1: builtin-file: /var/adm/messages 9959.857487100:1: template='/var/adm/messages' 9959.857504350:1: use async writer=0 9959.857521400:1: flush on TX end=1 9959.857538300:1: flush interval=1 9959.857555500:1: file cache size=10 9959.857572550:1: create directories: yes 9959.857589650:1: file owner 0, group 4 9959.857606550:1: force chown() for all files: no 9959.857623700:1: directory owner -1, group -1 9959.857641050:1: dir create mode 0755, file create mode 0640 9959.857658100:1: fail if owner/group can not be set: yes 9959.857675000:1: Instance data: 0x89548 9959.857691950:1: RepeatedMsgReduction: 0 9959.857708850:1: Resume Interval: 30 9959.857725900:1: Suspended: 0 9959.857748800:1: Disabled: 0 9959.857765900:1: Exec only when previous is suspended: 0 9959.857788450:1: 9959.857806100:1: 9959.857822900:1: 9959.857841800:1: rule 89da8: rsyslog rule: 9959.857864000:1: X X X 7F X X X X X X X X X X X X X X X X X X X X X Actions: 9959.858030300:1: builtin-file: /var/adm/messages 9959.858054300:1: template='/var/adm/messages' 9959.858071450:1: use async writer=0 9959.858088200:1: flush on TX end=1 9959.858104900:1: flush interval=1 9959.858121800:1: file cache size=10 9959.858138950:1: create directories: yes 9959.858156300:1: file owner 0, group 4 9959.858173200:1: force chown() for all files: no 9959.858190600:1: directory owner -1, group -1 9959.858208200:1: dir create mode 0755, file create mode 0640 9959.858225450:1: fail if owner/group can not be set: yes 9959.858242750:1: Instance data: 0x89e08 9959.858259800:1: RepeatedMsgReduction: 0 9959.858276850:1: Resume Interval: 30 9959.858293750:1: Suspended: 0 9959.858316600:1: Disabled: 0 9959.858333800:1: Exec only when previous is suspended: 0 9959.858400750:1: 9959.858420650:1: 9959.858437750:1: 9959.858455350:1: End of Rulesets. 9959.858473050:1: 9959.858492200:1: Template: Name='RSYSLOG_DebugFormat' 9959.858516450:1: Entry(75158): type 1, (CONSTANT), value: 'Debug line with all properties: FROMHOST: '' 9959.858547050:1: Entry(751b8): type 2, (FIELD), value: '7' 9959.858578050:1: Entry(752a0): type 1, (CONSTANT), value: '', fromhost-ip: '' 9959.858607950:1: Entry(75300): type 2, (FIELD), value: '8' 9959.858637800:1: Entry(753e8): type 1, (CONSTANT), value: '', HOSTNAME: '' 9959.858667700:1: Entry(75448): type 2, (FIELD), value: '3' 9959.858697700:1: Entry(75530): type 1, (CONSTANT), value: '', PRI: ' 9959.858727600:1: Entry(75590): type 2, (FIELD), value: '9' 9959.858757300:1: Entry(75678): type 1, (CONSTANT), value: ', syslogtag '' 9959.858787550:1: Entry(756d8): type 2, (FIELD), value: '4' 9959.858817100:1: Entry(757c0): type 1, (CONSTANT), value: '', programname: '' 9959.858847300:1: Entry(75820): type 2, (FIELD), value: '17' 9959.858884950:1: Entry(75908): type 1, (CONSTANT), value: '', APP-NAME: '' 9959.858915750:1: Entry(75968): type 2, (FIELD), value: '20' 9959.858945650:1: Entry(75a50): type 1, (CONSTANT), value: '', PROCID: '' 9959.858975350:1: Entry(75ab0): type 2, (FIELD), value: '21' 9959.859005500:1: Entry(75b98): type 1, (CONSTANT), value: '', MSGID: '' 9959.859035400:1: Entry(75bf8): type 2, (FIELD), value: '22' 9959.859065050:1: Entry(75ce0): type 1, (CONSTANT), value: '', TIMESTAMP: '' 9959.859094850:1: Entry(75d40): type 2, (FIELD), value: '2' 9959.859124450:1: Entry(75e28): type 1, (CONSTANT), value: '', STRUCTURED-DATA: '' 9959.859154450:1: Entry(75e88): type 2, (FIELD), value: '19' 9959.859184150:1: Entry(75f70): type 1, (CONSTANT), value: '', msg: '' 9959.859213800:1: Entry(75fd0): type 2, (FIELD), value: '1' 9959.859243500:1: Entry(760b8): type 1, (CONSTANT), value: '' escaped msg: '' 9959.859273600:1: Entry(76118): type 2, (FIELD), value: '1' [drop control-characters] 9959.859309850:1: Entry(76200): type 1, (CONSTANT), value: '' rawmsg: '' 9959.859347250:1: Entry(76260): type 2, (FIELD), value: '5' 9959.859377350:1: Entry(76348): type 1, (CONSTANT), value: '' ' 9959.859407100:1: Template: Name='RSYSLOG_SyslogProtocol23Format' 9959.859430750:1: Entry(76430): type 1, (CONSTANT), value: '<' 9959.859460850:1: Entry(76490): type 2, (FIELD), value: '9' 9959.859490550:1: Entry(76578): type 1, (CONSTANT), value: '>1 ' 9959.859520500:1: Entry(765d8): type 2, (FIELD), value: '2' [Format as RFC3339-Date] 9959.859556600:1: Entry(766c0): type 1, (CONSTANT), value: ' ' 9959.859586250:1: Entry(76720): type 2, (FIELD), value: '3' 9959.859616100:1: Entry(76808): type 1, (CONSTANT), value: ' ' 9959.859646100:1: Entry(76868): type 2, (FIELD), value: '20' 9959.859676150:1: Entry(76950): type 1, (CONSTANT), value: ' ' 9959.859706000:1: Entry(769b0): type 2, (FIELD), value: '21' 9959.859735950:1: Entry(76a98): type 1, (CONSTANT), value: ' ' 9959.859765800:1: Entry(76af8): type 2, (FIELD), value: '22' 9959.859802400:1: Entry(76be0): type 1, (CONSTANT), value: ' ' 9959.859832750:1: Entry(76c40): type 2, (FIELD), value: '19' 9959.859862600:1: Entry(76d28): type 1, (CONSTANT), value: ' ' 9959.859892350:1: Entry(76d88): type 2, (FIELD), value: '1' 9959.859922200:1: Entry(76e70): type 1, (CONSTANT), value: ' ' 9959.859952000:1: Template: Name='RSYSLOG_FileFormat' 9959.859975450:1: Entry(76ed0): type 2, (FIELD), value: '2' [Format as RFC3339-Date] 9959.860011600:1: Entry(76fb8): type 1, (CONSTANT), value: ' ' 9959.860041800:1: Entry(77018): type 2, (FIELD), value: '3' 9959.860071650:1: Entry(77100): type 1, (CONSTANT), value: ' ' 9959.860101400:1: Entry(77160): type 2, (FIELD), value: '4' 9959.860131050:1: Entry(771c0): type 2, (FIELD), value: '1' [SP iff no first SP] 9959.860167300:1: Entry(77220): type 2, (FIELD), value: '1' [drop last LF in msg] 9959.860203800:1: Entry(77308): type 1, (CONSTANT), value: ' ' 9959.860233750:1: Template: Name='RSYSLOG_TraditionalFileFormat' 9959.860263650:1: Entry(77368): type 2, (FIELD), value: '2' 9959.860294300:1: Entry(77450): type 1, (CONSTANT), value: ' ' 9959.860324200:1: Entry(774b0): type 2, (FIELD), value: '3' 9959.860353950:1: Entry(77598): type 1, (CONSTANT), value: ' ' 9959.860383900:1: Entry(775f8): type 2, (FIELD), value: '4' 9959.860414100:1: Entry(77658): type 2, (FIELD), value: '1' [SP iff no first SP] 9959.860449950:1: Entry(776b8): type 2, (FIELD), value: '1' [drop last LF in msg] 9959.860485950:1: Entry(777a0): type 1, (CONSTANT), value: ' ' 9959.860515850:1: Template: Name=' WallFmt' 9959.860539400:1: Entry(77888): type 1, (CONSTANT), value: ' Message from syslogd@' 9959.860569200:1: Entry(778e8): type 2, (FIELD), value: '3' 9959.860599000:1: Entry(779d0): type 1, (CONSTANT), value: ' at ' 9959.860629150:1: Entry(77a30): type 2, (FIELD), value: '16' 9959.860658950:1: Entry(77b18): type 1, (CONSTANT), value: ' ... ' 9959.860689000:1: Entry(77b78): type 2, (FIELD), value: '4' 9959.860719100:1: Entry(77bd8): type 2, (FIELD), value: '1' 9959.860755650:1: Entry(77cc0): type 1, (CONSTANT), value: ' ' 9959.860785600:1: Template: Name='RSYSLOG_ForwardFormat' 9959.860809150:1: Entry(77da8): type 1, (CONSTANT), value: '<' 9959.860839000:1: Entry(77e08): type 2, (FIELD), value: '9' 9959.860868900:1: Entry(77ef0): type 1, (CONSTANT), value: '>' 9959.860898600:1: Entry(77f50): type 2, (FIELD), value: '2' [Format as RFC3339-Date] 9959.860934650:1: Entry(78038): type 1, (CONSTANT), value: ' ' 9959.860964550:1: Entry(78098): type 2, (FIELD), value: '3' 9959.860994250:1: Entry(78180): type 1, (CONSTANT), value: ' ' 9959.861023800:1: Entry(781e0): type 2, (FIELD), value: '4' [substring, from character 1 to 32] 9959.861060900:1: Entry(78240): type 2, (FIELD), value: '1' [SP iff no first SP] 9959.861096550:1: Entry(782a0): type 2, (FIELD), value: '1' 9959.861126350:1: Template: Name='RSYSLOG_TraditionalForwardFormat' 9959.861149650:1: Entry(783b8): type 1, (CONSTANT), value: '<' 9959.861179650:1: Entry(78418): type 2, (FIELD), value: '9' 9959.861216000:1: Entry(78500): type 1, (CONSTANT), value: '>' 9959.861246100:1: Entry(78560): type 2, (FIELD), value: '2' 9959.861275700:1: Entry(78648): type 1, (CONSTANT), value: ' ' 9959.861305550:1: Entry(786a8): type 2, (FIELD), value: '3' 9959.861335350:1: Entry(78790): type 1, (CONSTANT), value: ' ' 9959.861365100:1: Entry(787f0): type 2, (FIELD), value: '4' [substring, from character 1 to 32] 9959.861401650:1: Entry(78850): type 2, (FIELD), value: '1' [SP iff no first SP] 9959.861437450:1: Entry(788b0): type 2, (FIELD), value: '1' 9959.861467300:1: Template: Name=' StdUsrMsgFmt' 9959.861490800:1: Entry(78998): type 1, (CONSTANT), value: ' ' 9959.861520600:1: Entry(789f8): type 2, (FIELD), value: '4' 9959.861550400:1: Entry(78a58): type 2, (FIELD), value: '1' 9959.861580350:1: Entry(78b40): type 1, (CONSTANT), value: ' ' 9959.861609700:1: Template: Name=' StdDBFmt' [SQL-Format (MySQL)] 9959.861639050:1: Entry(78d28): type 1, (CONSTANT), value: 'insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('' 9959.861675950:1: Entry(78d88): type 2, (FIELD), value: '1' 9959.861706650:1: Entry(78e70): type 1, (CONSTANT), value: '', ' 9959.861736550:1: Entry(78ed0): type 2, (FIELD), value: '12' 9959.861766400:1: Entry(78fb8): type 1, (CONSTANT), value: ', '' 9959.861796500:1: Entry(79018): type 2, (FIELD), value: '3' 9959.861826350:1: Entry(79100): type 1, (CONSTANT), value: '', ' 9959.861855850:1: Entry(79160): type 2, (FIELD), value: '14' 9959.861885700:1: Entry(79248): type 1, (CONSTANT), value: ', '' 9959.861915400:1: Entry(792a8): type 2, (FIELD), value: '2' [Format as MySQL-Date] 9959.861951500:1: Entry(79390): type 1, (CONSTANT), value: '', '' 9959.861981400:1: Entry(793f0): type 2, (FIELD), value: '16' [Format as MySQL-Date] 9959.862017350:1: Entry(794d8): type 1, (CONSTANT), value: '', ' 9959.862047000:1: Entry(79538): type 2, (FIELD), value: '11' 9959.862077500:1: Entry(79620): type 1, (CONSTANT), value: ', '' 9959.862159450:1: Entry(79680): type 2, (FIELD), value: '4' 9959.862189400:1: Entry(79768): type 1, (CONSTANT), value: '')' 9959.862219400:1: Template: Name=' StdPgSQLFmt' [SQL-Format (standard SQL)] 9959.862249250:1: Entry(79950): type 1, (CONSTANT), value: 'insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('' 9959.862280450:1: Entry(799b0): type 2, (FIELD), value: '1' 9959.862310750:1: Entry(79a98): type 1, (CONSTANT), value: '', ' 9959.862340600:1: Entry(79af8): type 2, (FIELD), value: '12' 9959.862370500:1: Entry(79be0): type 1, (CONSTANT), value: ', '' 9959.862400200:1: Entry(79c40): type 2, (FIELD), value: '3' 9959.862430100:1: Entry(79d28): type 1, (CONSTANT), value: '', ' 9959.862459800:1: Entry(79d88): type 2, (FIELD), value: '14' 9959.862489750:1: Entry(79e70): type 1, (CONSTANT), value: ', '' 9959.862519400:1: Entry(79ed0): type 2, (FIELD), value: '2' [Format as PgSQL-Date] 9959.862564200:1: Entry(79fb8): type 1, (CONSTANT), value: '', '' 9959.862594550:1: Entry(7a018): type 2, (FIELD), value: '16' [Format as PgSQL-Date] 9959.862630500:1: Entry(7a100): type 1, (CONSTANT), value: '', ' 9959.862660350:1: Entry(7a160): type 2, (FIELD), value: '11' 9959.862690900:1: Entry(7a248): type 1, (CONSTANT), value: ', '' 9959.862721050:1: Entry(7a2a8): type 2, (FIELD), value: '4' 9959.862777250:1: Entry(7a390): type 1, (CONSTANT), value: '')' 9959.862807750:1: Template: Name='DynAuth' 9959.862831350:1: Entry(80648): type 1, (CONSTANT), value: '/var/log/remote/' 9959.862861350:1: Entry(806a8): type 2, (FIELD), value: '3' 9959.862891000:1: Entry(80790): type 1, (CONSTANT), value: '/secure.log' 9959.862921200:1: Template: Name='DynMSG' 9959.862944650:1: Entry(80878): type 1, (CONSTANT), value: '/var/log/remote/' 9959.862974850:1: Entry(808d8): type 2, (FIELD), value: '3' 9959.863004750:1: Entry(809c0): type 1, (CONSTANT), value: '/messages' 9959.863042250:1: Template: Name='Dynmail' 9959.863065700:1: Entry(80aa8): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863095650:1: Entry(80b08): type 2, (FIELD), value: '3' 9959.863125150:1: Entry(80bf0): type 1, (CONSTANT), value: '/maillog' 9959.863154950:1: Template: Name='Dyncron' 9959.863178600:1: Entry(80cd8): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863208350:1: Entry(80d38): type 2, (FIELD), value: '3' 9959.863238300:1: Entry(80e20): type 1, (CONSTANT), value: '/cron' 9959.863267600:1: Template: Name='Dynspool' 9959.863290900:1: Entry(80f08): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863320800:1: Entry(80f68): type 2, (FIELD), value: '3' 9959.863350550:1: Entry(81050): type 1, (CONSTANT), value: '/spooler' 9959.863380350:1: Template: Name='Dynboot' 9959.863403500:1: Entry(81138): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863433200:1: Entry(81198): type 2, (FIELD), value: '3' 9959.863463050:1: Entry(81280): type 1, (CONSTANT), value: '/boot.log' 9959.863500050:1: Template: Name='Dynlocal3' 9959.863523950:1: Entry(81368): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863553800:1: Entry(813c8): type 2, (FIELD), value: '3' 9959.863583600:1: Entry(814b0): type 1, (CONSTANT), value: '/local3.log' 9959.863613300:1: Template: Name='Dynlocal4' 9959.863636450:1: Entry(81598): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863666600:1: Entry(815f8): type 2, (FIELD), value: '3' 9959.863696600:1: Entry(816e0): type 1, (CONSTANT), value: '/local4.log' 9959.863726000:1: Template: Name='Dynlocal5' 9959.863749200:1: Entry(817c8): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863779300:1: Entry(81828): type 2, (FIELD), value: '3' 9959.863809000:1: Entry(81910): type 1, (CONSTANT), value: '/local5.log' 9959.863838600:1: Template: Name='Dynlocal6' 9959.863862000:1: Entry(819f8): type 1, (CONSTANT), value: '/var/log/remote/' 9959.863891850:1: Entry(81a58): type 2, (FIELD), value: '3' 9959.863921550:1: Entry(81b40): type 1, (CONSTANT), value: '/local6.log' 9959.863957900:1: Template: Name='Dynlocal7' 9959.863982150:1: Entry(81c28): type 1, (CONSTANT), value: '/var/log/remote/' 9959.864012000:1: Entry(81c88): type 2, (FIELD), value: '3' 9959.864041750:1: Entry(81d70): type 1, (CONSTANT), value: '/local7.log' 9959.864073200:1: Loaded Module: Name='lmnet', IFVersion=5, type=library module. 9959.864109000:1: Entry points: 9959.864126400:1: queryEtryPt: 0xff0e3688 9959.864143650:1: doAction: 0x0 9959.864160850:1: parseSelectorAct: 0x0 9959.864178250:1: dbgPrintInstInfo: 0x0 9959.864195350:1: freeInstance: 0x0 9959.864212200:1: 9959.864229700:1: Loaded Module: Name='builtin-file', IFVersion=5, type=output module. 9959.864265300:1: Entry points: 9959.864282400:1: queryEtryPt: 0x1f6f0 9959.864299550:1: doAction: 0x1ec80 9959.864316950:1: parseSelectorAct: 0x1eff8 9959.864334050:1: dbgPrintInstInfo: 0x1e528 9959.864351100:1: freeInstance: 0x1ebc8 9959.864368000:1: 9959.864385450:1: Loaded Module: Name='builtin-pipe', IFVersion=5, type=output module. 9959.864427800:1: Entry points: 9959.864445200:1: queryEtryPt: 0x200f8 9959.864462650:1: doAction: 0x1fddc 9959.864480000:1: parseSelectorAct: 0x1ff24 9959.864497050:1: dbgPrintInstInfo: 0x1fd64 9959.864513950:1: freeInstance: 0x1fd9c 9959.864530700:1: 9959.864547950:1: Loaded Module: Name='builtin-fwd', IFVersion=5, type=output module. 9959.864583300:1: Entry points: 9959.864600450:1: queryEtryPt: 0x1e060 9959.864617550:1: doAction: 0x1d3e4 9959.864634500:1: parseSelectorAct: 0x1d7b0 9959.864651750:1: dbgPrintInstInfo: 0x1cf9c 9959.864669150:1: freeInstance: 0x1cebc 9959.864686000:1: 9959.864703350:1: Loaded Module: Name='builtin-shell', IFVersion=5, type=output module. 9959.864738500:1: Entry points: 9959.864755750:1: queryEtryPt: 0x1c3c8 9959.864773000:1: doAction: 0x1c248 9959.864789900:1: parseSelectorAct: 0x1c298 9959.864807150:1: dbgPrintInstInfo: 0x1c224 9959.864830000:1: freeInstance: 0x1c204 9959.864847900:1: 9959.864865800:1: Loaded Module: Name='builtin-discard', IFVersion=5, type=output module. 9959.864900900:1: Entry points: 9959.864917900:1: queryEtryPt: 0x20490 9959.864935000:1: doAction: 0x2039c 9959.864952150:1: parseSelectorAct: 0x203d4 9959.864969250:1: dbgPrintInstInfo: 0x20384 9959.864986600:1: freeInstance: 0x203b4 9959.865003200:1: 9959.865021000:1: Loaded Module: Name='builtin-usrmsg', IFVersion=5, type=output module. 9959.865056050:1: Entry points: 9959.865073400:1: queryEtryPt: 0x1cbdc 9959.865090350:1: doAction: 0x1c6c8 9959.865107550:1: parseSelectorAct: 0x1c8ec 9959.865124950:1: dbgPrintInstInfo: 0x1c668 9959.865141900:1: freeInstance: 0x1c648 9959.865158750:1: 9959.865176500:1: Loaded Module: Name='immark', IFVersion=5, type=input module. 9959.865211800:1: Entry points: 9959.865229050:1: queryEtryPt: 0xff080880 9959.865246600:1: doAction: 0xff08078c 9959.865269350:1: parseSelectorAct: 0xff080800 9959.865287850:1: dbgPrintInstInfo: 0x0 9959.865305250:1: freeInstance: 0x0 9959.865322100:1: 9959.865339450:1: Loaded Module: Name='imuxsock', IFVersion=5, type=input module. 9959.865375100:1: Entry points: 9959.865392250:1: queryEtryPt: 0xff061998 9959.865409850:1: doAction: 0xff060e90 9959.865427150:1: parseSelectorAct: 0xff061470 9959.865444400:1: dbgPrintInstInfo: 0x0 9959.865461600:1: freeInstance: 0x0 9959.865478350:1: 9959.865495600:1: Loaded Module: Name='lmnetstrms', IFVersion=5, type=library module. 9959.865531000:1: Entry points: 9959.865548150:1: queryEtryPt: 0xff01140c 9959.865565250:1: doAction: 0x0 9959.865582350:1: parseSelectorAct: 0x0 9959.865599550:1: dbgPrintInstInfo: 0x0 9959.865616850:1: freeInstance: 0x0 9959.865633500:1: 9959.865651300:1: Loaded Module: Name='lmtcpsrv', IFVersion=5, type=library module. 9959.865686800:1: Entry points: 9959.865703900:1: queryEtryPt: 0xfeff3ae4 9959.865775650:1: doAction: 0x0 9959.865793800:1: parseSelectorAct: 0x0 9959.865811150:1: dbgPrintInstInfo: 0x0 9959.865828500:1: freeInstance: 0x0 9959.865845300:1: 9959.865862950:1: Loaded Module: Name='imtcp.so', IFVersion=5, type=input module. 9959.865897900:1: Entry points: 9959.865914950:1: queryEtryPt: 0xff031234 9959.865932050:1: doAction: 0xff030eac 9959.865949250:1: parseSelectorAct: 0xff030f10 9959.865966250:1: dbgPrintInstInfo: 0x0 9959.865983550:1: freeInstance: 0x0 9959.866000500:1: 9959.866017800:1: Loaded Module: Name='imudp.so', IFVersion=5, type=input module. 9959.866052900:1: Entry points: 9959.866070000:1: queryEtryPt: 0xfefd1bb0 9959.866087100:1: doAction: 0xfefd0e18 9959.866104100:1: parseSelectorAct: 0xfefd1878 9959.866121200:1: dbgPrintInstInfo: 0x0 9959.866138400:1: freeInstance: 0x0 9959.866154850:1: 9959.866173450:1: Sytem Line Configuration Commands: 9959.866198600:1: Command 'workdirectory': 9959.866217750:1: type : 11 9959.866235350:1: pData: 0x69de8 9959.866252550:1: Hdlr : 0x0 9959.866269750:1: Owner: 0x0 9959.866286350:1: 9959.866303700:1: Command 'dropmsgswithmaliciousdnsptrrecords': 9959.866321100:1: type : 4 9959.866338200:1: pData: 0x69e04 9959.866355350:1: Hdlr : 0x0 9959.866372150:1: Owner: 0x0 9959.866388750:1: 9959.866406450:1: Command 'defaultnetstreamdriver': 9959.866423500:1: type : 11 9959.866440450:1: pData: 0x69df8 9959.866457550:1: Hdlr : 0x0 9959.866474600:1: Owner: 0x0 9959.866491500:1: 9959.866508900:1: Command 'defaultnetstreamdrivercafile': 9959.866526150:1: type : 11 9959.866543200:1: pData: 0x69df4 9959.866560350:1: Hdlr : 0x0 9959.866577400:1: Owner: 0x0 9959.866594250:1: 9959.866611400:1: Command 'defaultnetstreamdriverkeyfile': 9959.866628600:1: type : 11 9959.866645800:1: pData: 0x69df0 9959.866662750:1: Hdlr : 0x0 9959.866680100:1: Owner: 0x0 9959.866696800:1: 9959.866714000:1: Command 'defaultnetstreamdrivercertfile': 9959.866738450:1: type : 11 9959.866756300:1: pData: 0x69dec 9959.866773350:1: Hdlr : 0x0 9959.866790600:1: Owner: 0x0 9959.866807200:1: 9959.866824650:1: Command 'optimizeforuniprocessor': 9959.866841950:1: type : 4 9959.866859200:1: pData: 0x68088 9959.866876200:1: Hdlr : 0x0 9959.866893100:1: Owner: 0x0 9959.866909650:1: 9959.866926850:1: Command 'hupisrestart': 9959.866943950:1: type : 4 9959.866961200:1: pData: 0x69e00 9959.866978150:1: Hdlr : 0x0 9959.866995300:1: Owner: 0x0 9959.867011900:1: 9959.867029200:1: Command 'preservefqdn': 9959.867046150:1: type : 4 9959.867063400:1: pData: 0x69dfc 9959.867080650:1: Hdlr : 0x0 9959.867098000:1: Owner: 0x0 9959.867114400:1: 9959.867131850:1: Command 'resetconfigvariables': 9959.867149100:1: type : 1 9959.867166100:1: pData: 0x0 9959.867183300:1: Hdlr : 0x21850 9959.867200400:1: Owner: 0x0 9959.867217300:1: 9959.867234350:1: type : 1 9959.867251400:1: pData: 0x0 9959.867274250:1: Hdlr : 0x1f538 9959.867292450:1: Owner: 0x1f678 9959.867309450:1: 9959.867326550:1: type : 1 9959.867343550:1: pData: 0x0 9959.867360650:1: Hdlr : 0x1e1e8 9959.867377750:1: Owner: 0x1df5c 9959.867394550:1: 9959.867411750:1: type : 1 9959.867428950:1: pData: 0x0 9959.867445850:1: Hdlr : 0x16d84 9959.867463100:1: Owner: 0x0 9959.867479600:1: 9959.867496750:1: type : 1 9959.867513650:1: pData: 0x0 9959.867530700:1: Hdlr : 0xff0809d4 9959.867547900:1: Owner: 0xff080850 9959.867564500:1: 9959.867581450:1: type : 1 9959.867598600:1: pData: 0x0 9959.867615850:1: Hdlr : 0xff061aec 9959.867632900:1: Owner: 0xff0618c8 9959.867649850:1: 9959.867667050:1: type : 1 9959.867684000:1: pData: 0x0 9959.867700800:1: Hdlr : 0xff031168 9959.867718100:1: Owner: 0xff030f98 9959.867734650:1: 9959.867751750:1: type : 1 9959.867768650:1: pData: 0x0 9959.867785650:1: Hdlr : 0xfefd1d04 9959.867802500:1: Owner: 0xfefd1a88 9959.867819200:1: 9959.867836400:1: Command 'dynafilecachesize': 9959.867859100:1: type : 6 9959.867877150:1: pData: 0x0 9959.867894700:1: Hdlr : 0x1e68c 9959.867912100:1: Owner: 0x1f678 9959.867929050:1: 9959.867946350:1: Command 'omfileziplevel': 9959.867963700:1: type : 6 9959.867980800:1: pData: 0x69d90 9959.867997900:1: Hdlr : 0x0 9959.868014900:1: Owner: 0x1f678 9959.868031600:1: 9959.868049050:1: Command 'omfileflushinterval': 9959.868066250:1: type : 6 9959.868083300:1: pData: 0x67e9c 9959.868100200:1: Hdlr : 0x0 9959.868117250:1: Owner: 0x1f678 9959.868133800:1: 9959.868151100:1: Command 'omfileasyncwriting': 9959.868168300:1: type : 4 9959.868185550:1: pData: 0x69d8c 9959.868202550:1: Hdlr : 0x0 9959.868219750:1: Owner: 0x1f678 9959.868236350:1: 9959.868253600:1: Command 'omfileflushontxend': 9959.868270700:1: type : 4 9959.868288000:1: pData: 0x67e98 9959.868305000:1: Hdlr : 0x0 9959.868322350:1: Owner: 0x1f678 9959.868338900:1: 9959.868356200:1: Command 'omfileiobuffersize': 9959.868373300:1: type : 7 9959.868397000:1: pData: 0x67e90 9959.868414550:1: Hdlr : 0x0 9959.868432100:1: Owner: 0x1f678 9959.868448950:1: 9959.868466250:1: Command 'dirowner': 9959.868483500:1: type : 2 9959.868500550:1: pData: 0x69da8 9959.868517650:1: Hdlr : 0x0 9959.868534950:1: Owner: 0x1f678 9959.868551550:1: 9959.868568600:1: Command 'dirgroup': 9959.868585800:1: type : 3 9959.868602900:1: pData: 0x69dac 9959.868619900:1: Hdlr : 0x0 9959.868637000:1: Owner: 0x1f678 9959.868653600:1: 9959.868671100:1: Command 'fileowner': 9959.868688000:1: type : 2 9959.868705400:1: pData: 0x69da0 9959.868722350:1: Hdlr : 0x0 9959.868739700:1: Owner: 0x1f678 9959.868756400:1: 9959.868773750:1: Command 'filegroup': 9959.868790900:1: type : 3 9959.868808050:1: pData: 0x69da4 9959.868825100:1: Hdlr : 0x0 9959.868842450:1: Owner: 0x1f678 9959.868858950:1: 9959.868876350:1: Command 'dircreatemode': 9959.868893450:1: type : 5 9959.868910550:1: pData: 0x67e88 9959.868927500:1: Hdlr : 0x0 9959.868950100:1: Owner: 0x1f678 9959.868968100:1: 9959.868985900:1: Command 'filecreatemode': 9959.869003000:1: type : 5 9959.869020300:1: pData: 0x67e84 9959.869037400:1: Hdlr : 0x0 9959.869054750:1: Owner: 0x1f678 9959.869071450:1: 9959.869088750:1: Command 'createdirs': 9959.869105750:1: type : 4 9959.869123250:1: pData: 0x67e80 9959.869140100:1: Hdlr : 0x0 9959.869157200:1: Owner: 0x1f678 9959.869173900:1: 9959.869191250:1: Command 'failonchownfailure': 9959.869208450:1: type : 4 9959.869225750:1: pData: 0x69d9c 9959.869242850:1: Hdlr : 0x0 9959.869260100:1: Owner: 0x1f678 9959.869276700:1: 9959.869294050:1: Command 'omfileForceChown': 9959.869311050:1: type : 4 9959.869328300:1: pData: 0x69d88 9959.869345450:1: Hdlr : 0x0 9959.869362750:1: Owner: 0x1f678 9959.869379450:1: 9959.869396750:1: Command 'actionfileenablesync': 9959.869413550:1: type : 4 9959.869430500:1: pData: 0x69d84 9959.869447650:1: Hdlr : 0x0 9959.869464600:1: Owner: 0x1f678 9959.869481300:1: 9959.869505100:1: Command 'actionfiledefaulttemplate': 9959.869522850:1: type : 11 9959.869539800:1: pData: 0x69d80 9959.869557150:1: Hdlr : 0x0 9959.869574250:1: Owner: 0x0 9959.869591050:1: 9959.869608450:1: Command 'actionforwarddefaulttemplate': 9959.869625750:1: type : 11 9959.869642550:1: pData: 0x69d70 9959.869659600:1: Hdlr : 0x0 9959.869676650:1: Owner: 0x0 9959.869693400:1: 9959.869710650:1: Command 'actionsendtcprebindinterval': 9959.869727800:1: type : 6 9959.869745100:1: pData: 0x69d6c 9959.869762000:1: Hdlr : 0x0 9959.869779000:1: Owner: 0x0 9959.869795550:1: 9959.869812950:1: Command 'actionsendudprebindinterval': 9959.869830200:1: type : 6 9959.869847650:1: pData: 0x69d68 9959.869864700:1: Hdlr : 0x0 9959.869882000:1: Owner: 0x0 9959.869898550:1: 9959.869915800:1: Command 'actionsendstreamdriver': 9959.869933200:1: type : 11 9959.869950150:1: pData: 0x69d64 9959.869967150:1: Hdlr : 0x0 9959.869984250:1: Owner: 0x0 9959.870000750:1: 9959.870069900:1: Command 'actionsendstreamdrivermode': 9959.870088100:1: type : 6 9959.870105650:1: pData: 0x69d60 9959.870122950:1: Hdlr : 0x0 9959.870140100:1: Owner: 0x0 9959.870156700:1: 9959.870174200:1: Command 'actionsendstreamdriverauthmode': 9959.870191450:1: type : 11 9959.870208500:1: pData: 0x69d5c 9959.870225750:1: Hdlr : 0x0 9959.870243050:1: Owner: 0x0 9959.870259900:1: 9959.870277200:1: Command 'actionsendstreamdriverpermittedpeer': 9959.870294700:1: type : 11 9959.870311800:1: pData: 0x0 9959.870328850:1: Hdlr : 0x1cfb8 9959.870346000:1: Owner: 0x0 9959.870362600:1: 9959.870379800:1: Command 'actionsendresendlastmsgonreconnect': 9959.870397150:1: type : 4 9959.870414250:1: pData: 0x69d58 9959.870431250:1: Hdlr : 0x0 9959.870448300:1: Owner: 0x0 9959.870464900:1: 9959.870482300:1: Command 'logrsyslogstatusmessages': 9959.870499450:1: type : 4 9959.870516400:1: pData: 0x675f4 9959.870533350:1: Hdlr : 0x0 9959.870550550:1: Owner: 0x0 9959.870566950:1: 9959.870591950:1: Command 'actionresumeretrycount': 9959.870609850:1: type : 6 9959.870627200:1: pData: 0x6a5b0 9959.870644450:1: Hdlr : 0x0 9959.870661600:1: Owner: 0x0 9959.870678550:1: 9959.870695800:1: Command 'defaultruleset': 9959.870713200:1: type : 11 9959.870730200:1: pData: 0x0 9959.870747200:1: Hdlr : 0x19aa0 9959.870764050:1: Owner: 0x0 9959.870780900:1: 9959.870798300:1: Command 'ruleset': 9959.870815300:1: type : 11 9959.870832500:1: pData: 0x0 9959.870849400:1: Hdlr : 0x19b50 9959.870866650:1: Owner: 0x0 9959.870883350:1: 9959.870900900:1: Command 'sleep': 9959.870918000:1: type : 6 9959.870935000:1: pData: 0x0 9959.870951900:1: Hdlr : 0x19acc 9959.870968800:1: Owner: 0x0 9959.870985450:1: 9959.871003000:1: Command 'mainmsgqueuefilename': 9959.871020350:1: type : 11 9959.871037500:1: pData: 0x69d24 9959.871054450:1: Hdlr : 0x0 9959.871071550:1: Owner: 0x0 9959.871088150:1: 9959.871105500:1: Command 'mainmsgqueuesize': 9959.871122700:1: type : 6 9959.871146250:1: pData: 0x675f0 9959.871163750:1: Hdlr : 0x0 9959.871180950:1: Owner: 0x0 9959.871197750:1: 9959.871215400:1: Command 'mainmsgqueuehighwatermark': 9959.871232450:1: type : 6 9959.871249650:1: pData: 0x675ec 9959.871266600:1: Hdlr : 0x0 9959.871283600:1: Owner: 0x0 9959.871300300:1: 9959.871317850:1: Command 'mainmsgqueuelowwatermark': 9959.871335000:1: type : 6 9959.871352050:1: pData: 0x675e8 9959.871369150:1: Hdlr : 0x0 9959.871386250:1: Owner: 0x0 9959.871402850:1: 9959.871420300:1: Command 'mainmsgqueuediscardmark': 9959.871437450:1: type : 6 9959.871454200:1: pData: 0x675e4 9959.871471400:1: Hdlr : 0x0 9959.871488600:1: Owner: 0x0 9959.871505300:1: 9959.871522650:1: Command 'mainmsgqueuediscardseverity': 9959.871539850:1: type : 10 9959.871556950:1: pData: 0x675e0 9959.871574000:1: Hdlr : 0x0 9959.871591000:1: Owner: 0x0 9959.871607800:1: 9959.871625300:1: Command 'mainmsgqueuecheckpointinterval': 9959.871642650:1: type : 6 9959.871665300:1: pData: 0x69d20 9959.871683300:1: Hdlr : 0x0 9959.871700750:1: Owner: 0x0 9959.871717450:1: 9959.871734850:1: Command 'mainmsgqueuesyncqueuefiles': 9959.871752100:1: type : 4 9959.871769350:1: pData: 0x69d1c 9959.871786350:1: Hdlr : 0x0 9959.871803550:1: Owner: 0x0 9959.871819950:1: 9959.871837450:1: Command 'mainmsgqueuetype': 9959.871854350:1: type : 11 9959.871871400:1: pData: 0x0 9959.871888650:1: Hdlr : 0x19bfc 9959.871905700:1: Owner: 0x0 9959.871922400:1: 9959.871940000:1: Command 'mainmsgqueueworkerthreads': 9959.871957200:1: type : 6 9959.871974600:1: pData: 0x675dc 9959.871991400:1: Hdlr : 0x0 9959.872008550:1: Owner: 0x0 9959.872025100:1: 9959.872042400:1: Command 'mainmsgqueuetimeoutshutdown': 9959.872059600:1: type : 6 9959.872076750:1: pData: 0x675d8 9959.872093950:1: Hdlr : 0x0 9959.872110650:1: Owner: 0x0 9959.872127450:1: 9959.872144750:1: Command 'mainmsgqueuetimeoutactioncompletion': 9959.872162250:1: type : 6 9959.872179200:1: pData: 0x675d4 9959.872202550:1: Hdlr : 0x0 9959.872220000:1: Owner: 0x0 9959.872236900:1: 9959.872253950:1: Command 'mainmsgqueuetimeoutenqueue': 9959.872271000:1: type : 6 9959.872288050:1: pData: 0x675d0 9959.872305150:1: Hdlr : 0x0 9959.872322200:1: Owner: 0x0 9959.872339000:1: 9959.872356200:1: Command 'mainmsgqueueworkertimeoutthreadshutdown': 9959.872373500:1: type : 6 9959.872390750:1: pData: 0x675cc 9959.872407750:1: Hdlr : 0x0 9959.872424950:1: Owner: 0x0 9959.872441750:1: 9959.872459350:1: Command 'mainmsgqueuedequeueslowdown': 9959.872476500:1: type : 6 9959.872493800:1: pData: 0x69d18 9959.872510800:1: Hdlr : 0x0 9959.872527900:1: Owner: 0x0 9959.872544650:1: 9959.872561750:1: Command 'mainmsgqueueworkerthreadminimummessages': 9959.872579000:1: type : 6 9959.872596050:1: pData: 0x675c8 9959.872613050:1: Hdlr : 0x0 9959.872630200:1: Owner: 0x0 9959.872646750:1: 9959.872663950:1: Command 'mainmsgqueuemaxfilesize': 9959.872681050:1: type : 7 9959.872698100:1: pData: 0x675c0 9959.872721600:1: Hdlr : 0x0 9959.872769100:1: Owner: 0x0 9959.872786650:1: 9959.872804500:1: Command 'mainmsgqueuemaxdiskspace': 9959.872821950:1: type : 7 9959.872839100:1: pData: 0x69d10 9959.872856200:1: Hdlr : 0x0 9959.872873400:1: Owner: 0x0 9959.872889900:1: 9959.872907150:1: Command 'mainmsgqueuesaveonshutdown': 9959.872924300:1: type : 4 9959.872941600:1: pData: 0x675b8 9959.872958650:1: Hdlr : 0x0 9959.872975700:1: Owner: 0x0 9959.872992300:1: 9959.873010000:1: Command 'mainmsgqueuedequeuetimebegin': 9959.873094900:1: type : 6 9959.873113950:1: pData: 0x69d08 9959.873131300:1: Hdlr : 0x0 9959.873148550:1: Owner: 0x0 9959.873165150:1: 9959.873182600:1: Command 'mainmsgqueuedequeuetimeend': 9959.873199700:1: type : 6 9959.873216900:1: pData: 0x675b4 9959.873234150:1: Hdlr : 0x0 9959.873251050:1: Owner: 0x0 9959.873267850:1: 9959.873284900:1: Command 'repeatedmsgreduction': 9959.873302450:1: type : 4 9959.873319400:1: pData: 0x6a770 9959.873344800:1: Hdlr : 0x0 9959.873363250:1: Owner: 0x0 9959.873380250:1: 9959.873397750:1: Command 'actionexeconlywhenpreviousissuspended': 9959.873414900:1: type : 4 9959.873431950:1: pData: 0x6a98c 9959.873449250:1: Hdlr : 0x0 9959.873466400:1: Owner: 0x0 9959.873482800:1: 9959.873500300:1: Command 'actionexeconlyonceeveryinterval': 9959.873517600:1: type : 6 9959.873535000:1: pData: 0x69cd4 9959.873552050:1: Hdlr : 0x0 9959.873569300:1: Owner: 0x0 9959.873585850:1: 9959.873603000:1: Command 'actionresumeinterval': 9959.873620000:1: type : 6 9959.873637300:1: pData: 0x0 9959.873654400:1: Hdlr : 0x1898c 9959.873671450:1: Owner: 0x0 9959.873688050:1: 9959.873705750:1: Command 'controlcharacterescapeprefix': 9959.873723050:1: type : 8 9959.873739950:1: pData: 0x67314 9959.873757150:1: Hdlr : 0x0 9959.873774100:1: Owner: 0x0 9959.873790600:1: 9959.873808250:1: Command 'escapecontrolcharactersonreceive': 9959.873825550:1: type : 4 9959.873842550:1: pData: 0x67310 9959.873866300:1: Hdlr : 0x0 9959.873883900:1: Owner: 0x0 9959.873900850:1: 9959.873918300:1: Command 'escapecontrolcharactertab': 9959.873935650:1: type : 4 9959.873952600:1: pData: 0x6730c 9959.873969750:1: Hdlr : 0x0 9959.873986750:1: Owner: 0x0 9959.874003550:1: 9959.874021100:1: Command 'droptrailinglfonreception': 9959.874038100:1: type : 4 9959.874055200:1: pData: 0x67318 9959.874072300:1: Hdlr : 0x0 9959.874089350:1: Owner: 0x0 9959.874106200:1: 9959.874123150:1: Command 'template': 9959.874140550:1: type : 1 9959.874157550:1: pData: 0x0 9959.874174550:1: Hdlr : 0x21f70 9959.874191450:1: Owner: 0x0 9959.874208300:1: 9959.874225350:1: Command 'outchannel': 9959.874242650:1: type : 1 9959.874259650:1: pData: 0x1 9959.874276800:1: Hdlr : 0x21f70 9959.874293800:1: Owner: 0x0 9959.874310600:1: 9959.874327850:1: Command 'allowedsender': 9959.874345300:1: type : 1 9959.874362150:1: pData: 0x2 9959.874379350:1: Hdlr : 0x21f70 9959.874396350:1: Owner: 0x0 9959.874468400:1: 9959.874486800:1: Command 'modload': 9959.874504350:1: type : 1 9959.874521450:1: pData: 0x0 9959.874538650:1: Hdlr : 0x21eac 9959.874555850:1: Owner: 0x0 9959.874572700:1: 9959.874589850:1: Command 'includeconfig': 9959.874607050:1: type : 1 9959.874624350:1: pData: 0x0 9959.874641750:1: Hdlr : 0x237d8 9959.874658700:1: Owner: 0x0 9959.874675350:1: 9959.874692600:1: Command 'umask': 9959.874709550:1: type : 5 9959.874726750:1: pData: 0x0 9959.874744050:1: Hdlr : 0x18a50 9959.874761350:1: Owner: 0x0 9959.874778150:1: 9959.874795500:1: Command 'maxopenfiles': 9959.874812350:1: type : 6 9959.874829600:1: pData: 0x0 9959.874846800:1: Hdlr : 0x189a0 9959.874864050:1: Owner: 0x0 9959.874880900:1: 9959.874898300:1: Command 'debugprinttemplatelist': 9959.874915650:1: type : 4 9959.874932700:1: pData: 0x675b0 9959.874949750:1: Hdlr : 0x0 9959.874966950:1: Owner: 0x0 9959.874983750:1: 9959.875001250:1: Command 'debugprintmodulelist': 9959.875025300:1: type : 4 9959.875043900:1: pData: 0x675ac 9959.875061350:1: Hdlr : 0x0 9959.875078650:1: Owner: 0x0 9959.875095350:1: 9959.875113400:1: Command 'debugprintcfsyslinehandlerlist': 9959.875130750:1: type : 4 9959.875147800:1: pData: 0x675a8 9959.875165150:1: Hdlr : 0x0 9959.875182200:1: Owner: 0x0 9959.875199200:1: 9959.875216500:1: Command 'moddir': 9959.875233800:1: type : 11 9959.875250950:1: pData: 0x6a510 9959.875268200:1: Hdlr : 0x0 9959.875285400:1: Owner: 0x0 9959.875302200:1: 9959.875319500:1: Command 'generateconfiggraph': 9959.875337000:1: type : 11 9959.875354150:1: pData: 0x69d04 9959.875371350:1: Hdlr : 0x0 9959.875388250:1: Owner: 0x0 9959.875405100:1: 9959.875422500:1: Command 'errormessagestostderr': 9959.875439850:1: type : 4 9959.875457150:1: pData: 0x675a4 9959.875474600:1: Hdlr : 0x0 9959.875491750:1: Owner: 0x0 9959.875508350:1: 9959.875525950:1: Command 'maxmessagesize': 9959.875543200:1: type : 7 9959.875560150:1: pData: 0x0 9959.875583050:1: Hdlr : 0x18970 9959.875601400:1: Owner: 0x0 9959.875618450:1: 9959.875635600:1: Command 'privdroptouser': 9959.875653000:1: type : 2 9959.875669950:1: pData: 0x69d00 9959.875687350:1: Hdlr : 0x0 9959.875704350:1: Owner: 0x0 9959.875721000:1: 9959.875738300:1: Command 'privdroptouserid': 9959.875755550:1: type : 6 9959.875772800:1: pData: 0x69d00 9959.875790000:1: Hdlr : 0x0 9959.875807050:1: Owner: 0x0 9959.875823800:1: 9959.875841300:1: Command 'privdroptogroup': 9959.875858800:1: type : 3 9959.875875950:1: pData: 0x69cfc 9959.875893350:1: Hdlr : 0x0 9959.875910450:1: Owner: 0x0 9959.875927400:1: 9959.875944800:1: Command 'privdroptogroupid': 9959.875962400:1: type : 3 9959.875979350:1: pData: 0x69cfc 9959.875996700:1: Hdlr : 0x0 9959.876013700:1: Owner: 0x0 9959.876030600:1: 9959.876048150:1: Command 'actionname': 9959.876065550:1: type : 11 9959.876082750:1: pData: 0x6a5e8 9959.876100050:1: Hdlr : 0x0 9959.876117100:1: Owner: 0x0 9959.876139550:1: 9959.876157950:1: Command 'actionqueuefilename': 9959.876175450:1: type : 11 9959.876192550:1: pData: 0x6a5e0 9959.876209750:1: Hdlr : 0x0 9959.876226800:1: Owner: 0x0 9959.876243550:1: 9959.876260800:1: Command 'actionqueuesize': 9959.876278150:1: type : 6 9959.876295300:1: pData: 0x6933c 9959.876312200:1: Hdlr : 0x0 9959.876329200:1: Owner: 0x0 9959.876345900:1: 9959.876363350:1: Command 'actionqueuemaxdiskspace': 9959.876380650:1: type : 7 9959.876397650:1: pData: 0x6a5d8 9959.876414800:1: Hdlr : 0x0 9959.876432000:1: Owner: 0x0 9959.876448850:1: 9959.876466450:1: Command 'actionqueuehighwatermark': 9959.876483750:1: type : 6 9959.876500850:1: pData: 0x69338 9959.876517900:1: Hdlr : 0x0 9959.876534800:1: Owner: 0x0 9959.876551700:1: 9959.876569150:1: Command 'actionqueuelowwatermark': 9959.876586650:1: type : 6 9959.876603650:1: pData: 0x69334 9959.876620750:1: Hdlr : 0x0 9959.876637850:1: Owner: 0x0 9959.876654750:1: 9959.876672200:1: Command 'actionqueuediscardmark': 9959.876695750:1: type : 6 9959.876714150:1: pData: 0x69330 9959.876731550:1: Hdlr : 0x0 9959.876748750:1: Owner: 0x0 9959.876765450:1: 9959.876782800:1: Command 'actionqueuediscardseverity': 9959.876800000:1: type : 6 9959.876817200:1: pData: 0x6932c 9959.876834200:1: Hdlr : 0x0 9959.876851200:1: Owner: 0x0 9959.876867850:1: 9959.876885050:1: Command 'actionqueuecheckpointinterval': 9959.876902150:1: type : 6 9959.876919550:1: pData: 0x6a5d0 9959.876936650:1: Hdlr : 0x0 9959.876953850:1: Owner: 0x0 9959.876970500:1: 9959.876988100:1: Command 'actionqueuesyncqueuefiles': 9959.877005450:1: type : 4 9959.877022500:1: pData: 0x6a5cc 9959.877039750:1: Hdlr : 0x0 9959.877056700:1: Owner: 0x0 9959.877073550:1: 9959.877090900:1: Command 'actionqueuetype': 9959.877108250:1: type : 11 9959.877125100:1: pData: 0x0 9959.877142200:1: Hdlr : 0x4444c 9959.877159100:1: Owner: 0x0 9959.877175950:1: 9959.877193250:1: Command 'actionqueueworkerthreads': 9959.877216200:1: type : 6 9959.877234300:1: pData: 0x69328 9959.877251400:1: Hdlr : 0x0 9959.877268650:1: Owner: 0x0 9959.877285250:1: 9959.877302850:1: Command 'actionqueuetimeoutshutdown': 9959.877320200:1: type : 6 9959.877337400:1: pData: 0x6a5c8 9959.877354550:1: Hdlr : 0x0 9959.877371650:1: Owner: 0x0 9959.877388700:1: 9959.877405950:1: Command 'actionqueuetimeoutactioncompletion': 9959.877423300:1: type : 6 9959.877440700:1: pData: 0x69324 9959.877457600:1: Hdlr : 0x0 9959.877474800:1: Owner: 0x0 9959.877491700:1: 9959.877509050:1: Command 'actionqueuetimeoutenqueue': 9959.877526350:1: type : 6 9959.877543650:1: pData: 0x69320 9959.877560650:1: Hdlr : 0x0 9959.877577950:1: Owner: 0x0 9959.877594700:1: 9959.877612200:1: Command 'actionqueueworkertimeoutthreadshutdown': 9959.877629450:1: type : 6 9959.877646650:1: pData: 0x6931c 9959.877663900:1: Hdlr : 0x0 9959.877681000:1: Owner: 0x0 9959.877697700:1: 9959.877714850:1: Command 'actionqueueworkerthreadminimummessages': 9959.877738100:1: type : 6 9959.877756400:1: pData: 0x69318 9959.877773750:1: Hdlr : 0x0 9959.877791000:1: Owner: 0x0 9959.877807750:1: 9959.877825200:1: Command 'actionqueuemaxfilesize': 9959.877842600:1: type : 7 9959.877859850:1: pData: 0x69310 9959.877876950:1: Hdlr : 0x0 9959.877894400:1: Owner: 0x0 9959.877911000:1: 9959.877928650:1: Command 'actionqueuesaveonshutdown': 9959.877945800:1: type : 4 9959.877963000:1: pData: 0x69308 9959.877979950:1: Hdlr : 0x0 9959.877997400:1: Owner: 0x0 9959.878014000:1: 9959.878031250:1: Command 'actionqueuedequeueslowdown': 9959.878048450:1: type : 6 9959.878065650:1: pData: 0x6a5c4 9959.878082800:1: Hdlr : 0x0 9959.878099950:1: Owner: 0x0 9959.878116650:1: 9959.878133850:1: Command 'actionqueuedequeuetimebegin': 9959.878151100:1: type : 6 9959.878168000:1: pData: 0x6a5c0 9959.878185200:1: Hdlr : 0x0 9959.878202300:1: Owner: 0x0 9959.878219100:1: 9959.878236400:1: Command 'actionqueuedequeuetimeend': 9959.878259050:1: type : 6 9959.878277250:1: pData: 0x69304 9959.878294850:1: Hdlr : 0x0 9959.878311900:1: Owner: 0x0 9959.878328650:1: 9959.878346050:1: Command 'actionexeconlyeverynthtime': 9959.878363500:1: type : 6 9959.878380450:1: pData: 0x6a5bc 9959.878397800:1: Hdlr : 0x0 9959.878414800:1: Owner: 0x0 9959.878431800:1: 9959.878449300:1: Command 'actionexeconlyeverynthtimetimeout': 9959.878466600:1: type : 6 9959.878483750:1: pData: 0x6a5b8 9959.878500700:1: Hdlr : 0x0 9959.878517850:1: Owner: 0x0 9959.878534600:1: 9959.878551950:1: Command 'repeatedmsgcontainsoriginalmsg': 9959.878569200:1: type : 4 9959.878586450:1: pData: 0x6a5b4 9959.878603550:1: Hdlr : 0x0 9959.878620700:1: Owner: 0x0 9959.878637450:1: 9959.878654750:1: Command 'markmessageperiod': 9959.878671700:1: type : 6 9959.878689000:1: pData: 0xff090ec8 9959.878706150:1: Hdlr : 0x0 9959.878723500:1: Owner: 0xff080850 9959.878740250:1: 9959.878757600:1: Command 'omitlocallogging': 9959.878774950:1: type : 4 9959.878862250:1: pData: 0xff0729ec 9959.878883900:1: Hdlr : 0x0 9959.878904400:1: Owner: 0xff0618c8 9959.878924650:1: 9959.878945450:1: Command 'inputunixlistensocketignoremsgtimestamp': 9959.878966100:1: type : 4 9959.878986700:1: pData: 0xff072784 9959.879007150:1: Hdlr : 0x0 9959.879027850:1: Owner: 0xff0618c8 9959.879047750:1: 9959.879068200:1: Command 'systemlogsocketname': 9959.879088850:1: type : 11 9959.879109100:1: pData: 0xff0729e8 9959.879129600:1: Hdlr : 0x0 9959.879150100:1: Owner: 0xff0618c8 9959.879170100:1: 9959.879190750:1: Command 'inputunixlistensockethostname': 9959.879211150:1: type : 11 9959.879231600:1: pData: 0xff0729e4 9959.879252100:1: Hdlr : 0x0 9959.879272400:1: Owner: 0xff0618c8 9959.879292600:1: 9959.879312900:1: Command 'inputunixlistensocketflowcontrol': 9959.879333400:1: type : 4 9959.879353700:1: pData: 0xff0729e0 9959.879373950:1: Hdlr : 0x0 9959.879394450:1: Owner: 0xff0618c8 9959.879414200:1: 9959.879434800:1: Command 'inputunixlistensocketcreatepath': 9959.879455200:1: type : 4 9959.879475800:1: pData: 0xff0729dc 9959.879495900:1: Hdlr : 0x0 9959.879516350:1: Owner: 0xff0618c8 9959.879536350:1: 9959.879556550:1: Command 'addunixlistensocket': 9959.879577300:1: type : 11 9959.879597600:1: pData: 0x0 9959.879618400:1: Hdlr : 0xff060c88 9959.879638750:1: Owner: 0xff0618c8 9959.879658750:1: 9959.879679750:1: Command 'systemlogsocketignoremsgtimestamp': 9959.879700250:1: type : 4 9959.879720500:1: pData: 0x0 9959.879740800:1: Hdlr : 0xff060c24 9959.879761400:1: Owner: 0xff0618c8 9959.879781200:1: 9959.879801750:1: Command 'systemlogsocketflowcontrol': 9959.879822300:1: type : 4 9959.879842850:1: pData: 0x0 9959.879863150:1: Hdlr : 0xff060c58 9959.879883650:1: Owner: 0xff0618c8 9959.879903700:1: 9959.879924150:1: Command 'inputtcpserverrun': 9959.879944600:1: type : 11 9959.879965100:1: pData: 0x0 9959.879985650:1: Hdlr : 0xff030b90 9959.880005900:1: Owner: 0xff030f98 9959.880026050:1: 9959.880046700:1: Command 'inputtcpmaxsessions': 9959.880067300:1: type : 6 9959.880087750:1: pData: 0xff041e80 9959.880108250:1: Hdlr : 0x0 9959.880128350:1: Owner: 0xff030f98 9959.880148450:1: 9959.880168950:1: Command 'inputtcpmaxlisteners': 9959.880189550:1: type : 6 9959.880209750:1: pData: 0xff041e7c 9959.880230250:1: Hdlr : 0x0 9959.880250550:1: Owner: 0xff030f98 9959.880270850:1: 9959.880291300:1: Command 'inputtcpservernotifyonconnectionclose': 9959.880311800:1: type : 4 9959.880332350:1: pData: 0xff0420b0 9959.880352500:1: Hdlr : 0x0 9959.880372750:1: Owner: 0xff030f98 9959.880392900:1: 9959.880413650:1: Command 'inputtcpserverstreamdrivermode': 9959.880434250:1: type : 6 9959.880454450:1: pData: 0xff0420ac 9959.880475050:1: Hdlr : 0x0 9959.880495450:1: Owner: 0xff030f98 9959.880515350:1: 9959.880536000:1: Command 'inputtcpserverstreamdriverauthmode': 9959.880556450:1: type : 11 9959.880576850:1: pData: 0xff0420a8 9959.880597000:1: Hdlr : 0x0 9959.880617650:1: Owner: 0xff030f98 9959.880637550:1: 9959.880658200:1: Command 'inputtcpserverstreamdriverpermittedpeer': 9959.880678700:1: type : 11 9959.880698800:1: pData: 0x0 9959.880719150:1: Hdlr : 0xff030a74 9959.880739400:1: Owner: 0xff030f98 9959.880759500:1: 9959.880779900:1: Command 'inputtcpserveraddtlframedelimiter': 9959.880800500:1: type : 6 9959.880820900:1: pData: 0xff041e78 9959.880841150:1: Hdlr : 0x0 9959.880861750:1: Owner: 0xff030f98 9959.880881850:1: 9959.880902950:1: Command 'inputtcpserverinputname': 9959.880923250:1: type : 11 9959.880943800:1: pData: 0xff0420a4 9959.880964200:1: Hdlr : 0x0 9959.880984600:1: Owner: 0xff030f98 9959.881004700:1: 9959.881025350:1: Command 'inputtcpserverbindruleset': 9959.881045700:1: type : 11 9959.881065950:1: pData: 0x0 9959.881086350:1: Hdlr : 0xff030acc 9959.881106650:1: Owner: 0xff030f98 9959.881126750:1: 9959.881147000:1: Command 'udpserverrun': 9959.881167950:1: type : 11 9959.881188350:1: pData: 0x0 9959.881208600:1: Hdlr : 0xfefd0c0c 9959.881229150:1: Owner: 0xfefd1a88 9959.881249150:1: 9959.881270050:1: Command 'udpserveraddress': 9959.881290600:1: type : 11 9959.881311150:1: pData: 0xfefe2900 9959.881331650:1: Hdlr : 0x0 9959.881351950:1: Owner: 0xfefd1a88 9959.881371700:1: 9959.881392200:1: Command 'udpservertimerequery': 9959.881412500:1: type : 6 9959.881433100:1: pData: 0xfefe272c 9959.881453200:1: Hdlr : 0x0 9959.881473800:1: Owner: 0xfefd1a88 9959.881493650:1: 9959.881513550:1: 9959.881535950:1: Messages with malicious PTR DNS Records are not dropped. 9959.881557300:1: Control characters are replaced upon reception. 9959.881578650:1: Control character escape sequence prefix is '#'. 9959.881599700:1: Main queue size 10000 messages. 9959.881621550:1: Main queue worker threads: 1, wThread shutdown: 60000, Perists every 0 updates. 9959.881643550:1: Main queue timeouts: shutdown: 1500, action completion shutdown: 1000, enq: 2000 9959.881665450:1: Main queue watermarks: high: 8000, low: 2000, discard: 9800, discard-severity: 8 9959.881686950:1: Main queue save on shutdown 1, max disk space allowed 0 9959.881707850:1: Work Directory: ''. 9959.881923300:1: logmsg: flags 1, from 'fire03', msg [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start 9959.881952900:1: Message has legacy syslog format. 9959.881984500:1: main Q: entry added, size now 1 entries 9959.882014550:1: wtpAdviseMaxWorkers signals busy 9959.882056250:1: main Q: EnqueueMsg advised worker start 9959.882088050:2: main Q: entry deleted, state 0, size now 0 entries 9959.882157050:1: (re)started. 9959.882198400:1: Debugging enabled, SIGUSR1 to turn off debugging. 9959.882235350:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'seahorse_ssh_source_init': FALSE 9959.882281200:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'monitor the GPG home directory': FALSE 9959.882315100:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'dbus-daemon': FALSE 9959.882348000:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'freedesktop': FALSE 9959.882380650:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'gnome-keyring-daemon': FALSE 9959.882413250:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'quota_ufs': FALSE 9959.882446000:2: Filter: check for property 'msg' (value ' [origin software="rsyslogd" swVersion="4.7.2" x-pid="27445" x-info="http://www.rsyslog.com"] (re)start') contains 'HAL': FALSE 9959.882477150:2: testing filter, f_pmask 0 9959.882499400:1: initialization completed, transitioning to regular run mode 9959.882549650:2: testing filter, f_pmask 127 9959.882587200:2: Called action, logging to builtin-file 9959.882637850:2: file to log to: DynMSG 9959.882706350:4: --------imuxsock calling select, active file descriptors (max 0): 9959.882841300:2: file stream messages params: flush interval 0, async write 0 9959.882903550:5: caller requested object 'nsd_ptcp', not found (iRet -3003) 9959.882949850:5: Requested to load module 'lmnsd_ptcp' 9959.882986150:2: Added new entry 0 for file cache, file '/var/log/remote/fire03/messages'. 9959.883028200:2: doWrite, pData->pStrm 7b308, lenBuf 136 9959.883104900:6: Listening on UDP syslogd socket 5 (IPv6/port 514). 9959.883155450:2: strm 7b308: file -1(messages) flush, buflen 136 9959.883209450:5: loading module '/usr/local/rsyslog-4.7.2/lib/rsyslog/lmnsd_ptcp.so' 9959.883247950:2: file '/var/log/remote/fire03/messages' opened as #4 with mode 416 9959.883341600:6: Listening on UDP syslogd socket 6 (IPv4/port 514). 9959.883375000:6: --------imUDP calling select, active file descriptors (max 6): 5 6 9959.883896750:2: strm 7b308: opened file '/var/log/remote/fire03/messages' for WRITE as 4 9959.883940750:5: source file nsd_ptcp.c requested reference for module 'lmnetstrms', reference count now 4 9959.883981700:2: strm 7b308: file 4 write wrote 136 bytes 9959.884015650:5: module of type 2 being loaded. 9959.884055900:5: source file netstrms.c requested reference for module 'lmnsd_ptcp', reference count now 1 9959.884092700:2: testing filter, f_pmask 0 9959.884131300:2: testing filter, f_pmask 0 9959.884160650:2: testing filter, f_pmask 0 9959.884189200:2: testing filter, f_pmask 0 9959.884217250:2: testing filter, f_pmask 0 9959.884244450:2: testing filter, f_pmask 0 9959.884272400:2: testing filter, f_pmask 0 9959.884299700:2: testing filter, f_pmask 0 9959.884327250:2: testing filter, f_pmask 1 9959.884355150:2: testing filter, f_pmask 3 9959.884383150:2: testing filter, f_pmask 3 9959.884410950:2: testing filter, f_pmask 0 9959.884438650:2: testing filter, f_pmask 15 9959.884466350:2: testing filter, f_pmask 63 9959.884493850:2: testing filter, f_pmask 0 9959.884521600:2: testing filter, f_pmask 0 9959.884549100:2: testing filter, f_pmask 0 9959.884576600:2: testing filter, f_pmask 0 9959.884604150:2: testing filter, f_pmask 0 9959.884632000:2: testing filter, f_pmask 0 9959.884664500:5: creating tcp listen socket on port 10514 9959.884710700:2: main Q:Reg/w0: worker IDLE, waiting for work. 9959.884935850:5: creating tcp listen socket on port 514 9959.885090700:5: Allocating buffer for 200 TCP sessions. 9959.885230700:5: -------- calling select, active fds (max 10): 7 8 9 10 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From rgerhards at hq.adiscon.com Thu Nov 4 16:00:39 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 16:00:39 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <166441385.1129.1288863236907.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD657@GRFEXC.intern.adiscon.com> You can chain the actions via &. This video tutorial may be useful: http://www.rsyslog.com/article350/ Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Thursday, November 04, 2010 10:34 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? > > Hi, > > I have a rather complex config which puts logs into a database by using > several Templates with > different SQL-Statements and so I use several Actions which match > different log-lines. > > And I want rsyslog to do disk assisted queuing in case of DB-failure or > lag. > > Question: Is is possible in 4.6.4 to have ONE queue with MULTIPLE > actions so that > queued SQL-Statements will be processed in correct order like it would > be done without queueing? > > Or do I have to use the MainQueue for that which will affect log-files > in case of DB failure? > > TIA > -Marc > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Nov 4 16:36:22 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Nov 2010 16:36:22 +0100 Subject: [rsyslog] Rsyslog evaluation questions References: <4CD1CA9B.3090301@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD65A@GRFEXC.intern.adiscon.com> Dear Jason, Thank you for your interest in rsyslog. 1) Of course you can use regexes. For more information, take a look at the following links: http://www.rsyslog.com/doc/property_replacer.html http://www.rsyslog.com/doc/rsyslog_conf_filter.html http://www.rsyslog.com/regex/ 2) The documentation is here: http://www.rsyslog.com/doc/manual.html You can find it to be the first point from the dropdown menu when moving the mouse over "Help" on the website. 3. We do not have any information on that. On the rsyslog side, this is basically not a problem. But we are not sure if MySQL is capable of this and might be the bottleneck in this kind of setup. Florian -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Jason Antman Sent: Mittwoch, 3. November 2010 21:48 To: rsyslog at lists.adiscon.com Subject: [rsyslog] Rsyslog evaluation questions Hello, At $UNIVERSITY, we're planning to overhaul our centralized syslog infrastructure. At the moment, we're running syslog-ng on an aged Solaris/Sparc box and planning on replacing it with either syslog-ng or rsyslog on CentOS 5. We're currently handling approximately 3 million lines per day from 25 hosts. We can expect that to increase to approx. 5M+ lines from 40 hosts within 6 months. We're beginning an evaluation of rsyslog, and I have a few preliminary questions I can't find clear answers to in the docs, and was hoping perhaps someone with rsyslog in production could provide some input. 1) We do a lot of post-processing of logs to pull out relevant information. Specifically, we have quite a few scripts (PHP) that rely on preg_match pulling out capture groups and putting them in different columns in MySQL. Does rsyslog support any regexes in templates that would allow something like this? 2) I last visited rsyslog.com before the site redesign. Where has the documentation gone on the new site? I can't even seem to find a config file statement reference... 3) Assuming an even distribution over time (not quite accurate), any thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) with a single 2.8GHz CPU and 512MB RAM would go? Thanks for any advice, Jason Antman _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From epiphani at gmail.com Thu Nov 4 17:03:50 2010 From: epiphani at gmail.com (Aaron Wiebe) Date: Thu, 4 Nov 2010 12:03:50 -0400 Subject: [rsyslog] Rsyslog evaluation questions In-Reply-To: <4CD1CA9B.3090301@jasonantman.com> References: <4CD1CA9B.3090301@jasonantman.com> Message-ID: > 3) Assuming an even distribution over time (not quite accurate), any > thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) > with a single 2.8GHz CPU and 512MB RAM would go? 2 million lines a day is 70 lines per second assuming an 8 hour day. MySQL, properly configured, will eat that. Rsyslog won't even notice that log level. Your table sizes may be problematic over time, and your ability to query may impact things. Regardless, your concern should be mysql, not rsyslog. -Aaron From marc.schiffbauer at mightycare.de Thu Nov 4 17:33:33 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 17:33:33 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD657@GRFEXC.intern.adiscon.com> Message-ID: <1009060389.1184.1288888413183.JavaMail.root@zimbra> ----- Urspr?ngliche Mail ----- > You can chain the actions via &. But that is like a logical AND, right? What I would need then is an OR... Example: #---------------------- # Action 1 if $syslogtag startswith 'service[' \ and $msg contains 'FOO' \ then :ompgsql:localhost,db01,dbuser,dbpass;template_A1 # Action 2 if $syslogtag startswith 'service[' \ and $msg contains 'BAR' \ then :ompgsql:localhost,db01,dbuser,dbpass;template_A2 #---------------------- template_for_A1 does an INSERT on the database while template_for_A2 does an UPDATE to the row INSERTed by Action 1 So its fundamental that Action 2 is always executed AFTER Action 1. Without Queueing, this is working. But how to enable queuing so that behavior will be maintained? Ideally only those two Actions would build a single queue not affecting other things like file writing. > > This video tutorial may be useful: > > http://www.rsyslog.com/article350/ > Thanks for that hint. Nice thing. But it does not really help in that particular case :-/ -Marc > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > > Sent: Thursday, November 04, 2010 10:34 AM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? > > > > Hi, > > > > I have a rather complex config which puts logs into a database by > > using > > several Templates with > > different SQL-Statements and so I use several Actions which match > > different log-lines. > > > > And I want rsyslog to do disk assisted queuing in case of DB-failure > > or > > lag. > > > > Question: Is is possible in 4.6.4 to have ONE queue with MULTIPLE > > actions so that > > queued SQL-Statements will be processed in correct order like it > > would > > be done without queueing? > > > > Or do I have to use the MainQueue for that which will affect > > log-files > > in case of DB failure? > > > > TIA > > -Marc From rgerhards at hq.adiscon.com Thu Nov 4 17:35:44 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 17:35:44 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <1009060389.1184.1288888413183.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD65B@GRFEXC.intern.adiscon.com> ah, ok, I thought you wanted to chain two actions. Ruleset include is most probably the way to go. Have a look at omruleset. However, I remember that it is really hard to setup -- when I did this, it was the first time that I *really* noticed the need for a new config format myself ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Thursday, November 04, 2010 5:34 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > ----- Urspr?ngliche Mail ----- > > You can chain the actions via &. > > But that is like a logical AND, right? > What I would need then is an OR... > > Example: > #---------------------- > # Action 1 > if $syslogtag startswith 'service[' \ > and $msg contains 'FOO' \ > then :ompgsql:localhost,db01,dbuser,dbpass;template_A1 > > # Action 2 > if $syslogtag startswith 'service[' \ > and $msg contains 'BAR' \ > then :ompgsql:localhost,db01,dbuser,dbpass;template_A2 > #---------------------- > > template_for_A1 does an INSERT on the database while template_for_A2 > does an UPDATE to the row INSERTed by Action 1 > > So its fundamental that Action 2 is always executed AFTER Action 1. > > Without Queueing, this is working. But how to enable queuing so that > behavior > will be maintained? > > Ideally only those two Actions would build a single queue not affecting > other things > like file writing. > > > > > > This video tutorial may be useful: > > > > http://www.rsyslog.com/article350/ > > > > Thanks for that hint. Nice thing. But it does not really help in that > particular case :-/ > > -Marc > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > > > Sent: Thursday, November 04, 2010 10:34 AM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? > > > > > > Hi, > > > > > > I have a rather complex config which puts logs into a database by > > > using > > > several Templates with > > > different SQL-Statements and so I use several Actions which match > > > different log-lines. > > > > > > And I want rsyslog to do disk assisted queuing in case of DB- > failure > > > or > > > lag. > > > > > > Question: Is is possible in 4.6.4 to have ONE queue with MULTIPLE > > > actions so that > > > queued SQL-Statements will be processed in correct order like it > > > would > > > be done without queueing? > > > > > > Or do I have to use the MainQueue for that which will affect > > > log-files > > > in case of DB failure? > > > > > > TIA > > > -Marc > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marc.schiffbauer at mightycare.de Thu Nov 4 17:57:10 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 17:57:10 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD65B@GRFEXC.intern.adiscon.com> Message-ID: <219421162.1196.1288889829956.JavaMail.root@zimbra> Hi Rainer, > ah, ok, I thought you wanted to chain two actions. Ruleset include is > most > probably the way to go. Have a look at omruleset. thanks, but: "Available Since: 5.3.4" No way with 4.6.4? > However, I remember > that it > is really hard to setup -- when I did this, it was the first time that > I > *really* noticed the need for a new config format myself ;) > :-) -- Senior Consultant :: Solution Architect IT-Security :: Free Software :: GNU/Linux :: Virtualization Mobile +49_151_16227402 :: Fon +49_6187_9058695 :: Fax +49_2645_976896250 MightyCare Solutions GmbH http://www.mightycare.de Firmenangaben unter http://www.mightycare.de/impressum From rgerhards at hq.adiscon.com Thu Nov 4 17:58:22 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 17:58:22 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <219421162.1196.1288889829956.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD65C@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Thursday, November 04, 2010 5:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > Hi Rainer, > > > ah, ok, I thought you wanted to chain two actions. Ruleset include is > > most > > probably the way to go. Have a look at omruleset. > > thanks, but: > > "Available Since: 5.3.4" > > No way with 4.6.4? I don't think so (but others often have very creative ideas ;)) Rainer From marc.schiffbauer at mightycare.de Thu Nov 4 18:10:50 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 18:10:50 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD65C@GRFEXC.intern.adiscon.com> Message-ID: <505054804.1204.1288890650008.JavaMail.root@zimbra> Will the global MainMsgQueue maintain message order? Thanks -Marc ----- Urspr?ngliche Mail ----- > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > > Sent: Thursday, November 04, 2010 5:57 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > > > Hi Rainer, > > > > > ah, ok, I thought you wanted to chain two actions. Ruleset include > > > is > > > most > > > probably the way to go. Have a look at omruleset. > > > > thanks, but: > > > > "Available Since: 5.3.4" > > > > No way with 4.6.4? > > I don't think so (but others often have very creative ideas ;)) > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- Senior Consultant :: Solution Architect IT-Security :: Free Software :: GNU/Linux :: Virtualization Mobile +49_151_16227402 :: Fon +49_6187_9058695 :: Fax +49_2645_976896250 MightyCare Solutions GmbH http://www.mightycare.de Firmenangaben unter http://www.mightycare.de/impressum From david at lang.hm Thu Nov 4 19:53:50 2010 From: david at lang.hm (david at lang.hm) Date: Thu, 4 Nov 2010 11:53:50 -0700 (PDT) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <505054804.1204.1288890650008.JavaMail.root@zimbra> References: <505054804.1204.1288890650008.JavaMail.root@zimbra> Message-ID: On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > Will the global MainMsgQueue maintain message order? I believe that it will, but keep in mind that the settings needed to try and maintain message order are very different than the settings that you need to use to get best performance. what sort of message rate are you talking about using? David Lang > Thanks > -Marc > > ----- Urspr?ngliche Mail ----- >> > -----Original Message----- >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer >> > Sent: Thursday, November 04, 2010 5:57 PM >> > To: rsyslog-users >> > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? >> > >> > Hi Rainer, >> > >> > > ah, ok, I thought you wanted to chain two actions. Ruleset include >> > > is >> > > most >> > > probably the way to go. Have a look at omruleset. >> > >> > thanks, but: >> > >> > "Available Since: 5.3.4" >> > >> > No way with 4.6.4? >> >> I don't think so (but others often have very creative ideas ;)) >> >> Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > -- > Senior Consultant :: Solution Architect > IT-Security :: Free Software :: GNU/Linux :: Virtualization > Mobile +49_151_16227402 :: Fon +49_6187_9058695 :: Fax +49_2645_976896250 > MightyCare Solutions GmbH http://www.mightycare.de > Firmenangaben unter http://www.mightycare.de/impressum > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Nov 4 20:32:14 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Nov 2010 20:32:14 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <505054804.1204.1288890650008.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD65D@GRFEXC.intern.adiscon.com> I am not sure on the exact semantics of the 4.6.4 engine, but in any case you need to limit concurrency (and thus speed) as much as possible. If you really need sequence, that means you need to run it on a single thread (with a direct mode main queue). The defaults look like they "preserve sequence", and you will see the inevitable out of sequence only when heavy traffic occurs. However, any approach to trying to "preserve sequence" will not work if looked at closely enough, so there is nothing that can be preseverd. Details are in my Linux Kongress paper: http://blog.gerhards.net/2010/10/linux-kongress-2010-rsyslog-paper.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Thursday, November 04, 2010 7:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > > > Will the global MainMsgQueue maintain message order? > > I believe that it will, but keep in mind that the settings needed to > try > and maintain message order are very different than the settings that > you > need to use to get best performance. > > what sort of message rate are you talking about using? > > David Lang > > > Thanks > > -Marc > > > > ----- Urspr?ngliche Mail ----- > >> > -----Original Message----- > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > >> > Sent: Thursday, November 04, 2010 5:57 PM > >> > To: rsyslog-users > >> > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > >> > > >> > Hi Rainer, > >> > > >> > > ah, ok, I thought you wanted to chain two actions. Ruleset > include > >> > > is > >> > > most > >> > > probably the way to go. Have a look at omruleset. > >> > > >> > thanks, but: > >> > > >> > "Available Since: 5.3.4" > >> > > >> > No way with 4.6.4? > >> > >> I don't think so (but others often have very creative ideas ;)) > >> > >> Rainer > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > > -- > > Senior Consultant :: Solution Architect > > IT-Security :: Free Software :: GNU/Linux :: Virtualization > > Mobile +49_151_16227402 :: Fon +49_6187_9058695 :: Fax > +49_2645_976896250 > > MightyCare Solutions GmbH http://www.mightycare.de > > Firmenangaben unter http://www.mightycare.de/impressum > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marc.schiffbauer at mightycare.de Thu Nov 4 22:40:37 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 22:40:37 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: Message-ID: <441389232.1262.1288906837905.JavaMail.root@zimbra> > On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > > > Will the global MainMsgQueue maintain message order? > > I believe that it will, but keep in mind that the settings needed to > try > and maintain message order are very different than the settings that > you > need to use to get best performance. > > what sort of message rate are you talking about using? > About up to 3000 Messages/s -Marc From jason at jasonantman.com Thu Nov 4 22:47:27 2010 From: jason at jasonantman.com (Jason Antman) Date: Thu, 04 Nov 2010 17:47:27 -0400 Subject: [rsyslog] segfault with dynamic filenames (was: Rsyslog evaluation questions) In-Reply-To: References: <4CD1CA9B.3090301@jasonantman.com> Message-ID: <4CD329EF.7030001@jasonantman.com> Thanks to both of you guys for your answers. FYI, our days have 24 hours in them :) At this point I just got a config file worked up. I got the regexes working to do all of the matching I need into separate fields (the online regex tester was a big help), and upgraded to 5.6.0 (CentOS 5.5, in-house RPM build) in order to of bind imudp to a ruleset (so we can have one ruleset for localhost/unix socket, and one ruleset for all remote hosts). Unfortunately, I'm getting a segfault almost immediately on startup due to the following lines: $template RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility-text%.log" *.* ?RemoteHost I believe I've narrowed the problem down to any templated (dynamic) filename... even if I dump everything external directly to /var/log/remote and add the following in my local ruleset: $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" *.* ?LocalHost I still get a segfault. The last line in the output running with -dn is always: "file to log to: RemoteHost" (or LocalHost, when I used that config). Has anyone seen anything like this? I have rsyslog 2.0.6 running at another site and dynamic filenames work fine... sort of troubling that 5.6.0 is having issues with them... Thanks, Jason Antman Aaron Wiebe wrote: >> 3) Assuming an even distribution over time (not quite accurate), any >> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) >> with a single 2.8GHz CPU and 512MB RAM would go? >> > > 2 million lines a day is 70 lines per second assuming an 8 hour day. > MySQL, properly configured, will eat that. Rsyslog won't even notice > that log level. > > Your table sizes may be problematic over time, and your ability to > query may impact things. Regardless, your concern should be mysql, > not rsyslog. > > -Aaron > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From joe at joetify.com Thu Nov 4 22:43:35 2010 From: joe at joetify.com (Joe Williams) Date: Thu, 4 Nov 2010 14:43:35 -0700 Subject: [rsyslog] REST output plugin In-Reply-To: <25FE6B31-55B5-448B-B68C-85FFDBA95ABF@joetify.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD31D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD325@GRFEXC.intern.adiscon.com> <25FE6B31-55B5-448B-B68C-85FFDBA95ABF@joetify.com> Message-ID: <779305FE-58AA-439A-8114-DECD17D1C239@joetify.com> Rainer, Any thoughts on this? Thanks. -Joe On Sep 20, 2010, at 10:02 AM, Joe Williams wrote: > Rainer, > > I'm sure you have been busy with other stuff (stats subsystem, etc) but have you had a chance to look into this? > > Thanks. > > -Joe > > > On Sep 9, 2010, at 1:59 PM, Joe Williams wrote: > >> Fair enough. :P >> >> To be honest that was the first C I have written in years. If you want to off load some of the work I am happy to take it on with some guidance. >> >> -Joe >> >> >> On Sep 9, 2010, at 12:55 PM, Rainer Gerhards wrote: >> >>> Quite honestly, this looks like quite some work for me vs. no work for you. I >>> thought the major part of the work was on the party that gained the benefits >>> ;) >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Joe Williams >>>> Sent: Thursday, September 09, 2010 8:27 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] REST output plugin >>>> >>>> Rainier, >>>> >>>> Here is some basic code using libcurl for doing what I want: >>>> >>>> http://gist.github.com/572182 >>>> >>>> The data variable is json key/value pairs where I would like to put the >>>> syslog fields as in my aforementioned example. >>>> >>>> Let me know if you have any questions. >>>> >>>> Thanks for the help. >>>> >>>> -Joe >>>> >>>> >>>> >>>> On Sep 9, 2010, at 1:24 AM, Rainer Gerhards wrote: >>>> >>>>> If you know how to do the REST part, integrating this into a plugin >>>> is rather >>>>> simple. If you provide me working code for that part, I can setup a >>>> skeleton >>>>> and you just need to tweak the result. >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Joe Williams >>>>>> Sent: Thursday, September 09, 2010 2:43 AM >>>>>> To: rsyslog-users >>>>>> Subject: [rsyslog] REST output plugin >>>>>> >>>>>> >>>>>> Anyone have an idea of how difficult it would be to create an output >>>>>> plugin that uses REST to PUT log lines as JSON to a specific URL? >>>>>> Probably something equivalent to the following curl command. >>>>>> >>>>>> curl -X PUT http://host/ -d '{"app-name": "foo", "msg": "bar", >>>> "proc- >>>>>> id":"baz", "timestamp":"timestamp:::date-rfc3339", >>>>>> "hostname":"hostname", "severity":"syslogseverity-text"}' >>>>>> >>>>>> Thanks. >>>>>> >>>>>> -Joe >>>>>> >>>>>> >>>>>> Name: Joseph A. Williams >>>>>> Email: joe at joetify.com >>>>>> Blog: http://www.joeandmotorboat.com/ >>>>>> Twitter: http://twitter.com/williamsjoe >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> >>>> Name: Joseph A. Williams >>>> Email: joe at joetify.com >>>> Blog: http://www.joeandmotorboat.com/ >>>> Twitter: http://twitter.com/williamsjoe >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> Name: Joseph A. Williams >> Email: joe at joetify.com >> Blog: http://www.joeandmotorboat.com/ >> Twitter: http://twitter.com/williamsjoe >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > Twitter: http://twitter.com/williamsjoe > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From marc.schiffbauer at mightycare.de Thu Nov 4 23:25:23 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 23:25:23 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD65D@GRFEXC.intern.adiscon.com> Message-ID: <1241624757.1276.1288909523344.JavaMail.root@zimbra> > I am not sure on the exact semantics of the 4.6.4 engine, but in any > case you > need to limit concurrency (and thus speed) as much as possible. If you > really > need sequence, that means you need to run it on a single thread (with > a > direct mode main queue). The defaults look like they "preserve > sequence", and > you will see the inevitable out of sequence only when heavy traffic > occurs. > > However, any approach to trying to "preserve sequence" will not work > if > looked at closely enough, so there is nothing that can be preseverd. > Details > are in my Linux Kongress paper: > > http://blog.gerhards.net/2010/10/linux-kongress-2010-rsyslog-paper.html > Thanks Rainer. Interesting. I read chapter 7 about concurrency optimization. What I am looking for is indeed what is described there: preserved timestamp order. And this only with some restrictions: * in time windows within few seconds or so because those message will be processed with the need of chronological order. * only per system Of course there might be corner cases where this will not always work but this will then be only due to a system outage or something like that. I am now going to try 5.6.0 with omruleset. I hope it will work then. Are there any config options to control or affect timestamp ordering? In my case I guess it would be sufficient if only one thread would feed the database. TIA -Marc From david at lang.hm Thu Nov 4 23:26:07 2010 From: david at lang.hm (david at lang.hm) Date: Thu, 4 Nov 2010 15:26:07 -0700 (PDT) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <441389232.1262.1288906837905.JavaMail.root@zimbra> References: <441389232.1262.1288906837905.JavaMail.root@zimbra> Message-ID: On Thu, 4 Nov 2010, Marc Schiffbauer wrote: >> On Thu, 4 Nov 2010, Marc Schiffbauer wrote: >> >>> Will the global MainMsgQueue maintain message order? >> >> I believe that it will, but keep in mind that the settings needed to >> try >> and maintain message order are very different than the settings that >> you >> need to use to get best performance. >> >> what sort of message rate are you talking about using? >> > > About up to 3000 Messages/s at that data rate you will have a very hard time doing things the way you are trying. at the database level, doing 3000 inserts/sec requires _very_ high-end hardware. rsyslog supports inserting multiple messages in one command, and with that you could probably handle 3000 messages/sec on very low-end hardware (because rsyslog could insert messages in batches of 100 or more if needed). unfortunantly, when you enable this, you don't maintain the message order. at the rsyslog level, doing disk based queues with everything tuned to come as close to minimizing the chance of data loss is very hard to do at that traffic level. I ran tests last year on a 8 core box with 64G of ram and a fusion-io SSD pci card, and depending on the filesystem I used, I was able to get from 2K to 8K messages/sec where I wasn't trying to do anything more than write the message to a log file. you are going to have to think very hard about how critical it really is for you to maintain the same message order and what type of reliability you really need for your messages. it's counter-intuitive, but it may be that you end up with better overall reliability with a much faster configuration that has worse 'worst-case' data loss, but is fast enough that under normal conditions everything is processed really fast than you would under a configuration that is much slower all the time, but has a better worst-case data loss. David Lang From david at lang.hm Thu Nov 4 23:30:53 2010 From: david at lang.hm (david at lang.hm) Date: Thu, 4 Nov 2010 15:30:53 -0700 (PDT) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <1241624757.1276.1288909523344.JavaMail.root@zimbra> References: <1241624757.1276.1288909523344.JavaMail.root@zimbra> Message-ID: On Thu, 4 Nov 2010, Marc Schiffbauer wrote: >> I am not sure on the exact semantics of the 4.6.4 engine, but in any >> case you >> need to limit concurrency (and thus speed) as much as possible. If you >> really >> need sequence, that means you need to run it on a single thread (with >> a >> direct mode main queue). The defaults look like they "preserve >> sequence", and >> you will see the inevitable out of sequence only when heavy traffic >> occurs. >> >> However, any approach to trying to "preserve sequence" will not work >> if >> looked at closely enough, so there is nothing that can be preseverd. >> Details >> are in my Linux Kongress paper: >> >> http://blog.gerhards.net/2010/10/linux-kongress-2010-rsyslog-paper.html >> > > Thanks Rainer. Interesting. I read chapter 7 about concurrency optimization. > > What I am looking for is indeed what is described there: preserved timestamp > order. And this only with some restrictions: > * in time windows within few seconds or so > because those message will be processed with the need of chronological order. why is it so critical to process them in chronological order? are you absolutly sure that the messages will not be able to get re-ordered over the network (for example, UDP syslog messages could pass each other on the network) David Lang > * only per system > > Of course there might be corner cases where this will not always work but this > will then be only due to a system outage or something like that. > > I am now going to try 5.6.0 with omruleset. I hope it will work > then. > > Are there any config options to control or affect timestamp ordering? > > In my case I guess it would be sufficient if only one thread would feed the > database. > > TIA > -Marc > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From marc.schiffbauer at mightycare.de Thu Nov 4 23:51:49 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 23:51:49 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: Message-ID: <714991354.1290.1288911109819.JavaMail.root@zimbra> > > > > About up to 3000 Messages/s To be more precise: That will be the expected peak. So the average will be much lower. And we have decent hardware to do this. > > at that data rate you will have a very hard time doing things the way > you > are trying. > > at the database level, doing 3000 inserts/sec requires _very_ high-end > hardware. > rsyslog supports inserting multiple messages in one command, > and > with that you could probably handle 3000 messages/sec on very low-end > hardware (because rsyslog could insert messages in batches of 100 or > more > if needed). unfortunantly, when you enable this, you don't maintain > the > message order. > > at the rsyslog level, doing disk based queues with everything tuned to > come as close to minimizing the chance of data loss is very hard to do > at > that traffic level. I ran tests last year on a 8 core box with 64G of > ram > and a fusion-io SSD pci card, and depending on the filesystem I used, > I > was able to get from 2K to 8K messages/sec where I wasn't trying to do > anything more than write the message to a log file. > Thanks for the numbers. > you are going to have to think very hard about how critical it really > is > for you to maintain the same message order and what type of > reliability > you really need for your messages. > > it's counter-intuitive, but it may be that you end up with better > overall > reliability with a much faster configuration that has worse > 'worst-case' > data loss, but is fast enough that under normal conditions everything > is > processed really fast than you would under a configuration that is > much > slower all the time, but has a better worst-case data loss. Yes, I will keep that in mind, thanks. > > David Lang From marc.schiffbauer at mightycare.de Thu Nov 4 23:53:53 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 4 Nov 2010 23:53:53 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: Message-ID: <740504239.1292.1288911233802.JavaMail.root@zimbra> > > why is it so critical to process them in chronological order? are you > absolutly sure that the messages will not be able to get re-ordered > over > the network (for example, UDP syslog messages could pass each other on > the > network) > We use RELP for transmission. And order is important because some messages used for DB inserts must be seen and processed before other messages that will trigger an update to the row inserted before. -Marc From marc.schiffbauer at mightycare.de Fri Nov 5 00:24:32 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Fri, 5 Nov 2010 00:24:32 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: Message-ID: <1884162126.1294.1288913072423.JavaMail.root@zimbra> > rsyslog supports inserting multiple messages in one command, how can this be done? -Marc From david at lang.hm Fri Nov 5 02:25:18 2010 From: david at lang.hm (david at lang.hm) Date: Thu, 4 Nov 2010 18:25:18 -0700 (PDT) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <1884162126.1294.1288913072423.JavaMail.root@zimbra> References: <1884162126.1294.1288913072423.JavaMail.root@zimbra> Message-ID: On Fri, 5 Nov 2010, Marc Schiffbauer wrote: >> rsyslog supports inserting multiple messages in one command, > > how can this be done? in later versions it's on by default. search the config directives page for 'batch' and you will find two parameters, one for working with the main queue, and one with the action queue. the main queue defaults to working on 32 messages at a time, the action queue defaults to 16 messages at a time. another thing that will cause you grief (in terms of ordering) is if you enable more than one worker thread. David Lang From linuxmail at 4lin.net Fri Nov 5 08:23:26 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Fri, 05 Nov 2010 08:23:26 +0100 Subject: [rsyslog] Filter help Message-ID: <1288941806.490.35.camel@pcdenny.rbg.informatik.tu-darmstadt.de> hi, I want to build a filter, that: 1. match IP 2. all Messages, but NOT *.emerg and NOT *.err 3. To file /var/log/remote/host.log My tests: if :fromhost-ip, isequal, '130.83.160.22' and $syslogseverity-text != 'err' or $syslogseverity-text != 'emerg' then /var/log/nos.log & ~ or: if :fromhost-ip, isequal, '130.83.160.22' and ( $syslogseverity-text != 'err' or $syslogseverity-text != 'emerg') then /var/log/nos.log & ~ but I get only syntax error ... any suggestions? cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From david at lang.hm Fri Nov 5 08:32:21 2010 From: david at lang.hm (david at lang.hm) Date: Fri, 5 Nov 2010 00:32:21 -0700 (PDT) Subject: [rsyslog] Filter help In-Reply-To: <1288941806.490.35.camel@pcdenny.rbg.informatik.tu-darmstadt.de> References: <1288941806.490.35.camel@pcdenny.rbg.informatik.tu-darmstadt.de> Message-ID: >hi, > >I want to build a filter, that: > >1. match IP >2. all Messages, but NOT *.emerg and NOT *.err >3. To file /var/log/remote/host.log > >My tests: > > if :fromhost-ip, isequal, '130.83.160.22' and $syslogseverity-text != 'err' or $syslogseverity-text != 'emerg' then /var/log/nos.log > & ~ > >or: > >if :fromhost-ip, isequal, '130.83.160.22' and ( $syslogseverity-text != 'err' or $syslogseverity-text != 'emerg') then /var/log/nos.log >& ~ > >but I get only syntax error ... you are mixing different types of tests. you can't use the 'isequal' type of thing in an if statement you need to do something more like (untested): if ($fromhost-ip = '130.83.160.22' and ($syslogseverity-text != 'err' or $syslogseverity-text != 'emerg')) then /var/log/nos.log David Lang From linuxmail at 4lin.net Fri Nov 5 09:11:38 2010 From: linuxmail at 4lin.net (Denny Schierz) Date: Fri, 05 Nov 2010 09:11:38 +0100 Subject: [rsyslog] Filter help In-Reply-To: References: <1288941806.490.35.camel@pcdenny.rbg.informatik.tu-darmstadt.de> Message-ID: <1288944698.490.37.camel@pcdenny.rbg.informatik.tu-darmstadt.de> hi David, Am Freitag, den 05.11.2010, 00:32 -0700 schrieb david at lang.hm: > you can't use the 'isequal' type of thing in an if statement > you need to do something more like (untested): > > if ($fromhost-ip = '130.83.160.22' and ($syslogseverity-text != 'err' > or $syslogseverity-text != 'emerg')) then /var/log/nos.log thanks :-) seems to be working, only one '=' was missing in your example: if ($fromhost-ip == '130.83.160.22' and ($syslogseverity-text != 'err' or $syslogseverity-text != 'emerg')) then /var/log/nos.log :-) cu denny -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From marc.schiffbauer at mightycare.de Fri Nov 5 10:56:09 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Fri, 5 Nov 2010 10:56:09 +0100 (CET) Subject: [rsyslog] $ActionQueue* options for $RuleSet ? Message-ID: <1115648386.1360.1288950969071.JavaMail.root@zimbra> Hi all, now that I switched to stable 5.6.0 I am using a Ruleset to build one queue for several actions. Where do I place the $ActionQueue* options to control that queue? My test config looks currently like this: #--------------------------------------------------------- $ModLoad ompgsql $ModLoad omruleset $RuleSet dbAction $RulesetCreateMainQueue on if $syslogtag startswith 'postfix/smtpd[' \ and $msg contains 'client=' \ and not ($msg contains 'NOQUEUE: ') \ then :ompgsql:localhost,maillog,syslog,XXXXXX;template_1 if $syslogtag startswith 'postfix/cleanup[' \ and $msg contains 'header X-Foo:' \ then :ompgsql:localhost,maillog,syslog,XXXXXX;template_2 #switch back to default ruleset $ruleset RSYSLOG_DefaultRuleset # begin action $ActionOmrulesetRulesetName dbAction mail.* :omruleset: mail.* ~ #--------------------------------------------------------- -- Senior Consultant :: Solution Architect IT-Security :: Free Software :: GNU/Linux :: Virtualization Mobile +49_151_16227402 :: Fon +49_6187_9058695 :: Fax +49_2645_976896250 MightyCare Solutions GmbH http://www.mightycare.de Firmenangaben unter http://www.mightycare.de/impressum From rgerhards at hq.adiscon.com Fri Nov 5 11:56:59 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 11:56:59 +0100 Subject: [rsyslog] Could some autotools expert help with this bug report? Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD664@GRFEXC.intern.adiscon.com> The report says it all ;) http://bugzilla.adiscon.com/show_bug.cgi?id=202 Rainer From rgerhards at hq.adiscon.com Fri Nov 5 11:58:54 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 11:58:54 +0100 Subject: [rsyslog] REST output plugin References: <9B6E2A8877C38245BFB15CC491A11DA71DD31D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD325@GRFEXC.intern.adiscon.com><25FE6B31-55B5-448B-B68C-85FFDBA95ABF@joetify.com> <779305FE-58AA-439A-8114-DECD17D1C239@joetify.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD666@GRFEXC.intern.adiscon.com> Sorry, I am swamped with - from the project PoV - more important things at least for the next three to four month. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Joe Williams > Sent: Thursday, November 04, 2010 10:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] REST output plugin > > > Rainer, > > Any thoughts on this? > > Thanks. > -Joe > > > On Sep 20, 2010, at 10:02 AM, Joe Williams wrote: > > > Rainer, > > > > I'm sure you have been busy with other stuff (stats subsystem, etc) > but have you had a chance to look into this? > > > > Thanks. > > > > -Joe > > > > > > On Sep 9, 2010, at 1:59 PM, Joe Williams wrote: > > > >> Fair enough. :P > >> > >> To be honest that was the first C I have written in years. If you > want to off load some of the work I am happy to take it on with some > guidance. > >> > >> -Joe > >> > >> > >> On Sep 9, 2010, at 12:55 PM, Rainer Gerhards wrote: > >> > >>> Quite honestly, this looks like quite some work for me vs. no work > for you. I > >>> thought the major part of the work was on the party that gained the > benefits > >>> ;) > >>> > >>> Rainer > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Joe Williams > >>>> Sent: Thursday, September 09, 2010 8:27 PM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] REST output plugin > >>>> > >>>> Rainier, > >>>> > >>>> Here is some basic code using libcurl for doing what I want: > >>>> > >>>> http://gist.github.com/572182 > >>>> > >>>> The data variable is json key/value pairs where I would like to > put the > >>>> syslog fields as in my aforementioned example. > >>>> > >>>> Let me know if you have any questions. > >>>> > >>>> Thanks for the help. > >>>> > >>>> -Joe > >>>> > >>>> > >>>> > >>>> On Sep 9, 2010, at 1:24 AM, Rainer Gerhards wrote: > >>>> > >>>>> If you know how to do the REST part, integrating this into a > plugin > >>>> is rather > >>>>> simple. If you provide me working code for that part, I can setup > a > >>>> skeleton > >>>>> and you just need to tweak the result. > >>>>> > >>>>> Rainer > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>>> bounces at lists.adiscon.com] On Behalf Of Joe Williams > >>>>>> Sent: Thursday, September 09, 2010 2:43 AM > >>>>>> To: rsyslog-users > >>>>>> Subject: [rsyslog] REST output plugin > >>>>>> > >>>>>> > >>>>>> Anyone have an idea of how difficult it would be to create an > output > >>>>>> plugin that uses REST to PUT log lines as JSON to a specific > URL? > >>>>>> Probably something equivalent to the following curl command. > >>>>>> > >>>>>> curl -X PUT http://host/ -d '{"app-name": "foo", "msg": "bar", > >>>> "proc- > >>>>>> id":"baz", "timestamp":"timestamp:::date-rfc3339", > >>>>>> "hostname":"hostname", "severity":"syslogseverity-text"}' > >>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> -Joe > >>>>>> > >>>>>> > >>>>>> Name: Joseph A. Williams > >>>>>> Email: joe at joetify.com > >>>>>> Blog: http://www.joeandmotorboat.com/ > >>>>>> Twitter: http://twitter.com/williamsjoe > >>>>>> > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> > >>>> Name: Joseph A. Williams > >>>> Email: joe at joetify.com > >>>> Blog: http://www.joeandmotorboat.com/ > >>>> Twitter: http://twitter.com/williamsjoe > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> Name: Joseph A. Williams > >> Email: joe at joetify.com > >> Blog: http://www.joeandmotorboat.com/ > >> Twitter: http://twitter.com/williamsjoe > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > > Name: Joseph A. Williams > > Email: joe at joetify.com > > Blog: http://www.joeandmotorboat.com/ > > Twitter: http://twitter.com/williamsjoe > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > Twitter: http://twitter.com/williamsjoe > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Nov 5 12:01:06 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 12:01:06 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <441389232.1262.1288906837905.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD667@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Thursday, November 04, 2010 11:26 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > > >> On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > >> > >>> Will the global MainMsgQueue maintain message order? > >> > >> I believe that it will, but keep in mind that the settings needed to > >> try > >> and maintain message order are very different than the settings that > >> you > >> need to use to get best performance. > >> > >> what sort of message rate are you talking about using? > >> > > > > About up to 3000 Messages/s > > at that data rate you will have a very hard time doing things the way > you > are trying. > > at the database level, doing 3000 inserts/sec requires _very_ high-end > hardware. rsyslog supports inserting multiple messages in one command, > and > with that you could probably handle 3000 messages/sec on very low-end > hardware (because rsyslog could insert messages in batches of 100 or > more > if needed). unfortunantly, when you enable this, you don't maintain the > message order. > > at the rsyslog level, doing disk based queues with everything tuned to > come as close to minimizing the chance of data loss is very hard to do > at > that traffic level. I ran tests last year on a 8 core box with 64G of > ram > and a fusion-io SSD pci card, and depending on the filesystem I used, I > was able to get from 2K to 8K messages/sec where I wasn't trying to do > anything more than write the message to a log file. disk based queues *could* work much faster, but it requires a full rewrite. Some details already in the design paper, but no point in discussing that right now: I can't do this right at this time. Rainer From rgerhards at hq.adiscon.com Fri Nov 5 12:06:51 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 12:06:51 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <1241624757.1276.1288909523344.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD668@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Thursday, November 04, 2010 11:25 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > > I am not sure on the exact semantics of the 4.6.4 engine, but in any > > case you > > need to limit concurrency (and thus speed) as much as possible. If > you > > really > > need sequence, that means you need to run it on a single thread (with > > a > > direct mode main queue). The defaults look like they "preserve > > sequence", and > > you will see the inevitable out of sequence only when heavy traffic > > occurs. > > > > However, any approach to trying to "preserve sequence" will not work > > if > > looked at closely enough, so there is nothing that can be preseverd. > > Details > > are in my Linux Kongress paper: > > > > http://blog.gerhards.net/2010/10/linux-kongress-2010-rsyslog- > paper.html > > > > Thanks Rainer. Interesting. I read chapter 7 about concurrency > optimization. > > What I am looking for is indeed what is described there: preserved > timestamp > order. And this only with some restrictions: > * in time windows within few seconds or so > because those message will be processed with the need of > chronological order. > * only per system Well, I don't intend to duplicate the paper content, but the point is that you can not "preserve" order, because as soon as there is a bit of concurrency involved, there is no strict order that you could preserve. At best, you have a partial order. > Of course there might be corner cases where this will not always work > but this > will then be only due to a system outage or something like that. Read on the scheduling order ;) > I am now going to try 5.6.0 with omruleset. I hope it will work > then. > > Are there any config options to control or affect timestamp ordering? no, that's external to rsyslog > In my case I guess it would be sufficient if only one thread would feed > the > database. so that's the route to take: make sure you set the queue to single worker, use direct queue mode for the output action (this is the default). There is no way, though, to guard against scheduler-based reordering e.g. of messages arriving at the "same time" both in imuxsock and imrelp. But if I understood you correctly, this is not an issue for you. Note that the default config should already work pretty well for your case, but be sure to explicitely limit the workers to one. Rainer > > TIA > -Marc > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Nov 5 12:07:57 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 12:07:57 +0100 Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? References: <714991354.1290.1288911109819.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD669@GRFEXC.intern.adiscon.com> oh, one thing I forgot: you mentioned disk queues in some mail. You can not use disk-assisted queues if you totally insist on ordering, because there is some inherent re-ordering possible when starting up disk mode. So you need to use pure disk queues or pure memory queues. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Thursday, November 04, 2010 11:52 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > > > > > > About up to 3000 Messages/s > > To be more precise: That will be the expected peak. So the average will > be > much lower. > > And we have decent hardware to do this. > > > > > at that data rate you will have a very hard time doing things the way > > you > > are trying. > > > > at the database level, doing 3000 inserts/sec requires _very_ high- > end > > hardware. > > rsyslog supports inserting multiple messages in one command, > > and > > with that you could probably handle 3000 messages/sec on very low-end > > hardware (because rsyslog could insert messages in batches of 100 or > > more > > if needed). unfortunantly, when you enable this, you don't maintain > > the > > message order. > > > > at the rsyslog level, doing disk based queues with everything tuned > to > > come as close to minimizing the chance of data loss is very hard to > do > > at > > that traffic level. I ran tests last year on a 8 core box with 64G of > > ram > > and a fusion-io SSD pci card, and depending on the filesystem I used, > > I > > was able to get from 2K to 8K messages/sec where I wasn't trying to > do > > anything more than write the message to a log file. > > > > Thanks for the numbers. > > > you are going to have to think very hard about how critical it really > > is > > for you to maintain the same message order and what type of > > reliability > > you really need for your messages. > > > > it's counter-intuitive, but it may be that you end up with better > > overall > > reliability with a much faster configuration that has worse > > 'worst-case' > > data loss, but is fast enough that under normal conditions everything > > is > > processed really fast than you would under a configuration that is > > much > > slower all the time, but has a better worst-case data loss. > > Yes, I will keep that in mind, thanks. > > > > > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jsabo at criminal.org Fri Nov 5 15:42:08 2010 From: jsabo at criminal.org (Jonathan Sabo) Date: Fri, 5 Nov 2010 10:42:08 -0400 Subject: [rsyslog] substring matching in templates Message-ID: List, I want to organise my central syslog server based on physical site location. I can obtain the physical site location by matching a substring in the hostname message property. How do you do that in ryslog templates? Given a known list of site names that are part of the hostname for each system sending logs, what's the best way to match that in the hostname and then place them in directories based on that site name. /var/log/$sitename/$hostname/logs.log For example: web1-nyc Logs would placed in /var/log/nyc/web1-nyc/access.log I think I could say something like: $template NYCTemplate, /app/log/nyc/%HOSTNAME%/access.log if $hostname contains "nyc" then ?NYCTemplate But then I have to have each site name hard coded in the config... I have a naming convention of typeNum-site so I would like it to be more dynamic based on whatever follows the hyphen. Is there a way to match this convention and then just create the directory based on the string following the hyphen instead of having to hard code the names in the config? Thanks, Jonathan From tbergfeld at hq.adiscon.com Fri Nov 5 16:07:19 2010 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Fri, 5 Nov 2010 16:07:19 +0100 Subject: [rsyslog] segfault with dynamic filenames (was: Rsyslog evaluationquestions) References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> Hi Jason, unfortunately we were not able to reproduce your issue in Fedora 13. We will create an environment with centos and try it again. Tom -----Urspr?ngliche Nachricht----- Von: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason Antman Gesendet: Donnerstag, 4. November 2010 22:47 An: rsyslog-users Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog evaluationquestions) Thanks to both of you guys for your answers. FYI, our days have 24 hours in them :) At this point I just got a config file worked up. I got the regexes working to do all of the matching I need into separate fields (the online regex tester was a big help), and upgraded to 5.6.0 (CentOS 5.5, in-house RPM build) in order to of bind imudp to a ruleset (so we can have one ruleset for localhost/unix socket, and one ruleset for all remote hosts). Unfortunately, I'm getting a segfault almost immediately on startup due to the following lines: $template RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility -text%.log" *.* ?RemoteHost I believe I've narrowed the problem down to any templated (dynamic) filename... even if I dump everything external directly to /var/log/remote and add the following in my local ruleset: $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" *.* ?LocalHost I still get a segfault. The last line in the output running with -dn is always: "file to log to: RemoteHost" (or LocalHost, when I used that config). Has anyone seen anything like this? I have rsyslog 2.0.6 running at another site and dynamic filenames work fine... sort of troubling that 5.6.0 is having issues with them... Thanks, Jason Antman Aaron Wiebe wrote: >> 3) Assuming an even distribution over time (not quite accurate), any >> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) >> with a single 2.8GHz CPU and 512MB RAM would go? >> > > 2 million lines a day is 70 lines per second assuming an 8 hour day. > MySQL, properly configured, will eat that. Rsyslog won't even notice > that log level. > > Your table sizes may be problematic over time, and your ability to > query may impact things. Regardless, your concern should be mysql, > not rsyslog. > > -Aaron > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From jason at jasonantman.com Fri Nov 5 16:30:52 2010 From: jason at jasonantman.com (Jason Antman) Date: Fri, 05 Nov 2010 11:30:52 -0400 Subject: [rsyslog] segfault with dynamic filenames In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> Message-ID: <4CD4232C.3080100@jasonantman.com> I'm doing a local build of it with all of the debugging features. On a side note, since I'm not the one who built the RPM and there's no documentation on it (that I could find).... what is omtemplate? -Jason Tom Bergfeld wrote: > Hi Jason, > > unfortunately we were not able to reproduce your issue in Fedora 13. > We will create an environment with centos and try it again. > > Tom > > > -----Urspr?ngliche Nachricht----- > Von: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason Antman > Gesendet: Donnerstag, 4. November 2010 22:47 > An: rsyslog-users > Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > evaluationquestions) > > Thanks to both of you guys for your answers. FYI, our days have 24 hours > in them :) > > At this point I just got a config file worked up. I got the regexes > working to do all of the matching I need into separate fields (the > online regex tester was a big help), and upgraded to 5.6.0 (CentOS 5.5, > in-house RPM build) in order to of bind imudp to a ruleset (so we can > have one ruleset for localhost/unix socket, and one ruleset for all > remote hosts). > > Unfortunately, I'm getting a segfault almost immediately on startup due > to the following lines: > $template > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility > -text%.log" > *.* ?RemoteHost > > I believe I've narrowed the problem down to any templated (dynamic) > filename... even if I dump everything external directly to > /var/log/remote and add the following in my local ruleset: > $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > *.* ?LocalHost > > I still get a segfault. The last line in the output running with -dn is > always: > "file to log to: RemoteHost" (or LocalHost, when I used that config). > > Has anyone seen anything like this? I have rsyslog 2.0.6 running at > another site and dynamic filenames work fine... sort of troubling that > 5.6.0 is having issues with them... > > Thanks, > Jason Antman > > Aaron Wiebe wrote: > >>> 3) Assuming an even distribution over time (not quite accurate), any >>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) >>> with a single 2.8GHz CPU and 512MB RAM would go? >>> >>> >> 2 million lines a day is 70 lines per second assuming an 8 hour day. >> MySQL, properly configured, will eat that. Rsyslog won't even notice >> that log level. >> >> Your table sizes may be problematic over time, and your ability to >> query may impact things. Regardless, your concern should be mysql, >> not rsyslog. >> >> -Aaron >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From jsabo at criminal.org Fri Nov 5 17:03:10 2010 From: jsabo at criminal.org (Jonathan Sabo) Date: Fri, 5 Nov 2010 12:03:10 -0400 Subject: [rsyslog] substring matching in templates In-Reply-To: References: Message-ID: I think I can do something like this: %HOSTNAME:F,45:2% To match my convention where the location is the suffix of the hostname seperated by a hyphen. Ie web1-nyc But how do you use this property replacer to construct a path name in a template..... On Fri, Nov 5, 2010 at 10:42 AM, Jonathan Sabo wrote: > List, > > I want to organise my central syslog server based on physical site > location. ?I can obtain the physical site location by matching a > substring in the hostname message property. ?How do you do that in > ryslog templates? ?Given a known list of site names that are part of > the hostname for each system sending logs, what's the best way to > match that in the hostname and then place them in directories based on > that site name. > > /var/log/$sitename/$hostname/logs.log > > For example: > > web1-nyc > > Logs would placed in > > /var/log/nyc/web1-nyc/access.log > > I think I could say something like: > > $template NYCTemplate, /app/log/nyc/%HOSTNAME%/access.log > > if $hostname contains "nyc" then ?NYCTemplate > > But then I have to have each site name hard coded in the config... ? I > have a naming convention of typeNum-site so I would like it to be more > dynamic based on whatever follows the hyphen. ? Is there a way to > match this convention and then just create the directory based on the > string following the hyphen instead of having to hard code the names > in the config? > > Thanks, > > Jonathan > From rgerhards at hq.adiscon.com Fri Nov 5 17:14:00 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 17:14:00 +0100 Subject: [rsyslog] substring matching in templates References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD675@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jonathan Sabo > Sent: Friday, November 05, 2010 5:03 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] substring matching in templates > > I think I can do something like this: > > %HOSTNAME:F,45:2% > > To match my convention where the location is the suffix of the > hostname seperated by a hyphen. Ie web1-nyc > > But how do you use this property replacer to construct a path name in > a template..... Just like you already did: put "%HOSTNAME:F,45:2%" into the template. ;) This is where it is used. Rainer > > > > On Fri, Nov 5, 2010 at 10:42 AM, Jonathan Sabo > wrote: > > List, > > > > I want to organise my central syslog server based on physical site > > location. ?I can obtain the physical site location by matching a > > substring in the hostname message property. ?How do you do that in > > ryslog templates? ?Given a known list of site names that are part of > > the hostname for each system sending logs, what's the best way to > > match that in the hostname and then place them in directories based > on > > that site name. > > > > /var/log/$sitename/$hostname/logs.log > > > > For example: > > > > web1-nyc > > > > Logs would placed in > > > > /var/log/nyc/web1-nyc/access.log > > > > I think I could say something like: > > > > $template NYCTemplate, /app/log/nyc/%HOSTNAME%/access.log > > > > if $hostname contains "nyc" then ?NYCTemplate > > > > But then I have to have each site name hard coded in the config... > I > > have a naming convention of typeNum-site so I would like it to be > more > > dynamic based on whatever follows the hyphen. ? Is there a way to > > match this convention and then just create the directory based on the > > string following the hyphen instead of having to hard code the names > > in the config? > > > > Thanks, > > > > Jonathan > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Nov 5 17:16:48 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 17:16:48 +0100 Subject: [rsyslog] segfault with dynamic filenames References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4232C.3080100@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD676@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jason Antman > Sent: Friday, November 05, 2010 4:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] segfault with dynamic filenames > > I'm doing a local build of it with all of the debugging features. > > On a side note, since I'm not the one who built the RPM and there's no > documentation on it (that I could find).... what is omtemplate? That's a template for developers. Developers who want to implement an output module can copy it and use that as a starting point. Same for imtemplate for input modules. So not something you need to have a any production system. Rainer From jason at jasonantman.com Fri Nov 5 17:26:22 2010 From: jason at jasonantman.com (Jason Antman) Date: Fri, 05 Nov 2010 12:26:22 -0400 Subject: [rsyslog] segfault with dynamic filenames In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> Message-ID: <4CD4302E.4070803@jasonantman.com> Thanks for the info! Just sent this to Tom, forgot to CC the list.... I just rebuilt the RPM with all debugging features enabled and am experiencing the same issue. I have: - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env variable set to "LogFuncFlow FileTrace=omfile.c Debug") - strace output - RPM spec file including configure options, etc. - the Fedora patch that is being applied - the actual RPMs, if they're of interest, and a source rpm Which of this is of use? Should I email it? Open a ticket and attach? Here are the last few lines before the crash: 2954.318300688:b7f06b90: Processing next action 2954.318308532:b7f06b90: Called action(NotAllMark), logging to builtin-file 2954.318315864:b7f06b90: Called action(Batch), logging to builtin-file 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx 2954.318411997:b7f06b90: omfile.c:697: doAction: enter 2954.318418085:b7f06b90: file to log to: LocalHost 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter (and then it dies, in the middle of a select() call) I'm going to start looking through omfile.c and perhaps adding some of my own instrumentation just so I can localize it a bit more. Thanks for any advice you can provide, Jason Antman Tom Bergfeld wrote: > Hi Jason, > > unfortunately we were not able to reproduce your issue in Fedora 13. > We will create an environment with centos and try it again. > > Tom > > > -----Urspr?ngliche Nachricht----- > Von: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason Antman > Gesendet: Donnerstag, 4. November 2010 22:47 > An: rsyslog-users > Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > evaluationquestions) > > Thanks to both of you guys for your answers. FYI, our days have 24 hours > in them :) > > At this point I just got a config file worked up. I got the regexes > working to do all of the matching I need into separate fields (the > online regex tester was a big help), and upgraded to 5.6.0 (CentOS 5.5, > in-house RPM build) in order to of bind imudp to a ruleset (so we can > have one ruleset for localhost/unix socket, and one ruleset for all > remote hosts). > > Unfortunately, I'm getting a segfault almost immediately on startup due > to the following lines: > $template > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility > -text%.log" > *.* ?RemoteHost > > I believe I've narrowed the problem down to any templated (dynamic) > filename... even if I dump everything external directly to > /var/log/remote and add the following in my local ruleset: > $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > *.* ?LocalHost > > I still get a segfault. The last line in the output running with -dn is > always: > "file to log to: RemoteHost" (or LocalHost, when I used that config). > > Has anyone seen anything like this? I have rsyslog 2.0.6 running at > another site and dynamic filenames work fine... sort of troubling that > 5.6.0 is having issues with them... > > Thanks, > Jason Antman > > Aaron Wiebe wrote: > >>> 3) Assuming an even distribution over time (not quite accurate), any >>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) >>> with a single 2.8GHz CPU and 512MB RAM would go? >>> >>> >> 2 million lines a day is 70 lines per second assuming an 8 hour day. >> MySQL, properly configured, will eat that. Rsyslog won't even notice >> that log level. >> >> Your table sizes may be problematic over time, and your ability to >> query may impact things. Regardless, your concern should be mysql, >> not rsyslog. >> >> -Aaron >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Fri Nov 5 17:30:30 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 17:30:30 +0100 Subject: [rsyslog] segfault with dynamic filenames References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4302E.4070803@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com> An you run it under valgrind control? If the issue reappears, it should give us *very* good hints of where to look at... RAiner > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jason Antman > Sent: Friday, November 05, 2010 5:26 PM > To: rsyslog-users > Subject: Re: [rsyslog] segfault with dynamic filenames > > Thanks for the info! > > Just sent this to Tom, forgot to CC the list.... > > I just rebuilt the RPM with all debugging features enabled and am > experiencing the same issue. > > I have: > - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env variable > set to "LogFuncFlow FileTrace=omfile.c Debug") > - strace output > - RPM spec file including configure options, etc. > - the Fedora patch that is being applied > - the actual RPMs, if they're of interest, and a source rpm > > Which of this is of use? Should I email it? Open a ticket and attach? > > Here are the last few lines before the crash: > 2954.318300688:b7f06b90: Processing next action > 2954.318308532:b7f06b90: Called action(NotAllMark), logging to builtin- > file > 2954.318315864:b7f06b90: Called action(Batch), logging to builtin-file > 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter > 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 > 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx > 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx > 2954.318411997:b7f06b90: omfile.c:697: doAction: enter > 2954.318418085:b7f06b90: file to log to: LocalHost > 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter > 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter > (and then it dies, in the middle of a select() call) > > I'm going to start looking through omfile.c and perhaps adding some of > my own instrumentation just so I can localize it a bit more. > > Thanks for any advice you can provide, > Jason Antman > > Tom Bergfeld wrote: > > Hi Jason, > > > > unfortunately we were not able to reproduce your issue in Fedora 13. > > We will create an environment with centos and try it again. > > > > Tom > > > > > > -----Urspr?ngliche Nachricht----- > > Von: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason > Antman > > Gesendet: Donnerstag, 4. November 2010 22:47 > > An: rsyslog-users > > Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > > evaluationquestions) > > > > Thanks to both of you guys for your answers. FYI, our days have 24 > hours > > in them :) > > > > At this point I just got a config file worked up. I got the regexes > > working to do all of the matching I need into separate fields (the > > online regex tester was a big help), and upgraded to 5.6.0 (CentOS > 5.5, > > in-house RPM build) in order to of bind imudp to a ruleset (so we can > > have one ruleset for localhost/unix socket, and one ruleset for all > > remote hosts). > > > > Unfortunately, I'm getting a segfault almost immediately on startup > due > > to the following lines: > > $template > > > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa > cility > > -text%.log" > > *.* ?RemoteHost > > > > I believe I've narrowed the problem down to any templated (dynamic) > > filename... even if I dump everything external directly to > > /var/log/remote and add the following in my local ruleset: > > $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > > *.* ?LocalHost > > > > I still get a segfault. The last line in the output running with -dn > is > > always: > > "file to log to: RemoteHost" (or LocalHost, when I used that config). > > > > Has anyone seen anything like this? I have rsyslog 2.0.6 running at > > another site and dynamic filenames work fine... sort of troubling > that > > 5.6.0 is having issues with them... > > > > Thanks, > > Jason Antman > > > > Aaron Wiebe wrote: > > > >>> 3) Assuming an even distribution over time (not quite accurate), > any > >>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM > (Xen) > >>> with a single 2.8GHz CPU and 512MB RAM would go? > >>> > >>> > >> 2 million lines a day is 70 lines per second assuming an 8 hour day. > >> MySQL, properly configured, will eat that. Rsyslog won't even > notice > >> that log level. > >> > >> Your table sizes may be problematic over time, and your ability to > >> query may impact things. Regardless, your concern should be mysql, > >> not rsyslog. > >> > >> -Aaron > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jason at jasonantman.com Fri Nov 5 17:57:06 2010 From: jason at jasonantman.com (Jason Antman) Date: Fri, 05 Nov 2010 12:57:06 -0400 Subject: [rsyslog] segfault with dynamic filenames In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com> References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4302E.4070803@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com> Message-ID: <4CD43762.20603@jasonantman.com> I haven't run it with valgrind yet... honestly I don't think I've used valgrind more than once or twice, I'd need to do a bit of reading up to remember how to use it. This isn't a matter of *if* the issue reappears. Every time rsyslogd tries to write out a message matching a dynafile output, it dies (within a few ms of starting, or whenever it actually tries to write the first line out to disk). At the moment, I have the issue localized to the call to getClockFileAccess() on line 517 of tools/omfile.c, and am continuing to trace it through the code. I'll see if one of the more c-skilled guys can give me a hand with valgrind. -Jason Rainer Gerhards wrote: > An you run it under valgrind control? If the issue reappears, it should give > us *very* good hints of where to look at... > > RAiner > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Jason Antman >> Sent: Friday, November 05, 2010 5:26 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] segfault with dynamic filenames >> >> Thanks for the info! >> >> Just sent this to Tom, forgot to CC the list.... >> >> I just rebuilt the RPM with all debugging features enabled and am >> experiencing the same issue. >> >> I have: >> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env variable >> set to "LogFuncFlow FileTrace=omfile.c Debug") >> - strace output >> - RPM spec file including configure options, etc. >> - the Fedora patch that is being applied >> - the actual RPMs, if they're of interest, and a source rpm >> >> Which of this is of use? Should I email it? Open a ticket and attach? >> >> Here are the last few lines before the crash: >> 2954.318300688:b7f06b90: Processing next action >> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to builtin- >> file >> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin-file >> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter >> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 >> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx >> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx >> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter >> 2954.318418085:b7f06b90: file to log to: LocalHost >> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter >> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter >> (and then it dies, in the middle of a select() call) >> >> I'm going to start looking through omfile.c and perhaps adding some of >> my own instrumentation just so I can localize it a bit more. >> >> Thanks for any advice you can provide, >> Jason Antman >> >> Tom Bergfeld wrote: >> >>> Hi Jason, >>> >>> unfortunately we were not able to reproduce your issue in Fedora 13. >>> We will create an environment with centos and try it again. >>> >>> Tom >>> >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: rsyslog-bounces at lists.adiscon.com >>> [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason >>> >> Antman >> >>> Gesendet: Donnerstag, 4. November 2010 22:47 >>> An: rsyslog-users >>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog >>> evaluationquestions) >>> >>> Thanks to both of you guys for your answers. FYI, our days have 24 >>> >> hours >> >>> in them :) >>> >>> At this point I just got a config file worked up. I got the regexes >>> working to do all of the matching I need into separate fields (the >>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS >>> >> 5.5, >> >>> in-house RPM build) in order to of bind imudp to a ruleset (so we can >>> have one ruleset for localhost/unix socket, and one ruleset for all >>> remote hosts). >>> >>> Unfortunately, I'm getting a segfault almost immediately on startup >>> >> due >> >>> to the following lines: >>> $template >>> >>> >> RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa >> cility >> >>> -text%.log" >>> *.* ?RemoteHost >>> >>> I believe I've narrowed the problem down to any templated (dynamic) >>> filename... even if I dump everything external directly to >>> /var/log/remote and add the following in my local ruleset: >>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" >>> *.* ?LocalHost >>> >>> I still get a segfault. The last line in the output running with -dn >>> >> is >> >>> always: >>> "file to log to: RemoteHost" (or LocalHost, when I used that config). >>> >>> Has anyone seen anything like this? I have rsyslog 2.0.6 running at >>> another site and dynamic filenames work fine... sort of troubling >>> >> that >> >>> 5.6.0 is having issues with them... >>> >>> Thanks, >>> Jason Antman >>> >>> Aaron Wiebe wrote: >>> >>> >>>>> 3) Assuming an even distribution over time (not quite accurate), >>>>> >> any >> >>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM >>>>> >> (Xen) >> >>>>> with a single 2.8GHz CPU and 512MB RAM would go? >>>>> >>>>> >>>>> >>>> 2 million lines a day is 70 lines per second assuming an 8 hour day. >>>> MySQL, properly configured, will eat that. Rsyslog won't even >>>> >> notice >> >>>> that log level. >>>> >>>> Your table sizes may be problematic over time, and your ability to >>>> query may impact things. Regardless, your concern should be mysql, >>>> not rsyslog. >>>> >>>> -Aaron >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Fri Nov 5 18:00:59 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 18:00:59 +0100 Subject: [rsyslog] segfault with dynamic filenames References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4302E.4070803@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com> <4CD43762.20603@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD67A@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jason Antman > Sent: Friday, November 05, 2010 5:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] segfault with dynamic filenames > > I haven't run it with valgrind yet... honestly I don't think I've used > valgrind more than once or twice, I'd need to do a bit of reading up to > remember how to use it. It actually is extemely easy... yum install valgrind valgrind rsyslogd ... your regular rsyslogd command line that's it ;) Rainer > > This isn't a matter of *if* the issue reappears. Every time rsyslogd > tries to write out a message matching a dynafile output, it dies > (within > a few ms of starting, or whenever it actually tries to write the first > line out to disk). > > At the moment, I have the issue localized to the call to > getClockFileAccess() on line 517 of tools/omfile.c, and am continuing > to > trace it through the code. I'll see if one of the more c-skilled guys > can give me a hand with valgrind. > > -Jason > > Rainer Gerhards wrote: > > An you run it under valgrind control? If the issue reappears, it > should give > > us *very* good hints of where to look at... > > > > RAiner > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Jason Antman > >> Sent: Friday, November 05, 2010 5:26 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] segfault with dynamic filenames > >> > >> Thanks for the info! > >> > >> Just sent this to Tom, forgot to CC the list.... > >> > >> I just rebuilt the RPM with all debugging features enabled and am > >> experiencing the same issue. > >> > >> I have: > >> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env > variable > >> set to "LogFuncFlow FileTrace=omfile.c Debug") > >> - strace output > >> - RPM spec file including configure options, etc. > >> - the Fedora patch that is being applied > >> - the actual RPMs, if they're of interest, and a source rpm > >> > >> Which of this is of use? Should I email it? Open a ticket and > attach? > >> > >> Here are the last few lines before the crash: > >> 2954.318300688:b7f06b90: Processing next action > >> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to > builtin- > >> file > >> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin- > file > >> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter > >> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 > >> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx > >> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx > >> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter > >> 2954.318418085:b7f06b90: file to log to: LocalHost > >> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter > >> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter > >> (and then it dies, in the middle of a select() call) > >> > >> I'm going to start looking through omfile.c and perhaps adding some > of > >> my own instrumentation just so I can localize it a bit more. > >> > >> Thanks for any advice you can provide, > >> Jason Antman > >> > >> Tom Bergfeld wrote: > >> > >>> Hi Jason, > >>> > >>> unfortunately we were not able to reproduce your issue in Fedora > 13. > >>> We will create an environment with centos and try it again. > >>> > >>> Tom > >>> > >>> > >>> -----Urspr?ngliche Nachricht----- > >>> Von: rsyslog-bounces at lists.adiscon.com > >>> [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason > >>> > >> Antman > >> > >>> Gesendet: Donnerstag, 4. November 2010 22:47 > >>> An: rsyslog-users > >>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > >>> evaluationquestions) > >>> > >>> Thanks to both of you guys for your answers. FYI, our days have 24 > >>> > >> hours > >> > >>> in them :) > >>> > >>> At this point I just got a config file worked up. I got the regexes > >>> working to do all of the matching I need into separate fields (the > >>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS > >>> > >> 5.5, > >> > >>> in-house RPM build) in order to of bind imudp to a ruleset (so we > can > >>> have one ruleset for localhost/unix socket, and one ruleset for all > >>> remote hosts). > >>> > >>> Unfortunately, I'm getting a segfault almost immediately on startup > >>> > >> due > >> > >>> to the following lines: > >>> $template > >>> > >>> > >> > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa > >> cility > >> > >>> -text%.log" > >>> *.* ?RemoteHost > >>> > >>> I believe I've narrowed the problem down to any templated (dynamic) > >>> filename... even if I dump everything external directly to > >>> /var/log/remote and add the following in my local ruleset: > >>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > >>> *.* ?LocalHost > >>> > >>> I still get a segfault. The last line in the output running with - > dn > >>> > >> is > >> > >>> always: > >>> "file to log to: RemoteHost" (or LocalHost, when I used that > config). > >>> > >>> Has anyone seen anything like this? I have rsyslog 2.0.6 running at > >>> another site and dynamic filenames work fine... sort of troubling > >>> > >> that > >> > >>> 5.6.0 is having issues with them... > >>> > >>> Thanks, > >>> Jason Antman > >>> > >>> Aaron Wiebe wrote: > >>> > >>> > >>>>> 3) Assuming an even distribution over time (not quite accurate), > >>>>> > >> any > >> > >>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM > >>>>> > >> (Xen) > >> > >>>>> with a single 2.8GHz CPU and 512MB RAM would go? > >>>>> > >>>>> > >>>>> > >>>> 2 million lines a day is 70 lines per second assuming an 8 hour > day. > >>>> MySQL, properly configured, will eat that. Rsyslog won't even > >>>> > >> notice > >> > >>>> that log level. > >>>> > >>>> Your table sizes may be problematic over time, and your ability to > >>>> query may impact things. Regardless, your concern should be > mysql, > >>>> not rsyslog. > >>>> > >>>> -Aaron > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jason at jasonantman.com Fri Nov 5 18:13:09 2010 From: jason at jasonantman.com (Jason Antman) Date: Fri, 05 Nov 2010 13:13:09 -0400 Subject: [rsyslog] segfault with dynamic filenames In-Reply-To: <4CD43762.20603@jasonantman.com> References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4302E.4070803@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com> <4CD43762.20603@jasonantman.com> Message-ID: <4CD43B25.7080201@jasonantman.com> Ok... in output from configure script: checking whether the compiler provides atomic builtins... yes checking whether the compiler provides atomic builtins for 64 bit data types... yes in omfile.c, lines 89-102: #if HAVE_ATOMIC_BUILTINS_64BIT static uint64 clockFileAccess = 0; #else static unsigned clockFileAccess = 0; #endif /* and the "tick" function */ #ifndef HAVE_ATOMIC_BUILTINS static pthread_mutex_t mutClock; #endif static inline uint64 getClockFileAccess(void) { return ATOMIC_INC_AND_FETCH(&clockFileAccess, &mutClock); } Once again, my C skills are pretty poor... but if HAVE_ATOMIC_BUILTINS is defined.... what is mutClock?? Output, strace log, and valgrind output are being sent to rainer. Jason Antman wrote: > I haven't run it with valgrind yet... honestly I don't think I've used > valgrind more than once or twice, I'd need to do a bit of reading up to > remember how to use it. > > This isn't a matter of *if* the issue reappears. Every time rsyslogd > tries to write out a message matching a dynafile output, it dies (within > a few ms of starting, or whenever it actually tries to write the first > line out to disk). > > At the moment, I have the issue localized to the call to > getClockFileAccess() on line 517 of tools/omfile.c, and am continuing to > trace it through the code. I'll see if one of the more c-skilled guys > can give me a hand with valgrind. > > -Jason > > Rainer Gerhards wrote: > >> An you run it under valgrind control? If the issue reappears, it should give >> us *very* good hints of where to look at... >> >> RAiner >> >> >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Jason Antman >>> Sent: Friday, November 05, 2010 5:26 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] segfault with dynamic filenames >>> >>> Thanks for the info! >>> >>> Just sent this to Tom, forgot to CC the list.... >>> >>> I just rebuilt the RPM with all debugging features enabled and am >>> experiencing the same issue. >>> >>> I have: >>> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env variable >>> set to "LogFuncFlow FileTrace=omfile.c Debug") >>> - strace output >>> - RPM spec file including configure options, etc. >>> - the Fedora patch that is being applied >>> - the actual RPMs, if they're of interest, and a source rpm >>> >>> Which of this is of use? Should I email it? Open a ticket and attach? >>> >>> Here are the last few lines before the crash: >>> 2954.318300688:b7f06b90: Processing next action >>> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to builtin- >>> file >>> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin-file >>> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter >>> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 >>> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx >>> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx >>> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter >>> 2954.318418085:b7f06b90: file to log to: LocalHost >>> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter >>> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter >>> (and then it dies, in the middle of a select() call) >>> >>> I'm going to start looking through omfile.c and perhaps adding some of >>> my own instrumentation just so I can localize it a bit more. >>> >>> Thanks for any advice you can provide, >>> Jason Antman >>> >>> Tom Bergfeld wrote: >>> >>> >>>> Hi Jason, >>>> >>>> unfortunately we were not able to reproduce your issue in Fedora 13. >>>> We will create an environment with centos and try it again. >>>> >>>> Tom >>>> >>>> >>>> -----Urspr?ngliche Nachricht----- >>>> Von: rsyslog-bounces at lists.adiscon.com >>>> [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason >>>> >>>> >>> Antman >>> >>> >>>> Gesendet: Donnerstag, 4. November 2010 22:47 >>>> An: rsyslog-users >>>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog >>>> evaluationquestions) >>>> >>>> Thanks to both of you guys for your answers. FYI, our days have 24 >>>> >>>> >>> hours >>> >>> >>>> in them :) >>>> >>>> At this point I just got a config file worked up. I got the regexes >>>> working to do all of the matching I need into separate fields (the >>>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS >>>> >>>> >>> 5.5, >>> >>> >>>> in-house RPM build) in order to of bind imudp to a ruleset (so we can >>>> have one ruleset for localhost/unix socket, and one ruleset for all >>>> remote hosts). >>>> >>>> Unfortunately, I'm getting a segfault almost immediately on startup >>>> >>>> >>> due >>> >>> >>>> to the following lines: >>>> $template >>>> >>>> >>>> >>> RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa >>> cility >>> >>> >>>> -text%.log" >>>> *.* ?RemoteHost >>>> >>>> I believe I've narrowed the problem down to any templated (dynamic) >>>> filename... even if I dump everything external directly to >>>> /var/log/remote and add the following in my local ruleset: >>>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" >>>> *.* ?LocalHost >>>> >>>> I still get a segfault. The last line in the output running with -dn >>>> >>>> >>> is >>> >>> >>>> always: >>>> "file to log to: RemoteHost" (or LocalHost, when I used that config). >>>> >>>> Has anyone seen anything like this? I have rsyslog 2.0.6 running at >>>> another site and dynamic filenames work fine... sort of troubling >>>> >>>> >>> that >>> >>> >>>> 5.6.0 is having issues with them... >>>> >>>> Thanks, >>>> Jason Antman >>>> >>>> Aaron Wiebe wrote: >>>> >>>> >>>> >>>>>> 3) Assuming an even distribution over time (not quite accurate), >>>>>> >>>>>> >>> any >>> >>> >>>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM >>>>>> >>>>>> >>> (Xen) >>> >>> >>>>>> with a single 2.8GHz CPU and 512MB RAM would go? >>>>>> >>>>>> >>>>>> >>>>>> >>>>> 2 million lines a day is 70 lines per second assuming an 8 hour day. >>>>> MySQL, properly configured, will eat that. Rsyslog won't even >>>>> >>>>> >>> notice >>> >>> >>>>> that log level. >>>>> >>>>> Your table sizes may be problematic over time, and your ability to >>>>> query may impact things. Regardless, your concern should be mysql, >>>>> not rsyslog. >>>>> >>>>> -Aaron >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>>> >>>>> >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Fri Nov 5 18:15:09 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 5 Nov 2010 18:15:09 +0100 Subject: [rsyslog] segfault with dynamic filenames References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4302E.4070803@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com><4CD43762.20603@jasonantman.com> <4CD43B25.7080201@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD67D@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jason Antman > Sent: Friday, November 05, 2010 6:13 PM > To: rsyslog-users > Subject: Re: [rsyslog] segfault with dynamic filenames > > Ok... > > in output from configure script: > checking whether the compiler provides atomic builtins... yes > checking whether the compiler provides atomic builtins for 64 bit data > types... yes > > in omfile.c, lines 89-102: > #if HAVE_ATOMIC_BUILTINS_64BIT > static uint64 clockFileAccess = 0; > #else > static unsigned clockFileAccess = 0; > #endif > /* and the "tick" function */ > #ifndef HAVE_ATOMIC_BUILTINS > static pthread_mutex_t mutClock; > #endif > static inline uint64 > getClockFileAccess(void) > { > return ATOMIC_INC_AND_FETCH(&clockFileAccess, &mutClock); > } > > Once again, my C skills are pretty poor... but if HAVE_ATOMIC_BUILTINS > is defined.... what is mutClock?? A mutex name for a mutex that is not even created in this case ;) > > Output, strace log, and valgrind output are being sent to rainer. I'll probably not manage to have more than a peek at it today, but will look ASAP (it's getting a bit late over here ;)). Rainer > > Jason Antman wrote: > > I haven't run it with valgrind yet... honestly I don't think I've > used > > valgrind more than once or twice, I'd need to do a bit of reading up > to > > remember how to use it. > > > > This isn't a matter of *if* the issue reappears. Every time rsyslogd > > tries to write out a message matching a dynafile output, it dies > (within > > a few ms of starting, or whenever it actually tries to write the > first > > line out to disk). > > > > At the moment, I have the issue localized to the call to > > getClockFileAccess() on line 517 of tools/omfile.c, and am continuing > to > > trace it through the code. I'll see if one of the more c-skilled guys > > can give me a hand with valgrind. > > > > -Jason > > > > Rainer Gerhards wrote: > > > >> An you run it under valgrind control? If the issue reappears, it > should give > >> us *very* good hints of where to look at... > >> > >> RAiner > >> > >> > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of Jason Antman > >>> Sent: Friday, November 05, 2010 5:26 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] segfault with dynamic filenames > >>> > >>> Thanks for the info! > >>> > >>> Just sent this to Tom, forgot to CC the list.... > >>> > >>> I just rebuilt the RPM with all debugging features enabled and am > >>> experiencing the same issue. > >>> > >>> I have: > >>> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env > variable > >>> set to "LogFuncFlow FileTrace=omfile.c Debug") > >>> - strace output > >>> - RPM spec file including configure options, etc. > >>> - the Fedora patch that is being applied > >>> - the actual RPMs, if they're of interest, and a source rpm > >>> > >>> Which of this is of use? Should I email it? Open a ticket and > attach? > >>> > >>> Here are the last few lines before the crash: > >>> 2954.318300688:b7f06b90: Processing next action > >>> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to > builtin- > >>> file > >>> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin- > file > >>> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter > >>> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 > >>> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: > itx > >>> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx > >>> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter > >>> 2954.318418085:b7f06b90: file to log to: LocalHost > >>> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter > >>> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter > >>> (and then it dies, in the middle of a select() call) > >>> > >>> I'm going to start looking through omfile.c and perhaps adding some > of > >>> my own instrumentation just so I can localize it a bit more. > >>> > >>> Thanks for any advice you can provide, > >>> Jason Antman > >>> > >>> Tom Bergfeld wrote: > >>> > >>> > >>>> Hi Jason, > >>>> > >>>> unfortunately we were not able to reproduce your issue in Fedora > 13. > >>>> We will create an environment with centos and try it again. > >>>> > >>>> Tom > >>>> > >>>> > >>>> -----Urspr?ngliche Nachricht----- > >>>> Von: rsyslog-bounces at lists.adiscon.com > >>>> [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason > >>>> > >>>> > >>> Antman > >>> > >>> > >>>> Gesendet: Donnerstag, 4. November 2010 22:47 > >>>> An: rsyslog-users > >>>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > >>>> evaluationquestions) > >>>> > >>>> Thanks to both of you guys for your answers. FYI, our days have 24 > >>>> > >>>> > >>> hours > >>> > >>> > >>>> in them :) > >>>> > >>>> At this point I just got a config file worked up. I got the > regexes > >>>> working to do all of the matching I need into separate fields (the > >>>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS > >>>> > >>>> > >>> 5.5, > >>> > >>> > >>>> in-house RPM build) in order to of bind imudp to a ruleset (so we > can > >>>> have one ruleset for localhost/unix socket, and one ruleset for > all > >>>> remote hosts). > >>>> > >>>> Unfortunately, I'm getting a segfault almost immediately on > startup > >>>> > >>>> > >>> due > >>> > >>> > >>>> to the following lines: > >>>> $template > >>>> > >>>> > >>>> > >>> > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa > >>> cility > >>> > >>> > >>>> -text%.log" > >>>> *.* ?RemoteHost > >>>> > >>>> I believe I've narrowed the problem down to any templated > (dynamic) > >>>> filename... even if I dump everything external directly to > >>>> /var/log/remote and add the following in my local ruleset: > >>>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > >>>> *.* ?LocalHost > >>>> > >>>> I still get a segfault. The last line in the output running with - > dn > >>>> > >>>> > >>> is > >>> > >>> > >>>> always: > >>>> "file to log to: RemoteHost" (or LocalHost, when I used that > config). > >>>> > >>>> Has anyone seen anything like this? I have rsyslog 2.0.6 running > at > >>>> another site and dynamic filenames work fine... sort of troubling > >>>> > >>>> > >>> that > >>> > >>> > >>>> 5.6.0 is having issues with them... > >>>> > >>>> Thanks, > >>>> Jason Antman > >>>> > >>>> Aaron Wiebe wrote: > >>>> > >>>> > >>>> > >>>>>> 3) Assuming an even distribution over time (not quite accurate), > >>>>>> > >>>>>> > >>> any > >>> > >>> > >>>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM > >>>>>> > >>>>>> > >>> (Xen) > >>> > >>> > >>>>>> with a single 2.8GHz CPU and 512MB RAM would go? > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> 2 million lines a day is 70 lines per second assuming an 8 hour > day. > >>>>> MySQL, properly configured, will eat that. Rsyslog won't even > >>>>> > >>>>> > >>> notice > >>> > >>> > >>>>> that log level. > >>>>> > >>>>> Your table sizes may be problematic over time, and your ability > to > >>>>> query may impact things. Regardless, your concern should be > mysql, > >>>>> not rsyslog. > >>>>> > >>>>> -Aaron > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jason at jasonantman.com Fri Nov 5 18:28:24 2010 From: jason at jasonantman.com (Jason Antman) Date: Fri, 05 Nov 2010 13:28:24 -0400 Subject: [rsyslog] segfault with dynamic filenames In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD67D@GRFEXC.intern.adiscon.com> References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD66E@GRFEXC.intern.adiscon.com> <4CD4302E.4070803@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD678@GRFEXC.intern.adiscon.com><4CD43762.20603@jasonantman.com> <4CD43B25.7080201@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD67D@GRFEXC.intern.adiscon.com> Message-ID: <4CD43EB8.2040504@jasonantman.com> Ok. Thank you so much! -Jason Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Jason Antman >> Sent: Friday, November 05, 2010 6:13 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] segfault with dynamic filenames >> >> Ok... >> >> in output from configure script: >> checking whether the compiler provides atomic builtins... yes >> checking whether the compiler provides atomic builtins for 64 bit data >> types... yes >> >> in omfile.c, lines 89-102: >> #if HAVE_ATOMIC_BUILTINS_64BIT >> static uint64 clockFileAccess = 0; >> #else >> static unsigned clockFileAccess = 0; >> #endif >> /* and the "tick" function */ >> #ifndef HAVE_ATOMIC_BUILTINS >> static pthread_mutex_t mutClock; >> #endif >> static inline uint64 >> getClockFileAccess(void) >> { >> return ATOMIC_INC_AND_FETCH(&clockFileAccess, &mutClock); >> } >> >> Once again, my C skills are pretty poor... but if HAVE_ATOMIC_BUILTINS >> is defined.... what is mutClock?? >> > > A mutex name for a mutex that is not even created in this case ;) > > >> Output, strace log, and valgrind output are being sent to rainer. >> > > I'll probably not manage to have more than a peek at it today, but will look > ASAP (it's getting a bit late over here ;)). > > Rainer > >> Jason Antman wrote: >> >>> I haven't run it with valgrind yet... honestly I don't think I've >>> >> used >> >>> valgrind more than once or twice, I'd need to do a bit of reading up >>> >> to >> >>> remember how to use it. >>> >>> This isn't a matter of *if* the issue reappears. Every time rsyslogd >>> tries to write out a message matching a dynafile output, it dies >>> >> (within >> >>> a few ms of starting, or whenever it actually tries to write the >>> >> first >> >>> line out to disk). >>> >>> At the moment, I have the issue localized to the call to >>> getClockFileAccess() on line 517 of tools/omfile.c, and am continuing >>> >> to >> >>> trace it through the code. I'll see if one of the more c-skilled guys >>> can give me a hand with valgrind. >>> >>> -Jason >>> >>> Rainer Gerhards wrote: >>> >>> >>>> An you run it under valgrind control? If the issue reappears, it >>>> >> should give >> >>>> us *very* good hints of where to look at... >>>> >>>> RAiner >>>> >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Jason Antman >>>>> Sent: Friday, November 05, 2010 5:26 PM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] segfault with dynamic filenames >>>>> >>>>> Thanks for the info! >>>>> >>>>> Just sent this to Tom, forgot to CC the list.... >>>>> >>>>> I just rebuilt the RPM with all debugging features enabled and am >>>>> experiencing the same issue. >>>>> >>>>> I have: >>>>> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env >>>>> >> variable >> >>>>> set to "LogFuncFlow FileTrace=omfile.c Debug") >>>>> - strace output >>>>> - RPM spec file including configure options, etc. >>>>> - the Fedora patch that is being applied >>>>> - the actual RPMs, if they're of interest, and a source rpm >>>>> >>>>> Which of this is of use? Should I email it? Open a ticket and >>>>> >> attach? >> >>>>> Here are the last few lines before the crash: >>>>> 2954.318300688:b7f06b90: Processing next action >>>>> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to >>>>> >> builtin- >> >>>>> file >>>>> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin- >>>>> >> file >> >>>>> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter >>>>> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 >>>>> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: >>>>> >> itx >> >>>>> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx >>>>> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter >>>>> 2954.318418085:b7f06b90: file to log to: LocalHost >>>>> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter >>>>> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter >>>>> (and then it dies, in the middle of a select() call) >>>>> >>>>> I'm going to start looking through omfile.c and perhaps adding some >>>>> >> of >> >>>>> my own instrumentation just so I can localize it a bit more. >>>>> >>>>> Thanks for any advice you can provide, >>>>> Jason Antman >>>>> >>>>> Tom Bergfeld wrote: >>>>> >>>>> >>>>> >>>>>> Hi Jason, >>>>>> >>>>>> unfortunately we were not able to reproduce your issue in Fedora >>>>>> >> 13. >> >>>>>> We will create an environment with centos and try it again. >>>>>> >>>>>> Tom >>>>>> >>>>>> >>>>>> -----Urspr?ngliche Nachricht----- >>>>>> Von: rsyslog-bounces at lists.adiscon.com >>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason >>>>>> >>>>>> >>>>>> >>>>> Antman >>>>> >>>>> >>>>> >>>>>> Gesendet: Donnerstag, 4. November 2010 22:47 >>>>>> An: rsyslog-users >>>>>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog >>>>>> evaluationquestions) >>>>>> >>>>>> Thanks to both of you guys for your answers. FYI, our days have 24 >>>>>> >>>>>> >>>>>> >>>>> hours >>>>> >>>>> >>>>> >>>>>> in them :) >>>>>> >>>>>> At this point I just got a config file worked up. I got the >>>>>> >> regexes >> >>>>>> working to do all of the matching I need into separate fields (the >>>>>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS >>>>>> >>>>>> >>>>>> >>>>> 5.5, >>>>> >>>>> >>>>> >>>>>> in-house RPM build) in order to of bind imudp to a ruleset (so we >>>>>> >> can >> >>>>>> have one ruleset for localhost/unix socket, and one ruleset for >>>>>> >> all >> >>>>>> remote hosts). >>>>>> >>>>>> Unfortunately, I'm getting a segfault almost immediately on >>>>>> >> startup >> >>>>>> >>>>> due >>>>> >>>>> >>>>> >>>>>> to the following lines: >>>>>> $template >>>>>> >>>>>> >>>>>> >>>>>> >> RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa >> >>>>> cility >>>>> >>>>> >>>>> >>>>>> -text%.log" >>>>>> *.* ?RemoteHost >>>>>> >>>>>> I believe I've narrowed the problem down to any templated >>>>>> >> (dynamic) >> >>>>>> filename... even if I dump everything external directly to >>>>>> /var/log/remote and add the following in my local ruleset: >>>>>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" >>>>>> *.* ?LocalHost >>>>>> >>>>>> I still get a segfault. The last line in the output running with - >>>>>> >> dn >> >>>>>> >>>>> is >>>>> >>>>> >>>>> >>>>>> always: >>>>>> "file to log to: RemoteHost" (or LocalHost, when I used that >>>>>> >> config). >> >>>>>> Has anyone seen anything like this? I have rsyslog 2.0.6 running >>>>>> >> at >> >>>>>> another site and dynamic filenames work fine... sort of troubling >>>>>> >>>>>> >>>>>> >>>>> that >>>>> >>>>> >>>>> >>>>>> 5.6.0 is having issues with them... >>>>>> >>>>>> Thanks, >>>>>> Jason Antman >>>>>> >>>>>> Aaron Wiebe wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>> 3) Assuming an even distribution over time (not quite accurate), >>>>>>>> >>>>>>>> >>>>>>>> >>>>> any >>>>> >>>>> >>>>> >>>>>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM >>>>>>>> >>>>>>>> >>>>>>>> >>>>> (Xen) >>>>> >>>>> >>>>> >>>>>>>> with a single 2.8GHz CPU and 512MB RAM would go? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> 2 million lines a day is 70 lines per second assuming an 8 hour >>>>>>> >> day. >> >>>>>>> MySQL, properly configured, will eat that. Rsyslog won't even >>>>>>> >>>>>>> >>>>>>> >>>>> notice >>>>> >>>>> >>>>> >>>>>>> that log level. >>>>>>> >>>>>>> Your table sizes may be problematic over time, and your ability >>>>>>> >> to >> >>>>>>> query may impact things. Regardless, your concern should be >>>>>>> >> mysql, >> >>>>>>> not rsyslog. >>>>>>> >>>>>>> -Aaron >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From Brianr at mylife.com Sat Nov 6 00:43:54 2010 From: Brianr at mylife.com (Brian Rogoff) Date: Fri, 5 Nov 2010 16:43:54 -0700 Subject: [rsyslog] rsyslog conf file questions References: <615F3D61E5B3E74AB3C81CF3904B922503FD56B9@maximus.REUNION.COM> <615F3D61E5B3E74AB3C81CF3904B922503FD56BA@maximus.REUNION.COM> Message-ID: <615F3D61E5B3E74AB3C81CF3904B922503FD56C2@maximus.REUNION.COM> Hi, On my logging server named rack1a I have at the top of my rsyslog.conf these lines $template DynaFile,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" if $hostname <> 'rack1a' then -?DynaFile & ~ the intent being that I'd like to collect messages from some services (%PROGRAMNAME%) from other machines and store them in their own files. I'd like the conf files to not have a dependency on their name. However, when I write lines like if $hostname <> $myhostname then -?DynaFile or if $hostname <> $fromhostname then -?DynaFile or even if $hostname <> getenv('HOSTNAME') then -?DynaFile rsyslogd creates /var/log/rack1a/%PROGRAMNAME%.log files in addition to the logs for the external programs. What am I doing wrong here? I'm using rsyslog 5.6.0v5(stable). -- Brian -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 2906 bytes Desc: not available URL: From jsabo at criminal.org Sat Nov 6 16:41:17 2010 From: jsabo at criminal.org (Jonathan Sabo) Date: Sat, 6 Nov 2010 11:41:17 -0400 Subject: [rsyslog] rsyslog conf file questions In-Reply-To: <615F3D61E5B3E74AB3C81CF3904B922503FD56C2@maximus.REUNION.COM> References: <615F3D61E5B3E74AB3C81CF3904B922503FD56B9@maximus.REUNION.COM> <615F3D61E5B3E74AB3C81CF3904B922503FD56BA@maximus.REUNION.COM> <615F3D61E5B3E74AB3C81CF3904B922503FD56C2@maximus.REUNION.COM> Message-ID: I think the properties are case sensitive in certain versions of rsyslog. I think I read that that changed in newer versions. Try %programname% On Fri, Nov 5, 2010 at 7:43 PM, Brian Rogoff wrote: > Hi, > ? ?On my logging server named rack1a I have at the top of my rsyslog.conf these lines > > $template DynaFile,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" > if $hostname <> 'rack1a' then -?DynaFile > & ~ > > the intent being that I'd like to collect messages from some services (%PROGRAMNAME%) from other machines > and store them in their own files. I'd like the conf files to not have a dependency on their name. However, > when I write lines like > > if $hostname <> $myhostname then -?DynaFile > > or > > if $hostname <> $fromhostname then -?DynaFile > > or even > > if $hostname <> getenv('HOSTNAME') then -?DynaFile > > rsyslogd creates /var/log/rack1a/%PROGRAMNAME%.log files in addition to the logs for the external programs. > > What am I doing wrong here? I'm using rsyslog 5.6.0v5(stable). > > -- Brian > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From reader at newsguy.com Sun Nov 7 06:16:55 2010 From: reader at newsguy.com (Harry Putnam) Date: Sun, 07 Nov 2010 00:16:55 -0500 Subject: [rsyslog] difference from v. 3.22 and 5.4.0 Message-ID: <87bp61emrc.fsf@newsguy.com> I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 In the course of a system update today, rsyslog version 5.4.0 was installed, but it crashes immediately when started. I've done no testing or diagnostics as yet, just went back to version 3.22. I wanted to find out first if there have been major changes that might help diagnose the problem. Any tricky gotchas that might be involved? From david at lang.hm Sun Nov 7 08:35:57 2010 From: david at lang.hm (david at lang.hm) Date: Sun, 7 Nov 2010 00:35:57 -0700 (PDT) Subject: [rsyslog] difference from v. 3.22 and 5.4.0 In-Reply-To: <87bp61emrc.fsf@newsguy.com> References: <87bp61emrc.fsf@newsguy.com> Message-ID: On Sun, 7 Nov 2010, Harry Putnam wrote: > I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 > > In the course of a system update today, rsyslog version 5.4.0 was > installed, but it crashes immediately when started. > > I've done no testing or diagnostics as yet, just went back to version > 3.22. I wanted to find out first if there have been major changes that > might help diagnose the problem. the changes between 3.22 and 5.4 are huge. you would have to go though the git tree to see them all, but that's such a huge amont of development that it's not really practical doing a little digging through git. v5.7.1 has 835 files, 241044 lines, 997701 words, 8971232 characters the diff between v.3.22.2 and v5.7.1 is 71367 lines long, and the diffstat says 625 files changed, 45917 insertions(+), 9128 deletions(-) > Any tricky gotchas that might be involved? oh yes. the locking has changed (a couple times), threading has improved dramatically, parsing has been completely re-done, the ability to process multiple messages at once has been added, the ability to bind separate rulesets to inputs (and have the results ofone filter feed onto a ruleset, in fact the entire concept of rulesets..), it's gained the ability to load multiple parsers (either in a stack or tied to specific inputs). in the process it has gained a lot of capibilities and speed (improving by at least two orders of magnatude) while keeping the logs at least as safe as they were before. as for where to start troubleshooting, the best thing to do with something like this where it crashes on startup is to start it in debug mode and read through the output. odds are you have some sort of fatal typo in the config file and reading through the debug output as it parses the config file will identify the error. David Lang P.S. Rainer, you really do need to do some sort of feture timeline to show the drastic nature of what's changed. even if you just took a couple of hours to do a git log and then wed out the merges and trivial commits, the highlights that remain would be very impressinve and give people a good idea how much more advanced 5.x is from 3.x From marc.schiffbauer at mightycare.de Mon Nov 8 11:56:02 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Mon, 8 Nov 2010 11:56:02 +0100 (CET) Subject: [rsyslog] One Queue for multiple Actions in 4.6.4? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD669@GRFEXC.intern.adiscon.com> Message-ID: <848193718.1625.1289213762750.JavaMail.root@zimbra> ----- Urspr?ngliche Mail ----- > oh, one thing I forgot: you mentioned disk queues in some mail. You > can not > use disk-assisted queues if you totally insist on ordering, because > there is > some inherent re-ordering possible when starting up disk mode. So you > need to > use pure disk queues or pure memory queues. > Hi Rainer, I think if I set up matermarks and queue sizes in a way that disk-assisted queues will only use the disk files very seldom so that some single message re-order events when it starts using disk files could be tolerated. I could use some catch-all mechanism to find such occurences. Or will re-ordering take place all the time while it is using disk files? But I have another question about $Rulsets: When I have a ruleset with its own mainqueue and several actions within the ruleset: Where do I put the settings for that mainqueue? $RuleSet dbAction $RulesetCreateMainQueue on # <--- is this the place to put ruleset mainqueue params ($ActionQueue...)? # rulset action 1 if $syslogtag startswith # rulset action 2 if $syslogtag startswith #switch back to default ruleset $ruleset RSYSLOG_DefaultRuleset $ActionOmrulesetRulesetName dbAction # <------------------------------------------- or here? mail.* :omruleset: > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > > Sent: Thursday, November 04, 2010 11:52 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > > > > > > > > > About up to 3000 Messages/s > > > > To be more precise: That will be the expected peak. So the average > > will > > be > > much lower. > > > > And we have decent hardware to do this. > > > > > > > > at that data rate you will have a very hard time doing things the > > > way > > > you > > > are trying. > > > > > > at the database level, doing 3000 inserts/sec requires _very_ > > > high- > > end > > > hardware. > > > rsyslog supports inserting multiple messages in one command, > > > and > > > with that you could probably handle 3000 messages/sec on very > > > low-end > > > hardware (because rsyslog could insert messages in batches of 100 > > > or > > > more > > > if needed). unfortunantly, when you enable this, you don't > > > maintain > > > the > > > message order. > > > > > > at the rsyslog level, doing disk based queues with everything > > > tuned > > to > > > come as close to minimizing the chance of data loss is very hard > > > to > > do > > > at > > > that traffic level. I ran tests last year on a 8 core box with 64G > > > of > > > ram > > > and a fusion-io SSD pci card, and depending on the filesystem I > > > used, > > > I > > > was able to get from 2K to 8K messages/sec where I wasn't trying > > > to > > do > > > anything more than write the message to a log file. > > > > > > > Thanks for the numbers. > > > > > you are going to have to think very hard about how critical it > > > really > > > is > > > for you to maintain the same message order and what type of > > > reliability > > > you really need for your messages. > > > > > > it's counter-intuitive, but it may be that you end up with better > > > overall > > > reliability with a much faster configuration that has worse > > > 'worst-case' > > > data loss, but is fast enough that under normal conditions > > > everything > > > is > > > processed really fast than you would under a configuration that is > > > much > > > slower all the time, but has a better worst-case data loss. > > > > Yes, I will keep that in mind, thanks. > > > > > > > > > > David Lang From mmonitz at gmail.com Mon Nov 8 14:44:44 2010 From: mmonitz at gmail.com (matan monitz) Date: Mon, 8 Nov 2010 15:44:44 +0200 Subject: [rsyslog] rsyslog Text File Input Module losing seek? Message-ID: Hello all we have a bind server writing and rotating logs on file system (rotating when log file reaches 10Mb). we also have rsyslog configured with the text file input module reading the file and sending it over UDP to another rsyslog server. several times during the day the rsyslog on the bind machine starts reading the file from the top, causing a sudden high rate of duplicate and old events. in an attempt to identify the problem: 1.verified this using tcpdump to make sure the and saw the old logs coming down the wire to the rsyslog server 2.made sure state files are correctly configured and are being created when needed 3.made sure the rsyslog is not crashing and restarting by monitoring the pid googeling hasn't helped as well any ideas? From rgerhards at hq.adiscon.com Mon Nov 8 14:46:16 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Nov 2010 14:46:16 +0100 Subject: [rsyslog] rsyslog Text File Input Module losing seek? References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> which version do you use? I have recently addressed such a problem... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of matan monitz > Sent: Monday, November 08, 2010 2:45 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog Text File Input Module losing seek? > > Hello all > we have a bind server writing and rotating logs on file system > (rotating > when log file reaches 10Mb). > we also have rsyslog configured with the text file input module reading > the > file and sending it over UDP to another rsyslog server. > several times during the day the rsyslog on the bind machine starts > reading > the file from the top, causing a sudden high rate of duplicate and old > events. > in an attempt to identify the problem: > 1.verified this using tcpdump to make sure the and saw the old logs > coming > down the wire to the rsyslog server > 2.made sure state files are correctly configured and are being created > when > needed > 3.made sure the rsyslog is not crashing and restarting by monitoring > the pid > googeling hasn't helped as well > > any ideas? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From john at feurix.com Mon Nov 8 15:16:26 2010 From: john at feurix.com (John Feuerstein) Date: Mon, 08 Nov 2010 15:16:26 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) Message-ID: <4CD8063A.40004@feurix.com> Hello, running rsyslog version 5.6.0 for some days always results in one thread permanently running at 100% CPU, while there is no functional downside, everything in regards to logging works as expected. This rsyslogd instance is used as a RELP server for around 30 client machines also running rsyslog version 5.6.0. Note that this also happens with 5.4.0 and the same configuration. Build-time configure options: ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64 --enable-largefile --enable-regexp --enable-zlib --enable-pthreads --enable-klog --enable-unix --enable-inet --enable-fsstnd --enable-rsyslogrt --enable-rsyslogd --disable-imdiag --disable-omruleset --enable-relp --disable-debug --disable-rtinst --disable-valgrind --disable-diagtools Server Config: > $ModLoad imrelp > $InputRELPServerRun 600 > > $MainMsgQueueSize 100000 > $DynaFileCacheSize 500 ... and a lot of splitting with dynamic filenames like this: > $template SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" > > if $hostname == 'node6' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > if $hostname == 'node7' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > if $hostname == 'build' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ The server always has a lot of open files because of the dynamic filenames: $ lsof -p | grep /srv/log | wc -l 3217 Client Config: > $ModLoad omrelp > $WorkDirectory /var/spool/rsyslog > > $ActionQueueType LinkedList > $ActionQueueFileName queue > $ActionQueueSaveOnShutdown on > $ActionQueueSize 51200 > $ActionQueueMaxDiskSpace 2G > $ActionResumeRetryCount -1 > *.* :omrelp:log.example.com:600 Have a look at the TIME column of PID 32270: > PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command > 32267 syslog 20 0 184M 8044 504 S 0.0 0.1 0:00.00 `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf > 32270 syslog 20 0 184M 8044 504 R 99.0 0.1 33h05:00 | `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf > 32269 syslog 20 0 184M 8044 504 S 0.0 0.1 0:00.02 | `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf > 32268 syslog 20 0 184M 8044 504 S 0.0 0.1 12:47.15 | `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf Attaching with strace shows an endless select()-accept()-recvfrom() loop like this: > ... > select(2945, [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926 2931 2944], [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2158 2244 2245 2246 2249 2251 2252 2254 2258 2259 2261 2262 2373 2374 2375 2382 2383 2387 2388 2389 2390 2391 2393 2395 2396 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2414 2508 2510 2511 2514 2515 2516 2518 2520 2521 2523 2524 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2542 2560 2561 2625 2627 2628 2629 2630 2631 2632 2639 2640 2641 2646 2708], NULL, NULL) = 272 (in [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926], out [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143]) > accept(4, 0x5fff93ffebc0, [128]) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(3, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(6, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(8, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(10, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(12, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(17, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(22, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(25, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(28, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(29, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(59, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(62, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(68, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(71, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(76, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(81, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(86, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(91, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(96, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(100, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(111, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(114, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(117, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(120, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(123, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(126, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(131, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(136, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(139, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(142, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(145, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(148, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(151, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(156, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > select(2945, [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926 2931 2944], [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2158 2244 2245 2246 2249 2251 2252 2254 2258 2259 2261 2262 2373 2374 2375 2382 2383 2387 2388 2389 2390 2391 2393 2395 2396 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2414 2508 2510 2511 2514 2515 2516 2518 2520 2521 2523 2524 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2542 2560 2561 2625 2627 2628 2629 2630 2631 2632 2639 2640 2641 2646 2708], NULL, NULL) = 272 (in [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926], out [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143]) > accept(4, 0x5fff93ffebc0, [128]) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(3, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(6, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(8, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(10, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(12, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(17, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(22, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(25, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(28, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(29, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(59, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(62, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(68, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(71, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(76, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(81, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(86, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(91, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(96, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(100, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(111, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(114, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(117, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(120, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(123, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(126, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(131, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(136, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(139, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(142, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(145, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(148, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(151, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(156, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > select(2945, [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926 2931 2944], [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2158 2244 2245 2246 2249 2251 2252 2254 2258 2259 2261 2262 2373 2374 2375 2382 2383 2387 2388 2389 2390 2391 2393 2395 2396 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2414 2508 2510 2511 2514 2515 2516 2518 2520 2521 2523 2524 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2542 2560 2561 2625 2627 2628 2629 2630 2631 2632 2639 2640 2641 2646 2708], NULL, NULL) = 272 (in [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926], out [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143]) > accept(4, 0x5fff93ffebc0, [128]) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(3, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(6, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(8, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(10, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(12, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(17, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(22, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(25, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(28, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(29, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(59, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(62, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(68, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(71, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(76, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(81, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(86, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(91, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(96, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(100, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(111, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > recvfrom(114, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) > ... Can this be related to not using --enable-unlimited-select with RELP? Any other ideas? Thanks, John From mmonitz at gmail.com Mon Nov 8 15:50:43 2010 From: mmonitz at gmail.com (matan monitz) Date: Mon, 8 Nov 2010 16:50:43 +0200 Subject: [rsyslog] rsyslog Text File Input Module losing seek? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> Message-ID: hello rainer we are using 3.22.1-3 from rpm on rhel 5 On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards wrote: > which version do you use? I have recently addressed such a problem... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > Sent: Monday, November 08, 2010 2:45 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] rsyslog Text File Input Module losing seek? > > > > Hello all > > we have a bind server writing and rotating logs on file system > > (rotating > > when log file reaches 10Mb). > > we also have rsyslog configured with the text file input module reading > > the > > file and sending it over UDP to another rsyslog server. > > several times during the day the rsyslog on the bind machine starts > > reading > > the file from the top, causing a sudden high rate of duplicate and old > > events. > > in an attempt to identify the problem: > > 1.verified this using tcpdump to make sure the and saw the old logs > > coming > > down the wire to the rsyslog server > > 2.made sure state files are correctly configured and are being created > > when > > needed > > 3.made sure the rsyslog is not crashing and restarting by monitoring > > the pid > > googeling hasn't helped as well > > > > any ideas? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Nov 8 17:39:21 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Nov 2010 17:39:21 +0100 Subject: [rsyslog] rsyslog Text File Input Module losing seek? References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of matan monitz > Sent: Monday, November 08, 2010 3:51 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > hello rainer > we are using 3.22.1-3 from rpm on rhel 5 That's definitely too old. I maintain it when I am at it, but the fix went into v4+ so far. I suggest that you pull the latest v4-stable from git. If git doesn't work for you, let me know and I'll mail you a private tarball. I am just right now very busy (probably tomorrow as well, as I am writing a lot of new code in support for CEE). Rainer > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards > wrote: > > > which version do you use? I have recently addressed such a problem... > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > Sent: Monday, November 08, 2010 2:45 PM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > Hello all > > > we have a bind server writing and rotating logs on file system > > > (rotating > > > when log file reaches 10Mb). > > > we also have rsyslog configured with the text file input module > reading > > > the > > > file and sending it over UDP to another rsyslog server. > > > several times during the day the rsyslog on the bind machine starts > > > reading > > > the file from the top, causing a sudden high rate of duplicate and > old > > > events. > > > in an attempt to identify the problem: > > > 1.verified this using tcpdump to make sure the and saw the old logs > > > coming > > > down the wire to the rsyslog server > > > 2.made sure state files are correctly configured and are being > created > > > when > > > needed > > > 3.made sure the rsyslog is not crashing and restarting by > monitoring > > > the pid > > > googeling hasn't helped as well > > > > > > any ideas? > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Mon Nov 8 17:43:34 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 8 Nov 2010 17:43:34 +0100 Subject: [rsyslog] difference from v. 3.22 and 5.4.0 References: <87bp61emrc.fsf@newsguy.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD69B@GRFEXC.intern.adiscon.com> David, Rainer adressed me with the task to compile such a list. A first draft can be found here: http://www.rsyslog.com/features/ I will expand it a bit more and look up the version information that is yet missing, which is still a lot. Florian -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm Sent: Sonntag, 7. November 2010 08:36 To: rsyslog-users Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 On Sun, 7 Nov 2010, Harry Putnam wrote: > I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 > > In the course of a system update today, rsyslog version 5.4.0 was > installed, but it crashes immediately when started. > > I've done no testing or diagnostics as yet, just went back to version > 3.22. I wanted to find out first if there have been major changes that > might help diagnose the problem. the changes between 3.22 and 5.4 are huge. you would have to go though the git tree to see them all, but that's such a huge amont of development that it's not really practical doing a little digging through git. v5.7.1 has 835 files, 241044 lines, 997701 words, 8971232 characters the diff between v.3.22.2 and v5.7.1 is 71367 lines long, and the diffstat says 625 files changed, 45917 insertions(+), 9128 deletions(-) > Any tricky gotchas that might be involved? oh yes. the locking has changed (a couple times), threading has improved dramatically, parsing has been completely re-done, the ability to process multiple messages at once has been added, the ability to bind separate rulesets to inputs (and have the results ofone filter feed onto a ruleset, in fact the entire concept of rulesets..), it's gained the ability to load multiple parsers (either in a stack or tied to specific inputs). in the process it has gained a lot of capibilities and speed (improving by at least two orders of magnatude) while keeping the logs at least as safe as they were before. as for where to start troubleshooting, the best thing to do with something like this where it crashes on startup is to start it in debug mode and read through the output. odds are you have some sort of fatal typo in the config file and reading through the debug output as it parses the config file will identify the error. David Lang P.S. Rainer, you really do need to do some sort of feture timeline to show the drastic nature of what's changed. even if you just took a couple of hours to do a git log and then wed out the merges and trivial commits, the highlights that remain would be very impressinve and give people a good idea how much more advanced 5.x is from 3.x _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From Brianr at mylife.com Mon Nov 8 18:03:52 2010 From: Brianr at mylife.com (Brian Rogoff) Date: Mon, 8 Nov 2010 09:03:52 -0800 Subject: [rsyslog] rsyslog conf file questions References: <615F3D61E5B3E74AB3C81CF3904B922503FD56B9@maximus.REUNION.COM><615F3D61E5B3E74AB3C81CF3904B922503FD56BA@maximus.REUNION.COM><615F3D61E5B3E74AB3C81CF3904B922503FD56C2@maximus.REUNION.COM> Message-ID: <615F3D61E5B3E74AB3C81CF3904B922503FD56C5@maximus.REUNION.COM> Thanks, but the %PROGRAMNAME%.log file is being created with the correct name, so it doesn't appear to be a case issue in the template line, which works fine, rather in the expression filter line, with the comparison '$hostname <> $myhostname' and variations always being true unless I replace the RHS with the literal 'rack1a'. Playing with the cases there doesn't change a thing. Any other ideas? -- Brian -----Original Message----- From: rsyslog-bounces at lists.adiscon.com on behalf of Jonathan Sabo Sent: Sat 11/6/2010 8:41 AM To: rsyslog-users Subject: Re: [rsyslog] rsyslog conf file questions I think the properties are case sensitive in certain versions of rsyslog. I think I read that that changed in newer versions. Try %programname% On Fri, Nov 5, 2010 at 7:43 PM, Brian Rogoff wrote: > Hi, > ? ?On my logging server named rack1a I have at the top of my rsyslog.conf these lines > > $template DynaFile,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" > if $hostname <> 'rack1a' then -?DynaFile > & ~ > > the intent being that I'd like to collect messages from some services (%PROGRAMNAME%) from other machines > and store them in their own files. I'd like the conf files to not have a dependency on their name. However, > when I write lines like > > if $hostname <> $myhostname then -?DynaFile > > or > > if $hostname <> $fromhostname then -?DynaFile > > or even > > if $hostname <> getenv('HOSTNAME') then -?DynaFile > > rsyslogd creates /var/log/rack1a/%PROGRAMNAME%.log files in addition to the logs for the external programs. > > What am I doing wrong here? I'm using rsyslog 5.6.0v5(stable). > > -- Brian > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 3704 bytes Desc: not available URL: From mmonitz at gmail.com Mon Nov 8 19:07:36 2010 From: mmonitz at gmail.com (matan monitz) Date: Mon, 8 Nov 2010 20:07:36 +0200 Subject: [rsyslog] rsyslog Text File Input Module losing seek? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> Message-ID: hi rainer unfortunately this is not a server i'm directly in charge of so we can't really upgrade for while (they will only upgrade as part of the planned upgrade for the whole server) can you please elaborate on the cause of the problem so that maybe i can figure out a temporary workaround? On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > Sent: Monday, November 08, 2010 3:51 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > > > hello rainer > > we are using 3.22.1-3 from rpm on rhel 5 > > That's definitely too old. I maintain it when I am at it, but the fix went > into v4+ so far. I suggest that you pull the latest v4-stable from git. If > git doesn't work for you, let me know and I'll mail you a private tarball. > I > am just right now very busy (probably tomorrow as well, as I am writing a > lot > of new code in support for CEE). > > Rainer > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards > > wrote: > > > > > which version do you use? I have recently addressed such a problem... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > > Sent: Monday, November 08, 2010 2:45 PM > > > > To: rsyslog at lists.adiscon.com > > > > Subject: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > > > Hello all > > > > we have a bind server writing and rotating logs on file system > > > > (rotating > > > > when log file reaches 10Mb). > > > > we also have rsyslog configured with the text file input module > > reading > > > > the > > > > file and sending it over UDP to another rsyslog server. > > > > several times during the day the rsyslog on the bind machine starts > > > > reading > > > > the file from the top, causing a sudden high rate of duplicate and > > old > > > > events. > > > > in an attempt to identify the problem: > > > > 1.verified this using tcpdump to make sure the and saw the old logs > > > > coming > > > > down the wire to the rsyslog server > > > > 2.made sure state files are correctly configured and are being > > created > > > > when > > > > needed > > > > 3.made sure the rsyslog is not crashing and restarting by > > monitoring > > > > the pid > > > > googeling hasn't helped as well > > > > > > > > any ideas? > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Mon Nov 8 20:47:13 2010 From: david at lang.hm (david at lang.hm) Date: Mon, 8 Nov 2010 11:47:13 -0800 (PST) Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <4CD8063A.40004@feurix.com> References: <4CD8063A.40004@feurix.com> Message-ID: can you get the lsof output so that we can see what the file descriptor that it's having problems with is supposed to be? David Lang On Mon, 8 Nov 2010, John Feuerstein wrote: > Date: Mon, 08 Nov 2010 15:16:26 +0100 > From: John Feuerstein > Reply-To: rsyslog-users > To: rsyslog-users > Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) > > Hello, > > running rsyslog version 5.6.0 for some days always results in one thread > permanently running at 100% CPU, while there is no functional downside, > everything in regards to logging works as expected. > > This rsyslogd instance is used as a RELP server for around 30 client > machines also running rsyslog version 5.6.0. > > Note that this also happens with 5.4.0 and the same configuration. > > Build-time configure options: > > ./configure --prefix=/usr --build=x86_64-pc-linux-gnu > --host=x86_64-pc-linux-gnu --mandir=/usr/share/man > --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc > --localstatedir=/var/lib --libdir=/usr/lib64 --enable-largefile > --enable-regexp --enable-zlib --enable-pthreads --enable-klog > --enable-unix --enable-inet --enable-fsstnd --enable-rsyslogrt > --enable-rsyslogd --disable-imdiag --disable-omruleset --enable-relp > --disable-debug --disable-rtinst --disable-valgrind --disable-diagtools > > > Server Config: > >> $ModLoad imrelp >> $InputRELPServerRun 600 >> >> $MainMsgQueueSize 100000 >> $DynaFileCacheSize 500 > > ... and a lot of splitting with dynamic filenames like this: > >> $template SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" >> >> if $hostname == 'node6' and $syslogfacility == 16 then -?SplitServiceRsyncLog >> & ~ >> if $hostname == 'node7' and $syslogfacility == 16 then -?SplitServiceRsyncLog >> & ~ >> if $hostname == 'build' and $syslogfacility == 16 then -?SplitServiceRsyncLog >> & ~ > > The server always has a lot of open files because of the dynamic filenames: > > $ lsof -p | grep /srv/log | wc -l > 3217 > > > Client Config: > >> $ModLoad omrelp >> $WorkDirectory /var/spool/rsyslog >> >> $ActionQueueType LinkedList >> $ActionQueueFileName queue >> $ActionQueueSaveOnShutdown on >> $ActionQueueSize 51200 >> $ActionQueueMaxDiskSpace 2G >> $ActionResumeRetryCount -1 >> *.* :omrelp:log.example.com:600 > > > > > Have a look at the TIME column of PID 32270: > >> PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command >> 32267 syslog 20 0 184M 8044 504 S 0.0 0.1 0:00.00 `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf >> 32270 syslog 20 0 184M 8044 504 R 99.0 0.1 33h05:00 | `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf >> 32269 syslog 20 0 184M 8044 504 S 0.0 0.1 0:00.02 | `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf >> 32268 syslog 20 0 184M 8044 504 S 0.0 0.1 12:47.15 | `- /usr/sbin/rsyslogd -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf > > Attaching with strace shows an endless select()-accept()-recvfrom() loop > like this: > >> ... >> select(2945, [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926 2931 2944], [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 > 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2158 2244 2245 2246 2249 2251 2252 2254 2258 2259 2261 2262 2373 2374 2375 2382 2383 2387 2388 2389 2390 2391 2393 2395 2396 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2414 2508 2510 2511 2514 2515 2516 2518 2520 2521 2523 2524 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2542 2560 2561 2625 2627 2628 2629 2630 2631 2632 2639 2640 2641 2646 2708], NULL, NULL) = 272 (in [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1! 487 > 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926], out [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143]) >> accept(4, 0x5fff93ffebc0, [128]) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(3, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(6, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(8, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(10, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(12, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(17, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(22, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(25, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(28, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(29, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(59, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(62, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(68, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(71, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(76, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(81, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(86, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(91, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(96, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(100, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(111, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(114, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(117, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(120, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(123, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(126, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(131, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(136, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(139, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(142, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(145, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(148, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(151, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(156, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> select(2945, [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926 2931 2944], [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 > 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2158 2244 2245 2246 2249 2251 2252 2254 2258 2259 2261 2262 2373 2374 2375 2382 2383 2387 2388 2389 2390 2391 2393 2395 2396 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2414 2508 2510 2511 2514 2515 2516 2518 2520 2521 2523 2524 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2542 2560 2561 2625 2627 2628 2629 2630 2631 2632 2639 2640 2641 2646 2708], NULL, NULL) = 272 (in [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1! 487 > 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926], out [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143]) >> accept(4, 0x5fff93ffebc0, [128]) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(3, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(6, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(8, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(10, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(12, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(17, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(22, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(25, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(28, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(29, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(59, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(62, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(68, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(71, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(76, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(81, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(86, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(91, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(96, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(100, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(111, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(114, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(117, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(120, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(123, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(126, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(131, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(136, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(139, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(142, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(145, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(148, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(151, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(156, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> select(2945, [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1487 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926 2931 2944], [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 > 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2158 2244 2245 2246 2249 2251 2252 2254 2258 2259 2261 2262 2373 2374 2375 2382 2383 2387 2388 2389 2390 2391 2393 2395 2396 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2414 2508 2510 2511 2514 2515 2516 2518 2520 2521 2523 2524 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2542 2560 2561 2625 2627 2628 2629 2630 2631 2632 2639 2640 2641 2646 2708], NULL, NULL) = 272 (in [3 4 6 8 10 12 17 22 25 28 29 59 62 68 71 76 81 86 91 96 100 111 114 117 120 123 126 131 136 139 142 145 148 151 156 1093 1094 1096 1097 1099 1100 1105 1106 1114 1115 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1134 1220 1221 1222 1225 1227 1228 1230 1234 1235 1237 1238 1349 1350 1351 1358 1359 1363 1364 1365 1366 1367 1369 1371 1372 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1390 1484 1486 1! 487 > 1490 1491 1492 1494 1496 1497 1499 1500 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1518 1536 1537 1601 1603 1604 1605 1606 1607 1608 1615 1616 1617 1622 1684 2882 2885 2886 2887 2891 2892 2899 2900 2901 2902 2903 2905 2907 2908 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2926], out [1027 1028 1030 1032 1034 1036 1041 1046 1049 1052 1053 1083 1086 1092 1095 1100 1105 1110 1115 1120 1124 1135 1138 1141 1144 1147 1150 1155 1160 1163 1166 1169 1172 1175 1180 2117 2118 2120 2121 2123 2124 2129 2130 2138 2139 2143]) >> accept(4, 0x5fff93ffebc0, [128]) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(3, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(6, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(8, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(10, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(12, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(17, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(22, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(25, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(28, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(29, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(59, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(62, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(68, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(71, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(76, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(81, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(86, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(91, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(96, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(100, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(111, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> recvfrom(114, 0x5fff93ff6c80, 32768, 64, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) >> ... > > Can this be related to not using --enable-unlimited-select with RELP? > > Any other ideas? > > Thanks, > John > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From reader at newsguy.com Tue Nov 9 00:22:55 2010 From: reader at newsguy.com (Harry Putnam) Date: Mon, 08 Nov 2010 17:22:55 -0600 Subject: [rsyslog] difference from v. 3.22 and 5.4.0 References: <87bp61emrc.fsf@newsguy.com> Message-ID: <87r5evienk.fsf@newsguy.com> david at lang.hm writes: > the changes between 3.22 and 5.4 are huge. you would have to go though > the git tree to see them all, but that's such a huge amont of > development that it's not really practical Thanks... good info there. About starting in debug mode as suggested, is it possible to do that on a machine running a working version or will it really jack up existing logging? I guess what I'm asking is how to debug in a sandbox. Some way to run the newest version without bothering the current system logger. From john at feurix.com Tue Nov 9 00:31:34 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 00:31:34 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: References: <4CD8063A.40004@feurix.com> Message-ID: <4CD88856.6000705@feurix.com> Hello, On 11/08/2010 08:47 PM, david at lang.hm wrote: > can you get the lsof output so that we can see what the file descriptor > that it's having problems with is supposed to be? I've attached strace for about 10 seconds using: $ strace -ff -v -p 32267 -o rsyslogd ... and while strace was running, I've captured lsof output using: lsof -n -p 32267 > rsyslogd.lsof You can find all files and a tarball of them here: http://biz.baze.de/debug/rsyslog/ Hope this helps. FYI, I've also run this for about 10 seconds on the problematic thread: $ strace -p 32270 -c Process 32270 attached - interrupt to quit Process 32270 detached % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 60.95 0.003239 0 30405 select 36.24 0.001926 0 1033772 1033709 recvfrom 2.80 0.000149 0 30405 30405 accept 0.00 0.000000 0 64 sendto 0.00 0.000000 0 124 futex ------ ----------- ----------- --------- --------- ---------------- 100.00 0.005314 1094770 1064114 total Best, John From david at lang.hm Tue Nov 9 01:16:45 2010 From: david at lang.hm (david at lang.hm) Date: Mon, 8 Nov 2010 16:16:45 -0800 (PST) Subject: [rsyslog] difference from v. 3.22 and 5.4.0 In-Reply-To: <87r5evienk.fsf@newsguy.com> References: <87bp61emrc.fsf@newsguy.com> <87r5evienk.fsf@newsguy.com> Message-ID: On Mon, 8 Nov 2010, Harry Putnam wrote: > david at lang.hm writes: > >> the changes between 3.22 and 5.4 are huge. you would have to go though >> the git tree to see them all, but that's such a huge amont of >> development that it's not really practical > > Thanks... good info there. > > About starting in debug mode as suggested, is it possible to do that > on a machine running a working version or will it really jack up > existing logging? > > I guess what I'm asking is how to debug in a sandbox. Some way to run > the newest version without bothering the current system logger. if you add the -d option to the command line at startup time, rsyslog will spew out a HUGE amount of data to stdout, it won't affect anything else on the system. you may not even need to stop the running copy (if there is a conflict with the running copy, it will show up in the output, at which time you can re-run the test after stopping the running copy) In your case, I think the problem sounds like a config file error, so jus tthe startup debug messages are probably enough. David Lang From david at lang.hm Tue Nov 9 09:17:21 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 9 Nov 2010 00:17:21 -0800 (PST) Subject: [rsyslog] difference from v. 3.22 and 5.4.0 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD69B@GRFEXC.intern.adiscon.com> References: <87bp61emrc.fsf@newsguy.com> <9B6E2A8877C38245BFB15CC491A11DA71DD69B@GRFEXC.intern.adiscon.com> Message-ID: thanks for doing this,what you have is a good start. wnother tidbit from git, between 3.22.0 and 5.7.1 there were 1388 commits, with 11539 lines of commit logs (~6 lines per commit are boilerplate, but that's still averaging about 4 lines of comment per commit) DavidLang On Mon, 8 Nov 2010, Florian Riedl wrote: > David, > > Rainer adressed me with the task to compile such a list. > > A first draft can be found here: > http://www.rsyslog.com/features/ > > I will expand it a bit more and look up the version information that is yet > missing, which is still a lot. > > Florian > > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Sonntag, 7. November 2010 08:36 > To: rsyslog-users > Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > > On Sun, 7 Nov 2010, Harry Putnam wrote: > >> I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 >> >> In the course of a system update today, rsyslog version 5.4.0 was >> installed, but it crashes immediately when started. >> >> I've done no testing or diagnostics as yet, just went back to version >> 3.22. I wanted to find out first if there have been major changes that >> might help diagnose the problem. > > the changes between 3.22 and 5.4 are huge. you would have to go though the > git tree to see them all, but that's such a huge amont of development that > it's not really practical > > doing a little digging through git. > > v5.7.1 has 835 files, 241044 lines, 997701 words, 8971232 characters > > the diff between v.3.22.2 and v5.7.1 is 71367 lines long, and the diffstat > says 625 files changed, 45917 insertions(+), 9128 deletions(-) > >> Any tricky gotchas that might be involved? > > oh yes. the locking has changed (a couple times), threading has improved > dramatically, parsing has been completely re-done, the ability to process > multiple messages at once has been added, the ability to bind > separate rulesets to inputs (and have the results ofone filter feed onto a > ruleset, in fact the entire concept of rulesets..), it's gained the > ability to load multiple parsers (either in a stack or tied to specific > inputs). > > in the process it has gained a lot of capibilities and speed (improving by > at least two orders of magnatude) while keeping the logs at least as safe > as they were before. > > > > as for where to start troubleshooting, the best thing to do with something > like this where it crashes on startup is to start it in debug mode and > read through the output. odds are you have some sort of fatal typo in the > config file and reading through the debug output as it parses the config > file will identify the error. > > David Lang > > P.S. Rainer, you really do need to do some sort of feture timeline to show > the drastic nature of what's changed. even if you just took a couple of > hours to do a git log and then wed out the merges and trivial commits, the > highlights that remain would be very impressinve and give people a good > idea how much more advanced 5.x is from 3.x > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Tue Nov 9 09:19:55 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 09:19:55 +0100 Subject: [rsyslog] difference from v. 3.22 and 5.4.0 References: <87bp61emrc.fsf@newsguy.com><9B6E2A8877C38245BFB15CC491A11DA71DD69B@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6A6@GRFEXC.intern.adiscon.com> David, sorry for the sluggish response, but as you see right at the moment really a *lot* is going on. I really like your postings on this topic. As another measure, do you mind if I pull them together - almost literally - and create a blog post out of them (or would you like to do a guest posting?)? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, November 09, 2010 9:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > > > thanks for doing this,what you have is a good start. > > > wnother tidbit from git, between 3.22.0 and 5.7.1 there were 1388 > commits, > with 11539 lines of commit logs (~6 lines per commit are boilerplate, > but > that's still averaging about 4 lines of comment per commit) > > DavidLang > > On Mon, 8 Nov 2010, > Florian Riedl wrote: > > > David, > > > > Rainer adressed me with the task to compile such a list. > > > > A first draft can be found here: > > http://www.rsyslog.com/features/ > > > > I will expand it a bit more and look up the version information that > is yet > > missing, which is still a lot. > > > > Florian > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > > Sent: Sonntag, 7. November 2010 08:36 > > To: rsyslog-users > > Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > > > > On Sun, 7 Nov 2010, Harry Putnam wrote: > > > >> I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 > >> > >> In the course of a system update today, rsyslog version 5.4.0 was > >> installed, but it crashes immediately when started. > >> > >> I've done no testing or diagnostics as yet, just went back to > version > >> 3.22. I wanted to find out first if there have been major changes > that > >> might help diagnose the problem. > > > > the changes between 3.22 and 5.4 are huge. you would have to go > though the > > git tree to see them all, but that's such a huge amont of development > that > > it's not really practical > > > > doing a little digging through git. > > > > v5.7.1 has 835 files, 241044 lines, 997701 words, 8971232 characters > > > > the diff between v.3.22.2 and v5.7.1 is 71367 lines long, and the > diffstat > > says 625 files changed, 45917 insertions(+), 9128 deletions(-) > > > >> Any tricky gotchas that might be involved? > > > > oh yes. the locking has changed (a couple times), threading has > improved > > dramatically, parsing has been completely re-done, the ability to > process > > multiple messages at once has been added, the ability to bind > > separate rulesets to inputs (and have the results ofone filter feed > onto a > > ruleset, in fact the entire concept of rulesets..), it's gained the > > ability to load multiple parsers (either in a stack or tied to > specific > > inputs). > > > > in the process it has gained a lot of capibilities and speed > (improving by > > at least two orders of magnatude) while keeping the logs at least as > safe > > as they were before. > > > > > > > > as for where to start troubleshooting, the best thing to do with > something > > like this where it crashes on startup is to start it in debug mode > and > > read through the output. odds are you have some sort of fatal typo in > the > > config file and reading through the debug output as it parses the > config > > file will identify the error. > > > > David Lang > > > > P.S. Rainer, you really do need to do some sort of feture timeline to > show > > the drastic nature of what's changed. even if you just took a couple > of > > hours to do a git log and then wed out the merges and trivial > commits, the > > highlights that remain would be very impressinve and give people a > good > > idea how much more advanced 5.x is from 3.x > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Tue Nov 9 09:25:57 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 09:25:57 +0100 Subject: [rsyslog] liblognorm/libee Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6A7@GRFEXC.intern.adiscon.com> Hi all, especially to those interested in liblognorm: it looks like I already orphaned the project. This is a false impression ;) I'd like to utilize as many standard parts as possible, and thus I am looking very seriously into CEE and libee. Right now, I anticipate that around 80% to 90% of what liblognorm intends to do will actually be done by libee. Consequently, I am currently very actively working on libee. So if you are interested in liblognorm, you'd probably want to have a look at libee as well. Git: http://git.adiscon.com Web: http://www.libee.org As a side-note I am also implementing a new small string library ( http://doc.libestr.adiscon.com ). This new library may in the long term replace rsyslog's internal string handling, from which it is derived. I'll probably experiment with it in v6. If I am not mistaken, the new lib can provide some performance benefits. But this needs to be weighted. Rainer From david at lang.hm Tue Nov 9 09:33:28 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 9 Nov 2010 00:33:28 -0800 (PST) Subject: [rsyslog] difference from v. 3.22 and 5.4.0 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6A6@GRFEXC.intern.adiscon.com> References: <87bp61emrc.fsf@newsguy.com><9B6E2A8877C38245BFB15CC491A11DA71DD69B@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6A6@GRFEXC.intern.adiscon.com> Message-ID: go ahead and pull them togeather. David Lang On Tue, 9 Nov 2010, Rainer Gerhards wrote: > Date: Tue, 9 Nov 2010 09:19:55 +0100 > From: Rainer Gerhards > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > > David, > > sorry for the sluggish response, but as you see right at the moment really a > *lot* is going on. I really like your postings on this topic. As another > measure, do you mind if I pull them together - almost literally - and create > a blog post out of them (or would you like to do a guest posting?)? > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Tuesday, November 09, 2010 9:17 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 >> >> >> thanks for doing this,what you have is a good start. >> >> >> wnother tidbit from git, between 3.22.0 and 5.7.1 there were 1388 >> commits, >> with 11539 lines of commit logs (~6 lines per commit are boilerplate, >> but >> that's still averaging about 4 lines of comment per commit) >> >> DavidLang >> >> On Mon, 8 Nov 2010, >> Florian Riedl wrote: >> >>> David, >>> >>> Rainer adressed me with the task to compile such a list. >>> >>> A first draft can be found here: >>> http://www.rsyslog.com/features/ >>> >>> I will expand it a bit more and look up the version information that >> is yet >>> missing, which is still a lot. >>> >>> Florian >>> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com >>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Sonntag, 7. November 2010 08:36 >>> To: rsyslog-users >>> Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 >>> >>> On Sun, 7 Nov 2010, Harry Putnam wrote: >>> >>>> I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 >>>> >>>> In the course of a system update today, rsyslog version 5.4.0 was >>>> installed, but it crashes immediately when started. >>>> >>>> I've done no testing or diagnostics as yet, just went back to >> version >>>> 3.22. I wanted to find out first if there have been major changes >> that >>>> might help diagnose the problem. >>> >>> the changes between 3.22 and 5.4 are huge. you would have to go >> though the >>> git tree to see them all, but that's such a huge amont of development >> that >>> it's not really practical >>> >>> doing a little digging through git. >>> >>> v5.7.1 has 835 files, 241044 lines, 997701 words, 8971232 characters >>> >>> the diff between v.3.22.2 and v5.7.1 is 71367 lines long, and the >> diffstat >>> says 625 files changed, 45917 insertions(+), 9128 deletions(-) >>> >>>> Any tricky gotchas that might be involved? >>> >>> oh yes. the locking has changed (a couple times), threading has >> improved >>> dramatically, parsing has been completely re-done, the ability to >> process >>> multiple messages at once has been added, the ability to bind >>> separate rulesets to inputs (and have the results ofone filter feed >> onto a >>> ruleset, in fact the entire concept of rulesets..), it's gained the >>> ability to load multiple parsers (either in a stack or tied to >> specific >>> inputs). >>> >>> in the process it has gained a lot of capibilities and speed >> (improving by >>> at least two orders of magnatude) while keeping the logs at least as >> safe >>> as they were before. >>> >>> >>> >>> as for where to start troubleshooting, the best thing to do with >> something >>> like this where it crashes on startup is to start it in debug mode >> and >>> read through the output. odds are you have some sort of fatal typo in >> the >>> config file and reading through the debug output as it parses the >> config >>> file will identify the error. >>> >>> David Lang >>> >>> P.S. Rainer, you really do need to do some sort of feture timeline to >> show >>> the drastic nature of what's changed. even if you just took a couple >> of >>> hours to do a git log and then wed out the merges and trivial >> commits, the >>> highlights that remain would be very impressinve and give people a >> good >>> idea how much more advanced 5.x is from 3.x >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Tue Nov 9 09:33:57 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 09:33:57 +0100 Subject: [rsyslog] difference from v. 3.22 and 5.4.0 References: <87bp61emrc.fsf@newsguy.com><9B6E2A8877C38245BFB15CC491A11DA71DD69B@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD6A6@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6A8@GRFEXC.intern.adiscon.com> Thx! > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, November 09, 2010 9:33 AM > To: rsyslog-users > Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > > go ahead and pull them togeather. > > David Lang > > On Tue, 9 Nov 2010, Rainer Gerhards > wrote: > > > Date: Tue, 9 Nov 2010 09:19:55 +0100 > > From: Rainer Gerhards > > Reply-To: rsyslog-users > > To: rsyslog-users > > Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > > > > David, > > > > sorry for the sluggish response, but as you see right at the moment > really a > > *lot* is going on. I really like your postings on this topic. As > another > > measure, do you mind if I pull them together - almost literally - and > create > > a blog post out of them (or would you like to do a guest posting?)? > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >> Sent: Tuesday, November 09, 2010 9:17 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > >> > >> > >> thanks for doing this,what you have is a good start. > >> > >> > >> wnother tidbit from git, between 3.22.0 and 5.7.1 there were 1388 > >> commits, > >> with 11539 lines of commit logs (~6 lines per commit are > boilerplate, > >> but > >> that's still averaging about 4 lines of comment per commit) > >> > >> DavidLang > >> > >> On Mon, 8 Nov 2010, > >> Florian Riedl wrote: > >> > >>> David, > >>> > >>> Rainer adressed me with the task to compile such a list. > >>> > >>> A first draft can be found here: > >>> http://www.rsyslog.com/features/ > >>> > >>> I will expand it a bit more and look up the version information > that > >> is yet > >>> missing, which is still a lot. > >>> > >>> Florian > >>> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com > >>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >>> Sent: Sonntag, 7. November 2010 08:36 > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] difference from v. 3.22 and 5.4.0 > >>> > >>> On Sun, 7 Nov 2010, Harry Putnam wrote: > >>> > >>>> I run gentoo linux (kernel 2.6.33) and rsyslog version 3.22 > >>>> > >>>> In the course of a system update today, rsyslog version 5.4.0 was > >>>> installed, but it crashes immediately when started. > >>>> > >>>> I've done no testing or diagnostics as yet, just went back to > >> version > >>>> 3.22. I wanted to find out first if there have been major changes > >> that > >>>> might help diagnose the problem. > >>> > >>> the changes between 3.22 and 5.4 are huge. you would have to go > >> though the > >>> git tree to see them all, but that's such a huge amont of > development > >> that > >>> it's not really practical > >>> > >>> doing a little digging through git. > >>> > >>> v5.7.1 has 835 files, 241044 lines, 997701 words, 8971232 > characters > >>> > >>> the diff between v.3.22.2 and v5.7.1 is 71367 lines long, and the > >> diffstat > >>> says 625 files changed, 45917 insertions(+), 9128 deletions(-) > >>> > >>>> Any tricky gotchas that might be involved? > >>> > >>> oh yes. the locking has changed (a couple times), threading has > >> improved > >>> dramatically, parsing has been completely re-done, the ability to > >> process > >>> multiple messages at once has been added, the ability to bind > >>> separate rulesets to inputs (and have the results ofone filter feed > >> onto a > >>> ruleset, in fact the entire concept of rulesets..), it's gained the > >>> ability to load multiple parsers (either in a stack or tied to > >> specific > >>> inputs). > >>> > >>> in the process it has gained a lot of capibilities and speed > >> (improving by > >>> at least two orders of magnatude) while keeping the logs at least > as > >> safe > >>> as they were before. > >>> > >>> > >>> > >>> as for where to start troubleshooting, the best thing to do with > >> something > >>> like this where it crashes on startup is to start it in debug mode > >> and > >>> read through the output. odds are you have some sort of fatal typo > in > >> the > >>> config file and reading through the debug output as it parses the > >> config > >>> file will identify the error. > >>> > >>> David Lang > >>> > >>> P.S. Rainer, you really do need to do some sort of feture timeline > to > >> show > >>> the drastic nature of what's changed. even if you just took a > couple > >> of > >>> hours to do a git log and then wed out the merges and trivial > >> commits, the > >>> highlights that remain would be very impressinve and give people a > >> good > >>> idea how much more advanced 5.x is from 3.x > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Tue Nov 9 11:04:49 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 11:04:49 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> Hi, this could probably have something todo with the select limit. The question is if this is imrelp or not. While the CPU is busy, can you continue to receive RELP messages? If so, is output written? Also, is it possible to enable run rsyslog in debug mode? That would enable me to see where exactly the problem is. I guess it will be problematic, because debug logs quickly get rather large. But it would be useful. I would also appreciate if you could add a bug to the bug tracker at http://bugzilla.adiscon.com as this enables me to keep track of the issue (and others can better find it if they experience the same issue). Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 12:32 AM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > Hello, > > On 11/08/2010 08:47 PM, david at lang.hm wrote: > > can you get the lsof output so that we can see what the file > descriptor > > that it's having problems with is supposed to be? > > I've attached strace for about 10 seconds using: > > $ strace -ff -v -p 32267 -o rsyslogd > > ... and while strace was running, I've captured lsof output using: > > lsof -n -p 32267 > rsyslogd.lsof > > You can find all files and a tarball of them here: > http://biz.baze.de/debug/rsyslog/ > > Hope this helps. FYI, I've also run this for about 10 seconds on the > problematic thread: > > $ strace -p 32270 -c > Process 32270 attached - interrupt to quit > Process 32270 detached > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 60.95 0.003239 0 30405 select > 36.24 0.001926 0 1033772 1033709 recvfrom > 2.80 0.000149 0 30405 30405 accept > 0.00 0.000000 0 64 sendto > 0.00 0.000000 0 124 futex > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.005314 1094770 1064114 total > > > Best, > John > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marc.schiffbauer at mightycare.de Tue Nov 9 12:00:24 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Tue, 9 Nov 2010 12:00:24 +0100 (CET) Subject: [rsyslog] How to configure a RuleSets main queue? In-Reply-To: <2048657668.1935.1289298388478.JavaMail.root@zimbra> Message-ID: <7028519.1979.1289300424548.JavaMail.root@zimbra> Hi all, I have a question about $Rulsets: The example below is from the docs on http://www.rsyslog.com/doc/omruleset.html Question: How can I set queue paramaters as queue size, queue type, watermarks, queue filename etc for a rulesets main queue? I already tried several Queue* paramaters at several places in the config. So far without success ( = MainMsg and Action). Is it at all possible to configure a rulesets main queue? #-------------------------------------------------------------- $ModLoad omruleset # define ruleset for commonly written file $RuleSet commonAction $RulesetCreateMainQueue on *.* /path/to/file.log #switch back to default ruleset $ruleset RSYSLOG_DefaultRuleset # begin first action # note that we must first specify which ruleset to use for omruleset: $ActionOmrulesetRulesetName CommonAction mail.info :omruleset: #end first action # begin second action # note that we must first specify which ruleset to use for omruleset: $ActionOmrulesetRulesetName CommonAction :FROMHOST, isequal, "myhost.example.com" :omruleset: #end second action # of course, we can have "regular" actions alongside :omrulset: actions *.* /path/to/general-message-file.log #-------------------------------------------------------------- rsyslog version is 5.6.0 TIA -Marc From marc.schiffbauer at mightycare.de Tue Nov 9 16:29:19 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Tue, 9 Nov 2010 16:29:19 +0100 (CET) Subject: [rsyslog] DA-queuing bug reported with 4.4.3 still present in recent versions In-Reply-To: <473563494.2083.1289312945030.JavaMail.root@zimbra> Message-ID: <124663436.2116.1289316559335.JavaMail.root@zimbra> Hi Rainer et al, round about a year ago we had a relativly long thread about a bug with DA queues where one message always gets lost. -> "rsyslog 4.5.x queue file cleanup" This bug is still present in 4.6.4 ... I can trigger the bug the following way: Setup: DA MainMsgQueue + ompgsql Action Steps to reproduce with 4.6.4: 1. Stop the Database, rsyslogd still running 2. Produce 50 log messages for the database 3. Shutdown rsyslogd, queue gets saved to disk 4. Startup rsyslogd 5. Startup databae again. Result: only 49 Messages arrive at the Database. (These are only INSERT statements where message ORDER does not matter at all) At that time we had another issue with that setup and you found a bug and fixed it: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=8e45fd5c8e7df45532c5d239676cd92107c9e942 With that patch it worked but with one issue left which is described here or later in the mentioned thread. Additionally (maybe I just confuse something now) it seems that the MainMsgQueue is not DA at all, but only saves mem-Q-messages to disk on shutdown but not when the mem-queue is full... This is the relevant part of the config: $WorkDirectory /var/spool/rsyslog # default location for work (spool) files $MainMsgQueueType LinkedList # use asynchronous processing $MainMsgQueueFileName mainq # set file name, also enables disk mode $MainMsgQueueMaxFileSize 10M # when to start new file $MainMsgQueueSaveOnShutdown on # save queued message to DA queue on shutdown $MainMsgQueueSize 100 # very small for testing only $ModLoad ompgsql $ActionResumeRetryCount -1 $ActionResumeInterval 5 if $syslogtag startswith 'postfix/smtpd[' \ then :ompgsql:localhost,maillog,syslog,XXXXX;mail_t1 [ .. some more actions .. ] BTW: I tested the same config with v5 stable (5.6.0) and noticed that this version does not save the queue to disk at all. When stopping rsyslogd the process consumes some time but does not write anything to disk. regards -Marc -- Senior Consultant :: Solution Architect IT-Security :: Free Software :: GNU/Linux :: Virtualization Mobile +49_151_16227402 :: Fon +49_6187_9058695 :: Fax +49_2645_976896250 MightyCare Solutions GmbH http://www.mightycare.de Firmenangaben unter http://www.mightycare.de/impressum From john at feurix.com Tue Nov 9 16:46:59 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 16:46:59 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> Message-ID: <4CD96CF3.2090306@feurix.com> Hi, > this could probably have something todo with the select limit. The question > is if this is imrelp or not. > > While the CPU is busy, can you continue to receive RELP messages? If so, is > output written? new RELP messages are received, the conditional splitting and the dynamic filenames work correctly, and yes, new output is written to the correct files. (At least that's what I can see when testing with custom messages from different hosts to different facilities, there are so many active logfiles on the production server that it could be possible that something is lost without being noticed) > Also, is it possible to enable run rsyslog in debug mode? That would enable > me to see where exactly the problem is. I guess it will be problematic, > because debug logs quickly get rather large. But it would be useful. I'm a bit skeptic at this, because the problem only comes up after some days of usage. The rsyslog server and some clients are also in production and produce a lot of traffic. So I guess the debug log would grow to several GiB already before the problem even arises. Perhaps it's better to try to reproduce it first on a test-bed. I'll look into it, but I will need something like tcpflood.c for RELP with the ability to pass custom (so I can emulate the real traffic and cause all splitting/dynamic stuff to occur) :-) BTW, thinking more about it... since the problem always occurs after some days - perhaps it is related to using %$now% within the filename templates in combination with the $DynaFileCacheSize? The lsof shows a lot of open FDs ("REG") for "(deleted)" files of previous days. FD leaking? > I would also appreciate if you could add a bug to the bug tracker at > http://bugzilla.adiscon.com as this enables me to keep track of the issue > (and others can better find it if they experience the same issue). http://bugzilla.adiscon.com/show_bug.cgi?id=208 Thanks, John From rgerhards at hq.adiscon.com Tue Nov 9 17:01:24 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 17:01:24 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6B9@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 4:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > Hi, > > > this could probably have something todo with the select limit. The > question > > is if this is imrelp or not. > > > > While the CPU is busy, can you continue to receive RELP messages? If > so, is > > output written? > > new RELP messages are received, the conditional splitting and the > dynamic filenames work correctly, and yes, new output is written to the > correct files. (At least that's what I can see when testing with custom > messages from different hosts to different facilities, there are so > many > active logfiles on the production server that it could be possible that > something is lost without being noticed) > > > Also, is it possible to enable run rsyslog in debug mode? That would > enable > > me to see where exactly the problem is. I guess it will be > problematic, > > because debug logs quickly get rather large. But it would be useful. > > I'm a bit skeptic at this, because the problem only comes up after some > days of usage. The rsyslog server and some clients are also in > production and produce a lot of traffic. So I guess the debug log would > grow to several GiB already before the problem even arises. > > Perhaps it's better to try to reproduce it first on a test-bed. I'll > look into it, but I will need something like tcpflood.c for RELP with > the ability to pass custom > (so I can emulate the real traffic and cause all splitting/dynamic > stuff > to occur) :-) > > BTW, thinking more about it... since the problem always occurs after > some days - perhaps it is related to using %$now% within the filename > templates in combination with the $DynaFileCacheSize? > The lsof shows a lot of open FDs ("REG") for "(deleted)" files of > previous days. FD leaking? ah! this is interesting. How have you set your dynafile cache sizes? Rainer > > > I would also appreciate if you could add a bug to the bug tracker at > > http://bugzilla.adiscon.com as this enables me to keep track of the > issue > > (and others can better find it if they experience the same issue). > > http://bugzilla.adiscon.com/show_bug.cgi?id=208 > > Thanks, > John > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Tue Nov 9 17:34:58 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 9 Nov 2010 08:34:58 -0800 (PST) Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <4CD96CF3.2090306@feurix.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com> Message-ID: On Tue, 9 Nov 2010, John Feuerstein wrote: > BTW, thinking more about it... since the problem always occurs after > some days - perhaps it is related to using %$now% within the filename > templates in combination with the $DynaFileCacheSize? > The lsof shows a lot of open FDs ("REG") for "(deleted)" files of > previous days. FD leaking? does a kill -HUP (which tells rsyslog to close and re-open all output files) clear up the problem? David Lang From john at feurix.com Tue Nov 9 17:37:04 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 17:37:04 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6B9@GRFEXC.intern.adiscon.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6B9@GRFEXC.intern.adiscon.com> Message-ID: <4CD978B0.9040003@feurix.com> >> BTW, thinking more about it... since the problem always occurs after >> some days - perhaps it is related to using %$now% within the filename >> templates in combination with the $DynaFileCacheSize? >> The lsof shows a lot of open FDs ("REG") for "(deleted)" files of >> previous days. FD leaking? > > ah! this is interesting. How have you set your dynafile cache sizes? Yes, I've set: $DynaFileCacheSize 500 ... because with %$now% in the filenames, I expect around 500 active logfiles per day (plus some low-traffic ones that may get closed). From champ at softwink.com Tue Nov 9 17:51:07 2010 From: champ at softwink.com (Champ Clark III [Softwink]) Date: Tue, 9 Nov 2010 11:51:07 -0500 Subject: [rsyslog] liblognorm/libee In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6A7@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD6A7@GRFEXC.intern.adiscon.com> Message-ID: <20101109165107.GB1145@bundy.vistech.net> On Tue, Nov 09, 2010 at 09:25:57AM +0100, Rainer Gerhards wrote: > Hi all, > > especially to those interested in liblognorm: it looks like I already > orphaned the project. This is a false impression ;) I'd like to utilize as > many standard parts as possible, and thus I am looking very seriously into > CEE and libee. Right now, I anticipate that around 80% to 90% of what > liblognorm intends to do will actually be done by libee. Consequently, I am > currently very actively working on libee. So if you are interested in > liblognorm, you'd probably want to have a look at libee as well. > > Git: http://git.adiscon.com > Web: http://www.libee.org > > As a side-note I am also implementing a new small string library ( > http://doc.libestr.adiscon.com ). This new library may in the long term > replace rsyslog's internal string handling, from which it is derived. I'll > probably experiment with it in v6. If I am not mistaken, the new lib can > provide some performance benefits. But this needs to be weighted. Definitely not forgotten about on my end. However, I got caught up with some work and haven't had a chance to even look/dive into any code to see where libee liblognorm are headed. Hopefully, that'll change soon. Thanks for the update. -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From john at feurix.com Tue Nov 9 17:54:50 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 17:54:50 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com> Message-ID: <4CD97CDA.6000700@feurix.com> On 11/09/2010 05:34 PM, david at lang.hm wrote: > On Tue, 9 Nov 2010, John Feuerstein wrote: > >> BTW, thinking more about it... since the problem always occurs after >> some days - perhaps it is related to using %$now% within the filename >> templates in combination with the $DynaFileCacheSize? >> The lsof shows a lot of open FDs ("REG") for "(deleted)" files of >> previous days. FD leaking? > > does a kill -HUP (which tells rsyslog to close and re-open all output > files) clear up the problem? This does indeed do a *LOT* to lsof output. The differencte between the pre- and post- lsof outputs is huge. I guess that we are on the right track here, the old FDs marked as "(deleted)" are not closed properly by rsyslogd? lsof output before sending SIGHUP: http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/pre-lsof.txt lsof output after sending SIGHUP: http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/post-lsof.txt Note that I'm using rsyslogd's %$now% dynamic file template only to rotate logs without the need of any external helper. So I never send SIGHUP, because I expect rsyslogd to handle that internally? However, the original problem of the permanent 100% CPU thread is still there, see the pre- and post-process-tree.txt files at: http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/ From rgerhards at hq.adiscon.com Tue Nov 9 17:57:53 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 17:57:53 +0100 Subject: [rsyslog] liblognorm/libee References: <9B6E2A8877C38245BFB15CC491A11DA71DD6A7@GRFEXC.intern.adiscon.com> <20101109165107.GB1145@bundy.vistech.net> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6BB@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink] > Sent: Tuesday, November 09, 2010 5:51 PM > To: rsyslog-users > Subject: Re: [rsyslog] liblognorm/libee > > On Tue, Nov 09, 2010 at 09:25:57AM +0100, Rainer Gerhards wrote: > > Hi all, > > > > especially to those interested in liblognorm: it looks like I already > > orphaned the project. This is a false impression ;) I'd like to > utilize as > > many standard parts as possible, and thus I am looking very seriously > into > > CEE and libee. Right now, I anticipate that around 80% to 90% of what > > liblognorm intends to do will actually be done by libee. > Consequently, I am > > currently very actively working on libee. So if you are interested in > > liblognorm, you'd probably want to have a look at libee as well. > > > > Git: http://git.adiscon.com > > Web: http://www.libee.org > > > > As a side-note I am also implementing a new small string library ( > > http://doc.libestr.adiscon.com ). This new library may in the long > term > > replace rsyslog's internal string handling, from which it is derived. > I'll > > probably experiment with it in v6. If I am not mistaken, the new lib > can > > provide some performance benefits. But this needs to be weighted. > > Definitely not forgotten about on my end. However, I got > caught up with some work and haven't had a chance to even look/dive > into > any code to see where libee liblognorm are headed. Hopefully, that'll > change soon. Thanks for the update. Take your time, it's still pretty preliminary. I just wanted to make sure that the effort is avlive (I orignally had hoped to finish it within a month ... that was a month ago ;)). Rainer From rgerhards at hq.adiscon.com Tue Nov 9 18:02:43 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 18:02:43 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6B9@GRFEXC.intern.adiscon.com> <4CD978B0.9040003@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6BC@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 5:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > >> BTW, thinking more about it... since the problem always occurs after > >> some days - perhaps it is related to using %$now% within the > filename > >> templates in combination with the $DynaFileCacheSize? > >> The lsof shows a lot of open FDs ("REG") for "(deleted)" files of > >> previous days. FD leaking? > > > > ah! this is interesting. How have you set your dynafile cache sizes? > > Yes, I've set: > > $DynaFileCacheSize 500 Keep in mind that this setting is ona per-action (!) level. So if you have 20 actions, you have a total limit of 500*20 = 10,000 files that can be open at any time (but not more than 500 for a single action). This is action-specific so that you can tune the size of the cache to what is expected for the action. Rainer > > ... because with %$now% in the filenames, I expect around 500 active > logfiles per day (plus some low-traffic ones that may get closed). > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Tue Nov 9 18:38:13 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 18:38:13 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 12:32 AM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > Hello, > > On 11/08/2010 08:47 PM, david at lang.hm wrote: > > can you get the lsof output so that we can see what the file > descriptor > > that it's having problems with is supposed to be? > > I've attached strace for about 10 seconds using: > > $ strace -ff -v -p 32267 -o rsyslogd > > ... and while strace was running, I've captured lsof output using: > > lsof -n -p 32267 > rsyslogd.lsof > > You can find all files and a tarball of them here: > http://biz.baze.de/debug/rsyslog/ > > Hope this helps. FYI, I've also run this for about 10 seconds on the > problematic thread: > > $ strace -p 32270 -c > Process 32270 attached - interrupt to quit > Process 32270 detached > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 60.95 0.003239 0 30405 select > 36.24 0.001926 0 1033772 1033709 recvfrom > 2.80 0.000149 0 30405 30405 accept > 0.00 0.000000 0 64 sendto > 0.00 0.000000 0 124 futex > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.005314 1094770 1064114 total which inputs do you run? Only imrelp? The bottom line is that we need to actually see what rsyslog is doing. If you version is not too old, we could use debug on demand (in the link, Florian says he tested it with 6.x, but left out the more interesting fact of which is the minimum version ;)): http://www.rsyslog.com/how-to-use-debug-on-demand/ Rainer From john at feurix.com Tue Nov 9 18:54:20 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 18:54:20 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6BC@GRFEXC.intern.adiscon.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6B9@GRFEXC.intern.adiscon.com> <4CD978B0.9040003@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6BC@GRFEXC.intern.adiscon.com> Message-ID: <4CD98ACC.4020509@feurix.com> >> $DynaFileCacheSize 500 > > Keep in mind that this setting is ona per-action (!) level. So if you have 20 > actions, you have a total limit of 500*20 = 10,000 files that can be open at > any time (but not more than 500 for a single action). > > This is action-specific so that you can tune the size of the cache to what is > expected for the action. Wow, this is a very important detail. Can you add this to the documentation[1]? The scoping doc[2] is also very interesting in this regard. A document with an overview of parameters with a specific scope would be very helpful for version 5.x. So if I understand this correctly, I was mistaken that rsyslogd should have closed all rotated files. Instead, it keeps them open until the DynaFileCache is full... which was wrongly configured by me as you pointed out (set way too high before all actions in the global scope). Is it correct with versions 5.x (no scoping) to do the following, as an example: > $template SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" > > $DynaFileCacheSize 1 > > if $hostname == 'node6' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > if $hostname == 'node7' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > if $hostname == 'build' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ ... because I expect only one logfile per host per day which should be held open? And in contrast, for the following selector with two actions: > $template SystemAuthLog,"/srv/log/system/CLUSTER/%$now%/auth.log" > $template SplitSystemAuthLog,"/srv/log/system/%hostname%/%$now%/auth.log" > > $DynaFileCacheSize 30 > > auth,authpriv.* ?SystemAuthLog > & ?SplitSystemAuthLog I would set it to 30 because I expect 30 open logfiles per action (one per host per day)? A documented example would be very helpful for this. Thanks, John [1] http://www.rsyslog.com/doc/rsconf1_dynafilecachesize.html [2] http://www.rsyslog.com/doc/scoping.html From rgerhards at hq.adiscon.com Tue Nov 9 18:58:09 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 18:58:09 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6B9@GRFEXC.intern.adiscon.com> <4CD978B0.9040003@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6BC@GRFEXC.intern.adiscon.com> <4CD98ACC.4020509@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6C0@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 6:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > >> $DynaFileCacheSize 500 > > > > Keep in mind that this setting is ona per-action (!) level. So if you > have 20 > > actions, you have a total limit of 500*20 = 10,000 files that can be > open at > > any time (but not more than 500 for a single action). > > > > This is action-specific so that you can tune the size of the cache to > what is > > expected for the action. > > Wow, this is a very important detail. Can you add this to the > documentation[1]? The scoping doc[2] is also very interesting in this > regard. A document with an overview of parameters with a specific scope > would be very helpful for version 5.x. > > So if I understand this correctly, I was mistaken that rsyslogd should > have closed all rotated files. Instead, it keeps them open until the > DynaFileCache is full... which was wrongly configured by me as you > pointed out (set way too high before all actions in the global scope). > > Is it correct with versions 5.x (no scoping) to do the following, as an > example: > > > $template > SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" > > > > $DynaFileCacheSize 1 > > > > if $hostname == 'node6' and $syslogfacility == 16 then - > ?SplitServiceRsyncLog > > & ~ > > if $hostname == 'node7' and $syslogfacility == 16 then - > ?SplitServiceRsyncLog > > & ~ > > if $hostname == 'build' and $syslogfacility == 16 then - > ?SplitServiceRsyncLog > > & ~ > > ... because I expect only one logfile per host per day which should be > held open? > > > And in contrast, for the following selector with two actions: > > > $template SystemAuthLog,"/srv/log/system/CLUSTER/%$now%/auth.log" > > $template > SplitSystemAuthLog,"/srv/log/system/%hostname%/%$now%/auth.log" > > > > $DynaFileCacheSize 30 > > > > auth,authpriv.* ?SystemAuthLog > > & ?SplitSystemAuthLog > > I would set it to 30 because I expect 30 open logfiles per action (one > per host per day)? In short words: you are right. > > A documented example would be very helpful for this. > I agree, unfortunately, it's a constant struggle. Basically, I am a one-man show except when someone purchases some services. If you look just at the mailing list, it's quite a bit to do. And that's only part of what is on the agenda... Doc usually falls behind :( If someone is up to writing the samples etc. I promise he or she will get edit access to the web site within one working day ;) Rainer > Thanks, > John > > [1] http://www.rsyslog.com/doc/rsconf1_dynafilecachesize.html > [2] http://www.rsyslog.com/doc/scoping.html > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From john at feurix.com Tue Nov 9 19:06:51 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 19:06:51 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> Message-ID: <4CD98DBB.1020808@feurix.com> > which inputs do you run? Only imrelp? I'm running imuxsock and imrelp on the rsyslog server. The imuxsock doesn't really get any load, it's just for conformance to have a working /dev/log for vixie-cron and nullmailer etc. Here is the whole server config: > $ModLoad imuxsock > > # Default syslog file format > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # Never filter duplicated messages > $RepeatedMsgReduction off > > # We need these for further analysis > $EscapeControlCharactersOnReceive off > > $FileOwner syslog > $FileGroup syslog > > $FileCreateMode 0640 > $DirCreateMode 0750 > > $Umask 0027 > > $PrivDropToUser syslog > $PrivDropToGroup syslog > > # RELP > $ModLoad imrelp > $InputRELPServerRun 600 > > $MainMsgQueueSize 100000 ... followed by a lot of splitting with dynamic filenames like this: > $template SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" > > if $hostname == 'node6' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > if $hostname == 'node7' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > if $hostname == 'build' and $syslogfacility == 16 then -?SplitServiceRsyncLog > & ~ > The bottom line is that we need to actually see what rsyslog is doing. If you > version is not too old, we could use debug on demand (in the link, Florian > says he tested it with 6.x, but left out the more interesting fact of which > is the minimum version ;)): > http://www.rsyslog.com/how-to-use-debug-on-demand/ I'm using 5.6.0. Thanks, John From david at lang.hm Tue Nov 9 19:23:13 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 9 Nov 2010 10:23:13 -0800 (PST) Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <4CD98DBB.1020808@feurix.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> <4CD98DBB.1020808@feurix.com> Message-ID: On Tue, 9 Nov 2010, John Feuerstein wrote: > Here is the whole server config: > >> $ModLoad imuxsock >> >> # Default syslog file format >> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >> >> # Never filter duplicated messages >> $RepeatedMsgReduction off >> >> # We need these for further analysis >> $EscapeControlCharactersOnReceive off >> >> $FileOwner syslog >> $FileGroup syslog >> >> $FileCreateMode 0640 >> $DirCreateMode 0750 >> >> $Umask 0027 >> >> $PrivDropToUser syslog >> $PrivDropToGroup syslog you may try skipping the privdrop stuff, this is an area that rsyslog does not do especially well with. it may or may not cause this problem, but it's worth trying without it. David Lang >> # RELP >> $ModLoad imrelp >> $InputRELPServerRun 600 >> >> $MainMsgQueueSize 100000 > > > ... followed by a lot of splitting with dynamic filenames like this: > > >> $template SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" >> >> if $hostname == 'node6' and $syslogfacility == 16 then -?SplitServiceRsyncLog >> & ~ >> if $hostname == 'node7' and $syslogfacility == 16 then -?SplitServiceRsyncLog >> & ~ >> if $hostname == 'build' and $syslogfacility == 16 then -?SplitServiceRsyncLog >> & ~ > > >> The bottom line is that we need to actually see what rsyslog is doing. If you >> version is not too old, we could use debug on demand (in the link, Florian >> says he tested it with 6.x, but left out the more interesting fact of which >> is the minimum version ;)): >> http://www.rsyslog.com/how-to-use-debug-on-demand/ > > I'm using 5.6.0. > > Thanks, > John > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Tue Nov 9 19:28:09 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 9 Nov 2010 10:28:09 -0800 (PST) Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <4CD97CDA.6000700@feurix.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com> <4CD97CDA.6000700@feurix.com> Message-ID: On Tue, 9 Nov 2010, John Feuerstein wrote: > On 11/09/2010 05:34 PM, david at lang.hm wrote: >> On Tue, 9 Nov 2010, John Feuerstein wrote: >> >>> BTW, thinking more about it... since the problem always occurs after >>> some days - perhaps it is related to using %$now% within the filename >>> templates in combination with the $DynaFileCacheSize? >>> The lsof shows a lot of open FDs ("REG") for "(deleted)" files of >>> previous days. FD leaking? >> >> does a kill -HUP (which tells rsyslog to close and re-open all output >> files) clear up the problem? > > This does indeed do a *LOT* to lsof output. The differencte between the > pre- and post- lsof outputs is huge. I guess that we are on the right > track here, the old FDs marked as "(deleted)" are not closed properly by > rsyslogd? > > lsof output before sending SIGHUP: > > http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/pre-lsof.txt > > > lsof output after sending SIGHUP: > > http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/post-lsof.txt > > Note that I'm using rsyslogd's %$now% dynamic file template only to > rotate logs without the need of any external helper. So I never send > SIGHUP, because I expect rsyslogd to handle that internally? > > However, the original problem of the permanent 100% CPU thread is still > there, see the pre- and post-process-tree.txt files at: > > http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/ one other thing that I find useful to understand things is to do a strace on the other threads to identify what each ofthem are. you have the parent thread (doesn't do much) each input will have it's own thread listening. you will have one (or more if you have multiple worker threads configured) output threads. if you can identify what is happening on the other threads, you may be able to figure out what's happening on the problem thread by process of elimination. David Lang From david at lang.hm Tue Nov 9 19:34:37 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 9 Nov 2010 10:34:37 -0800 (PST) Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <4CD97CDA.6000700@feurix.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6AC@GRFEXC.intern.adiscon.com> <4CD96CF3.2090306@feurix.com> <4CD97CDA.6000700@feurix.com> Message-ID: hmm, in looking at the list of FDs in the select giving -EAGAIN, they all look like they are TCP connections, not file outputs. so I think the problem is not in the dynafiles output, but in the input moduleyou are using (although i won't rule out the possibilitythat a huge number of open files triggered the problem) David Lang On Tue, 9 Nov 2010, John Feuerstein wrote: > Date: Tue, 09 Nov 2010 17:54:50 +0100 > From: John Feuerstein > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time (spinlock?) > > On 11/09/2010 05:34 PM, david at lang.hm wrote: >> On Tue, 9 Nov 2010, John Feuerstein wrote: >> >>> BTW, thinking more about it... since the problem always occurs after >>> some days - perhaps it is related to using %$now% within the filename >>> templates in combination with the $DynaFileCacheSize? >>> The lsof shows a lot of open FDs ("REG") for "(deleted)" files of >>> previous days. FD leaking? >> >> does a kill -HUP (which tells rsyslog to close and re-open all output >> files) clear up the problem? > > This does indeed do a *LOT* to lsof output. The differencte between the > pre- and post- lsof outputs is huge. I guess that we are on the right > track here, the old FDs marked as "(deleted)" are not closed properly by > rsyslogd? > > lsof output before sending SIGHUP: > > http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/pre-lsof.txt > > > lsof output after sending SIGHUP: > > http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/post-lsof.txt > > Note that I'm using rsyslogd's %$now% dynamic file template only to > rotate logs without the need of any external helper. So I never send > SIGHUP, because I expect rsyslogd to handle that internally? > > However, the original problem of the permanent 100% CPU thread is still > there, see the pre- and post-process-tree.txt files at: > > http://biz.baze.de/debug/rsyslog/pre-post-SIGHUP/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From john at feurix.com Tue Nov 9 20:01:30 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 20:01:30 +0100 Subject: [rsyslog] rsyslog Text File Input Module losing seek? In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> Message-ID: <4CD99A8A.4010306@feurix.com> Hello, just curious... why don't you let bind use syslog() directly, e.g. receiving the messages with rsyslog locally using imuxsock. Then let rsyslog write it to your local file (if needed) and additionally send it to the remote host? Does the file input module really make sense for a service that can log to syslog? For example, here is part of my bind logging config: logging { channel default { syslog local0; severity debug; }; channel query { syslog local1; severity debug; }; category default { default; }; category general { default; }; category queries { query; }; }; Best regards, John From rgerhards at hq.adiscon.com Tue Nov 9 20:56:12 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 9 Nov 2010 20:56:12 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> <4CD98DBB.1020808@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6C3@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 7:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > > which inputs do you run? Only imrelp? > > I'm running imuxsock and imrelp on the rsyslog server. OK, this really smells like a problem with imrelp, and probably with librelp. I now also tend to think that the issue is rooted in select(). I'll try to implement epoll() in librelp, but can't promise I make it this week. But... > > The imuxsock doesn't really get any load, it's just for conformance to > have a working /dev/log for vixie-cron and nullmailer etc. > > Here is the whole server config: > > > $ModLoad imuxsock > > > > # Default syslog file format > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > > > # Never filter duplicated messages > > $RepeatedMsgReduction off > > > > # We need these for further analysis > > $EscapeControlCharactersOnReceive off > > > > $FileOwner syslog > > $FileGroup syslog > > > > $FileCreateMode 0640 > > $DirCreateMode 0750 > > > > $Umask 0027 > > > > $PrivDropToUser syslog > > $PrivDropToGroup syslog > > > > # RELP > > $ModLoad imrelp > > $InputRELPServerRun 600 > > > > $MainMsgQueueSize 100000 > > > ... followed by a lot of splitting with dynamic filenames like this: > > > > $template > SplitServiceRsyncLog,"/srv/log/system/%hostname%/%$now%/rsyncd.log" > > > > if $hostname == 'node6' and $syslogfacility == 16 then - > ?SplitServiceRsyncLog > > & ~ > > if $hostname == 'node7' and $syslogfacility == 16 then - > ?SplitServiceRsyncLog > > & ~ > > if $hostname == 'build' and $syslogfacility == 16 then - > ?SplitServiceRsyncLog > > & ~ > > > > The bottom line is that we need to actually see what rsyslog is > doing. If you > > version is not too old, we could use debug on demand (in the link, > Florian > > says he tested it with 6.x, but left out the more interesting fact of > which > > is the minimum version ;)): > > http://www.rsyslog.com/how-to-use-debug-on-demand/ > > I'm using 5.6.0. That version definitely has it. So it would be worth trying out debug on demand, and turn it on when the issue appears. This will definietely give me some useful clues. Rainer From john at feurix.com Tue Nov 9 23:34:33 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 09 Nov 2010 23:34:33 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6C3@GRFEXC.intern.adiscon.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> <4CD98DBB.1020808@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6C3@GRFEXC.intern.adiscon.com> Message-ID: <4CD9CC79.9070000@feurix.com> >>> http://www.rsyslog.com/how-to-use-debug-on-demand/ >> >> I'm using 5.6.0. > > That version definitely has it. So it would be worth trying out debug on > demand, and turn it on when the issue appears. This will definietely give me > some useful clues. OK, I've stopped rsyslogd and started it in a screen session: --- $ export RSYSLOG_DEBUG="DebugOnDemand NoStdOut" $ export RSYSLOG_DEBUGLOG="/srv/log/debug.log" $ rsyslogd -n -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf 1558.843688400:70395b0cb700: Note: debug on demand turned on via configuraton file, use USR1 signal to activate. --- I'll have a look on it for the next few days and will activate the debug log if necessary. Thanks all for the help so far! :-) John From rgerhards at hq.adiscon.com Wed Nov 10 07:13:34 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 10 Nov 2010 07:13:34 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> <4CD98DBB.1020808@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6C3@GRFEXC.intern.adiscon.com> <4CD9CC79.9070000@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6C4@GRFEXC.intern.adiscon.com> Excellent, thx! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Tuesday, November 09, 2010 11:35 PM > To: rsyslog-users > Subject: Re: [rsyslog] One thread eating up infinite CPU time > (spinlock?) > > >>> http://www.rsyslog.com/how-to-use-debug-on-demand/ > >> > >> I'm using 5.6.0. > > > > That version definitely has it. So it would be worth trying out debug > on > > demand, and turn it on when the issue appears. This will definietely > give me > > some useful clues. > > OK, I've stopped rsyslogd and started it in a screen session: > > --- > > $ export RSYSLOG_DEBUG="DebugOnDemand NoStdOut" > $ export RSYSLOG_DEBUGLOG="/srv/log/debug.log" > $ rsyslogd -n -c5 -i /var/run/rsyslogd.pid -f /etc/rsyslog.conf > > 1558.843688400:70395b0cb700: Note: debug on demand turned on via > configuraton file, use USR1 signal to activate. > > --- > > I'll have a look on it for the next few days and will activate the > debug > log if necessary. > > Thanks all for the help so far! :-) > > John > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mmonitz at gmail.com Wed Nov 10 09:27:58 2010 From: mmonitz at gmail.com (matan monitz) Date: Wed, 10 Nov 2010 10:27:58 +0200 Subject: [rsyslog] rsyslog Text File Input Module losing seek? In-Reply-To: <4CD99A8A.4010306@feurix.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> <4CD99A8A.4010306@feurix.com> Message-ID: hello john and thanks for the tip however, we had already tried it the guy in charge of the server claims it caused a significant decrease in bind performance On Tue, Nov 9, 2010 at 9:01 PM, John Feuerstein wrote: > Hello, > > just curious... why don't you let bind use syslog() directly, e.g. > receiving the messages with rsyslog locally using imuxsock. Then let > rsyslog write it to your local file (if needed) and additionally send it > to the remote host? Does the file input module really make sense for a > service that can log to syslog? > > For example, here is part of my bind logging config: > > logging { > channel default { > syslog local0; > severity debug; > }; > channel query { > syslog local1; > severity debug; > }; > category default { default; }; > category general { default; }; > category queries { query; }; > }; > > Best regards, > John > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Nov 10 10:39:08 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 10 Nov 2010 10:39:08 +0100 Subject: [rsyslog] rsyslog Text File Input Module losing seek? References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of matan monitz > Sent: Monday, November 08, 2010 7:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > hi rainer > unfortunately this is not a server i'm directly in charge of so we > can't > really upgrade for while > (they will only upgrade as part of the planned upgrade for the whole > server) Does that also mean you would not be able to apply a fix if I I provide one for 3.22.2 (the current v3-stable)? > can you please elaborate on the cause of the problem so that maybe i > can > figure out a temporary workaround? There definitely is a problem when the file size is larger than 2GB. The rest may be related to dirty shutdown when the state file is deleted. See http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536b3dc4a 28b94f80a361b41d http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2124ba7 d830652e9b1ead96 I have not yet receive feedback that makes me sure it's 100% fixed, so I'd actually be quite interested in another test case. However, quite honestly, if we can't run a better version on your box, there is no point for me in further looking at that case (because it can't be solved anyway...). Rainer > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards > wrote: > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > Sent: Monday, November 08, 2010 3:51 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > hello rainer > > > we are using 3.22.1-3 from rpm on rhel 5 > > > > That's definitely too old. I maintain it when I am at it, but the fix > went > > into v4+ so far. I suggest that you pull the latest v4-stable from > git. If > > git doesn't work for you, let me know and I'll mail you a private > tarball. > > I > > am just right now very busy (probably tomorrow as well, as I am > writing a > > lot > > of new code in support for CEE). > > > > Rainer > > > > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards > > > wrote: > > > > > > > which version do you use? I have recently addressed such a > problem... > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > > > Sent: Monday, November 08, 2010 2:45 PM > > > > > To: rsyslog at lists.adiscon.com > > > > > Subject: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > > > > > Hello all > > > > > we have a bind server writing and rotating logs on file system > > > > > (rotating > > > > > when log file reaches 10Mb). > > > > > we also have rsyslog configured with the text file input module > > > reading > > > > > the > > > > > file and sending it over UDP to another rsyslog server. > > > > > several times during the day the rsyslog on the bind machine > starts > > > > > reading > > > > > the file from the top, causing a sudden high rate of duplicate > and > > > old > > > > > events. > > > > > in an attempt to identify the problem: > > > > > 1.verified this using tcpdump to make sure the and saw the old > logs > > > > > coming > > > > > down the wire to the rsyslog server > > > > > 2.made sure state files are correctly configured and are being > > > created > > > > > when > > > > > needed > > > > > 3.made sure the rsyslog is not crashing and restarting by > > > monitoring > > > > > the pid > > > > > googeling hasn't helped as well > > > > > > > > > > any ideas? > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mmonitz at gmail.com Wed Nov 10 18:45:49 2010 From: mmonitz at gmail.com (matan monitz) Date: Wed, 10 Nov 2010 19:45:49 +0200 Subject: [rsyslog] rsyslog Text File Input Module losing seek? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com> Message-ID: hello rainer unfortunately we will not be able to apply a fix if provided, if an upgrade will eventually happen it will likely be to Version 5. i have reviewed the references you provided and i don't believe they are relevant to our case out files rotate at 10MB which is miles off from 2GB. also, as i mentioned,i think this is not related to state files because we made sure they are being created and that rsyslog is not crashing i agree there is not much point in providing a fix at this point, however i would like to hear any other guesses you may have regarding the cause of the situation so that i may figure out a way to work around it outside of rsyslog also i will be happy to provide technical data and logs as instructed if you need another test case thanks in advance On Wed, Nov 10, 2010 at 11:39 AM, Rainer Gerhards wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > Sent: Monday, November 08, 2010 7:08 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > > > hi rainer > > unfortunately this is not a server i'm directly in charge of so we > > can't > > really upgrade for while > > (they will only upgrade as part of the planned upgrade for the whole > > server) > > Does that also mean you would not be able to apply a fix if I I provide one > for 3.22.2 (the current v3-stable)? > > > can you please elaborate on the cause of the problem so that maybe i > > can > > figure out a temporary workaround? > > There definitely is a problem when the file size is larger than 2GB. The > rest > may be related to dirty shutdown when the state file is deleted. See > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536b3dc4a > 28b94f80a361b41d > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2124ba7 > d830652e9b1ead96 > > I have not yet receive feedback that makes me sure it's 100% fixed, so I'd > actually be quite interested in another test case. However, quite honestly, > if we can't run a better version on your box, there is no point for me in > further looking at that case (because it can't be solved anyway...). > > Rainer > > > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards > > wrote: > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > > Sent: Monday, November 08, 2010 3:51 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > > > hello rainer > > > > we are using 3.22.1-3 from rpm on rhel 5 > > > > > > That's definitely too old. I maintain it when I am at it, but the fix > > went > > > into v4+ so far. I suggest that you pull the latest v4-stable from > > git. If > > > git doesn't work for you, let me know and I'll mail you a private > > tarball. > > > I > > > am just right now very busy (probably tomorrow as well, as I am > > writing a > > > lot > > > of new code in support for CEE). > > > > > > Rainer > > > > > > > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards > > > > wrote: > > > > > > > > > which version do you use? I have recently addressed such a > > problem... > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > > > > Sent: Monday, November 08, 2010 2:45 PM > > > > > > To: rsyslog at lists.adiscon.com > > > > > > Subject: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > > > > > > > Hello all > > > > > > we have a bind server writing and rotating logs on file system > > > > > > (rotating > > > > > > when log file reaches 10Mb). > > > > > > we also have rsyslog configured with the text file input module > > > > reading > > > > > > the > > > > > > file and sending it over UDP to another rsyslog server. > > > > > > several times during the day the rsyslog on the bind machine > > starts > > > > > > reading > > > > > > the file from the top, causing a sudden high rate of duplicate > > and > > > > old > > > > > > events. > > > > > > in an attempt to identify the problem: > > > > > > 1.verified this using tcpdump to make sure the and saw the old > > logs > > > > > > coming > > > > > > down the wire to the rsyslog server > > > > > > 2.made sure state files are correctly configured and are being > > > > created > > > > > > when > > > > > > needed > > > > > > 3.made sure the rsyslog is not crashing and restarting by > > > > monitoring > > > > > > the pid > > > > > > googeling hasn't helped as well > > > > > > > > > > > > any ideas? > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Nov 10 18:47:28 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 10 Nov 2010 18:47:28 +0100 Subject: [rsyslog] rsyslog Text File Input Module losing seek? References: <9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6D8@GRFEXC.intern.adiscon.com> Could you run it in debug mode and provide me a log covering a situation where the problem occurs? I may be able to gain some information from that. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of matan monitz > Sent: Wednesday, November 10, 2010 6:46 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > hello rainer > unfortunately we will not be able to apply a fix if provided, > if an upgrade will eventually happen it will likely be to Version 5. > > i have reviewed the references you provided and i don't believe they > are > relevant to our case > out files rotate at 10MB which is miles off from 2GB. > also, as i mentioned,i think this is not related to state files because > we > made sure they are being created and that rsyslog is not crashing > i agree there is not much point in providing a fix at this point, > however i > would like to hear any other guesses you may have regarding the cause > of the > situation so that i may figure out a way to work around it outside of > rsyslog > also i will be happy to provide technical data and logs as instructed > if you > need another test case > > thanks in advance > > > > > On Wed, Nov 10, 2010 at 11:39 AM, Rainer Gerhards > wrote: > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > Sent: Monday, November 08, 2010 7:08 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek? > > > > > > hi rainer > > > unfortunately this is not a server i'm directly in charge of so we > > > can't > > > really upgrade for while > > > (they will only upgrade as part of the planned upgrade for the > whole > > > server) > > > > Does that also mean you would not be able to apply a fix if I I > provide one > > for 3.22.2 (the current v3-stable)? > > > > > can you please elaborate on the cause of the problem so that maybe > i > > > can > > > figure out a temporary workaround? > > > > There definitely is a problem when the file size is larger than 2GB. > The > > rest > > may be related to dirty shutdown when the state file is deleted. See > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536 > b3dc4a > > > 28b94f80a361b41d 0814bb0307f9d8536b3dc4a%0A28b94f80a361b41d> > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2 > 124ba7 > > > d830652e9b1ead96 0933057bc2f014fd2124ba7%0Ad830652e9b1ead96> > > > > I have not yet receive feedback that makes me sure it's 100% fixed, > so I'd > > actually be quite interested in another test case. However, quite > honestly, > > if we can't run a better version on your box, there is no point for > me in > > further looking at that case (because it can't be solved anyway...). > > > > Rainer > > > > > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards > > > wrote: > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > > > Sent: Monday, November 08, 2010 3:51 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing > seek? > > > > > > > > > > hello rainer > > > > > we are using 3.22.1-3 from rpm on rhel 5 > > > > > > > > That's definitely too old. I maintain it when I am at it, but the > fix > > > went > > > > into v4+ so far. I suggest that you pull the latest v4-stable > from > > > git. If > > > > git doesn't work for you, let me know and I'll mail you a private > > > tarball. > > > > I > > > > am just right now very busy (probably tomorrow as well, as I am > > > writing a > > > > lot > > > > of new code in support for CEE). > > > > > > > > Rainer > > > > > > > > > > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards > > > > > wrote: > > > > > > > > > > > which version do you use? I have recently addressed such a > > > problem... > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz > > > > > > > Sent: Monday, November 08, 2010 2:45 PM > > > > > > > To: rsyslog at lists.adiscon.com > > > > > > > Subject: [rsyslog] rsyslog Text File Input Module losing > seek? > > > > > > > > > > > > > > Hello all > > > > > > > we have a bind server writing and rotating logs on file > system > > > > > > > (rotating > > > > > > > when log file reaches 10Mb). > > > > > > > we also have rsyslog configured with the text file input > module > > > > > reading > > > > > > > the > > > > > > > file and sending it over UDP to another rsyslog server. > > > > > > > several times during the day the rsyslog on the bind > machine > > > starts > > > > > > > reading > > > > > > > the file from the top, causing a sudden high rate of > duplicate > > > and > > > > > old > > > > > > > events. > > > > > > > in an attempt to identify the problem: > > > > > > > 1.verified this using tcpdump to make sure the and saw the > old > > > logs > > > > > > > coming > > > > > > > down the wire to the rsyslog server > > > > > > > 2.made sure state files are correctly configured and are > being > > > > > created > > > > > > > when > > > > > > > needed > > > > > > > 3.made sure the rsyslog is not crashing and restarting by > > > > > monitoring > > > > > > > the pid > > > > > > > googeling hasn't helped as well > > > > > > > > > > > > > > any ideas? > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > > > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From k at kevinkevin.com Thu Nov 11 21:09:26 2010 From: k at kevinkevin.com (kevin) Date: Thu, 11 Nov 2010 15:09:26 -0500 Subject: [rsyslog] filtering fromhost-ip to ommysql Message-ID: <002201cb81dc$587ee310$097ca930$@com> Hello, I am trying to filter only specific remote rsyslog hosts to send the data to my ommysql database. Currently the following configuration directive produces an error : *.*:fromhost-ip, isequal, "xx.xx.xx.xx" :ommysql:127.0.0.1,Syslog,loguser,logpassword The errors I get : rsyslogd: unknown priority name "*:fromhost-ip" [try http://www.rsyslog.com/e/3000 ] rsyslogd: the last error occured in /etc/rsyslog.conf, line 143:"*.*:fromhost-ip, isequal, "xx.xx.xx.xx" :ommysql:127.0.0.1,Syslog,loguser,logpassword " rsyslogd: warning: selector line without actions will be discarded rsyslogd: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ] There is nothing in the rsyslog manual pages that specify exactly HOW you can go about doing this. I just want to filter out specific hosts from sending the syslog data to my mysql database. Any help is greatly appreciated! Thanks, Kevin From john at feurix.com Thu Nov 11 21:54:00 2010 From: john at feurix.com (John Feuerstein) Date: Thu, 11 Nov 2010 21:54:00 +0100 Subject: [rsyslog] filtering fromhost-ip to ommysql In-Reply-To: <002201cb81dc$587ee310$097ca930$@com> References: <002201cb81dc$587ee310$097ca930$@com> Message-ID: <4CDC57E8.6080807@feurix.com> Hi, quoting the filter docs[1]: A property-based filter must start with a colon in column 0. This tells rsyslogd that it is the new filter type. The colon must be followed by the property name, a comma, the name of the compare operation to carry out, another comma and then the value to compare against. This value must be quoted. There can be spaces and tabs between the commas. Property names and compare operations are case-sensitive, so "msg" works, while "MSG" is an invalid property name. In brief, the syntax is as follows: :property, [!]compare-operation, "value" To find out the available properties, check out [2]. Then read up on [3] for the ommysql syntax. Untested example: :fromhost-ip, isequal, "xxx.xxx.xxx.xxx" :ommysql:host:db:user:pass Hope this helps, John [1] http://www.rsyslog.com/doc/rsyslog_conf_filter.html [2] http://www.rsyslog.com/doc/property_replacer.html [3] http://www.rsyslog.com/doc/rsyslog_mysql.html From k at kevinkevin.com Thu Nov 11 22:14:57 2010 From: k at kevinkevin.com (kevin) Date: Thu, 11 Nov 2010 16:14:57 -0500 Subject: [rsyslog] filtering fromhost-ip to ommysql In-Reply-To: <4CDC57E8.6080807@feurix.com> References: <002201cb81dc$587ee310$097ca930$@com> <4CDC57E8.6080807@feurix.com> Message-ID: <003301cb81e5$80334a50$8099def0$@com> >Untested example: > >:fromhost-ip, isequal, "xxx.xxx.xxx.xxx" :ommysql:host:db:user:pass > > >Hope this helps, >John John, Unfortunately that example you provided doesn't work. Here's what I put : :fromhost-ip, isequal, "xx.xx.xx.xx" :ommysql:127.0.0.1:dbname:dbuser:dbpassword Produced the following error : Nov 11 16:10:16 hostname rsyslogd-2016: Trouble with MySQL connection properties. -MySQL logging disabled [try http://www.rsyslog.com/e/2016 ] Nov 11 16:10:16 hostname rsyslogd: the last error occured in /etc/rsyslog.conf, line 142:":fromhost-ip, isequal, "xx.xx.xx.xx" :ommysql:127.0.0.1:dbname:dbuser:dbpassword" Nov 11 16:10:16 hostname rsyslogd: warning: selector line without actions will be discarded Nov 11 16:10:16 hostname rsyslogd-2124: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ] Thoughts? From john at feurix.com Thu Nov 11 22:25:17 2010 From: john at feurix.com (John Feuerstein) Date: Thu, 11 Nov 2010 22:25:17 +0100 Subject: [rsyslog] filtering fromhost-ip to ommysql In-Reply-To: <003301cb81e5$80334a50$8099def0$@com> References: <002201cb81dc$587ee310$097ca930$@com> <4CDC57E8.6080807@feurix.com> <003301cb81e5$80334a50$8099def0$@com> Message-ID: <4CDC5F3D.3090707@feurix.com> Hi, >> Untested example: >> >> :fromhost-ip, isequal, "xxx.xxx.xxx.xxx" :ommysql:host:db:user:pass >> > :fromhost-ip, isequal, "xx.xx.xx.xx" > :ommysql:127.0.0.1:dbname:dbuser:dbpassword I'm out of ideas (never used ommysql)... did you put it on the same line? (your two lines are hard-wrapped in the mail source, so this could be the problem). From david at lang.hm Fri Nov 12 00:38:29 2010 From: david at lang.hm (david at lang.hm) Date: Thu, 11 Nov 2010 15:38:29 -0800 (PST) Subject: [rsyslog] filtering fromhost-ip to ommysql In-Reply-To: <002201cb81dc$587ee310$097ca930$@com> References: <002201cb81dc$587ee310$097ca930$@com> Message-ID: On Thu, 11 Nov 2010, kevin wrote: > Hello, > > I am trying to filter only specific remote rsyslog hosts to send the data to > my ommysql database. Currently the following configuration directive > produces an error : > > *.*:fromhost-ip, isequal, "xx.xx.xx.xx" > :ommysql:127.0.0.1,Syslog,loguser,logpassword as John points out, having the *.* at the beginning of the line is wrong. there are four types of ways to define output lines in rsyslog last match (do this if the last test matched) i.e. & /var/log/messages facility.severity (i.e. traditional syslog) i.e. *.* /var/log/messages property tests i.e. :fromhost, isequal, "value" /var/log/messages scripted i.e. if ($fromhost eq "value") /var/log/messages the later items are slower than the earlier ones. you combined two types of entries (a common mistake), and the result will not work. what I would do is first fix the problem, and then attempt a simplified version to make sure you have the test correct, writing to a file, before you add in the database complexity. so try: :fromhost-ip, isequal, "xx.xx.xx.xx" /var/log/test then try adding the database piece. Davi dLang > > > The errors I get : > > rsyslogd: unknown priority name "*:fromhost-ip" [try > http://www.rsyslog.com/e/3000 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line > 143:"*.*:fromhost-ip, isequal, "xx.xx.xx.xx" > :ommysql:127.0.0.1,Syslog,loguser,logpassword " > rsyslogd: warning: selector line without actions will be discarded > rsyslogd: CONFIG ERROR: could not interpret master config file > '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ] > > > There is nothing in the rsyslog manual pages that specify exactly HOW you > can go about doing this. I just want to filter out specific hosts from > sending the syslog data to my mysql database. > > Any help is greatly appreciated! > > Thanks, > > Kevin > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Nov 12 07:13:40 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Nov 2010 07:13:40 +0100 Subject: [rsyslog] filtering fromhost-ip to ommysql References: <002201cb81dc$587ee310$097ca930$@com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6E7@GRFEXC.intern.adiscon.com> I may be wrong, but I think ommysql was somewhat buggish in that it required a semicolon after the config parameters even if no template was to be specified. It's actually the earliest-ever module and it was written when almost no plumbing existed. In theory, this problem should not have survived until now, but it may have (and I will have no time to fix/check for roughly the next week and a half). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, November 12, 2010 12:38 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering fromhost-ip to ommysql > > On Thu, 11 Nov 2010, kevin wrote: > > > Hello, > > > > I am trying to filter only specific remote rsyslog hosts to send the > data to > > my ommysql database. Currently the following configuration directive > > produces an error : > > > > *.*:fromhost-ip, isequal, "xx.xx.xx.xx" > > :ommysql:127.0.0.1,Syslog,loguser,logpassword > > as John points out, having the *.* at the beginning of the line is > wrong. > > there are four types of ways to define output lines in rsyslog > > last match (do this if the last test matched) > i.e. > & /var/log/messages > > facility.severity (i.e. traditional syslog) > i.e. > *.* /var/log/messages > > property tests > i.e. > :fromhost, isequal, "value" /var/log/messages > > scripted > i.e. > if ($fromhost eq "value") /var/log/messages > > the later items are slower than the earlier ones. > > you combined two types of entries (a common mistake), and the result > will > not work. > > what I would do is first fix the problem, and then attempt a simplified > version to make sure you have the test correct, writing to a file, > before > you add in the database complexity. > > so try: > > :fromhost-ip, isequal, "xx.xx.xx.xx" /var/log/test > > then try adding the database piece. > > Davi dLang > > > > > > > The errors I get : > > > > rsyslogd: unknown priority name "*:fromhost-ip" [try > > http://www.rsyslog.com/e/3000 ] > > rsyslogd: the last error occured in /etc/rsyslog.conf, line > > 143:"*.*:fromhost-ip, isequal, "xx.xx.xx.xx" > > :ommysql:127.0.0.1,Syslog,loguser,logpassword " > > rsyslogd: warning: selector line without actions will be discarded > > rsyslogd: CONFIG ERROR: could not interpret master config file > > '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ] > > > > > > There is nothing in the rsyslog manual pages that specify exactly HOW > you > > can go about doing this. I just want to filter out specific hosts > from > > sending the syslog data to my mysql database. > > > > Any help is greatly appreciated! > > > > Thanks, > > > > Kevin > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From dave at fly.srk.fer.hr Fri Nov 12 13:37:46 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Fri, 12 Nov 2010 13:37:46 +0100 Subject: [rsyslog] SIGSEGV because of double free in msgDestruct Message-ID: <20101112123746.GA26430@fly.srk.fer.hr> Hello. I have rsyslog 5.6.0 on CentOS 5.5 with a slightly complex configuration and it's crashing. The complete configuration file is attached. The crash is perfectly reproducible and it happens very soon after the data starts arriving. The program was started with: rsyslogd -c5 -x -f rsyslog-datasink.conf I have two queues in order to have two thread pools. Input queue just takes the message from UDP or TCP socket and uses omruleset to pass it to the output queue. The output queue then uses omprog to pass the message to the external program. Omprog blocks when the pipe to the external program is full, so I wanted to have unblocked threads to accept incoming messages (which will mostly use UDP). Hence, the configuration has two queues. It's possible that I made some error in the configuration and rsyslogd is crashing because I'm doing something that I wasn't supposed to do, but it didn't detect the faulty configuration early on. The whole thing works fine when I have only one queue (created with $Ruleset) and input and omprog modules on it. But I'd really like to use two thread pools. It should be possible to reproduce this problem with cat as the omprog binary, although I haven't tried. One curiosity (probably unrelated to the problem): $GenerateConfigGraph at the end of the config file creates a picture which has only the main queue, but the queues I configured with $Ruleset directives are not on the picture. The below is from gdb. The process was started from gdb, so there's no call to sigsegvHdlr(), which can be seen in the core file when I start rsyslogd on its own. (gdb) info threads * 8 Thread 0xb4debb90 (LWP 11149) ConsumerReg (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1679 7 Thread 0xb57ecb90 (LWP 11148) 0x00d46402 in __kernel_vsyscall () 6 Thread 0xb61edb90 (LWP 11147) msgDestruct (ppThis=0xb61ed1d4) at msg.c:790 5 Thread 0xb6beeb90 (LWP 11146) 0x00d46402 in __kernel_vsyscall () 4 Thread 0xb75efb90 (LWP 11145) 0x00d46402 in __kernel_vsyscall () 3 Thread 0xb7ff0b90 (LWP 11144) 0x00d46402 in __kernel_vsyscall () 2 Thread 0xb7ff1ac0 (LWP 11111) 0x00d46402 in __kernel_vsyscall () (gdb) bt #0 0x00d46402 in __kernel_vsyscall () #1 0x00b2f040 in raise () from /lib/i686/nosegneg/libc.so.6 #2 0x00b30a21 in abort () from /lib/i686/nosegneg/libc.so.6 #3 0x00b67e3b in __libc_message () from /lib/i686/nosegneg/libc.so.6 #4 0x00b70758 in free () from /lib/i686/nosegneg/libc.so.6 #5 0x080612ee in msgDestruct (ppThis=0xb4deb1d4) at msg.c:816 #6 0x08079e35 in DeleteProcessedBatch (pThis=0x80b7988, pBatch=0x80b7cd0) at queue.c:1404 #7 0x0807a3b9 in DequeueConsumableElements (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1441 #8 DequeueConsumable (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1489 #9 0x0807a5d7 in DequeueForConsumer (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1626 #10 ConsumerReg (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1679 #11 0x0807350e in wtiWorker (pThis=0x80b7cb8) at wti.c:315 #12 0x08072e1f in wtpWorker (arg=0x80b7cb8) at wtp.c:381 #13 0x00c9b869 in start_thread () from /lib/i686/nosegneg/libpthread.so.0 #14 0x00bd9e9e in clone () from /lib/i686/nosegneg/libc.so.6 The crash happens in msgDestruct() when it tries to free pThis->rcvFrom.pfrominet. Valgrind says it's a double free problem. The queue mutex used by DequeueForConsumer seems to be properly locked my thread 8. From stack frame 10: (gdb) p *pThis->mut $261 = {__data = {__lock = 2, __count = 0, __owner = 11149, __kind = 0, __nusers = 1, {__spins = 0, __list = {__next = 0x0}}}, __size = "\002\000\000\000\000\000\000\000\215+\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 2} The value for __lock is curious. It's usually 1 for locked or 0 for unlocked, but it might have something to do with gdb. It's 1 in the core files. pThis->mutThrdMgmt is unlocked. I've checked omruleset code and it does a proper deep copy, as far as I can tell. All the code in msg.c also seems fine. So I don't know what's happening. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr -------------- next part -------------- ################# #### MODULES #### ################# #$ModLoad imuxsock # provides support for local system logging #$ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability #$KLogPath /proc/kmsg # provides UDP syslog reception $ModLoad imudp # provides TCP syslog reception $ModLoad imtcp # Provides reception in another program's stdin $ModLoad omprog # Provides requeuing $ModLoad omruleset ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use traditional timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # DO not filter duplicated messages $RepeatedMsgReduction off # # Set the default permissions for all log files. # $FileOwner kkovac $FileGroup users $FileCreateMode 0644 $DirCreateMode 0755 $Umask 0022 #$PrivDropToUser syslog #$PrivDropToGroup syslog $template message,"%msg%\n" # Write data to omprog $Ruleset outdata $RulesetCreateMainQueue on $WorkDirectory /opt/bulb/var/rsyslog/out $ActionQueueType LinkedList $ActionQueueFileName srvrfwd $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on $ActionOMProgBinary /opt/bulb/bin/avr_parser.sh *.* :omprog:;message # Queue incoming messages $Ruleset indata $RulesetCreateMainQueue on $InputTCPServerBindRuleset indata $InputTCPServerRun 1514 $InputUDPServerBindRuleset indata $UDPServerRun 1514 $ActionQueueType Direct # $WorkDirectory /opt/bulb/var/rsyslog/in # location for work (spool) files # # $ActionQueueType LinkedList # use asynchronous processing # $ActionQueueFileName srvrfwd # set file name, also enables disk mode # $ActionResumeRetryCount -1 # infinite retries on insert failure # $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down $ActionOmrulesetRulesetName outdata local1.*;local2.*;local3.*;local4.*;local5.*;local6.* :omruleset: & ~ $Ruleset RSYSLOG_DefaultRuleset *.* /opt/bulb/var/rsyslog/log $GenerateConfigGraph /home/dave/rsyslog.dot From rgerhards at hq.adiscon.com Fri Nov 12 14:24:07 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Nov 2010 14:24:07 +0100 Subject: [rsyslog] SIGSEGV because of double free in msgDestruct References: <20101112123746.GA26430@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6EB@GRFEXC.intern.adiscon.com> Thanks for the excellent bug report. I have one idea based on a case I had in the past. Can it be that Centos GCC version does not properly support atomic instructions or is set to an to-old version? I have one other case open, and I think it was on centos. If atomics do no work right, nothing works well at all in rsyslog. Note that the build system seems to be unable to correctly detect this, as can be seen in this bug tracker here: http://bugzilla.adiscon.com/show_bug.cgi?id=202 Could you please rebuild it with -march=i586 added to the CFLAGS. If my idea is right, it will probably run with that. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Friday, November 12, 2010 1:38 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] SIGSEGV because of double free in msgDestruct > > Hello. > > I have rsyslog 5.6.0 on CentOS 5.5 with a slightly complex > configuration > and it's crashing. The complete configuration file is attached. The > crash > is perfectly reproducible and it happens very soon after the data > starts > arriving. The program was started with: > > rsyslogd -c5 -x -f rsyslog-datasink.conf > > I have two queues in order to have two thread pools. Input queue just > takes the message from UDP or TCP socket and uses omruleset to pass it > to > the output queue. The output queue then uses omprog to pass the message > to > the external program. Omprog blocks when the pipe to the external > program > is full, so I wanted to have unblocked threads to accept incoming > messages > (which will mostly use UDP). Hence, the configuration has two queues. > > It's possible that I made some error in the configuration and rsyslogd > is > crashing because I'm doing something that I wasn't supposed to do, but > it > didn't detect the faulty configuration early on. > > The whole thing works fine when I have only one queue (created with > $Ruleset) and input and omprog modules on it. But I'd really like to > use > two thread pools. It should be possible to reproduce this problem with > cat > as the omprog binary, although I haven't tried. > > One curiosity (probably unrelated to the problem): $GenerateConfigGraph > at > the end of the config file creates a picture which has only the main > queue, but the queues I configured with $Ruleset directives are not on > the > picture. > > The below is from gdb. The process was started from gdb, so there's no > call to sigsegvHdlr(), which can be seen in the core file when I start > rsyslogd on its own. > > (gdb) info threads > * 8 Thread 0xb4debb90 (LWP 11149) ConsumerReg (pThis=0x80b7988, > pWti=0x80b7cb8) at queue.c:1679 > 7 Thread 0xb57ecb90 (LWP 11148) 0x00d46402 in __kernel_vsyscall () > 6 Thread 0xb61edb90 (LWP 11147) msgDestruct (ppThis=0xb61ed1d4) at > msg.c:790 > 5 Thread 0xb6beeb90 (LWP 11146) 0x00d46402 in __kernel_vsyscall () > 4 Thread 0xb75efb90 (LWP 11145) 0x00d46402 in __kernel_vsyscall () > 3 Thread 0xb7ff0b90 (LWP 11144) 0x00d46402 in __kernel_vsyscall () > 2 Thread 0xb7ff1ac0 (LWP 11111) 0x00d46402 in __kernel_vsyscall () > (gdb) bt > #0 0x00d46402 in __kernel_vsyscall () > #1 0x00b2f040 in raise () from /lib/i686/nosegneg/libc.so.6 > #2 0x00b30a21 in abort () from /lib/i686/nosegneg/libc.so.6 > #3 0x00b67e3b in __libc_message () from /lib/i686/nosegneg/libc.so.6 > #4 0x00b70758 in free () from /lib/i686/nosegneg/libc.so.6 > #5 0x080612ee in msgDestruct (ppThis=0xb4deb1d4) at msg.c:816 > #6 0x08079e35 in DeleteProcessedBatch (pThis=0x80b7988, > pBatch=0x80b7cd0) > at queue.c:1404 > #7 0x0807a3b9 in DequeueConsumableElements (pThis=0x80b7988, > pWti=0x80b7cb8) > at queue.c:1441 > #8 DequeueConsumable (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1489 > #9 0x0807a5d7 in DequeueForConsumer (pThis=0x80b7988, pWti=0x80b7cb8) > at queue.c:1626 > #10 ConsumerReg (pThis=0x80b7988, pWti=0x80b7cb8) at queue.c:1679 > #11 0x0807350e in wtiWorker (pThis=0x80b7cb8) at wti.c:315 > #12 0x08072e1f in wtpWorker (arg=0x80b7cb8) at wtp.c:381 > #13 0x00c9b869 in start_thread () from > /lib/i686/nosegneg/libpthread.so.0 > #14 0x00bd9e9e in clone () from /lib/i686/nosegneg/libc.so.6 > > The crash happens in msgDestruct() when it tries to free > pThis->rcvFrom.pfrominet. Valgrind says it's a double free problem. > > The queue mutex used by DequeueForConsumer seems to be properly locked > my > thread 8. From stack frame 10: > > (gdb) p *pThis->mut > $261 = {__data = {__lock = 2, __count = 0, __owner = 11149, __kind = 0, > __nusers = 1, {__spins = 0, __list = {__next = 0x0}}}, > __size = > "\002\000\000\000\000\000\000\000\215+\000\000\000\000\000\000\001\000\ > 000\000\000\000\000", __align = 2} > > The value for __lock is curious. It's usually 1 for locked or 0 for > unlocked, but it might have something to do with gdb. It's 1 in the > core > files. pThis->mutThrdMgmt is unlocked. > > I've checked omruleset code and it does a proper deep copy, as far as I > can tell. All the code in msg.c also seems fine. So I don't know what's > happening. > > -- > .-. .-. Yes, I am an agent of Satan, but my duties are largely > (_ \ / _) ceremonial. > | > | dave at fly.srk.fer.hr From dave at fly.srk.fer.hr Fri Nov 12 15:18:02 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Fri, 12 Nov 2010 15:18:02 +0100 Subject: [rsyslog] SIGSEGV because of double free in msgDestruct In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6EB@GRFEXC.intern.adiscon.com> References: <20101112123746.GA26430@fly.srk.fer.hr> <9B6E2A8877C38245BFB15CC491A11DA71DD6EB@GRFEXC.intern.adiscon.com> Message-ID: <20101112141801.GA30712@fly.srk.fer.hr> Rainer Gerhards wrote: > Thanks for the excellent bug report. I have one idea based on a case I had in > the past. Can it be that Centos GCC version does not properly support atomic > instructions or is set to an to-old version? That has crossed my mind as well and I've checked generated config.h which contains: /* Define if compiler provides atomic builtins */ /* #undef HAVE_ATOMIC_BUILTINS */ /* Define if compiler provides 64 bit atomic builtins */ /* #undef HAVE_ATOMIC_BUILTINS_64BIT */ So I thought the mutexes were being used and everything was fine with that. > I have one other case open, and > I think it was on centos. If atomics do no work right, nothing works well at > all in rsyslog. You mean that the #ifdef directives which fall back to mutex locking aren't good enough? > Note that the build system seems to be unable to correctly > detect this, as can be seen in this bug tracker here: > > http://bugzilla.adiscon.com/show_bug.cgi?id=202 Well, even if the configure check is not correct, it doesn't exibit false positive in my case. > Could you please rebuild it with -march=i586 added to the CFLAGS. If my idea > is right, it will probably run with that. I've tried and now config.h contains: /* Define if compiler provides atomic builtins */ #define HAVE_ATOMIC_BUILTINS 1 /* Define if compiler provides 64 bit atomic builtins */ #define HAVE_ATOMIC_BUILTINS_64BIT 1 However, it still crashes in the same way. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From tbergfeld at hq.adiscon.com Fri Nov 12 15:34:25 2010 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Fri, 12 Nov 2010 15:34:25 +0100 Subject: [rsyslog] segfault with dynamic filenames (was: Rsyslog evaluationquestions) References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6EC@GRFEXC.intern.adiscon.com> Hi Jason, unfortunately I am still not able to reproduce your issue. (My system: Centos 5.5 rsyslog 5.6.0). Would it be possible that you create a debug log? In the link below you will find some help for creating a debug log http://www.rsyslog.com/doc/troubleshoot.html Just a question, but did you created the folders you mentioned in your templates? (/var/log/HOSTS or /var/log/LOCAL) Rsyslog is not able to create folders, as far as I know. Best regards, Tom -----Urspr?ngliche Nachricht----- Von: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason Antman Gesendet: Donnerstag, 4. November 2010 22:47 An: rsyslog-users Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog evaluationquestions) Thanks to both of you guys for your answers. FYI, our days have 24 hours in them :) At this point I just got a config file worked up. I got the regexes working to do all of the matching I need into separate fields (the online regex tester was a big help), and upgraded to 5.6.0 (CentOS 5.5, in-house RPM build) in order to of bind imudp to a ruleset (so we can have one ruleset for localhost/unix socket, and one ruleset for all remote hosts). Unfortunately, I'm getting a segfault almost immediately on startup due to the following lines: $template RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility -text%.log" *.* ?RemoteHost I believe I've narrowed the problem down to any templated (dynamic) filename... even if I dump everything external directly to /var/log/remote and add the following in my local ruleset: $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" *.* ?LocalHost I still get a segfault. The last line in the output running with -dn is always: "file to log to: RemoteHost" (or LocalHost, when I used that config). Has anyone seen anything like this? I have rsyslog 2.0.6 running at another site and dynamic filenames work fine... sort of troubling that 5.6.0 is having issues with them... Thanks, Jason Antman Aaron Wiebe wrote: >> 3) Assuming an even distribution over time (not quite accurate), any >> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) >> with a single 2.8GHz CPU and 512MB RAM would go? >> > > 2 million lines a day is 70 lines per second assuming an 8 hour day. > MySQL, properly configured, will eat that. Rsyslog won't even notice > that log level. > > Your table sizes may be problematic over time, and your ability to > query may impact things. Regardless, your concern should be mysql, > not rsyslog. > > -Aaron > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Nov 12 17:15:17 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Nov 2010 17:15:17 +0100 Subject: [rsyslog] SIGSEGV because of double free in msgDestruct References: <20101112123746.GA26430@fly.srk.fer.hr><9B6E2A8877C38245BFB15CC491A11DA71DD6EB@GRFEXC.intern.adiscon.com> <20101112141801.GA30712@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6EE@GRFEXC.intern.adiscon.com> OK, so this is not the issue. We are trying to reproduce the issue, I hope that will work thanks to your detailled report. I guess you will hear back on monday. Raienr > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Friday, November 12, 2010 3:18 PM > To: rsyslog-users > Subject: Re: [rsyslog] SIGSEGV because of double free in msgDestruct > > Rainer Gerhards wrote: > > Thanks for the excellent bug report. I have one idea based on a case > I had in > > the past. Can it be that Centos GCC version does not properly support > atomic > > instructions or is set to an to-old version? > > That has crossed my mind as well and I've checked generated config.h > which > contains: > > /* Define if compiler provides atomic builtins */ > /* #undef HAVE_ATOMIC_BUILTINS */ > > /* Define if compiler provides 64 bit atomic builtins */ > /* #undef HAVE_ATOMIC_BUILTINS_64BIT */ > > So I thought the mutexes were being used and everything was fine with > that. > > > I have one other case open, and > > I think it was on centos. If atomics do no work right, nothing works > well at > > all in rsyslog. > > You mean that the #ifdef directives which fall back to mutex locking > aren't good enough? > > > Note that the build system seems to be unable to correctly > > detect this, as can be seen in this bug tracker here: > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=202 > > Well, even if the configure check is not correct, it doesn't exibit > false > positive in my case. > > > Could you please rebuild it with -march=i586 added to the CFLAGS. If > my idea > > is right, it will probably run with that. > > I've tried and now config.h contains: > > /* Define if compiler provides atomic builtins */ > #define HAVE_ATOMIC_BUILTINS 1 > > /* Define if compiler provides 64 bit atomic builtins */ > #define HAVE_ATOMIC_BUILTINS_64BIT 1 > > However, it still crashes in the same way. > > -- > .-. .-. Yes, I am an agent of Satan, but my duties are largely > (_ \ / _) ceremonial. > | > | dave at fly.srk.fer.hr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Nov 12 18:40:45 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Nov 2010 18:40:45 +0100 Subject: [rsyslog] liblognorm has grown fast... Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> Hi all, a bit unexpectedly, I could finish a first very rough proof of concept of the log normalizer. Details are available here: http://blog.gerhards.net/2010/11/log-normalization-first-results.html Rainer From champ at softwink.com Fri Nov 12 19:05:35 2010 From: champ at softwink.com (Champ Clark III [Softwink]) Date: Fri, 12 Nov 2010 13:05:35 -0500 Subject: [rsyslog] liblognorm has grown fast... In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> Message-ID: <20101112180535.GA4009@bundy.vistech.net> On Fri, Nov 12, 2010 at 06:40:45PM +0100, Rainer Gerhards wrote: > Hi all, > > a bit unexpectedly, I could finish a first very rough proof of concept of the > log normalizer. Details are available here: > > http://blog.gerhards.net/2010/11/log-normalization-first-results.html Very cool stuff. Good job on the PoC. After this weekend, I intend to do a bit a 'playing' with it. Woo! -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From joe at joetify.com Sat Nov 13 02:52:58 2010 From: joe at joetify.com (Joe Williams) Date: Fri, 12 Nov 2010 17:52:58 -0800 Subject: [rsyslog] liblognorm has grown fast... In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> Message-ID: <879AC628-81A5-4D93-A1BE-B88CE1D289D2@joetify.com> On Nov 12, 2010, at 9:40 AM, Rainer Gerhards wrote: > Hi all, > > a bit unexpectedly, I could finish a first very rough proof of concept of the > log normalizer. Details are available here: > > http://blog.gerhards.net/2010/11/log-normalization-first-results.html Very cool, I assume it's no mistake that the normalized output is json? If so, perhaps you could make use of the entire json spec (http://www.json.org/). -Joe Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From rgerhards at hq.adiscon.com Sat Nov 13 10:23:47 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 13 Nov 2010 10:23:47 +0100 Subject: [rsyslog] liblognorm has grown fast... References: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> <879AC628-81A5-4D93-A1BE-B88CE1D289D2@joetify.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6F3@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Joe Williams > Sent: Saturday, November 13, 2010 2:53 AM > To: rsyslog-users > Subject: Re: [rsyslog] liblognorm has grown fast... > > > On Nov 12, 2010, at 9:40 AM, Rainer Gerhards wrote: > > > Hi all, > > > > a bit unexpectedly, I could finish a first very rough proof of > concept of the > > log normalizer. Details are available here: > > > > http://blog.gerhards.net/2010/11/log-normalization-first-results.html > > > Very cool, I assume it's no mistake that the normalized output is json? > If so, perhaps you could make use of the entire json spec > (http://www.json.org/). That's "just" one oft he encodings the CEE specifies. The actual output format can be selected in the final version by a switch. At the moment, I support JSON and syslog structured data. I am also curios what you are missing from the JSON spec. Rainer > > -Joe > > > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > Twitter: http://twitter.com/williamsjoe > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From joe at joetify.com Sat Nov 13 10:43:41 2010 From: joe at joetify.com (Joe Williams) Date: Sat, 13 Nov 2010 01:43:41 -0800 Subject: [rsyslog] liblognorm has grown fast... In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6F3@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com> <879AC628-81A5-4D93-A1BE-B88CE1D289D2@joetify.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6F3@GRFEXC.intern.adiscon.com> Message-ID: <4140D432-1395-45B0-AF31-6A67BD9D27D2@joetify.com> > I am also curios what you are missing from the JSON spec. Not sure I follow what you mean. Regardless it looks like you are producing valid JSON, I was just thinking it might be interesting to use arrays, objects, etc to represent log structures. BTW, I really like this idea, it would fit perfectly with my idea of a rsyslog REST output plugin. ;) -Joe Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From rgerhards at hq.adiscon.com Sat Nov 13 11:06:32 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 13 Nov 2010 11:06:32 +0100 Subject: [rsyslog] liblognorm has grown fast... References: <9B6E2A8877C38245BFB15CC491A11DA71DD6F1@GRFEXC.intern.adiscon.com><879AC628-81A5-4D93-A1BE-B88CE1D289D2@joetify.com><9B6E2A8877C38245BFB15CC491A11DA71DD6F3@GRFEXC.intern.adiscon.com> <4140D432-1395-45B0-AF31-6A67BD9D27D2@joetify.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6F5@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Joe Williams > Sent: Saturday, November 13, 2010 10:44 AM > To: rsyslog-users > Subject: Re: [rsyslog] liblognorm has grown fast... > > > > I am also curios what you are missing from the JSON spec. > > Not sure I follow what you mean. Regardless it looks like you are > producing valid JSON, I was just thinking it might be interesting to > use arrays, objects, etc to represent log structures. That's actually a discussion within CEE. So far, it looks we will not have objects within the CEE model (because that introduces a number of subtle issue). Arrays, however, are already supported. Multi-valued attributes (lists) will be encoded in that way. Libee already can do that, but liblognorm is not yet able to generate multi-valued attributes. > BTW, I really like this idea, it would fit perfectly with my idea of a > rsyslog REST output plugin. ;) Good to be persistent :) I think even better would be a XML representation, which will also be supported by CEE and libee. I've just not yet implemented it. Rainer > > -Joe > > > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > Twitter: http://twitter.com/williamsjoe > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From bernard.fay at gmail.com Sun Nov 14 02:47:17 2010 From: bernard.fay at gmail.com (Bernard Fay) Date: Sat, 13 Nov 2010 20:47:17 -0500 Subject: [rsyslog] Empty log RSYSLOG_DebugFormat Message-ID: Hi I added this to my config file : *.* /var/log/rsyslog_debug.log;RSYSLOG_DebugFormat When I start rsyslogd, the log file /var/log/rsyslog_debug.log is created but empty. I checked with lsof and there is not open file under this name. Any idea about what could be the problem? Thanks, Bernard beryllium:/var/log# ls -ltr -rw-r--r-- 1 root root 608492 2010-11-13 20:36 atop.log -rw-r----- 1 root adm 0 2010-11-13 20:39 *rsyslog_debug.log* -rw-r----- 1 root adm 25977 2010-11-13 20:39 messages -rw-r----- 1 root adm 1620 2010-11-13 20:39 kern.log -rw-r----- 1 root adm 45538 2010-11-13 20:45 syslog -rw-r----- 1 root adm 475829 2010-11-13 20:45 auth.log # lsof | grep rsyslog | fgrep /var/log rsyslogd 21679 root 5w REG 8,6 475634 28086 /var/log/auth.log rsyslogd 21679 root 6w REG 8,6 45378 27872 /var/log/syslog rsyslogd 21679 root 7w REG 8,6 14879 28084 /var/log/daemon.log rsyslogd 21679 root 8w REG 8,6 1620 28085 /var/log/kern.log rsyslogd 21679 root 9w REG 8,6 0 27875 /var/log/lpr.log rsyslogd 21679 root 10w REG 8,6 14420 27876 /var/log/mail.log rsyslogd 21679 root 11w REG 8,6 21839 28087 /var/log/user.log rsyslogd 21679 root 12w REG 8,6 14420 27878 /var/log/mail.info rsyslogd 21679 root 13w REG 8,6 2372 27879 /var/log/mail.warn rsyslogd 21679 root 14w REG 8,6 0 28188 /var/log/mail.err rsyslogd 21679 root 15w REG 8,6 0 27882 /var/log/news/news.crit rsyslogd 21679 root 16w REG 8,6 0 27883 /var/log/news/news.err rsyslogd 21679 root 17w REG 8,6 0 27884 /var/log/news/news.notice rsyslogd 21679 root 18w REG 8,6 108 27885 /var/log/debug rsyslogd 21679 root 19w REG 8,6 25977 28089 /var/log/messages From rgerhards at hq.adiscon.com Sun Nov 14 11:05:41 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 14 Nov 2010 11:05:41 +0100 Subject: [rsyslog] Empty log RSYSLOG_DebugFormat References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6FB@GRFEXC.intern.adiscon.com> Maybe your rsyslog version is too old, this canned template is not available in old versions. But then you should have seen an error message on startup (provided you do not throw them away, what unfortunately many distros do by default...). If that's the problem, you can use this template (I have NOT checked it is 100% accurate!): $template debug,"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n" Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Bernard Fay > Sent: Sunday, November 14, 2010 2:47 AM > To: rsyslog-users > Subject: [rsyslog] Empty log RSYSLOG_DebugFormat > > Hi > > I added this to my config file : *.* > /var/log/rsyslog_debug.log;RSYSLOG_DebugFormat > > When I start rsyslogd, the log file /var/log/rsyslog_debug.log is > created > but empty. I checked with lsof and there is not open file under this > name. > > Any idea about what could be the problem? > > Thanks, > Bernard > > > beryllium:/var/log# ls -ltr > -rw-r--r-- 1 root root 608492 2010-11-13 20:36 atop.log > -rw-r----- 1 root adm 0 2010-11-13 20:39 *rsyslog_debug.log* > -rw-r----- 1 root adm 25977 2010-11-13 20:39 messages > -rw-r----- 1 root adm 1620 2010-11-13 20:39 kern.log > -rw-r----- 1 root adm 45538 2010-11-13 20:45 syslog > -rw-r----- 1 root adm 475829 2010-11-13 20:45 auth.log > > > # lsof | grep rsyslog | fgrep /var/log > rsyslogd 21679 root 5w REG 8,6 475634 28086 > /var/log/auth.log > rsyslogd 21679 root 6w REG 8,6 45378 27872 > /var/log/syslog > rsyslogd 21679 root 7w REG 8,6 14879 28084 > /var/log/daemon.log > rsyslogd 21679 root 8w REG 8,6 1620 28085 > /var/log/kern.log > rsyslogd 21679 root 9w REG 8,6 0 27875 > /var/log/lpr.log > rsyslogd 21679 root 10w REG 8,6 14420 27876 > /var/log/mail.log > rsyslogd 21679 root 11w REG 8,6 21839 28087 > /var/log/user.log > rsyslogd 21679 root 12w REG 8,6 14420 27878 > /var/log/mail.info > rsyslogd 21679 root 13w REG 8,6 2372 27879 > /var/log/mail.warn > rsyslogd 21679 root 14w REG 8,6 0 28188 > /var/log/mail.err > rsyslogd 21679 root 15w REG 8,6 0 27882 > /var/log/news/news.crit > rsyslogd 21679 root 16w REG 8,6 0 27883 > /var/log/news/news.err > rsyslogd 21679 root 17w REG 8,6 0 27884 > /var/log/news/news.notice > rsyslogd 21679 root 18w REG 8,6 108 27885 > /var/log/debug > rsyslogd 21679 root 19w REG 8,6 25977 28089 > /var/log/messages > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Sun Nov 14 11:08:11 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 14 Nov 2010 11:08:11 +0100 Subject: [rsyslog] SIGSEGV because of double free in msgDestruct References: <20101112123746.GA26430@fly.srk.fer.hr><9B6E2A8877C38245BFB15CC491A11DA71DD6EB@GRFEXC.intern.adiscon.com> <20101112141801.GA30712@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD6FC@GRFEXC.intern.adiscon.com> > > I have one other case open, and > > I think it was on centos. If atomics do no work right, nothing works > well at > > all in rsyslog. > > You mean that the #ifdef directives which fall back to mutex locking > aren't good enough? Just for clarification: I have talked about problems inside the compiler and its atomic ops support. That it is claimed the compiler has it while it not actually has it. If rsyslog correctly detects atomic ops are not available, it will fall back to the mutex locking, which works correctly. Rainer From bernard.fay at gmail.com Mon Nov 15 00:18:24 2010 From: bernard.fay at gmail.com (Bernard Fay) Date: Sun, 14 Nov 2010 18:18:24 -0500 Subject: [rsyslog] Empty log RSYSLOG_DebugFormat In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6FB@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD6FB@GRFEXC.intern.adiscon.com> Message-ID: That is my version on Debian Lenny 5.0.6: # rsyslogd -v rsyslogd 3.18.6, compiled with: FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes GSSAPI Kerberos 5 support: No FEATURE_DEBUG (debug build, slow code): No Runtime Instrumentation (slow code): No I tried your template and it works. Thanks a lot Rainer! On Sun, Nov 14, 2010 at 5:05 AM, Rainer Gerhards wrote: > Maybe your rsyslog version is too old, this canned template is not > available > in old versions. But then you should have seen an error message on startup > (provided you do not throw them away, what unfortunately many distros do by > default...). > > If that's the problem, you can use this template (I have NOT checked it is > 100% accurate!): > > $template debug,"Debug line with all properties:\nFROMHOST: '%FROMHOST%', > fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: > %PRI%,\nsyslogtag > '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', > PROCID: > '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: > '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: > '%rawmsg%'\n\n" > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Bernard Fay > > Sent: Sunday, November 14, 2010 2:47 AM > > To: rsyslog-users > > Subject: [rsyslog] Empty log RSYSLOG_DebugFormat > > > > Hi > > > > I added this to my config file : *.* > > /var/log/rsyslog_debug.log;RSYSLOG_DebugFormat > > > > When I start rsyslogd, the log file /var/log/rsyslog_debug.log is > > created > > but empty. I checked with lsof and there is not open file under this > > name. > > > > Any idea about what could be the problem? > > > > Thanks, > > Bernard > > > > > > beryllium:/var/log# ls -ltr > > -rw-r--r-- 1 root root 608492 2010-11-13 20:36 atop.log > > -rw-r----- 1 root adm 0 2010-11-13 20:39 *rsyslog_debug.log* > > -rw-r----- 1 root adm 25977 2010-11-13 20:39 messages > > -rw-r----- 1 root adm 1620 2010-11-13 20:39 kern.log > > -rw-r----- 1 root adm 45538 2010-11-13 20:45 syslog > > -rw-r----- 1 root adm 475829 2010-11-13 20:45 auth.log > > > > > > # lsof | grep rsyslog | fgrep /var/log > > rsyslogd 21679 root 5w REG 8,6 475634 28086 > > /var/log/auth.log > > rsyslogd 21679 root 6w REG 8,6 45378 27872 > > /var/log/syslog > > rsyslogd 21679 root 7w REG 8,6 14879 28084 > > /var/log/daemon.log > > rsyslogd 21679 root 8w REG 8,6 1620 28085 > > /var/log/kern.log > > rsyslogd 21679 root 9w REG 8,6 0 27875 > > /var/log/lpr.log > > rsyslogd 21679 root 10w REG 8,6 14420 27876 > > /var/log/mail.log > > rsyslogd 21679 root 11w REG 8,6 21839 28087 > > /var/log/user.log > > rsyslogd 21679 root 12w REG 8,6 14420 27878 > > /var/log/mail.info > > rsyslogd 21679 root 13w REG 8,6 2372 27879 > > /var/log/mail.warn > > rsyslogd 21679 root 14w REG 8,6 0 28188 > > /var/log/mail.err > > rsyslogd 21679 root 15w REG 8,6 0 27882 > > /var/log/news/news.crit > > rsyslogd 21679 root 16w REG 8,6 0 27883 > > /var/log/news/news.err > > rsyslogd 21679 root 17w REG 8,6 0 27884 > > /var/log/news/news.notice > > rsyslogd 21679 root 18w REG 8,6 108 27885 > > /var/log/debug > > rsyslogd 21679 root 19w REG 8,6 25977 28089 > > /var/log/messages > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From dave at fly.srk.fer.hr Mon Nov 15 13:59:32 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Mon, 15 Nov 2010 13:59:32 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO Message-ID: <20101115125932.GA13723@fly.srk.fer.hr> Hello. I have rsyslogd 5.6.0 on CentOS 5.5. I've tried to send some of its output to FIFO and it doesn't work quite the way I'd like it to. It seems I can reproduce the problem only when the FIFO reader is started after rsyslogd, so the reader's end of FIFO is not opened yet. It should be possible to reproduce the problem regardless, but it didn't exibit itself when I tried in that sequence (which was just once). What happens is that rsyslogd tries to open the FIFO in non-blocking mode, doesn't succeed and waits for notification that the reader has opened its end. So far, so good. Then the reader starts, opens the reading end, so rsyslogd opens the writing end and starts sending data. That's fine as well. Then rsyslogd starts getting EGAIN. I suppose that's because the rate of incoming data is high enough to fill the kernel FIFO buffer before the reader program gets the CPU time. The reader has no problem with keeping up with the data rate in other configurations I've tried, but the FIFO buffer in kernel is relatively small, writing to it won't block the writing process untill the FIFO bufer is full, so it can use it's whole CPU timeslot to fill the buffer and the reader's processing speed doesn't matter. I'm seeing a large number of write() attempts by rsyslogd (all failing with EAGAIN) and eventually it stops trying to write and goes to sleep. strace shows: 11109 gettimeofday({1289467839, 591385}, NULL) = 0 11109 select(0, NULL, NULL, NULL, {30, 0} That is a pure 30 second pause which doesn't try to detect the state on file descriptors. The reader manages to emty the pipe and then, after 30 seconds, rsyslogd starts writing again. It writes several lines of data and then the whole process repeats. The net result is that the data transfer rate is extremly slow and the system can't nearly keep up with the incoming data because rsyslogd spends most of its time in that 30 second sleep in select(). I can see two ways out of that: 1. Instead of going to sleep with hard-coded time-out value, go to sleep on select/poll/epoll/whatever that wakes up immediately after the FIFO becomes writable. 2. Turn the FIFO file descriptor to blocking mode right after successful open. This should be a lot easier to implement than the first option and it's in-line with the way some other output modules behave (omprog has blocking writes, for example). I do not know if some part of the code expects FIFO file descriptor to be in the non-blocking mode, though. Blocking writes should enable me to use two queues and buffer the incoming data on disk for (transient) cases when the consumer can't empty the pipe fast enough. The configuration with the two queues doesn't work for me at the moment because of a crash I reported in another mail, but I don't think that's related to this problem. Once that is fixed, FIFOs should become usable. As it is, I can't use FIFO output at all, even with only one queue. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From rgerhards at hq.adiscon.com Mon Nov 15 14:29:13 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 15 Nov 2010 14:29:13 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO References: <20101115125932.GA13723@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD70B@GRFEXC.intern.adiscon.com> I am not sure there is any simple solution to this (without writing a new module). A core problem is that the pipe interface is being abused in a lot of default configs to write to a pipe that usually fails, for xconsole. So rsyslog must handle pipes in a way that does not thrash performance on these default configs. I think we will probably see a real failure case if we look at a debug log. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Monday, November 15, 2010 2:00 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Huge pauses when writing to FIFO > > Hello. > > I have rsyslogd 5.6.0 on CentOS 5.5. > > I've tried to send some of its output to FIFO and it doesn't work quite > the way I'd like it to. It seems I can reproduce the problem only when > the > FIFO reader is started after rsyslogd, so the reader's end of FIFO is > not > opened yet. It should be possible to reproduce the problem regardless, > but > it didn't exibit itself when I tried in that sequence (which was just > once). > > What happens is that rsyslogd tries to open the FIFO in non-blocking > mode, > doesn't succeed and waits for notification that the reader has opened > its > end. So far, so good. Then the reader starts, opens the reading end, > so rsyslogd opens the writing end and starts sending data. That's fine > as > well. > > Then rsyslogd starts getting EGAIN. I suppose that's because the rate > of > incoming data is high enough to fill the kernel FIFO buffer before the > reader program gets the CPU time. The reader has no problem with > keeping > up with the data rate in other configurations I've tried, but the FIFO > buffer in kernel is relatively small, writing to it won't block the > writing process untill the FIFO bufer is full, so it can use it's whole > CPU timeslot to fill the buffer and the reader's processing speed > doesn't > matter. > > I'm seeing a large number of write() attempts by rsyslogd (all failing > with EAGAIN) and eventually it stops trying to write and goes to sleep. > strace shows: > > 11109 gettimeofday({1289467839, 591385}, NULL) = 0 > 11109 select(0, NULL, NULL, NULL, {30, 0} > > That is a pure 30 second pause which doesn't try to detect the state on > file descriptors. > > The reader manages to emty the pipe and then, after 30 seconds, > rsyslogd > starts writing again. It writes several lines of data and then the > whole > process repeats. The net result is that the data transfer rate is > extremly > slow and the system can't nearly keep up with the incoming data because > rsyslogd spends most of its time in that 30 second sleep in select(). > > I can see two ways out of that: > > 1. Instead of going to sleep with hard-coded time-out value, go to > sleep > on select/poll/epoll/whatever that wakes up immediately after the > FIFO > becomes writable. > > 2. Turn the FIFO file descriptor to blocking mode right after > successful > open. This should be a lot easier to implement than the first option > and it's in-line with the way some other output modules behave > (omprog > has blocking writes, for example). I do not know if some part of the > code expects FIFO file descriptor to be in the non-blocking mode, > though. > > Blocking writes should enable me to use two queues and buffer the > incoming > data on disk for (transient) cases when the consumer can't empty the > pipe > fast enough. The configuration with the two queues doesn't work for me > at the moment because of a crash I reported in another mail, but I > don't > think that's related to this problem. Once that is fixed, FIFOs should > become usable. > > As it is, I can't use FIFO output at all, even with only one queue. > > -- > .-. .-. Yes, I am an agent of Satan, but my duties are largely > (_ \ / _) ceremonial. > | > | dave at fly.srk.fer.hr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Nov 15 14:45:05 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 15 Nov 2010 14:45:05 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO References: <20101115125932.GA13723@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD70C@GRFEXC.intern.adiscon.com> OK, I have just checked the source code. Probably the most simple and quick solution would be to add a config directive that tells ompipe to run in blocking mode. That will work well, except that the pathetic xconsole default config case will then run into troubles again. But on the other hand, if you enable blocking mode, you can also remove that pipe from the config. Should I add this option? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Monday, November 15, 2010 2:00 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Huge pauses when writing to FIFO > > Hello. > > I have rsyslogd 5.6.0 on CentOS 5.5. > > I've tried to send some of its output to FIFO and it doesn't work quite > the way I'd like it to. It seems I can reproduce the problem only when > the > FIFO reader is started after rsyslogd, so the reader's end of FIFO is > not > opened yet. It should be possible to reproduce the problem regardless, > but > it didn't exibit itself when I tried in that sequence (which was just > once). > > What happens is that rsyslogd tries to open the FIFO in non-blocking > mode, > doesn't succeed and waits for notification that the reader has opened > its > end. So far, so good. Then the reader starts, opens the reading end, > so rsyslogd opens the writing end and starts sending data. That's fine > as > well. > > Then rsyslogd starts getting EGAIN. I suppose that's because the rate > of > incoming data is high enough to fill the kernel FIFO buffer before the > reader program gets the CPU time. The reader has no problem with > keeping > up with the data rate in other configurations I've tried, but the FIFO > buffer in kernel is relatively small, writing to it won't block the > writing process untill the FIFO bufer is full, so it can use it's whole > CPU timeslot to fill the buffer and the reader's processing speed > doesn't > matter. > > I'm seeing a large number of write() attempts by rsyslogd (all failing > with EAGAIN) and eventually it stops trying to write and goes to sleep. > strace shows: > > 11109 gettimeofday({1289467839, 591385}, NULL) = 0 > 11109 select(0, NULL, NULL, NULL, {30, 0} > > That is a pure 30 second pause which doesn't try to detect the state on > file descriptors. > > The reader manages to emty the pipe and then, after 30 seconds, > rsyslogd > starts writing again. It writes several lines of data and then the > whole > process repeats. The net result is that the data transfer rate is > extremly > slow and the system can't nearly keep up with the incoming data because > rsyslogd spends most of its time in that 30 second sleep in select(). > > I can see two ways out of that: > > 1. Instead of going to sleep with hard-coded time-out value, go to > sleep > on select/poll/epoll/whatever that wakes up immediately after the > FIFO > becomes writable. > > 2. Turn the FIFO file descriptor to blocking mode right after > successful > open. This should be a lot easier to implement than the first option > and it's in-line with the way some other output modules behave > (omprog > has blocking writes, for example). I do not know if some part of the > code expects FIFO file descriptor to be in the non-blocking mode, > though. > > Blocking writes should enable me to use two queues and buffer the > incoming > data on disk for (transient) cases when the consumer can't empty the > pipe > fast enough. The configuration with the two queues doesn't work for me > at the moment because of a crash I reported in another mail, but I > don't > think that's related to this problem. Once that is fixed, FIFOs should > become usable. > > As it is, I can't use FIFO output at all, even with only one queue. > > -- > .-. .-. Yes, I am an agent of Satan, but my duties are largely > (_ \ / _) ceremonial. > | > | dave at fly.srk.fer.hr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From dave at fly.srk.fer.hr Mon Nov 15 15:54:11 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Mon, 15 Nov 2010 15:54:11 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD70C@GRFEXC.intern.adiscon.com> References: <20101115125932.GA13723@fly.srk.fer.hr> <9B6E2A8877C38245BFB15CC491A11DA71DD70C@GRFEXC.intern.adiscon.com> Message-ID: <20101115145411.GA21726@fly.srk.fer.hr> Configuration option would be perfect for my purposes. I'd just need a patch unless you intend to release a new version in the next few days. Thanks. Rainer Gerhards wrote: > OK, I have just checked the source code. Probably the most simple and quick > solution would be to add a config directive that tells ompipe to run in > blocking mode. That will work well, except that the pathetic xconsole default > config case will then run into troubles again. But on the other hand, if you > enable blocking mode, you can also remove that pipe from the config. > > Should I add this option? > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > > Sent: Monday, November 15, 2010 2:00 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] Huge pauses when writing to FIFO > > > > Hello. > > > > I have rsyslogd 5.6.0 on CentOS 5.5. > > > > I've tried to send some of its output to FIFO and it doesn't work quite > > the way I'd like it to. It seems I can reproduce the problem only when > > the > > FIFO reader is started after rsyslogd, so the reader's end of FIFO is > > not > > opened yet. It should be possible to reproduce the problem regardless, > > but > > it didn't exibit itself when I tried in that sequence (which was just > > once). > > > > What happens is that rsyslogd tries to open the FIFO in non-blocking > > mode, > > doesn't succeed and waits for notification that the reader has opened > > its > > end. So far, so good. Then the reader starts, opens the reading end, > > so rsyslogd opens the writing end and starts sending data. That's fine > > as > > well. > > > > Then rsyslogd starts getting EGAIN. I suppose that's because the rate > > of > > incoming data is high enough to fill the kernel FIFO buffer before the > > reader program gets the CPU time. The reader has no problem with > > keeping > > up with the data rate in other configurations I've tried, but the FIFO > > buffer in kernel is relatively small, writing to it won't block the > > writing process untill the FIFO bufer is full, so it can use it's whole > > CPU timeslot to fill the buffer and the reader's processing speed > > doesn't > > matter. > > > > I'm seeing a large number of write() attempts by rsyslogd (all failing > > with EAGAIN) and eventually it stops trying to write and goes to sleep. > > strace shows: > > > > 11109 gettimeofday({1289467839, 591385}, NULL) = 0 > > 11109 select(0, NULL, NULL, NULL, {30, 0} > > > > That is a pure 30 second pause which doesn't try to detect the state on > > file descriptors. > > > > The reader manages to emty the pipe and then, after 30 seconds, > > rsyslogd > > starts writing again. It writes several lines of data and then the > > whole > > process repeats. The net result is that the data transfer rate is > > extremly > > slow and the system can't nearly keep up with the incoming data because > > rsyslogd spends most of its time in that 30 second sleep in select(). > > > > I can see two ways out of that: > > > > 1. Instead of going to sleep with hard-coded time-out value, go to > > sleep > > on select/poll/epoll/whatever that wakes up immediately after the > > FIFO > > becomes writable. > > > > 2. Turn the FIFO file descriptor to blocking mode right after > > successful > > open. This should be a lot easier to implement than the first option > > and it's in-line with the way some other output modules behave > > (omprog > > has blocking writes, for example). I do not know if some part of the > > code expects FIFO file descriptor to be in the non-blocking mode, > > though. > > > > Blocking writes should enable me to use two queues and buffer the > > incoming > > data on disk for (transient) cases when the consumer can't empty the > > pipe > > fast enough. The configuration with the two queues doesn't work for me > > at the moment because of a crash I reported in another mail, but I > > don't > > think that's related to this problem. Once that is fixed, FIFOs should > > become usable. > > > > As it is, I can't use FIFO output at all, even with only one queue. > > > > -- > > .-. .-. Yes, I am an agent of Satan, but my duties are largely > > (_ \ / _) ceremonial. > > | > > | dave at fly.srk.fer.hr > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From friedl at hq.adiscon.com Mon Nov 15 16:19:26 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 15 Nov 2010 16:19:26 +0100 Subject: [rsyslog] Installation directories for rsyslog Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD710@GRFEXC.intern.adiscon.com> Dear Mailinglist-Users. I am contacting you, because we are in need of your help. This is nothing big, just some information gathering. I am currently making a list of installation directories for rsyslog. Perhaps I mean this wrong, so please forgive me. What I mean, is the different directories you usually need to add in the configure command. E.g. ./configure --libdir=/lib --sbindir=/sbin In this example, this works on Fedora. But what about other distributions? We are really appreciating your help. Best regards, Florian From rgerhards at hq.adiscon.com Mon Nov 15 17:46:12 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 15 Nov 2010 17:46:12 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <20101115145411.GA21726@fly.srk.fer.hr> References: <20101115125932.GA13723@fly.srk.fer.hr><9B6E2A8877C38245BFB15CC491A11DA71DD70C@GRFEXC.intern.adiscon.com> <20101115145411.GA21726@fly.srk.fer.hr> Message-ID: <4CE163D4.4070001@hq.adiscon.com> On 11/15/2010 03:54 PM, Dra?en Ka?ar wrote: > Configuration option would be perfect for my purposes. I'd just need a > patch unless you intend to release a new version in the next few days. I have attached a patch that *could* work. Unfortunately, I currently do not have the time to really test it out. So I'd appreciate if you could give it a try and report back your results. As a unrelated side-note, I have set up CentOS 5.5 in a VM and tried to compile rsyslog from git there. The autotools version seems to be very old. Is there any way to get 2.61 on that platform without building from source (I want to be as close to a standard system as possible and so I'd like not to build too many different things...). Rainer -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ompipeblockingmode.patch URL: From champ at softwink.com Mon Nov 15 18:52:22 2010 From: champ at softwink.com (Champ Clark III [Softwink]) Date: Mon, 15 Nov 2010 12:52:22 -0500 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <4CE163D4.4070001@hq.adiscon.com> References: <20101115125932.GA13723@fly.srk.fer.hr> <9B6E2A8877C38245BFB15CC491A11DA71DD70C@GRFEXC.intern.adiscon.com> <20101115145411.GA21726@fly.srk.fer.hr> <4CE163D4.4070001@hq.adiscon.com> Message-ID: <20101115175222.GA26751@bundy.vistech.net> > As a unrelated side-note, I have set up CentOS 5.5 in a VM and tried to > compile rsyslog from git there. The autotools version seems to be very > old. Is there any way to get 2.61 on that platform without building from > source (I want to be as close to a standard system as possible and so > I'd like not to build too many different things...). Thanks for the info. I have a user having issues with CentOS 5.5 & compiling Sagan. I suspect it's the same issue, but haven't had a chance to load up CentOS 5.5 in a VM. Please, let me know what you find & what the "fix" is. Like you, I'd like to keep the install to a "standard system" as possible. -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From taro4cycle at gmail.com Mon Nov 15 23:03:59 2010 From: taro4cycle at gmail.com (Taro F.) Date: Mon, 15 Nov 2010 14:03:59 -0800 Subject: [rsyslog] EscapeControlCharactersOnRecieve problem Message-ID: I have a problem with EscapeControlCharactersOnRecieve being on. When it's on (which it is by default) I get a control character \012 at the end of the lines, which I don't want. When it's off, I get a line that just has ';'. I don't want \012 and I don't want ';' clogging up my log files. syslog doesn't seem to have this problem. I am using rsyslog-3.22.1, part of RedHat Linux. Thanks, Taro From dave at fly.srk.fer.hr Tue Nov 16 11:57:45 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Tue, 16 Nov 2010 11:57:45 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <20101115175222.GA26751@bundy.vistech.net> References: <20101115125932.GA13723@fly.srk.fer.hr> <9B6E2A8877C38245BFB15CC491A11DA71DD70C@GRFEXC.intern.adiscon.com> <20101115145411.GA21726@fly.srk.fer.hr> <4CE163D4.4070001@hq.adiscon.com> <20101115175222.GA26751@bundy.vistech.net> Message-ID: <20101116105745.GA26159@fly.srk.fer.hr> Champ Clark III [Softwink] wrote: > Thanks for the info. I have a user having issues with CentOS > 5.5 & compiling Sagan. I suspect it's the same issue, but haven't had > a chance to load up CentOS 5.5 in a VM. Please, let me know what you > find & what the "fix" is. Like you, I'd like to keep the install > to a "standard system" as possible. I'm attaching a patch you need to compile rsyslogd 5.6.0 on CentOS 5.5. The configure script has a check for epoll_create1(), but there's #ifdef missing at one place. I'm not sure if the constant 10 in epoll_create(10) is the correct substitution. The man page says the value is ignored since kernel 2.6.8, so any value is fine for CentOS 5.5. I don't know what's appropriate for older kernels. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr -------------- next part -------------- A non-text attachment was scrubbed... Name: rsyslog-5.6.0-epoll_create1.diff Type: text/x-diff Size: 490 bytes Desc: not available URL: From dave at fly.srk.fer.hr Tue Nov 16 13:56:45 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Tue, 16 Nov 2010 13:56:45 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <4CE163D4.4070001@hq.adiscon.com> References: <20101115145411.GA21726@fly.srk.fer.hr> <4CE163D4.4070001@hq.adiscon.com> Message-ID: <20101116125645.GB26159@fly.srk.fer.hr> Rainer Gerhards wrote: > On 11/15/2010 03:54 PM, Dra?en Ka?ar wrote: >> Configuration option would be perfect for my purposes. I'd just need a >> patch unless you intend to release a new version in the next few days. > > I have attached a patch that *could* work. Unfortunately, I currently do > not have the time to really test it out. So I'd appreciate if you could > give it a try and report back your results. It doesn't work for me. In preparePipe() you forgot to use the variable mode in the subsequent open(). I've fixed that, but it still doesn't work. Linux lsof can't show file descriptor flags and I don't know any other way to get the O_NONBLOCK flag from the kernel, so I'm inferring from the behaviour. strace still shows 30 second sleep in select(). I suppose that the code from runtime/stream.c is being used and it has unconditional non-blocking mode for STREAMTYPE_NAMED_PIPE. I don't know enough about architecture to be able to write a correct patch for that. I'm not even sure I'm using ompipe module. I don't load it explicitely in the config file, version 5.6.0 doesn't have documentation for it, so I wasn't even aware it existed before I've seen your patch. In the configuration file I'm using FIFO like this: local1.* |/opt/foo/my_named_pipe And that seems to use the code from stream.c. I'm not aware of any other way to use a FIFO as a destination. > As a unrelated side-note, I have set up CentOS 5.5 in a VM and tried to > compile rsyslog from git there. The autotools version seems to be very > old. Is there any way to get 2.61 on that platform without building from > source (I want to be as close to a standard system as possible and so > I'd like not to build too many different things...). I'm not aware of a repository with the newer autotools packages. I didn't need them so far, so I wasn't looking. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From marc.schiffbauer at mightycare.de Tue Nov 16 14:17:54 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Tue, 16 Nov 2010 14:17:54 +0100 (CET) Subject: [rsyslog] Handle unsuccessful SQL statements In-Reply-To: <1842827877.3551.1289913403531.JavaMail.root@zimbra> Message-ID: <929020446.3553.1289913474420.JavaMail.root@zimbra> Hi all, Can rsyslog distinguish between "DB-Service not available" and single "SQL Statement failed" conditions? Background is that I want rsyslog to to retry for a infinite numer of times if the postgresql is down but I do not want it to DOS the PG-server until the end of time if a single SQL statements fails. Is this possible? TIA -Marc From rgerhards at hq.adiscon.com Tue Nov 16 14:24:12 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 16 Nov 2010 14:24:12 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO References: <20101115145411.GA21726@fly.srk.fer.hr><4CE163D4.4070001@hq.adiscon.com> <20101116125645.GB26159@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD726@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Tuesday, November 16, 2010 1:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] Huge pauses when writing to FIFO > > Rainer Gerhards wrote: > > On 11/15/2010 03:54 PM, Dra?en Ka?ar wrote: > >> Configuration option would be perfect for my purposes. I'd just need > a > >> patch unless you intend to release a new version in the next few > days. > > > > I have attached a patch that *could* work. Unfortunately, I currently > do > > not have the time to really test it out. So I'd appreciate if you > could > > give it a try and report back your results. > > It doesn't work for me. In preparePipe() you forgot to use the variable > mode in the subsequent open(). I've fixed that, but it still doesn't > work. I think I forgot to mention an important condition: by default, ompipe works in nonblocking mode. So you need to set it to blocking mode for the action in question. This is done via $OMPipeBlockingMode on in front of the action, like here: $OMPipeBlockingMode on local1.* |/opt/foo/my_named_pipe > Linux lsof can't show file descriptor flags and I don't know any other > way > to get the O_NONBLOCK flag from the kernel, so I'm inferring from the > behaviour. strace still shows 30 second sleep in select(). > > I suppose that the code from runtime/stream.c is being used no > and it has > unconditional non-blocking mode for STREAMTYPE_NAMED_PIPE. I don't know > enough about architecture to be able to write a correct patch for that. > > I'm not even sure I'm using ompipe module. I don't load it explicitely > in > the config file, version 5.6.0 doesn't have documentation for it, so I > wasn't even aware it existed before I've seen your patch. It is a built-in module that does not need to be loaded. There is a small set of core outputs that are always present, like omfwd, omfile and ompipe. There is no module-specific documentation for their functionality, because it is documented with the core functionality. > > In the configuration file I'm using FIFO like this: > > local1.* |/opt/foo/my_named_pipe > > And that seems to use the code from stream.c. I'm not aware of any > other > way to use a FIFO as a destination. you have probably looked into omfile, but since recently ompipe was extracted from it. The reason were problems with that xconsole pipe. > > > As a unrelated side-note, I have set up CentOS 5.5 in a VM and tried > to > > compile rsyslog from git there. The autotools version seems to be > very > > old. Is there any way to get 2.61 on that platform without building > from > > source (I want to be as close to a standard system as possible and so > > I'd like not to build too many different things...). > > I'm not aware of a repository with the newer autotools packages. I > didn't need them so far, so I wasn't looking. OK, thanks, so I probably need to dump the idea of working with a "normal" git environment there and will developed based on a release tarball. Rainer From dave at fly.srk.fer.hr Tue Nov 16 15:34:04 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Tue, 16 Nov 2010 15:34:04 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD726@GRFEXC.intern.adiscon.com> References: <20101116125645.GB26159@fly.srk.fer.hr> <9B6E2A8877C38245BFB15CC491A11DA71DD726@GRFEXC.intern.adiscon.com> Message-ID: <20101116143404.GA1167@fly.srk.fer.hr> Rainer Gerhards wrote: > I think I forgot to mention an important condition: by default, ompipe works > in nonblocking mode. So you need to set it to blocking mode for the action in > question. This is done via > > $OMPipeBlockingMode on I added that, just forgor to write it in the previous mail. The syntax check passed ok and there are no errors generated from rsyslogd. > in front of the action, like here: > $OMPipeBlockingMode on > local1.* |/opt/foo/my_named_pipe I still have 30 second sleep in select() and there is a number of write() calls to FIFO (checked with lsof) before that which fail with EAGAIN. The output of "strace -f -e trace=file rsyslogd ..." shows: 25169 open("/opt/bulb/var/rsyslog/to_parser", O_RDWR|O_NONBLOCK|O_LARGEFILE) = 5 Then I negated the condition in preparePipe() to: if(!pData->bBlocking) mode |= O_NONBLOCK; and now strace doesn't show O_NONBLOCK and there is no sleep in select(). I'll run more comprehensive tests just to be sure, but this looks ok. > > I'm not aware of a repository with the newer autotools packages. I > > didn't need them so far, so I wasn't looking. > > OK, thanks, so I probably need to dump the idea of working with a "normal" > git environment there and will developed based on a release tarball. There is rpmfind.net which will probably find the packages you want, maybe even for a Linux distribution you want it for. But the quality of the packages isn't always as good as it could be, so I tend to stay away from random sources for production systems. I'd try it for the development purposes, which is what you need. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From david at lang.hm Tue Nov 16 19:21:00 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 16 Nov 2010 10:21:00 -0800 (PST) Subject: [rsyslog] Handle unsuccessful SQL statements In-Reply-To: <929020446.3553.1289913474420.JavaMail.root@zimbra> References: <929020446.3553.1289913474420.JavaMail.root@zimbra> Message-ID: On Tue, 16 Nov 2010, Marc Schiffbauer wrote: > Hi all, > > Can rsyslog distinguish between "DB-Service not available" and single "SQL Statement failed" conditions? > > Background is that I want rsyslog to to retry for a infinite numer of times if the postgresql is down > but I do not want it to DOS the PG-server until the end of time if a single SQL statements fails. > > Is this possible? it is possible in theory at least. When we were developing the new batched (vector) interface for database plugins this was one of the issues that we discussed. This is implemented in the om database module because it can tell the difference between the two types of errors. unfortunantly I don't remember the details of making this work, but I think that if you set rsyslog to retry forever, but it the om module will not keep retrying a bad statement. I would suggest giving it a try. setup a output format that does not auto-quote the parameters and then try sending 'good' log messages through to verify things are working, then send a 'bad' log message through (say something with a quote in it) and see if that jams things up or if it gets dropped (hopefully with an error being logged by rsyslgo) and then following log messages go through. David Lang From joe at joetify.com Tue Nov 16 23:27:22 2010 From: joe at joetify.com (Joe Williams) Date: Tue, 16 Nov 2010 14:27:22 -0800 Subject: [rsyslog] hostnames Message-ID: <69D56D0F-9118-4185-BAB2-288C553CD1AB@joetify.com> List, I am trying to standardize the hostnames that we see in our logs. It seems that the services (haproxy, etc) that log directly to the rsyslog server do this differently. Here's an example: ./ec2-.us-west-1.compute.amazonaws.com/haproxy ./domu-/haproxy ./domu-/haproxy ./domu-/haproxy ./ip-.ec2.internal/haproxy All the standard logs (user.log, syslog, messages, etc) all use a standard format like: ./domu-/syslog ./ip-/syslog As you see like in the case of ip-, ".ec2.internal" gets appended on with haproxy. In the case of ec2-.us-west-1.compute.amazonaws.com is actually coming from a host with a hostname like ip-. Hopefully this makes sense. Across the board I am using %hostname:::lowercase% to create the directories. As a test to see what the application sees vs rsyslog we added code to log the hostname in one of our applications: ec2-.us-west-1.compute.amazonaws.com/ellison:Nov 16 20:34:22 ec2-.us-west-1.compute.amazonaws.com local3: 2010-11-16 20:34:22,123 INFO [main] ejje.Ejje - Address ip- As you can see rsyslog is logging this with the "ec2-" style hostname but what the application is seeing for the hostname is the "ip-" style. Interestingly the hostname style that rsyslog sees seems to depend on what EC2 availability zone the node is in. "ec2-" hostnames are only in the us-west-1 and ap-southeast-1 zones, "ip-*.ec2.internal" and "domu-*" hostnames in us-east-1. So I think this would suggest something with DNS configurations in different zones. Additionally the "ec2-" style hostnames are actually public hostnames that aren't assigned to the machines but to a MIP or VIP, which again suggests some sort of DNS lookup. I have tried using %fromhost% with the same results. Any thoughts on what might be going on and how to fix it? Thanks. -Joe Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From david at lang.hm Tue Nov 16 23:39:21 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 16 Nov 2010 14:39:21 -0800 (PST) Subject: [rsyslog] hostnames In-Reply-To: <69D56D0F-9118-4185-BAB2-288C553CD1AB@joetify.com> References: <69D56D0F-9118-4185-BAB2-288C553CD1AB@joetify.com> Message-ID: On Tue, 16 Nov 2010, Joe Williams wrote: > List, > > I am trying to standardize the hostnames that we see in our logs. It seems that the services (haproxy, etc) that log directly to the rsyslog server do this differently. Here's an example: > > ./ec2-.us-west-1.compute.amazonaws.com/haproxy > ./domu-/haproxy > ./domu-/haproxy > ./domu-/haproxy > ./ip-.ec2.internal/haproxy > > All the standard logs (user.log, syslog, messages, etc) all use a standard format like: > > ./domu-/syslog > ./ip-/syslog > > As you see like in the case of ip-, ".ec2.internal" gets appended on with haproxy. In the case of ec2-.us-west-1.compute.amazonaws.com is actually coming from a host with a hostname like ip-. Hopefully this makes sense. > > Across the board I am using %hostname:::lowercase% to create the directories. As a test to see what the application sees vs rsyslog we added code to log the hostname in one of our applications: > > ec2-.us-west-1.compute.amazonaws.com/ellison:Nov 16 20:34:22 ec2-.us-west-1.compute.amazonaws.com local3: 2010-11-16 20:34:22,123 INFO [main] ejje.Ejje - Address ip- > > As you can see rsyslog is logging this with the "ec2-" style hostname but what the application is seeing for the hostname is the "ip-" style. > > Interestingly the hostname style that rsyslog sees seems to depend on what EC2 availability zone the node is in. "ec2-" hostnames are only in the us-west-1 and ap-southeast-1 zones, "ip-*.ec2.internal" and "domu-*" hostnames in us-east-1. So I think this would suggest something with DNS configurations in different zones. Additionally the "ec2-" style hostnames are actually public hostnames that aren't assigned to the machines but to a MIP or VIP, which again suggests some sort of DNS lookup. > > I have tried using %fromhost% with the same results. Any thoughts on what might be going on and how to fix it? fromhost is the DNS lookup of the IP address of the machine that last touched the logs. if the sending host set hostname in it's logs, then hostname is that value. If the sending host did not put something that looks like a hostname in the log messages, the first instance of rsyslog that receives the message fills the hostname field with fromhost. it sounds as if your sending systems are not setting the hostname in the logs, so rsyslog is filling in the fromhost. If you setup /etc/hosts entries for the IP addresses of these machines with a short name first, I believe that rsyslog will use that as the result of the name lookup. The better option is to go to the sending machines and figure out why they aren't putting hostname in their outbound logs. David Lang From rgerhards at hq.adiscon.com Wed Nov 17 09:47:10 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 09:47:10 +0100 Subject: [rsyslog] hostnames References: <69D56D0F-9118-4185-BAB2-288C553CD1AB@joetify.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD736@GRFEXC.intern.adiscon.com> And if nothing else helps, you can use the property replacer to e.g. use a regexp to dig out the part that you are interested in. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, November 16, 2010 11:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] hostnames > > On Tue, 16 Nov 2010, Joe Williams wrote: > > > List, > > > > I am trying to standardize the hostnames that we see in our logs. It > seems that the services (haproxy, etc) that log directly to the rsyslog > server do this differently. Here's an example: > > > > ./ec2-.us-west-1.compute.amazonaws.com/haproxy > > ./domu-/haproxy > > ./domu-/haproxy > > ./domu-/haproxy > > ./ip-.ec2.internal/haproxy > > > > All the standard logs (user.log, syslog, messages, etc) all use a > standard format like: > > > > ./domu-/syslog > > ./ip-/syslog > > > > As you see like in the case of ip-, ".ec2.internal" gets > appended on with haproxy. In the case of ec2-.us-west- > 1.compute.amazonaws.com is actually coming from a host with a hostname > like ip-. Hopefully this makes sense. > > > > Across the board I am using %hostname:::lowercase% to create the > directories. As a test to see what the application sees vs rsyslog we > added code to log the hostname in one of our applications: > > > > ec2-.us-west-1.compute.amazonaws.com/ellison:Nov 16 20:34:22 > ec2-.us-west-1.compute.amazonaws.com local3: 2010-11-16 > 20:34:22,123 INFO [main] ejje.Ejje - Address ip- > > > > As you can see rsyslog is logging this with the "ec2-" style hostname > but what the application is seeing for the hostname is the "ip-" style. > > > > Interestingly the hostname style that rsyslog sees seems to depend on > what EC2 availability zone the node is in. "ec2-" hostnames are only in > the us-west-1 and ap-southeast-1 zones, "ip-*.ec2.internal" and "domu- > *" hostnames in us-east-1. So I think this would suggest something > with DNS configurations in different zones. Additionally the "ec2-" > style hostnames are actually public hostnames that aren't assigned to > the machines but to a MIP or VIP, which again suggests some sort of DNS > lookup. > > > > I have tried using %fromhost% with the same results. Any thoughts on > what might be going on and how to fix it? > > fromhost is the DNS lookup of the IP address of the machine that last > touched the logs. > > if the sending host set hostname in it's logs, then hostname is that > value. > > If the sending host did not put something that looks like a hostname in > the log messages, the first instance of rsyslog that receives the > message > fills the hostname field with fromhost. > > it sounds as if your sending systems are not setting the hostname in > the > logs, so rsyslog is filling in the fromhost. > > If you setup /etc/hosts entries for the IP addresses of these machines > with a short name first, I believe that rsyslog will use that as the > result of the name lookup. > > The better option is to go to the sending machines and figure out why > they > aren't putting hostname in their outbound logs. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Nov 17 10:18:19 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 10:18:19 +0100 Subject: [rsyslog] Handle unsuccessful SQL statements References: <929020446.3553.1289913474420.JavaMail.root@zimbra> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD739@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, November 16, 2010 7:21 PM > To: rsyslog-users > Subject: Re: [rsyslog] Handle unsuccessful SQL statements > > On Tue, 16 Nov 2010, Marc Schiffbauer wrote: > > > Hi all, > > > > Can rsyslog distinguish between "DB-Service not available" and single > "SQL Statement failed" conditions? > > > > Background is that I want rsyslog to to retry for a infinite numer of > times if the postgresql is down > > but I do not want it to DOS the PG-server until the end of time if a > single SQL statements fails. > > > > Is this possible? > > it is possible in theory at least. When we were developing the new > batched > (vector) interface for database plugins this was one of the issues that > we > discussed. This is implemented in the om database module because it can > tell the difference between the two types of errors. > > unfortunantly I don't remember the details of making this work, but I > think that if you set rsyslog to retry forever, but it the om module > will > not keep retrying a bad statement. Yup, the engine is capable of that, but the capability depends on the return codes from the ouput module. I think I at least tried to modify ompgsql so that it returns the required codes (even though I am no postres expert at all). I'd say there is a 70% chance the module supports it. Rainer > > I would suggest giving it a try. setup a output format that does not > auto-quote the parameters and then try sending 'good' log messages > through to verify things are working, then send a 'bad' log message > through (say something with a quote in it) and see if that jams things > up > or if it gets dropped (hopefully with an error being logged by rsyslgo) > and then following log messages go through. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marc.schiffbauer at mightycare.de Wed Nov 17 11:15:33 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Wed, 17 Nov 2010 11:15:33 +0100 Subject: [rsyslog] Handle unsuccessful SQL statements In-Reply-To: References: <929020446.3553.1289913474420.JavaMail.root@zimbra> Message-ID: <201011171115.33673.marc.schiffbauer@mightycare.de> On Tuesday 16 November 2010 19:21:00 david at lang.hm wrote: > On Tue, 16 Nov 2010, Marc Schiffbauer wrote: > > Hi all, > > > > Can rsyslog distinguish between "DB-Service not available" and single > > "SQL Statement failed" conditions? > > > > Background is that I want rsyslog to to retry for a infinite numer of > > times if the postgresql is down but I do not want it to DOS the > > PG-server until the end of time if a single SQL statements fails. > > > > Is this possible? > Hi David, > it is possible in theory at least. When we were developing the new batched > (vector) interface for database plugins this was one of the issues that we > discussed. This is implemented in the om database module because it can > tell the difference between the two types of errors. > > unfortunantly I don't remember the details of making this work, but I > think that if you set rsyslog to retry forever, but it the om module will > not keep retrying a bad statement. > > I would suggest giving it a try. setup a output format that does not > auto-quote the parameters and then try sending 'good' log messages > through to verify things are working, then send a 'bad' log message > through (say something with a quote in it) and see if that jams things up > or if it gets dropped (hopefully with an error being logged by rsyslgo) > and then following log messages go through. > I tried this out. Bad statements will be tried again and again in 4.6.4 :-/ So I guess I will have to deal with that by using as many casts as possible in the SQL statements. Thanks for your help. -Marc From marc.schiffbauer at mightycare.de Wed Nov 17 11:18:25 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Wed, 17 Nov 2010 11:18:25 +0100 Subject: [rsyslog] Handle unsuccessful SQL statements In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD739@GRFEXC.intern.adiscon.com> References: <929020446.3553.1289913474420.JavaMail.root@zimbra> <9B6E2A8877C38245BFB15CC491A11DA71DD739@GRFEXC.intern.adiscon.com> Message-ID: <201011171118.25915.marc.schiffbauer@mightycare.de> On Wednesday 17 November 2010 10:18:19 Rainer Gerhards wrote: > > Yup, the engine is capable of that, but the capability depends on the > return codes from the ouput module. I think I at least tried to modify > ompgsql so that it returns the required codes (even though I am no postres > expert at all). I'd say there is a 70% chance the module supports it. > Hello Rainer, David, I am glad to hear that the code itself is able to do it. But in 4.6.4 it does not work with ompgsql. What can I do to help fixing it for onpgsql? -Marc From rgerhards at hq.adiscon.com Wed Nov 17 11:22:02 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 11:22:02 +0100 Subject: [rsyslog] Handle unsuccessful SQL statements References: <929020446.3553.1289913474420.JavaMail.root@zimbra><9B6E2A8877C38245BFB15CC491A11DA71DD739@GRFEXC.intern.adiscon.com> <201011171118.25915.marc.schiffbauer@mightycare.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD741@GRFEXC.intern.adiscon.com> It's a v5 feature. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Wednesday, November 17, 2010 11:18 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Handle unsuccessful SQL statements > > On Wednesday 17 November 2010 10:18:19 Rainer Gerhards wrote: > > > > Yup, the engine is capable of that, but the capability depends on the > > return codes from the ouput module. I think I at least tried to > modify > > ompgsql so that it returns the required codes (even though I am no > postres > > expert at all). I'd say there is a 70% chance the module supports it. > > > > Hello Rainer, David, > > I am glad to hear that the code itself is able to do it. But in 4.6.4 > it does > not work with ompgsql. > > What can I do to help fixing it for onpgsql? > > -Marc > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marcin at mejor.pl Wed Nov 17 13:20:34 2010 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Wed, 17 Nov 2010 13:20:34 +0100 Subject: [rsyslog] dealing with broken spool Message-ID: <4CE3C892.5080503@mejor.pl> Hello! Yesterday my box crashed then i started to have problem with spool/workdirectory. It's defined in this way: $WorkDirectory /var/spool/rsyslog $ActionQueueType LinkedList $ActionQueueFileName dbq $ActionResumeRetryCount -1 $MainMsgQueueMaxFileSize 512M $MainMsgQueueSaveOnShutdown on At the moment of crash, there were a few files dbq.00000xxxx . In first file, with lowest number (dbq.00000098), appears zeroes form start of file to position #001000F8 (after reboot). I'm wondering why so much file is zeroed? Is it theme of filesystem? (its ext4, mount options: barrier=1,data=ordered) or don't know, rsyslog keeps open file too long? Next question is what can i do it with broken file? - probably i should delete zeroed part of file and modify dbq.qi? - how many messages could be lost? - is it possible for rsyslog to skip broken part of file? # rsyslogd -v rsyslogd 6.1.1, compiled with: FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes GSSAPI Kerberos 5 support: No FEATURE_DEBUG (debug build, slow code): No Atomic operations supported: Yes Runtime Instrumentation (slow code): No git version: 950fe206f84322bdff755988403a9d4017ec26f4 Host is on xen, paravirtualization. Regards! From rgerhards at hq.adiscon.com Wed Nov 17 13:47:11 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 13:47:11 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 1:21 PM > To: rsyslog-users > Subject: [rsyslog] dealing with broken spool > > Hello! > Yesterday my box crashed then i started to have problem with > spool/workdirectory. It's defined in this way: > $WorkDirectory /var/spool/rsyslog > $ActionQueueType LinkedList > $ActionQueueFileName dbq > $ActionResumeRetryCount -1 > $MainMsgQueueMaxFileSize 512M > $MainMsgQueueSaveOnShutdown on > > At the moment of crash, there were a few files dbq.00000xxxx . In first > file, with lowest number (dbq.00000098), appears zeroes form start of > file to position #001000F8 (after reboot). I'm wondering why so much > file is zeroed? Is it theme of filesystem? (its ext4, mount options: > barrier=1,data=ordered) or don't know, rsyslog keeps open file too > long? rsyslog does not write zeros. > Next question is what can i do it with broken file? > - probably i should delete zeroed part of file and modify dbq.qi? > - how many messages could be lost? > - is it possible for rsyslog to skip broken part of file? Unfortunately there is no tool yet to recover from such a situation. If the log data is important enough, you need to manually clean up the control structures, which is probably quite a bit of work. The part that zeroed out is lost in any case. Probably the best idea is to remove the zeroed-out part and delete everything up to the next object header (check obj.c or so for details). Then adjust the input part of the qi file to a zero offset. As long as the ouput is not malformed, that should require no action. Rainer > > # rsyslogd -v > rsyslogd 6.1.1, compiled with: > FEATURE_REGEXP: Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > GSSAPI Kerberos 5 support: No > FEATURE_DEBUG (debug build, slow code): No > Atomic operations supported: Yes > Runtime Instrumentation (slow code): No > > git version: 950fe206f84322bdff755988403a9d4017ec26f4 > Host is on xen, paravirtualization. > Regards! > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marcin at mejor.pl Wed Nov 17 14:33:38 2010 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Wed, 17 Nov 2010 14:33:38 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> References: <4CE3C892.5080503@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> Message-ID: <4CE3D9B2.60606@mejor.pl> W dniu 17.11.2010 13:47, Rainer Gerhards pisze: > Unfortunately there is no tool yet to recover from such a situation. If the Is this near future or distant future? > log data is important enough, you need to manually clean up the control > structures, which is probably quite a bit of work. The part that zeroed out > is lost in any case. Probably the best idea is to remove the zeroed-out part > and delete everything up to the next object header (check obj.c or so for > details). Then adjust the input part of the qi file to a zero offset. As long > as the ouput is not malformed, that should require no action. I don't know why so big part of file is zeroed ba replaying journal? There is quite small traffic, something about ~5 msg/sec, it doesn't need so much space. I would expect that zeroed should has size ~ one-three * blocksize. Could be related to way rsyslog opens file? probably last question, why after crash, always first file from queue is corrupted, not last, which is open while resyslog works? Does rsyslog write not only to last file and .qi but to diffrent files? Thanks a lot. From rgerhards at hq.adiscon.com Wed Nov 17 14:36:42 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 14:36:42 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> <4CE3D9B2.60606@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD74A@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 2:34 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] dealing with broken spool > > W dniu 17.11.2010 13:47, Rainer Gerhards pisze: > > Unfortunately there is no tool yet to recover from such a situation. > If the > > Is this near future or distant future? distant, I probably need a sponsor to do that to the current code base. In the medium term, I'll re-write the disk queue subsystem, because it has outgrown its initial requirements base. It can be done much better (including recovery). > > > log data is important enough, you need to manually clean up the > control > > structures, which is probably quite a bit of work. The part that > zeroed out > > is lost in any case. Probably the best idea is to remove the zeroed- > out part > > and delete everything up to the next object header (check obj.c or so > for > > details). Then adjust the input part of the qi file to a zero offset. > As long > > as the ouput is not malformed, that should require no action. > > I don't know why so big part of file is zeroed ba replaying journal? I don't know either, rsyslog writes pure text data to the queue files (one thing to be changed in the new subsystem), so I know for sure rsyslog does not write zeros. > There is quite small traffic, something about ~5 msg/sec, it doesn't > need so much space. I would expect that zeroed should has size ~ > one-three * blocksize. Could be related to way rsyslog opens file? > probably last question, why after crash, always first file from queue > is > corrupted, not last, which is open while resyslog works? Does rsyslog > write not only to last file and .qi but to diffrent files? It writes to the .qi and the last (highest numbered) file. non-highest numbered files are only read. Rainer > Thanks a lot. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Nov 17 14:56:33 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 14:56:33 +0100 Subject: [rsyslog] normalizing Apache logs to JSON, XML and syslog Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD74B@GRFEXC.intern.adiscon.com> Primarily for those interested in log normalization: I have written a small addition to libee that permits to convert Apache logs (actually any Apache-like log) to various other formats. Details in my blog post at: http://blog.gerhards.net/2010/11/normalizing-apache-access-logs-to-json.html Rainer From dave at fly.srk.fer.hr Wed Nov 17 15:15:38 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Wed, 17 Nov 2010 15:15:38 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <20101116143404.GA1167@fly.srk.fer.hr> References: <20101116125645.GB26159@fly.srk.fer.hr> <9B6E2A8877C38245BFB15CC491A11DA71DD726@GRFEXC.intern.adiscon.com> <20101116143404.GA1167@fly.srk.fer.hr> Message-ID: <20101117141538.GA16632@fly.srk.fer.hr> Dra?en Ka?ar wrote: > There is rpmfind.net which will probably find the packages you want, maybe > even for a Linux distribution you want it for. But the quality of the > packages isn't always as good as it could be, so I tend to stay away from > random sources for production systems. I'd try it for the development > purposes, which is what you need. I just found RPMForge: http://wiki.centos.org/AdditionalResources/Repositories/RPMForge It contains packages which are not part of RHEL or CentOS, but are designed to work in that environment. The repository contains newer autotools, so it might be helpful. -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From marc.schiffbauer at mightycare.de Wed Nov 17 15:32:23 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Wed, 17 Nov 2010 15:32:23 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <4CE3D9B2.60606@mejor.pl> References: <4CE3C892.5080503@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> <4CE3D9B2.60606@mejor.pl> Message-ID: <201011171532.23292.marc.schiffbauer@mightycare.de> On Wednesday 17 November 2010 14:33:38 Marcin Miros?aw wrote: > > I don't know why so big part of file is zeroed ba replaying journal? XFS Filesystem? From marcin at mejor.pl Wed Nov 17 15:37:50 2010 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Wed, 17 Nov 2010 15:37:50 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <201011171532.23292.marc.schiffbauer@mightycare.de> References: <4CE3C892.5080503@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> <4CE3D9B2.60606@mejor.pl> <201011171532.23292.marc.schiffbauer@mightycare.de> Message-ID: <4CE3E8BE.7000401@mejor.pl> W dniu 17.11.2010 15:32, Marc Schiffbauer pisze: > On Wednesday 17 November 2010 14:33:38 Marcin Miros?aw wrote: >> >> I don't know why so big part of file is zeroed ba replaying journal? > > XFS Filesystem? No, ext4, btw, problem with xfs and zeroes went away some time ago, as far as i know. From jason at jasonantman.com Wed Nov 17 15:50:26 2010 From: jason at jasonantman.com (Jason Antman) Date: Wed, 17 Nov 2010 09:50:26 -0500 Subject: [rsyslog] segfault with dynamic filenames In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6EC@GRFEXC.intern.adiscon.com> References: <4CD1CA9B.3090301@jasonantman.com> <4CD329EF.7030001@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6EC@GRFEXC.intern.adiscon.com> Message-ID: <4CE3EBB2.4020007@jasonantman.com> Tom, See the bug I just opened (CCed you on it) for the full details. What version of GCC are you using on CentOS 5.5? I'm starting to think it may be an issue with GCC 4.1.2. I'd be more than happy to give you the kickstart file we're using and whatever other info, but some of our packages are being installed from a local repo with local builds. -Jason Tom Bergfeld wrote: > Hi Jason, > > unfortunately I am still not able to reproduce your issue. (My system: Centos > 5.5 rsyslog 5.6.0). > Would it be possible that you create a debug log? > > In the link below you will find some help for creating a debug log > http://www.rsyslog.com/doc/troubleshoot.html > > Just a question, but did you created the folders you mentioned in your > templates? (/var/log/HOSTS or /var/log/LOCAL) > Rsyslog is not able to create folders, as far as I know. > > Best regards, > Tom > > -----Urspr?ngliche Nachricht----- > Von: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Jason Antman > Gesendet: Donnerstag, 4. November 2010 22:47 > An: rsyslog-users > Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > evaluationquestions) > > Thanks to both of you guys for your answers. FYI, our days have 24 hours > in them :) > > At this point I just got a config file worked up. I got the regexes > working to do all of the matching I need into separate fields (the > online regex tester was a big help), and upgraded to 5.6.0 (CentOS 5.5, > in-house RPM build) in order to of bind imudp to a ruleset (so we can > have one ruleset for localhost/unix socket, and one ruleset for all > remote hosts). > > Unfortunately, I'm getting a segfault almost immediately on startup due > to the following lines: > $template > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfacility > -text%.log" > *.* ?RemoteHost > > I believe I've narrowed the problem down to any templated (dynamic) > filename... even if I dump everything external directly to > /var/log/remote and add the following in my local ruleset: > $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > *.* ?LocalHost > > I still get a segfault. The last line in the output running with -dn is > always: > "file to log to: RemoteHost" (or LocalHost, when I used that config). > > Has anyone seen anything like this? I have rsyslog 2.0.6 running at > another site and dynamic filenames work fine... sort of troubling that > 5.6.0 is having issues with them... > > Thanks, > Jason Antman > > Aaron Wiebe wrote: > >>> 3) Assuming an even distribution over time (not quite accurate), any >>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM (Xen) >>> with a single 2.8GHz CPU and 512MB RAM would go? >>> >>> >> 2 million lines a day is 70 lines per second assuming an 8 hour day. >> MySQL, properly configured, will eat that. Rsyslog won't even notice >> that log level. >> >> Your table sizes may be problematic over time, and your ability to >> query may impact things. Regardless, your concern should be mysql, >> not rsyslog. >> >> -Aaron >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From marcin at mejor.pl Wed Nov 17 16:16:09 2010 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Wed, 17 Nov 2010 16:16:09 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD74A@GRFEXC.intern.adiscon.com> References: <4CE3C892.5080503@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> <4CE3D9B2.60606@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD74A@GRFEXC.intern.adiscon.com> Message-ID: <4CE3F1B9.3030806@mejor.pl> W dniu 17.11.2010 14:36, Rainer Gerhards pisze: > distant, I probably need a sponsor to do that to the current code base. It doesn't sounds good. > It writes to the .qi and the last (highest numbered) file. non-highest > numbered files are only read. This is very strange, i'll try to look closer, what's happening to oldest file. Thank you for anwsers, regards. From rgerhards at hq.adiscon.com Wed Nov 17 16:17:28 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 16:17:28 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD747@GRFEXC.intern.adiscon.com> <4CE3D9B2.60606@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD74A@GRFEXC.intern.adiscon.com> <4CE3F1B9.3030806@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD74D@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 4:16 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] dealing with broken spool > > W dniu 17.11.2010 14:36, Rainer Gerhards pisze: > > distant, I probably need a sponsor to do that to the current code > base. > > It doesn't sounds good. I know, but there is only so much I can do in a single day :( > > > It writes to the .qi and the last (highest numbered) file. non- > highest > > numbered files are only read. > > This is very strange, i'll try to look closer, what's happening to > oldest file. > Thank you for anwsers, > regards. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marcin at mejor.pl Wed Nov 17 18:11:20 2010 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Wed, 17 Nov 2010 18:11:20 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <4CE3C892.5080503@mejor.pl> References: <4CE3C892.5080503@mejor.pl> Message-ID: <4CE40CB8.90606@mejor.pl> Supplement: Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), I touched dbq.00000097 and all queue was relayed to the next rsyslog. Zeros wasn't problematic. So question could be written now as: "Why dbq.qi has incorrect information about oldest file? or Was the oldest file deleted by fsck?" I'd like to ask another question, about configuration this time. I'm not sure about meanings parameters: $MainMsgQueueLowWaterMark and $MainMsgQueueHighWaterMark . Could them be usefull to have queue more often flushed to disk? For example if i set: $MainMsgQueueLowWaterMark 3 $MainMsgQueueHighWaterMark 8 should queue workes flush to 5 messages to disk, if number of messages in memory reach value 8? From rgerhards at hq.adiscon.com Wed Nov 17 18:14:45 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 18:14:45 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl> <4CE40CB8.90606@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD759@GRFEXC.intern.adiscon.com> There is a dedicated setting (checkpoint interval or so) to set how often the queue file should be written. It's explained in the queue doc. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 6:11 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] dealing with broken spool > > Supplement: > Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), I > touched dbq.00000097 and all queue was relayed to the next rsyslog. > Zeros wasn't problematic. So question could be written now as: "Why > dbq.qi has incorrect information about oldest file? or Was the oldest > file deleted by fsck?" > I'd like to ask another question, about configuration this time. > I'm not sure about meanings parameters: > $MainMsgQueueLowWaterMark and $MainMsgQueueHighWaterMark . Could them > be > usefull to have queue more often flushed to disk? For example if i set: > $MainMsgQueueLowWaterMark 3 > $MainMsgQueueHighWaterMark 8 > should queue workes flush to 5 messages to disk, if number of messages > in memory reach value 8? > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Nov 17 18:15:51 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 18:15:51 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl> <4CE40CB8.90606@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD759@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD75A@GRFEXC.intern.adiscon.com> should have read: queue *info* file (.qi) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, November 17, 2010 6:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] dealing with broken spool > > There is a dedicated setting (checkpoint interval or so) to set how > often the > queue file should be written. It's explained in the queue doc. > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > > Sent: Wednesday, November 17, 2010 6:11 PM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] dealing with broken spool > > > > Supplement: > > Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), I > > touched dbq.00000097 and all queue was relayed to the next rsyslog. > > Zeros wasn't problematic. So question could be written now as: "Why > > dbq.qi has incorrect information about oldest file? or Was the oldest > > file deleted by fsck?" > > I'd like to ask another question, about configuration this time. > > I'm not sure about meanings parameters: > > $MainMsgQueueLowWaterMark and $MainMsgQueueHighWaterMark . Could them > > be > > usefull to have queue more often flushed to disk? For example if i > set: > > $MainMsgQueueLowWaterMark 3 > > $MainMsgQueueHighWaterMark 8 > > should queue workes flush to 5 messages to disk, if number of > messages > > in memory reach value 8? > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marcin at mejor.pl Wed Nov 17 18:28:21 2010 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Wed, 17 Nov 2010 18:28:21 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD759@GRFEXC.intern.adiscon.com> References: <4CE3C892.5080503@mejor.pl> <4CE40CB8.90606@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD759@GRFEXC.intern.adiscon.com> Message-ID: <4CE410B5.3020509@mejor.pl> W dniu 2010-11-17 18:14, Rainer Gerhards pisze: > There is a dedicated setting (checkpoint interval or so) to set how often the > queue file should be written. It's explained in the queue doc. I found it, thank you. From marcin at mejor.pl Wed Nov 17 18:36:21 2010 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Wed, 17 Nov 2010 18:36:21 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <4CE40CB8.90606@mejor.pl> References: <4CE3C892.5080503@mejor.pl> <4CE40CB8.90606@mejor.pl> Message-ID: <4CE41295.2040102@mejor.pl> Ech:( Another question, i hope the last. FIle dbq.qi contains values: +bDeleteOnClose:2:1:0: and +bDeleteOnClose:2:1:1: What does it means? Could it be related somehow to mine situation? From rgerhards at hq.adiscon.com Wed Nov 17 18:37:42 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 18:37:42 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl> <4CE40CB8.90606@mejor.pl> <4CE41295.2040102@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> This controls if a file should be deleted on close. This is set by the commit-reader task to remove files that have finally been procssed. Then, they will be unlinked, not zeroed out. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 6:36 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] dealing with broken spool > > Ech:( Another question, i hope the last. FIle dbq.qi contains values: > +bDeleteOnClose:2:1:0: > and > +bDeleteOnClose:2:1:1: > > What does it means? Could it be related somehow to mine situation? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marcin at mejor.pl Wed Nov 17 18:41:24 2010 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Wed, 17 Nov 2010 18:41:24 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> References: <4CE3C892.5080503@mejor.pl> <4CE40CB8.90606@mejor.pl> <4CE41295.2040102@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> Message-ID: <4CE413C4.4000800@mejor.pl> W dniu 2010-11-17 18:37, Rainer Gerhards pisze: > This controls if a file should be deleted on close. This is set by the > commit-reader task to remove files that have finally been procssed. Then, > they will be unlinked, not zeroed out. Exactly, i wrote earlier: "Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), I touched dbq.00000097 and all queue was relayed to the next rsyslog. " and this looks similar to my problem. From rgerhards at hq.adiscon.com Wed Nov 17 18:43:03 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 18:43:03 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl><4CE40CB8.90606@mejor.pl> <4CE41295.2040102@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> <4CE413C4.4000800@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD75E@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 6:41 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] dealing with broken spool > > W dniu 2010-11-17 18:37, Rainer Gerhards pisze: > > This controls if a file should be deleted on close. This is set by > the > > commit-reader task to remove files that have finally been procssed. > Then, > > they will be unlinked, not zeroed out. > > Exactly, i wrote earlier: > "Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), I > touched dbq.00000097 and all queue was relayed to the next rsyslog. > " quite honestly, I did not get the meaning of that sentence > and this looks similar to my problem. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From marcin at mejor.pl Wed Nov 17 18:55:04 2010 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Wed, 17 Nov 2010 18:55:04 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD75E@GRFEXC.intern.adiscon.com> References: <4CE3C892.5080503@mejor.pl><4CE40CB8.90606@mejor.pl> <4CE41295.2040102@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> <4CE413C4.4000800@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD75E@GRFEXC.intern.adiscon.com> Message-ID: <4CE416F8.3020308@mejor.pl> W dniu 2010-11-17 18:43, Rainer Gerhards pisze: >> Exactly, i wrote earlier: >> "Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), I >> touched dbq.00000097 and all queue was relayed to the next rsyslog. >> " > quite honestly, I did not get the meaning of that sentence Previously, i thought that problem is in file dbq.00000098, which was zeroed in half. When i run rsyslog with debug, i saw that rsyslog was looking for file dbq.00000097. I executed: "touch dbq.00000097", rerun rsylogd. Next rsyslog was happy, because has all needed files and relayed all messages to the next host. Recently i've discovered value "+DeleteOnClose" in dbq.qi , and i'm wondering if is it possible that file dbq.00000097 was unlinked when system became unstable (out of memory). From rgerhards at hq.adiscon.com Wed Nov 17 19:06:46 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 19:06:46 +0100 Subject: [rsyslog] dealing with broken spool References: <4CE3C892.5080503@mejor.pl><4CE40CB8.90606@mejor.pl> <4CE41295.2040102@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> <4CE413C4.4000800@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD75E@GRFEXC.intern.adiscon.com> <4CE416F8.3020308@mejor.pl> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD75F@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw > Sent: Wednesday, November 17, 2010 6:55 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] dealing with broken spool > > W dniu 2010-11-17 18:43, Rainer Gerhards pisze: > >> Exactly, i wrote earlier: > >> "Rsyslog was looking for dbq.00000097 (this was writtent in dbq.qi), > I > >> touched dbq.00000097 and all queue was relayed to the next rsyslog. > >> " > > quite honestly, I did not get the meaning of that sentence > > Previously, i thought that problem is in file dbq.00000098, which was > zeroed in half. When i run rsyslog with debug, i saw that rsyslog was > looking for file dbq.00000097. I executed: "touch dbq.00000097", rerun > rsylogd. Next rsyslog was happy, because has all needed files and > relayed all messages to the next host. Recently i've discovered value > "+DeleteOnClose" in dbq.qi , and i'm wondering if is it possible that > file dbq.00000097 was unlinked when system became unstable (out of > memory). Ah, ok, so you actually mean touch, as in create file. Makes sense. Yes, that could be and typically is the case. Rsyslog works on the assumption that the queue file is only persisted after the specified interval, and it may unlink a file without updating the queue file. Obviously, this could be improved, but requires more work than seems obvious (because all of this happens at different layers of the stack - still a good idea...). Rainer From marcin at mejor.pl Wed Nov 17 19:37:52 2010 From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=) Date: Wed, 17 Nov 2010 19:37:52 +0100 Subject: [rsyslog] dealing with broken spool In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD75F@GRFEXC.intern.adiscon.com> References: <4CE3C892.5080503@mejor.pl><4CE40CB8.90606@mejor.pl> <4CE41295.2040102@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD75D@GRFEXC.intern.adiscon.com> <4CE413C4.4000800@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA71DD75E@GRFEXC.intern.adiscon.com> <4CE416F8.3020308@mejor.pl> <9B6E2A8877C38245BFB15CC491A11DA71DD75F@GRFEXC.intern.adiscon.com> Message-ID: <4CE42100.4010604@mejor.pl> W dniu 2010-11-17 19:06, Rainer Gerhards pisze: > Ah, ok, so you actually mean touch, as in create file. Makes sense. Yes, that > could be and typically is the case. Rsyslog works on the assumption that the > queue file is only persisted after the specified interval, and it may unlink > a file without updating the queue file. Obviously, this could be improved, > but requires more work than seems obvious (because all of this happens at > different layers of the stack - still a good idea...). As always, not enough time, not enough people to write code ;) I hope at some it will be improved. In meantime i try to use checkpoint. Regards From rgerhards at hq.adiscon.com Wed Nov 17 21:08:24 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 17 Nov 2010 21:08:24 +0100 Subject: [rsyslog] FW: CEE Overview White Paper Published Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD761@GRFEXC.intern.adiscon.com> A forward for those interested in CEE... Rainer > -----Original Message----- > From: owner-cee-board-list at LISTS.MITRE.ORG [mailto:owner-cee-board- > list at LISTS.MITRE.ORG] On Behalf Of Eric Fitzgerald > Sent: Wednesday, November 17, 2010 8:46 PM > To: CEE Announcements; CEE Discussions > Subject: CEE Overview White Paper Published > > Hi Everyone, > > I'd like to make you aware that the CEE editorial board has published a > short overview white paper describing the overall CEE effort including > the problems and approaches that CEE is taking. > > If you want a quick summary of what CEE is and how the different parts > of the effort work, we'd encourage you to take a look at this paper. > > The document is available for download in PDF form on the CEE web site: > http://cee.mitre.org/documents.html. > > And as always, we'd encourage your feedback- please feel free to post > any comments to the CEE discussion mailing list at cee-discussion- > list at lists.mitre.org. > > Best regards, > Eric and the CEE Editorial Board > > Eric Fitzgerald > Microsoft Corporation > ericf at microsoft.com > 425-705-9601 From joe at joetify.com Wed Nov 17 23:38:42 2010 From: joe at joetify.com (Joe Williams) Date: Wed, 17 Nov 2010 14:38:42 -0800 Subject: [rsyslog] hostnames In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD736@GRFEXC.intern.adiscon.com> References: <69D56D0F-9118-4185-BAB2-288C553CD1AB@joetify.com> <9B6E2A8877C38245BFB15CC491A11DA71DD736@GRFEXC.intern.adiscon.com> Message-ID: Thanks pointers guys, I figured out that log4j doesn't set the syslog headers by default and haproxy doesn't set them at all. I assume they expect them to be relayed. -Joe On Nov 17, 2010, at 12:47 AM, Rainer Gerhards wrote: > And if nothing else helps, you can use the property replacer to e.g. use a > regexp to dig out the part that you are interested in. > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Tuesday, November 16, 2010 11:39 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] hostnames >> >> On Tue, 16 Nov 2010, Joe Williams wrote: >> >>> List, >>> >>> I am trying to standardize the hostnames that we see in our logs. It >> seems that the services (haproxy, etc) that log directly to the rsyslog >> server do this differently. Here's an example: >>> >>> ./ec2-.us-west-1.compute.amazonaws.com/haproxy >>> ./domu-/haproxy >>> ./domu-/haproxy >>> ./domu-/haproxy >>> ./ip-.ec2.internal/haproxy >>> >>> All the standard logs (user.log, syslog, messages, etc) all use a >> standard format like: >>> >>> ./domu-/syslog >>> ./ip-/syslog >>> >>> As you see like in the case of ip-, ".ec2.internal" gets >> appended on with haproxy. In the case of ec2-.us-west- >> 1.compute.amazonaws.com is actually coming from a host with a hostname >> like ip-. Hopefully this makes sense. >>> >>> Across the board I am using %hostname:::lowercase% to create the >> directories. As a test to see what the application sees vs rsyslog we >> added code to log the hostname in one of our applications: >>> >>> ec2-.us-west-1.compute.amazonaws.com/ellison:Nov 16 20:34:22 >> ec2-.us-west-1.compute.amazonaws.com local3: 2010-11-16 >> 20:34:22,123 INFO [main] ejje.Ejje - Address ip- >>> >>> As you can see rsyslog is logging this with the "ec2-" style hostname >> but what the application is seeing for the hostname is the "ip-" style. >>> >>> Interestingly the hostname style that rsyslog sees seems to depend on >> what EC2 availability zone the node is in. "ec2-" hostnames are only in >> the us-west-1 and ap-southeast-1 zones, "ip-*.ec2.internal" and "domu- >> *" hostnames in us-east-1. So I think this would suggest something >> with DNS configurations in different zones. Additionally the "ec2-" >> style hostnames are actually public hostnames that aren't assigned to >> the machines but to a MIP or VIP, which again suggests some sort of DNS >> lookup. >>> >>> I have tried using %fromhost% with the same results. Any thoughts on >> what might be going on and how to fix it? >> >> fromhost is the DNS lookup of the IP address of the machine that last >> touched the logs. >> >> if the sending host set hostname in it's logs, then hostname is that >> value. >> >> If the sending host did not put something that looks like a hostname in >> the log messages, the first instance of rsyslog that receives the >> message >> fills the hostname field with fromhost. >> >> it sounds as if your sending systems are not setting the hostname in >> the >> logs, so rsyslog is filling in the fromhost. >> >> If you setup /etc/hosts entries for the IP addresses of these machines >> with a short name first, I believe that rsyslog will use that as the >> result of the name lookup. >> >> The better option is to go to the sending machines and figure out why >> they >> aren't putting hostname in their outbound logs. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From rgerhards at hq.adiscon.com Thu Nov 18 07:56:15 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Nov 2010 07:56:15 +0100 Subject: [rsyslog] rsyslog wiki Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> Hi all, I am currently thinking to shut down the wiki and migrate content to the main site. While this somewhat saddens me, the volume of spam inside the wiki is increasing quite dramatically. There is a lot of manual effort required to keep it clean. Any suggestions on how to handle this other than shutting down the wiki (or volunteer editors with time to do the task ;)) are highly appreciated. Rainer From r.bhatia at ipax.at Thu Nov 18 08:53:43 2010 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 18 Nov 2010 08:53:43 +0100 Subject: [rsyslog] rsyslog wiki In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> Message-ID: <4CE4DB87.9040501@ipax.at> On 11/18/2010 07:56 AM, Rainer Gerhards wrote: > I am currently thinking to shut down the wiki and migrate content to the main > site. While this somewhat saddens me, the volume of spam inside the wiki is > increasing quite dramatically. There is a lot of manual effort required to > keep it clean. Any suggestions on how to handle this other than shutting down > the wiki (or volunteer editors with time to do the task ;)) are highly > appreciated. my suggestions: 1. some change in the signup procedure - normally, only few people will really contribute on a regular basis. 2. spam filtering using some captcha or aksimet-like blacklisting service? cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From marc.schiffbauer at mightycare.de Thu Nov 18 10:50:18 2010 From: marc.schiffbauer at mightycare.de (Marc Schiffbauer) Date: Thu, 18 Nov 2010 10:50:18 +0100 Subject: [rsyslog] rsyslog wiki In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> Message-ID: <201011181050.19176.marc.schiffbauer@mightycare.de> On Thursday 18 November 2010 07:56:15 Rainer Gerhards wrote: > Hi all, > > I am currently thinking to shut down the wiki and migrate content to the > main site. While this somewhat saddens me, the volume of spam inside the > wiki is increasing quite dramatically. There is a lot of manual effort > required to keep it clean. Any suggestions on how to handle this other > than shutting down the wiki (or volunteer editors with time to do the task > ;)) are highly appreciated. sign-in required + captcha For example on tha packman wiki (also a mediawiki) we had lots of spam, too. Until I changed the "create login" procedure. Since then: no more spam. It looks like that: http://wiki.links2linux.de/index.php?title=Special:UserLogin&type=signup&returnto=Main_Page -Marc From champ at softwink.com Thu Nov 18 18:48:12 2010 From: champ at softwink.com (Champ Clark III [Softwink]) Date: Thu, 18 Nov 2010 12:48:12 -0500 Subject: [rsyslog] rsyslog wiki In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> Message-ID: <20101118174812.GD31702@bundy.vistech.net> On Thu, Nov 18, 2010 at 07:56:15AM +0100, Rainer Gerhards wrote: > Hi all, > > I am currently thinking to shut down the wiki and migrate content to the main > site. While this somewhat saddens me, the volume of spam inside the wiki is > increasing quite dramatically. There is a lot of manual effort required to > keep it clean. Any suggestions on how to handle this other than shutting down > the wiki (or volunteer editors with time to do the task ;)) are highly > appreciated. Spam is a big issue. I run twiki (for various technical reasons) and I have it locked down for edits from only a hand full of people. Have you thought about doing that? -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From david at lang.hm Fri Nov 19 19:46:49 2010 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Nov 2010 10:46:49 -0800 (PST) Subject: [rsyslog] how to have imuxsock listen in multiple places? Message-ID: I need to have it listen on /dev/log and /var/jail/dev/log, how can I do this? I see an option for defining the path to listen to, but I don't see how to define multiple of them (I may just be missing something obvious here) David Lang From john at feurix.com Fri Nov 19 21:09:32 2010 From: john at feurix.com (John Feuerstein) Date: Fri, 19 Nov 2010 21:09:32 +0100 Subject: [rsyslog] how to have imuxsock listen in multiple places? In-Reply-To: References: Message-ID: <4CE6D97C.7080103@feurix.com> On 11/19/2010 07:46 PM, david at lang.hm wrote: > I need to have it listen on /dev/log and /var/jail/dev/log, how can I do > this? > > I see an option for defining the path to listen to, but I don't see how > to define multiple of them (I may just be missing something obvious here) $ModLoad imuxsock # needs to be done just once $InputUnixListenSocketHostName jail1.example.net $AddUnixListenSocket /jail/1/dev/log $InputUnixListenSocketHostName jail2.example.net $AddUnixListenSocket /jail/2/dev/log ... See http://www.rsyslog.com/doc/imuxsock.html From david at lang.hm Fri Nov 19 22:00:39 2010 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Nov 2010 13:00:39 -0800 (PST) Subject: [rsyslog] how to have imuxsock listen in multiple places? In-Reply-To: <4CE6D97C.7080103@feurix.com> References: <4CE6D97C.7080103@feurix.com> Message-ID: thanks. David Lang On Fri, 19 Nov 2010, John Feuerstein wrote: > Date: Fri, 19 Nov 2010 21:09:32 +0100 > From: John Feuerstein > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] how to have imuxsock listen in multiple places? > > On 11/19/2010 07:46 PM, david at lang.hm wrote: >> I need to have it listen on /dev/log and /var/jail/dev/log, how can I do >> this? >> >> I see an option for defining the path to listen to, but I don't see how >> to define multiple of them (I may just be missing something obvious here) > > $ModLoad imuxsock # needs to be done just once > > $InputUnixListenSocketHostName jail1.example.net > $AddUnixListenSocket /jail/1/dev/log > > $InputUnixListenSocketHostName jail2.example.net > $AddUnixListenSocket /jail/2/dev/log > > ... > > See http://www.rsyslog.com/doc/imuxsock.html > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From gbonser at seven.com Sat Nov 20 11:24:08 2010 From: gbonser at seven.com (George Bonser) Date: Sat, 20 Nov 2010 02:24:08 -0800 Subject: [rsyslog] rsyslog wiki In-Reply-To: <20101118174812.GD31702@bundy.vistech.net> References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> <20101118174812.GD31702@bundy.vistech.net> Message-ID: <5A6D953473350C4B9995546AFE9939EE0B14CAB4@RWC-EX1.corp.seven.com> > > Spam is a big issue. I run twiki (for various technical > reasons) and I have it locked down for edits from only a hand full of > people. Have you thought about doing that? In my experience, requiring users to use their real names and requiring registration where you send them their password by snail mail greatly cuts down on the spam. You still allow anyone to register, but they must provide a mailing address and possibly a phone number. That is not a problem for a legitimate user (and might actually help build a sense of community by initiating some more personal contact between people) but will generally filter out the spammers. From rgerhards at hq.adiscon.com Sat Nov 20 12:33:41 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 20 Nov 2010 12:33:41 +0100 Subject: [rsyslog] rsyslog wiki References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com><20101118174812.GD31702@bundy.vistech.net> <5A6D953473350C4B9995546AFE9939EE0B14CAB4@RWC-EX1.corp.seven.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD76C@GRFEXC.intern.adiscon.com> That sounds draconian, but really looks like a solution :) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of George Bonser > Sent: Saturday, November 20, 2010 11:24 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog wiki > > > > > Spam is a big issue. I run twiki (for various technical > > reasons) and I have it locked down for edits from only a hand full of > > people. Have you thought about doing that? > > In my experience, requiring users to use their real names and requiring > registration where you send them their password by snail mail greatly > cuts down on the spam. You still allow anyone to register, but they > must provide a mailing address and possibly a phone number. That is > not > a problem for a legitimate user (and might actually help build a sense > of community by initiating some more personal contact between people) > but will generally filter out the spammers. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From gbonser at seven.com Sat Nov 20 23:35:33 2010 From: gbonser at seven.com (George Bonser) Date: Sat, 20 Nov 2010 14:35:33 -0800 Subject: [rsyslog] rsyslog wiki In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD76C@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com><20101118174812.GD31702@bundy.vistech.net><5A6D953473350C4B9995546AFE9939EE0B14CAB4@RWC-EX1.corp.seven.com> <9B6E2A8877C38245BFB15CC491A11DA71DD76C@GRFEXC.intern.adiscon.com> Message-ID: <5A6D953473350C4B9995546AFE9939EE0B14CABC@RWC-EX1.corp.seven.com> It also helps filter out "sock puppets" (user registered under several different identities that "gang up" on people with a different opinion or come to the support of the primary account) but that probably isn't an issue on this list. 20 registrations to the same mailing address (or SMS the password to a phone number) or phone number would be suspicious. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Saturday, November 20, 2010 3:34 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog wiki > > That sounds draconian, but really looks like a solution :) > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of George Bonser > > Sent: Saturday, November 20, 2010 11:24 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog wiki > > > > > > > > Spam is a big issue. I run twiki (for various technical > > > reasons) and I have it locked down for edits from only a hand full > of > > > people. Have you thought about doing that? > > > > In my experience, requiring users to use their real names and > requiring > > registration where you send them their password by snail mail greatly > > cuts down on the spam. You still allow anyone to register, but they > > must provide a mailing address and possibly a phone number. That is > > not > > a problem for a legitimate user (and might actually help build a > sense > > of community by initiating some more personal contact between people) > > but will generally filter out the spammers. > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From gbonser at seven.com Sun Nov 21 13:27:39 2010 From: gbonser at seven.com (George Bonser) Date: Sun, 21 Nov 2010 04:27:39 -0800 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <20101116143404.GA1167@fly.srk.fer.hr> References: <20101116125645.GB26159@fly.srk.fer.hr><9B6E2A8877C38245BFB15CC491A11DA71DD726@GRFEXC.intern.adiscon.com> <20101116143404.GA1167@fly.srk.fer.hr> Message-ID: <5A6D953473350C4B9995546AFE9939EE0B14CACC@RWC-EX1.corp.seven.com> Just curious, is the program at the other end of the pipe a java program? If so, it might be going into "garbage collection" and becomes "deaf" during that period. George > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Tuesday, November 16, 2010 6:34 AM > To: rsyslog-users > Subject: Re: [rsyslog] Huge pauses when writing to FIFO > > Rainer Gerhards wrote: > > > I think I forgot to mention an important condition: by default, > ompipe works > > in nonblocking mode. So you need to set it to blocking mode for the > action in > > question. This is done via > > > > $OMPipeBlockingMode on > > I added that, just forgor to write it in the previous mail. The syntax > check passed ok and there are no errors generated from rsyslogd. > > > in front of the action, like here: > > $OMPipeBlockingMode on > > local1.* |/opt/foo/my_named_pipe > > I still have 30 second sleep in select() and there is a number of > write() > calls to FIFO (checked with lsof) before that which fail with EAGAIN. > > The output of "strace -f -e trace=file rsyslogd ..." shows: > > 25169 open("/opt/bulb/var/rsyslog/to_parser", > O_RDWR|O_NONBLOCK|O_LARGEFILE) = 5 > > Then I negated the condition in preparePipe() to: > > if(!pData->bBlocking) > mode |= O_NONBLOCK; > > and now strace doesn't show O_NONBLOCK and there is no sleep in > select(). > I'll run more comprehensive tests just to be sure, but this looks ok. > > > > I'm not aware of a repository with the newer autotools packages. I > > > didn't need them so far, so I wasn't looking. > > > > OK, thanks, so I probably need to dump the idea of working with a > "normal" > > git environment there and will developed based on a release tarball. > > There is rpmfind.net which will probably find the packages you want, > maybe > even for a Linux distribution you want it for. But the quality of the > packages isn't always as good as it could be, so I tend to stay away > from > random sources for production systems. I'd try it for the development > purposes, which is what you need. > > -- > .-. .-. Yes, I am an agent of Satan, but my duties are largely > (_ \ / _) ceremonial. > | > | dave at fly.srk.fer.hr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From sivan at omniqueue.com Sun Nov 21 15:41:58 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Sun, 21 Nov 2010 16:41:58 +0200 Subject: [rsyslog] handling two collectors without failover Message-ID: Hi List. I have 2 collectors and a bunch of clients the log to each one of them in parallel. I want to find a way to have them get back into sync, if one of them drops or is cut network and comes back after a while. Currently I use reliable disk queues for each forward action, so each client has 2 forward actions per each one of the two collectors. What would be the way to make those two collector get back to sync if one of them is offline for a period of time and messages have been logged to the other ? I require this in order to be able to have 2 backups or as a form of redundancy for message storage. Many thanks! -Sivan From sivan at omniqueue.com Sun Nov 21 16:31:48 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Sun, 21 Nov 2010 17:31:48 +0200 Subject: [rsyslog] logAnalyzer best practice Message-ID: Hi list, again :) I would like to know if it is best to feed loganalyzer through a mysql storage or filesyste, files? It seems to me, that using mysql could be better for log view performance, e.g. since mysql supports proper indexing and free text search? Anybody have any experience with something similar? I would like to have the log analyzer have the best setup scenario for quick search and analysis. also I would like to know, if using flat files, it is using indexes and the like for better searches and real time analysis? (I'm asking as I'm going to set what's best for logAnalyzer and then follow suite to adopt the rest of the infrastructure accordingly) Many thanks, -Sivan From david at lang.hm Sun Nov 21 23:46:35 2010 From: david at lang.hm (david at lang.hm) Date: Sun, 21 Nov 2010 14:46:35 -0800 (PST) Subject: [rsyslog] logAnalyzer best practice In-Reply-To: References: Message-ID: On Sun, 21 Nov 2010, Sivan Greenberg wrote: > Hi list, again :) > > I would like to know if it is best to feed loganalyzer through a > mysql storage or filesyste, files? > > It seems to me, that using mysql could be better for log view > performance, e.g. since mysql supports proper indexing and free text > search? Anybody have any experience with something similar? > > I would like to have the log analyzer have the best setup scenario > for quick search and analysis. also I would like to know, if using > flat files, it is using indexes and the like for better searches and > real time analysis? (I'm asking as I'm going to set what's best for > logAnalyzer and then follow suite to adopt the rest of the > infrastructure accordingly) this depends a lot on your log analyser tool. there are a lot of different things that you can do when analysing logs, and searching through them is only one of them (and arguably the least useful one) what sort of volume of logs are you talking about dealing with? do you have a particular tool in mind? (if so what is the URL for that tool?) In general, I tend to not use tools that plan to do their analysis by doing searches through the data, they just don't scale well. I prefer to have tools that make as few passes through the data as possible, extracting a lot of information as they do each pass (and at the very least, makes it so that futher passes don't have to look at all the data) David Lang From alorbach at ro1.adiscon.com Mon Nov 22 09:42:09 2010 From: alorbach at ro1.adiscon.com (Andre Lorbach) Date: Mon, 22 Nov 2010 09:42:09 +0100 Subject: [rsyslog] logAnalyzer best practice In-Reply-To: References: Message-ID: Hi, it depends on how much logdata you are going to view and search with LogAnalyzer. For example on single Servers where you want to do some quick searches through daily logs (like mail.log), it can be fully sufficient to use file logging. We are using LogAnalyzer this way on our postfix gateways when we search for missing mails or failures. For any usage above this, I would recommend to use a MSQL database for the logdata. It will improve search speed. However you may need to clean up the database from time to time, a database with millions of log entries will become slow at some point. Best regards, Andre Lorbach > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sivan Greenberg > Sent: Sonntag, 21. November 2010 16:32 > To: rsyslog-users > Subject: [rsyslog] logAnalyzer best practice > > Hi list, again :) > > I would like to know if it is best to feed loganalyzer through a mysql storage > or filesyste, files? > > It seems to me, that using mysql could be better for log view performance, > e.g. since mysql supports proper indexing and free text search? Anybody > have any experience with something similar? > > I would like to have the log analyzer have the best setup scenario for quick > search and analysis. also I would like to know, if using flat files, it is using > indexes and the like for better searches and real time analysis? (I'm asking as > I'm going to set what's best for logAnalyzer and then follow suite to adopt > the rest of the infrastructure accordingly) > > Many thanks, > > -Sivan > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From sivan at omniqueue.com Mon Nov 22 11:20:15 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Mon, 22 Nov 2010 12:20:15 +0200 Subject: [rsyslog] logAnalyzer best practice In-Reply-To: References: Message-ID: On Mon, Nov 22, 2010 at 12:46 AM, wrote: > > this depends a lot on your log analyser tool. there are a lot of different > things that you can do when analysing logs, and searching through them is > only one of them (and arguably the least useful one) > > what sort of volume of logs are you talking about dealing with? We are still investigating this, but consider both logs and tailored msgs we would be passing through rsyslog. it would be a lot of traffic but we want to have different log-rotate intervals so some logs would be kept for a week and some for months or years. > > do you have a particular tool in mind? (if so what is the URL for that > tool?) > Yes, Adiscon's http://loganalyzer.adiscon.com/ > In general, I tend to not use tools that plan to do their analysis by doing > searches through the data, they just don't scale well. I prefer to have > tools that make as few passes through the data as possible, extracting a lot > of information as they do each pass (and at the very least, makes it so that > futher passes don't have to look at all the data) Yes, it seems that Splunk is one such tool. Thanks! -Sivan From dave at fly.srk.fer.hr Mon Nov 22 11:46:25 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Mon, 22 Nov 2010 11:46:25 +0100 Subject: [rsyslog] Huge pauses when writing to FIFO In-Reply-To: <5A6D953473350C4B9995546AFE9939EE0B14CACC@RWC-EX1.corp.seven.com> References: <20101116143404.GA1167@fly.srk.fer.hr> <5A6D953473350C4B9995546AFE9939EE0B14CACC@RWC-EX1.corp.seven.com> Message-ID: <20101122104625.GA12801@fly.srk.fer.hr> It is Java (which I didn't write, so I don't know how it operates exactly). I don't know if garbage collection is responsible for anything. It only took several lines of rsyslog output to exibit pause scenario and I'm not sure if garbage collection kicks in so fast. Besides, I'm normally testing with sending SIGSTOP to the consumer process, so it doesn't matter whether it's performing garbage collection or not. After receiving SIGCONT it should start processing input, which it cannot do if rsyslog doesn't send anything. Everything works George Bonser wrote: > Just curious, is the program at the other end of the pipe a java program? If so, it might be going into "garbage collection" and becomes "deaf" during that period. > > George > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > > Sent: Tuesday, November 16, 2010 6:34 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] Huge pauses when writing to FIFO > > > > Rainer Gerhards wrote: > > > > > I think I forgot to mention an important condition: by default, > > ompipe works > > > in nonblocking mode. So you need to set it to blocking mode for the > > action in > > > question. This is done via > > > > > > $OMPipeBlockingMode on > > > > I added that, just forgor to write it in the previous mail. The syntax > > check passed ok and there are no errors generated from rsyslogd. > > > > > in front of the action, like here: > > > $OMPipeBlockingMode on > > > local1.* |/opt/foo/my_named_pipe > > > > I still have 30 second sleep in select() and there is a number of > > write() > > calls to FIFO (checked with lsof) before that which fail with EAGAIN. > > > > The output of "strace -f -e trace=file rsyslogd ..." shows: > > > > 25169 open("/opt/bulb/var/rsyslog/to_parser", > > O_RDWR|O_NONBLOCK|O_LARGEFILE) = 5 > > > > Then I negated the condition in preparePipe() to: > > > > if(!pData->bBlocking) > > mode |= O_NONBLOCK; > > > > and now strace doesn't show O_NONBLOCK and there is no sleep in > > select(). > > I'll run more comprehensive tests just to be sure, but this looks ok. > > > > > > I'm not aware of a repository with the newer autotools packages. I > > > > didn't need them so far, so I wasn't looking. > > > > > > OK, thanks, so I probably need to dump the idea of working with a > > "normal" > > > git environment there and will developed based on a release tarball. > > > > There is rpmfind.net which will probably find the packages you want, > > maybe > > even for a Linux distribution you want it for. But the quality of the > > packages isn't always as good as it could be, so I tend to stay away > > from > > random sources for production systems. I'd try it for the development > > purposes, which is what you need. > > > > -- > > .-. .-. Yes, I am an agent of Satan, but my duties are largely > > (_ \ / _) ceremonial. > > | > > | dave at fly.srk.fer.hr > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr From alorbach at ro1.adiscon.com Mon Nov 22 12:23:55 2010 From: alorbach at ro1.adiscon.com (Andre Lorbach) Date: Mon, 22 Nov 2010 12:23:55 +0100 Subject: [rsyslog] rsyslog wiki In-Reply-To: <201011181050.19176.marc.schiffbauer@mightycare.de> References: <9B6E2A8877C38245BFB15CC491A11DA71DD766@GRFEXC.intern.adiscon.com> <201011181050.19176.marc.schiffbauer@mightycare.de> Message-ID: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer > Sent: Donnerstag, 18. November 2010 10:50 > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog wiki > > On Thursday 18 November 2010 07:56:15 Rainer Gerhards wrote: > > Hi all, > > > > I am currently thinking to shut down the wiki and migrate content to > > the main site. While this somewhat saddens me, the volume of spam > > inside the wiki is increasing quite dramatically. There is a lot of > > manual effort required to keep it clean. Any suggestions on how to > > handle this other than shutting down the wiki (or volunteer editors > > with time to do the task > > ;)) are highly appreciated. > > > sign-in required + captcha > > For example on tha packman wiki (also a mediawiki) we had lots of spam, > too. > > Until I changed the "create login" procedure. Since then: no more spam. > > It looks like that: > http://wiki.links2linux.de/index.php?title=Special:UserLogin&type=signup&r > eturnto=Main_Page Excellent extension, was a little bit difficult to find and add all needed packages, but it works now in the rsyslog wiki. Best regards, Andre Lorbach From cmetcalf at tilera.com Mon Nov 22 15:59:12 2010 From: cmetcalf at tilera.com (Chris Metcalf) Date: Mon, 22 Nov 2010 09:59:12 -0500 Subject: [rsyslog] [PATCH] avoid use of non-word size atomics Message-ID: <4CEA8540.4000700@tilera.com> The Tile platform doesn't natively support atomic operations other than 4-byte and 8-byte. Although our runtime can handle subword atomics (by doing a word-aligned read, inserting the sub-word properly, and trying compare-and-exchange) it is more efficient to use word-size atomics where possible. I suspect this may also be true for other non-Intel platforms, and certainly the top of rsyslog's runtime/atomic.h does say "THESE MACROS MUST ONLY BE USED WITH WORD-SIZED DATA TYPES!". The attached patch against 6.1.0 converts msg_t's iRefCount from short to int, and moves it in the structure so that the neighboring sbool and short types can be packed more efficiently. -- Chris Metcalf, Tilera Corp. http://www.tilera.com -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: word-atomics.patch URL: From sivan at omniqueue.com Mon Nov 22 16:40:14 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Mon, 22 Nov 2010 17:40:14 +0200 Subject: [rsyslog] Issue when mysql starts before rsyslog using rsyslog 4.6.4 and ommysql. Message-ID: Hi List, We have an issue when using ommysql to send stuff to mysql. If the mysql server is already up then everything is good, messages get's inserted into mysql. However, since rsyslog provide fundamental logging facilities for the server, there is not much option for us to have it start after mysql. When more messages are sent to rsyslog, everything that was accumulated in the queue finally get's saved into mysql but that is suboptimal as we would like to have stuff into mysql as long as mysql is up and running. We also need this to reliably allow mysql to go down and up and messages still sent to it as soon as it comes up. In my understanding rsyslog supports that, but perhaps a special config parameter should be supplied to enable this? Many thanks! -Sivan From sivan at omniqueue.com Tue Nov 23 18:54:45 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Tue, 23 Nov 2010 19:54:45 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. Message-ID: Hi all, I have two rsyslog nodes set up as follows: CLIENT: $ModLoad omrelp $ModLoad imuxsock $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" $WorkDirectory /root/rsyslog/spool $MainMsgQueueSaveOnShutDown on $MainMsgQueueType LinkedList $MainMsgQueueFileName mainq $MainMsgQueueSize 5 $MainMsgQueueCheckpointInterval 1 $MainMsgQueueHighWatermark 4 $MainMsgQueueLowWatermark 2 $MainMsgQueueMaxDiskSpace 1g $WorkDirectory /root/rsyslog/spool $ActionQueueType Disk # making the queue pure disk type $ActionQueueSyncQueueFiles on $ActionQueueFileName forwq $ActionResumeRetryCount -1 # infinite retires on insert failure :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid SERVER: $ModLoad omrelp $ModLoad imuxsock $ModLoad imrelp $ModLoad ommysql $InputRELPServerRun 10514 $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" $WorkDirectory /root/rsyslog/spool $MainMsgQueueSaveOnShutDown on $MainMsgQueueType LinkedList $MainMsgQueueFileName mainq $MainMsgQueueSize 5 $MainMsgQueueCheckpointInterval 1 $MainMsgQueueHighWatermark 4 $MainMsgQueueLowWatermark 2 $MainMsgQueueMaxDiskSpace 1g $WorkDirectory /root/rsyslog/spool $ActionQueueType Disk # making the queue pure disk type $ActionQueueSyncQueueFiles on $ActionQueueFileName forwq $ActionResumeRetryCount -1 # infinite retires on insert failure :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog :msg, contains, "lead" /var/log/leads.log ------------- Message are delivered fine in normal case where server is always up, however, rsyslog according to the configuration should support resending messages in the disk queue if they have been stuck there due to the server being down. So when I intentionally stop the server, and bring it up again ($ reboot) messages waiting in the client's disk queue seem to never get delivered to the server. Also I confirmed that messages are still in the disk queue at the client side and rsyslog never seems to try and send them again after it lost connection to the server; 1) Is there a way to set the interval by which rsyslog retries? 2) If not how to fix this? maybe upgrading the 5.6.0 can solve this? Many thanks, -Sivan From david at lang.hm Tue Nov 23 19:20:44 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 23 Nov 2010 10:20:44 -0800 (PST) Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: References: Message-ID: look at ActionResumeInterval, it looks like what you are needing. David Lang On Tue, 23 Nov 2010, Sivan Greenberg wrote: > Hi all, > > I have two rsyslog nodes set up as follows: > > CLIENT: > > > $ModLoad omrelp > $ModLoad imuxsock > > $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 > %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" > $WorkDirectory /root/rsyslog/spool > $MainMsgQueueSaveOnShutDown on > $MainMsgQueueType LinkedList > $MainMsgQueueFileName mainq > $MainMsgQueueSize 5 > $MainMsgQueueCheckpointInterval 1 > $MainMsgQueueHighWatermark 4 > $MainMsgQueueLowWatermark 2 > $MainMsgQueueMaxDiskSpace 1g > > $WorkDirectory /root/rsyslog/spool > $ActionQueueType Disk # making the queue pure disk type > $ActionQueueSyncQueueFiles on > $ActionQueueFileName forwq > $ActionResumeRetryCount -1 # infinite retires on insert failure > :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid > > > SERVER: > $ModLoad omrelp > $ModLoad imuxsock > $ModLoad imrelp > $ModLoad ommysql > > $InputRELPServerRun 10514 > > $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 > %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" > $WorkDirectory /root/rsyslog/spool > $MainMsgQueueSaveOnShutDown on > $MainMsgQueueType LinkedList > $MainMsgQueueFileName mainq > $MainMsgQueueSize 5 > $MainMsgQueueCheckpointInterval 1 > $MainMsgQueueHighWatermark 4 > $MainMsgQueueLowWatermark 2 > $MainMsgQueueMaxDiskSpace 1g > > $WorkDirectory /root/rsyslog/spool > $ActionQueueType Disk # making the queue pure disk type > $ActionQueueSyncQueueFiles on > $ActionQueueFileName forwq > $ActionResumeRetryCount -1 # infinite retires on insert failure > :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog > :msg, contains, "lead" /var/log/leads.log > > > ------------- > > Message are delivered fine in normal case where server is always up, > however, rsyslog according to the configuration should support > resending messages in the disk queue if they have been stuck there due > to the server being down. So when I intentionally stop the server, and > bring it up again ($ reboot) messages waiting in the client's disk > queue seem to never get delivered to the server. Also I confirmed that > messages are still in the disk queue at the client side and rsyslog > never seems to try and send them again after it lost connection to the > server; > 1) Is there a way to set the interval by which rsyslog retries? > 2) If not how to fix this? maybe upgrading the 5.6.0 can solve this? > > Many thanks, > > -Sivan > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From sivan at omniqueue.com Tue Nov 23 19:48:06 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Tue, 23 Nov 2010 20:48:06 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: References: Message-ID: it is already set to -1 as i understood this controls infinite retries but what about intervals ? On Tue, Nov 23, 2010 at 8:20 PM, wrote: > look at ActionResumeInterval, it looks like what you are needing. > > David Lang > > On Tue, 23 Nov 2010, Sivan Greenberg wrote: > >> Hi all, >> >> I have two rsyslog nodes set up as follows: >> >> CLIENT: >> >> >> $ModLoad omrelp >> $ModLoad imuxsock >> >> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 >> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >> $WorkDirectory /root/rsyslog/spool >> $MainMsgQueueSaveOnShutDown on >> $MainMsgQueueType LinkedList >> $MainMsgQueueFileName mainq >> $MainMsgQueueSize 5 >> $MainMsgQueueCheckpointInterval 1 >> $MainMsgQueueHighWatermark 4 >> $MainMsgQueueLowWatermark ?2 >> $MainMsgQueueMaxDiskSpace 1g >> >> $WorkDirectory /root/rsyslog/spool >> $ActionQueueType Disk # making the queue pure disk type >> $ActionQueueSyncQueueFiles on >> $ActionQueueFileName forwq >> $ActionResumeRetryCount -1 # infinite retires on insert failure >> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid >> >> >> SERVER: >> $ModLoad omrelp >> $ModLoad imuxsock >> $ModLoad imrelp >> $ModLoad ommysql >> >> $InputRELPServerRun 10514 >> >> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 >> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >> $WorkDirectory /root/rsyslog/spool >> $MainMsgQueueSaveOnShutDown on >> $MainMsgQueueType LinkedList >> $MainMsgQueueFileName mainq >> $MainMsgQueueSize 5 >> $MainMsgQueueCheckpointInterval 1 >> $MainMsgQueueHighWatermark 4 >> $MainMsgQueueLowWatermark ?2 >> $MainMsgQueueMaxDiskSpace 1g >> >> $WorkDirectory /root/rsyslog/spool >> $ActionQueueType Disk # making the queue pure disk type >> $ActionQueueSyncQueueFiles on >> $ActionQueueFileName forwq >> $ActionResumeRetryCount -1 # infinite retires on insert failure >> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog >> :msg, contains, "lead" /var/log/leads.log >> >> >> ------------- >> >> Message are delivered fine in normal case where server is always up, >> however, rsyslog according to the configuration should support >> resending messages in the disk queue if they have been stuck there due >> to the server being down. So when I intentionally stop the server, and >> bring it up again ($ reboot) messages waiting in the client's disk >> queue seem to never get delivered to the server. Also I confirmed that >> messages are still in the disk queue at the client side and rsyslog >> never seems to try and send them again after it lost connection to the >> server; >> 1) Is there a way to set the interval by which rsyslog retries? >> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve this? >> >> Many thanks, >> >> -Sivan >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Tue Nov 23 19:56:11 2010 From: david at lang.hm (david at lang.hm) Date: Tue, 23 Nov 2010 10:56:11 -0800 (PST) Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: References: Message-ID: On Tue, 23 Nov 2010, Sivan Greenberg wrote: > it is already set to -1 as i understood this controls infinite retries > but what about intervals ? retry count is set to -1, but you don't have retry interval set. interval sets how frequently you retry (or at least that's how I read the documentation) David Lang > On Tue, Nov 23, 2010 at 8:20 PM, wrote: >> look at ActionResumeInterval, it looks like what you are needing. >> >> David Lang >> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >> >>> Hi all, >>> >>> I have two rsyslog nodes set up as follows: >>> >>> CLIENT: >>> >>> >>> $ModLoad omrelp >>> $ModLoad imuxsock >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>> $WorkDirectory /root/rsyslog/spool >>> $MainMsgQueueSaveOnShutDown on >>> $MainMsgQueueType LinkedList >>> $MainMsgQueueFileName mainq >>> $MainMsgQueueSize 5 >>> $MainMsgQueueCheckpointInterval 1 >>> $MainMsgQueueHighWatermark 4 >>> $MainMsgQueueLowWatermark ?2 >>> $MainMsgQueueMaxDiskSpace 1g >>> >>> $WorkDirectory /root/rsyslog/spool >>> $ActionQueueType Disk # making the queue pure disk type >>> $ActionQueueSyncQueueFiles on >>> $ActionQueueFileName forwq >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid >>> >>> >>> SERVER: >>> $ModLoad omrelp >>> $ModLoad imuxsock >>> $ModLoad imrelp >>> $ModLoad ommysql >>> >>> $InputRELPServerRun 10514 >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>> $WorkDirectory /root/rsyslog/spool >>> $MainMsgQueueSaveOnShutDown on >>> $MainMsgQueueType LinkedList >>> $MainMsgQueueFileName mainq >>> $MainMsgQueueSize 5 >>> $MainMsgQueueCheckpointInterval 1 >>> $MainMsgQueueHighWatermark 4 >>> $MainMsgQueueLowWatermark ?2 >>> $MainMsgQueueMaxDiskSpace 1g >>> >>> $WorkDirectory /root/rsyslog/spool >>> $ActionQueueType Disk # making the queue pure disk type >>> $ActionQueueSyncQueueFiles on >>> $ActionQueueFileName forwq >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog >>> :msg, contains, "lead" /var/log/leads.log >>> >>> >>> ------------- >>> >>> Message are delivered fine in normal case where server is always up, >>> however, rsyslog according to the configuration should support >>> resending messages in the disk queue if they have been stuck there due >>> to the server being down. So when I intentionally stop the server, and >>> bring it up again ($ reboot) messages waiting in the client's disk >>> queue seem to never get delivered to the server. Also I confirmed that >>> messages are still in the disk queue at the client side and rsyslog >>> never seems to try and send them again after it lost connection to the >>> server; >>> 1) Is there a way to set the interval by which rsyslog retries? >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve this? >>> >>> Many thanks, >>> >>> -Sivan >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From john at feurix.com Tue Nov 23 21:55:52 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 23 Nov 2010 21:55:52 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6C4@GRFEXC.intern.adiscon.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> <4CD98DBB.1020808@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6C3@GRFEXC.intern.adiscon.com> <4CD9CC79.9070000@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6C4@GRFEXC.intern.adiscon.com> Message-ID: <4CEC2A58.6030000@feurix.com> Hi, so here we go... :-) After running for some days, the rsyslogd RELP input thread (pid 26858) is spinning at constant 100% cpu utilization again: http://biz.baze.de/debug/rsyslog/debug-on-demand/rsyslogd-pstree.txt I've uploaded various debug files: http://biz.baze.de/debug/rsyslog/debug-on-demand/ Sorry that I can't open up the rsyslog-debug.log for everyone (debug on demand logfile), because it contains sensitive stuff... I have one new and interesting fact: The RELP input thread started going crazy at the time (+- some minutes) where several clients where completely disconnected from the network (someone pulled the plug). However, there is no way to bring this thread back to normal operation. Even after restoring the network connection to these clients and getting them reconnected, the thread is broken. New messages from these reconnected clients arrive and make it into the logfiles on disk just fine. Hope this helps, John From john at feurix.com Tue Nov 23 22:08:02 2010 From: john at feurix.com (John Feuerstein) Date: Tue, 23 Nov 2010 22:08:02 +0100 Subject: [rsyslog] One thread eating up infinite CPU time (spinlock?) In-Reply-To: <4CEC2A58.6030000@feurix.com> References: <4CD8063A.40004@feurix.com> <4CD88856.6000705@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6BF@GRFEXC.intern.adiscon.com> <4CD98DBB.1020808@feurix.com><9B6E2A8877C38245BFB15CC491A11DA71DD6C3@GRFEXC.intern.adiscon.com> <4CD9CC79.9070000@feurix.com> <9B6E2A8877C38245BFB15CC491A11DA71DD6C4@GRFEXC.intern.adiscon.com> <4CEC2A58.6030000@feurix.com> Message-ID: <4CEC2D32.9070802@feurix.com> > I have one new and interesting fact: The RELP input thread started going > crazy at the time (+- some minutes) where several clients where > completely disconnected from the network (someone pulled the plug). FYI: The disconnected clients were *all* clients in 10.10.2.0/24. This might be useful to know (you can get the socket fds from the strace files). From rgerhards at hq.adiscon.com Wed Nov 24 07:50:32 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Nov 2010 07:50:32 +0100 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, November 23, 2010 7:56 PM > To: rsyslog-users > Subject: Re: [rsyslog] messages stuck on a client, using RELP between > client and server, after a restart of the server. > > On Tue, 23 Nov 2010, Sivan Greenberg wrote: > > > it is already set to -1 as i understood this controls infinite > retries > > but what about intervals ? > > retry count is set to -1, but you don't have retry interval set. > interval > sets how frequently you retry (or at least that's how I read the > documentation) Yes, that's right. There are two settings. Rsyslog does not try for each and every message, but suspends the action for some time, in order to guard the rest of the system against failure (think about the timeout associated with trying to connect to a gone-down remote system). The suspension window is increased after each unsuccessful retry in sequence (up to some upper limit). Think of it as graceful degradation of some system components in order to preserve overall health. Rainer > > David Lang > > > On Tue, Nov 23, 2010 at 8:20 PM, wrote: > >> look at ActionResumeInterval, it looks like what you are needing. > >> > >> David Lang > >> > >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: > >> > >>> Hi all, > >>> > >>> I have two rsyslog nodes set up as follows: > >>> > >>> CLIENT: > >>> > >>> > >>> $ModLoad omrelp > >>> $ModLoad imuxsock > >>> > >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 > >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" > >>> $WorkDirectory /root/rsyslog/spool > >>> $MainMsgQueueSaveOnShutDown on > >>> $MainMsgQueueType LinkedList > >>> $MainMsgQueueFileName mainq > >>> $MainMsgQueueSize 5 > >>> $MainMsgQueueCheckpointInterval 1 > >>> $MainMsgQueueHighWatermark 4 > >>> $MainMsgQueueLowWatermark ?2 > >>> $MainMsgQueueMaxDiskSpace 1g > >>> > >>> $WorkDirectory /root/rsyslog/spool > >>> $ActionQueueType Disk # making the queue pure disk type > >>> $ActionQueueSyncQueueFiles on > >>> $ActionQueueFileName forwq > >>> $ActionResumeRetryCount -1 # infinite retires on insert failure > >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid > >>> > >>> > >>> SERVER: > >>> $ModLoad omrelp > >>> $ModLoad imuxsock > >>> $ModLoad imrelp > >>> $ModLoad ommysql > >>> > >>> $InputRELPServerRun 10514 > >>> > >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 > >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" > >>> $WorkDirectory /root/rsyslog/spool > >>> $MainMsgQueueSaveOnShutDown on > >>> $MainMsgQueueType LinkedList > >>> $MainMsgQueueFileName mainq > >>> $MainMsgQueueSize 5 > >>> $MainMsgQueueCheckpointInterval 1 > >>> $MainMsgQueueHighWatermark 4 > >>> $MainMsgQueueLowWatermark ?2 > >>> $MainMsgQueueMaxDiskSpace 1g > >>> > >>> $WorkDirectory /root/rsyslog/spool > >>> $ActionQueueType Disk # making the queue pure disk type > >>> $ActionQueueSyncQueueFiles on > >>> $ActionQueueFileName forwq > >>> $ActionResumeRetryCount -1 # infinite retires on insert failure > >>> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog > >>> :msg, contains, "lead" /var/log/leads.log > >>> > >>> > >>> ------------- > >>> > >>> Message are delivered fine in normal case where server is always > up, > >>> however, rsyslog according to the configuration should support > >>> resending messages in the disk queue if they have been stuck there > due > >>> to the server being down. So when I intentionally stop the server, > and > >>> bring it up again ($ reboot) messages waiting in the client's disk > >>> queue seem to never get delivered to the server. Also I confirmed > that > >>> messages are still in the disk queue at the client side and rsyslog > >>> never seems to try and send them again after it lost connection to > the > >>> server; > >>> 1) Is there a way to set the interval by which rsyslog retries? > >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve > this? > >>> > >>> Many thanks, > >>> > >>> -Sivan > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > From unclemonty at gmail.com Wed Nov 24 12:25:22 2010 From: unclemonty at gmail.com (Monty) Date: Wed, 24 Nov 2010 11:25:22 +0000 Subject: [rsyslog] hostname / fromhost showing only node Message-ID: Hi, I'm trying to filter based on the host which is sending the syslog data. I've tried this based around hostname and fromhost, but both only contain the node name, not the full hostname. I suspected this could be because the rsyslog server is running on the same domain as the host doing the reporting, but my /etc/resolv.conf does not have any reference to the domain, so I cannot see why rsyslog would shorten the full hostname to just the node name. Resolving the IP address from the command line returns the full hostname. I've seen messages from Rainer saying that rsyslog uses the systems resolving capacity, so I can't see why it should be any different. Does anyone have any idea why this is happening? From sivan at omniqueue.com Wed Nov 24 13:17:26 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 24 Nov 2010 14:17:26 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> Message-ID: I am playing with the interval setting, and still messages seem to get stuck or delivery continues only when firing a new message a new client, in which case the previous messages remain in the disk queue but never get forwarded, so they are "discarded" in a sense. Any idea how to move forward? -Sivan On Wed, Nov 24, 2010 at 8:50 AM, Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Tuesday, November 23, 2010 7:56 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] messages stuck on a client, using RELP between >> client and server, after a restart of the server. >> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >> >> > it is already set to -1 as i understood this controls infinite >> retries >> > but what about intervals ? >> >> retry count is set to -1, but you don't have retry interval set. >> interval >> sets how frequently you retry (or at least that's how I read the >> documentation) > > Yes, that's right. There are two settings. Rsyslog does not try for each and > every message, but suspends the action for some time, in order to guard the > rest of the system against failure (think about the timeout associated with > trying to connect to a gone-down remote system). The suspension window is > increased after each unsuccessful retry in sequence (up to some upper limit). > Think of it as graceful degradation of some system components in order to > preserve overall health. > > Rainer >> >> David Lang >> >> > On Tue, Nov 23, 2010 at 8:20 PM, ? wrote: >> >> look at ActionResumeInterval, it looks like what you are needing. >> >> >> >> David Lang >> >> >> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >> >> >> >>> Hi all, >> >>> >> >>> I have two rsyslog nodes set up as follows: >> >>> >> >>> CLIENT: >> >>> >> >>> >> >>> $ModLoad omrelp >> >>> $ModLoad imuxsock >> >>> >> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 >> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >> >>> $WorkDirectory /root/rsyslog/spool >> >>> $MainMsgQueueSaveOnShutDown on >> >>> $MainMsgQueueType LinkedList >> >>> $MainMsgQueueFileName mainq >> >>> $MainMsgQueueSize 5 >> >>> $MainMsgQueueCheckpointInterval 1 >> >>> $MainMsgQueueHighWatermark 4 >> >>> $MainMsgQueueLowWatermark ?2 >> >>> $MainMsgQueueMaxDiskSpace 1g >> >>> >> >>> $WorkDirectory /root/rsyslog/spool >> >>> $ActionQueueType Disk # making the queue pure disk type >> >>> $ActionQueueSyncQueueFiles on >> >>> $ActionQueueFileName forwq >> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >> >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid >> >>> >> >>> >> >>> SERVER: >> >>> $ModLoad omrelp >> >>> $ModLoad imuxsock >> >>> $ModLoad imrelp >> >>> $ModLoad ommysql >> >>> >> >>> $InputRELPServerRun 10514 >> >>> >> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 >> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >> >>> $WorkDirectory /root/rsyslog/spool >> >>> $MainMsgQueueSaveOnShutDown on >> >>> $MainMsgQueueType LinkedList >> >>> $MainMsgQueueFileName mainq >> >>> $MainMsgQueueSize 5 >> >>> $MainMsgQueueCheckpointInterval 1 >> >>> $MainMsgQueueHighWatermark 4 >> >>> $MainMsgQueueLowWatermark ?2 >> >>> $MainMsgQueueMaxDiskSpace 1g >> >>> >> >>> $WorkDirectory /root/rsyslog/spool >> >>> $ActionQueueType Disk # making the queue pure disk type >> >>> $ActionQueueSyncQueueFiles on >> >>> $ActionQueueFileName forwq >> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >> >>> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog >> >>> :msg, contains, "lead" /var/log/leads.log >> >>> >> >>> >> >>> ------------- >> >>> >> >>> Message are delivered fine in normal case where server is always >> up, >> >>> however, rsyslog according to the configuration should support >> >>> resending messages in the disk queue if they have been stuck there >> due >> >>> to the server being down. So when I intentionally stop the server, >> and >> >>> bring it up again ($ reboot) messages waiting in the client's disk >> >>> queue seem to never get delivered to the server. Also I confirmed >> that >> >>> messages are still in the disk queue at the client side and rsyslog >> >>> never seems to try and send them again after it lost connection to >> the >> >>> server; >> >>> 1) Is there a way to set the interval by which rsyslog retries? >> >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve >> this? >> >>> >> >>> Many thanks, >> >>> >> >>> -Sivan >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> http://www.rsyslog.com >> >>> >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> >> >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From sivan at omniqueue.com Wed Nov 24 13:18:44 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 24 Nov 2010 14:18:44 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> Message-ID: On a related note, what does 5.6.0 adds in bug fixes to 4.6.4 ? Thanks, -Sivan On Wed, Nov 24, 2010 at 2:17 PM, Sivan Greenberg wrote: > I am playing with the interval setting, and still messages seem to get > stuck or delivery continues only when firing a new message a new > client, in which case the previous messages remain in the disk queue > but never get forwarded, so they are "discarded" in a sense. > > Any idea how to move forward? > > -Sivan > > On Wed, Nov 24, 2010 at 8:50 AM, Rainer Gerhards > wrote: >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Tuesday, November 23, 2010 7:56 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] messages stuck on a client, using RELP between >>> client and server, after a restart of the server. >>> >>> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >>> >>> > it is already set to -1 as i understood this controls infinite >>> retries >>> > but what about intervals ? >>> >>> retry count is set to -1, but you don't have retry interval set. >>> interval >>> sets how frequently you retry (or at least that's how I read the >>> documentation) >> >> Yes, that's right. There are two settings. Rsyslog does not try for each and >> every message, but suspends the action for some time, in order to guard the >> rest of the system against failure (think about the timeout associated with >> trying to connect to a gone-down remote system). The suspension window is >> increased after each unsuccessful retry in sequence (up to some upper limit). >> Think of it as graceful degradation of some system components in order to >> preserve overall health. >> >> Rainer >>> >>> David Lang >>> >>> > On Tue, Nov 23, 2010 at 8:20 PM, ? wrote: >>> >> look at ActionResumeInterval, it looks like what you are needing. >>> >> >>> >> David Lang >>> >> >>> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >>> >> >>> >>> Hi all, >>> >>> >>> >>> I have two rsyslog nodes set up as follows: >>> >>> >>> >>> CLIENT: >>> >>> >>> >>> >>> >>> $ModLoad omrelp >>> >>> $ModLoad imuxsock >>> >>> >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 >>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $MainMsgQueueSaveOnShutDown on >>> >>> $MainMsgQueueType LinkedList >>> >>> $MainMsgQueueFileName mainq >>> >>> $MainMsgQueueSize 5 >>> >>> $MainMsgQueueCheckpointInterval 1 >>> >>> $MainMsgQueueHighWatermark 4 >>> >>> $MainMsgQueueLowWatermark ?2 >>> >>> $MainMsgQueueMaxDiskSpace 1g >>> >>> >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $ActionQueueType Disk # making the queue pure disk type >>> >>> $ActionQueueSyncQueueFiles on >>> >>> $ActionQueueFileName forwq >>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>> >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid >>> >>> >>> >>> >>> >>> SERVER: >>> >>> $ModLoad omrelp >>> >>> $ModLoad imuxsock >>> >>> $ModLoad imrelp >>> >>> $ModLoad ommysql >>> >>> >>> >>> $InputRELPServerRun 10514 >>> >>> >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 >>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $MainMsgQueueSaveOnShutDown on >>> >>> $MainMsgQueueType LinkedList >>> >>> $MainMsgQueueFileName mainq >>> >>> $MainMsgQueueSize 5 >>> >>> $MainMsgQueueCheckpointInterval 1 >>> >>> $MainMsgQueueHighWatermark 4 >>> >>> $MainMsgQueueLowWatermark ?2 >>> >>> $MainMsgQueueMaxDiskSpace 1g >>> >>> >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $ActionQueueType Disk # making the queue pure disk type >>> >>> $ActionQueueSyncQueueFiles on >>> >>> $ActionQueueFileName forwq >>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>> >>> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog >>> >>> :msg, contains, "lead" /var/log/leads.log >>> >>> >>> >>> >>> >>> ------------- >>> >>> >>> >>> Message are delivered fine in normal case where server is always >>> up, >>> >>> however, rsyslog according to the configuration should support >>> >>> resending messages in the disk queue if they have been stuck there >>> due >>> >>> to the server being down. So when I intentionally stop the server, >>> and >>> >>> bring it up again ($ reboot) messages waiting in the client's disk >>> >>> queue seem to never get delivered to the server. Also I confirmed >>> that >>> >>> messages are still in the disk queue at the client side and rsyslog >>> >>> never seems to try and send them again after it lost connection to >>> the >>> >>> server; >>> >>> 1) Is there a way to set the interval by which rsyslog retries? >>> >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve >>> this? >>> >>> >>> >>> Many thanks, >>> >>> >>> >>> -Sivan >>> >>> _______________________________________________ >>> >>> rsyslog mailing list >>> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> http://www.rsyslog.com >>> >>> >>> >> _______________________________________________ >>> >> rsyslog mailing list >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> http://www.rsyslog.com >>> >> >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com >>> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > From sivan at omniqueue.com Wed Nov 24 13:22:19 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 24 Nov 2010 14:22:19 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> Message-ID: My setup now is as before, only I added the "$ActionResumeInterval 10". I could not experience message loss now, but still a new messages had to be sent on the client for it to work. It seems a bit non-deterministic ;) What's the smallest value that's still okay to use for the interval? -Sivan On Wed, Nov 24, 2010 at 2:17 PM, Sivan Greenberg wrote: > I am playing with the interval setting, and still messages seem to get > stuck or delivery continues only when firing a new message a new > client, in which case the previous messages remain in the disk queue > but never get forwarded, so they are "discarded" in a sense. > > Any idea how to move forward? > > -Sivan > > On Wed, Nov 24, 2010 at 8:50 AM, Rainer Gerhards > wrote: >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Tuesday, November 23, 2010 7:56 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] messages stuck on a client, using RELP between >>> client and server, after a restart of the server. >>> >>> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >>> >>> > it is already set to -1 as i understood this controls infinite >>> retries >>> > but what about intervals ? >>> >>> retry count is set to -1, but you don't have retry interval set. >>> interval >>> sets how frequently you retry (or at least that's how I read the >>> documentation) >> >> Yes, that's right. There are two settings. Rsyslog does not try for each and >> every message, but suspends the action for some time, in order to guard the >> rest of the system against failure (think about the timeout associated with >> trying to connect to a gone-down remote system). The suspension window is >> increased after each unsuccessful retry in sequence (up to some upper limit). >> Think of it as graceful degradation of some system components in order to >> preserve overall health. >> >> Rainer >>> >>> David Lang >>> >>> > On Tue, Nov 23, 2010 at 8:20 PM, ? wrote: >>> >> look at ActionResumeInterval, it looks like what you are needing. >>> >> >>> >> David Lang >>> >> >>> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >>> >> >>> >>> Hi all, >>> >>> >>> >>> I have two rsyslog nodes set up as follows: >>> >>> >>> >>> CLIENT: >>> >>> >>> >>> >>> >>> $ModLoad omrelp >>> >>> $ModLoad imuxsock >>> >>> >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 >>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $MainMsgQueueSaveOnShutDown on >>> >>> $MainMsgQueueType LinkedList >>> >>> $MainMsgQueueFileName mainq >>> >>> $MainMsgQueueSize 5 >>> >>> $MainMsgQueueCheckpointInterval 1 >>> >>> $MainMsgQueueHighWatermark 4 >>> >>> $MainMsgQueueLowWatermark ?2 >>> >>> $MainMsgQueueMaxDiskSpace 1g >>> >>> >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $ActionQueueType Disk # making the queue pure disk type >>> >>> $ActionQueueSyncQueueFiles on >>> >>> $ActionQueueFileName forwq >>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>> >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid >>> >>> >>> >>> >>> >>> SERVER: >>> >>> $ModLoad omrelp >>> >>> $ModLoad imuxsock >>> >>> $ModLoad imrelp >>> >>> $ModLoad ommysql >>> >>> >>> >>> $InputRELPServerRun 10514 >>> >>> >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 >>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $MainMsgQueueSaveOnShutDown on >>> >>> $MainMsgQueueType LinkedList >>> >>> $MainMsgQueueFileName mainq >>> >>> $MainMsgQueueSize 5 >>> >>> $MainMsgQueueCheckpointInterval 1 >>> >>> $MainMsgQueueHighWatermark 4 >>> >>> $MainMsgQueueLowWatermark ?2 >>> >>> $MainMsgQueueMaxDiskSpace 1g >>> >>> >>> >>> $WorkDirectory /root/rsyslog/spool >>> >>> $ActionQueueType Disk # making the queue pure disk type >>> >>> $ActionQueueSyncQueueFiles on >>> >>> $ActionQueueFileName forwq >>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>> >>> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog >>> >>> :msg, contains, "lead" /var/log/leads.log >>> >>> >>> >>> >>> >>> ------------- >>> >>> >>> >>> Message are delivered fine in normal case where server is always >>> up, >>> >>> however, rsyslog according to the configuration should support >>> >>> resending messages in the disk queue if they have been stuck there >>> due >>> >>> to the server being down. So when I intentionally stop the server, >>> and >>> >>> bring it up again ($ reboot) messages waiting in the client's disk >>> >>> queue seem to never get delivered to the server. Also I confirmed >>> that >>> >>> messages are still in the disk queue at the client side and rsyslog >>> >>> never seems to try and send them again after it lost connection to >>> the >>> >>> server; >>> >>> 1) Is there a way to set the interval by which rsyslog retries? >>> >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve >>> this? >>> >>> >>> >>> Many thanks, >>> >>> >>> >>> -Sivan >>> >>> _______________________________________________ >>> >>> rsyslog mailing list >>> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> http://www.rsyslog.com >>> >>> >>> >> _______________________________________________ >>> >> rsyslog mailing list >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> http://www.rsyslog.com >>> >> >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com >>> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > From sivan at omniqueue.com Wed Nov 24 13:49:36 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 24 Nov 2010 14:49:36 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> Message-ID: I've just confirmed in my tests that messages do not get sent unless a new message is added after the rsyslog receiver goes up... It seem to ignore the resume interval. -Sivan On Wed, Nov 24, 2010 at 2:22 PM, Sivan Greenberg wrote: > My setup now is as before, only I added the "$ActionResumeInterval 10". > > I could not experience message loss now, but still a new messages had > to be sent on the client for it to work. It seems a bit > non-deterministic ;) What's the smallest value that's still okay to > use for the interval? > > -Sivan > > On Wed, Nov 24, 2010 at 2:17 PM, Sivan Greenberg wrote: >> I am playing with the interval setting, and still messages seem to get >> stuck or delivery continues only when firing a new message a new >> client, in which case the previous messages remain in the disk queue >> but never get forwarded, so they are "discarded" in a sense. >> >> Any idea how to move forward? >> >> -Sivan >> >> On Wed, Nov 24, 2010 at 8:50 AM, Rainer Gerhards >> wrote: >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>>> Sent: Tuesday, November 23, 2010 7:56 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] messages stuck on a client, using RELP between >>>> client and server, after a restart of the server. >>>> >>>> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >>>> >>>> > it is already set to -1 as i understood this controls infinite >>>> retries >>>> > but what about intervals ? >>>> >>>> retry count is set to -1, but you don't have retry interval set. >>>> interval >>>> sets how frequently you retry (or at least that's how I read the >>>> documentation) >>> >>> Yes, that's right. There are two settings. Rsyslog does not try for each and >>> every message, but suspends the action for some time, in order to guard the >>> rest of the system against failure (think about the timeout associated with >>> trying to connect to a gone-down remote system). The suspension window is >>> increased after each unsuccessful retry in sequence (up to some upper limit). >>> Think of it as graceful degradation of some system components in order to >>> preserve overall health. >>> >>> Rainer >>>> >>>> David Lang >>>> >>>> > On Tue, Nov 23, 2010 at 8:20 PM, ? wrote: >>>> >> look at ActionResumeInterval, it looks like what you are needing. >>>> >> >>>> >> David Lang >>>> >> >>>> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: >>>> >> >>>> >>> Hi all, >>>> >>> >>>> >>> I have two rsyslog nodes set up as follows: >>>> >>> >>>> >>> CLIENT: >>>> >>> >>>> >>> >>>> >>> $ModLoad omrelp >>>> >>> $ModLoad imuxsock >>>> >>> >>>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 >>>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>>> >>> $WorkDirectory /root/rsyslog/spool >>>> >>> $MainMsgQueueSaveOnShutDown on >>>> >>> $MainMsgQueueType LinkedList >>>> >>> $MainMsgQueueFileName mainq >>>> >>> $MainMsgQueueSize 5 >>>> >>> $MainMsgQueueCheckpointInterval 1 >>>> >>> $MainMsgQueueHighWatermark 4 >>>> >>> $MainMsgQueueLowWatermark ?2 >>>> >>> $MainMsgQueueMaxDiskSpace 1g >>>> >>> >>>> >>> $WorkDirectory /root/rsyslog/spool >>>> >>> $ActionQueueType Disk # making the queue pure disk type >>>> >>> $ActionQueueSyncQueueFiles on >>>> >>> $ActionQueueFileName forwq >>>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>>> >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid >>>> >>> >>>> >>> >>>> >>> SERVER: >>>> >>> $ModLoad omrelp >>>> >>> $ModLoad imuxsock >>>> >>> $ModLoad imrelp >>>> >>> $ModLoad ommysql >>>> >>> >>>> >>> $InputRELPServerRun 10514 >>>> >>> >>>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% ReplicaFrontEnd109 >>>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" >>>> >>> $WorkDirectory /root/rsyslog/spool >>>> >>> $MainMsgQueueSaveOnShutDown on >>>> >>> $MainMsgQueueType LinkedList >>>> >>> $MainMsgQueueFileName mainq >>>> >>> $MainMsgQueueSize 5 >>>> >>> $MainMsgQueueCheckpointInterval 1 >>>> >>> $MainMsgQueueHighWatermark 4 >>>> >>> $MainMsgQueueLowWatermark ?2 >>>> >>> $MainMsgQueueMaxDiskSpace 1g >>>> >>> >>>> >>> $WorkDirectory /root/rsyslog/spool >>>> >>> $ActionQueueType Disk # making the queue pure disk type >>>> >>> $ActionQueueSyncQueueFiles on >>>> >>> $ActionQueueFileName forwq >>>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure >>>> >>> :msg, contains, "lead" :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog >>>> >>> :msg, contains, "lead" /var/log/leads.log >>>> >>> >>>> >>> >>>> >>> ------------- >>>> >>> >>>> >>> Message are delivered fine in normal case where server is always >>>> up, >>>> >>> however, rsyslog according to the configuration should support >>>> >>> resending messages in the disk queue if they have been stuck there >>>> due >>>> >>> to the server being down. So when I intentionally stop the server, >>>> and >>>> >>> bring it up again ($ reboot) messages waiting in the client's disk >>>> >>> queue seem to never get delivered to the server. Also I confirmed >>>> that >>>> >>> messages are still in the disk queue at the client side and rsyslog >>>> >>> never seems to try and send them again after it lost connection to >>>> the >>>> >>> server; >>>> >>> 1) Is there a way to set the interval by which rsyslog retries? >>>> >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve >>>> this? >>>> >>> >>>> >>> Many thanks, >>>> >>> >>>> >>> -Sivan >>>> >>> _______________________________________________ >>>> >>> rsyslog mailing list >>>> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>> http://www.rsyslog.com >>>> >>> >>>> >> _______________________________________________ >>>> >> rsyslog mailing list >>>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >> http://www.rsyslog.com >>>> >> >>>> > _______________________________________________ >>>> > rsyslog mailing list >>>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> > http://www.rsyslog.com >>>> > >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >> > From rgerhards at hq.adiscon.com Wed Nov 24 14:01:36 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Nov 2010 14:01:36 +0100 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD78F@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sivan Greenberg > Sent: Wednesday, November 24, 2010 1:50 PM > To: rsyslog-users > Subject: Re: [rsyslog] messages stuck on a client, using RELP between > client and server, after a restart of the server. > > I've just confirmed in my tests that messages do not get sent unless a > new message is added after the rsyslog receiver goes up... It seem to > ignore the resume interval. Which version do you use? If it is a current one, we need a debug log to see what is going on. Rainer From rgerhards at hq.adiscon.com Wed Nov 24 14:01:57 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Nov 2010 14:01:57 +0100 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD790@GRFEXC.intern.adiscon.com> Please have a look at ./ChangeLog > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sivan Greenberg > Sent: Wednesday, November 24, 2010 1:19 PM > To: rsyslog-users > Subject: Re: [rsyslog] messages stuck on a client, using RELP between > client and server, after a restart of the server. > > On a related note, what does 5.6.0 adds in bug fixes to 4.6.4 ? > > Thanks, > > -Sivan > > On Wed, Nov 24, 2010 at 2:17 PM, Sivan Greenberg > wrote: > > I am playing with the interval setting, and still messages seem to > get > > stuck or delivery continues only when firing a new message a new > > client, in which case the previous messages remain in the disk queue > > but never get forwarded, so they are "discarded" in a sense. > > > > Any idea how to move forward? > > > > -Sivan > > > > On Wed, Nov 24, 2010 at 8:50 AM, Rainer Gerhards > > wrote: > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Tuesday, November 23, 2010 7:56 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] messages stuck on a client, using RELP > between > >>> client and server, after a restart of the server. > >>> > >>> On Tue, 23 Nov 2010, Sivan Greenberg wrote: > >>> > >>> > it is already set to -1 as i understood this controls infinite > >>> retries > >>> > but what about intervals ? > >>> > >>> retry count is set to -1, but you don't have retry interval set. > >>> interval > >>> sets how frequently you retry (or at least that's how I read the > >>> documentation) > >> > >> Yes, that's right. There are two settings. Rsyslog does not try for > each and > >> every message, but suspends the action for some time, in order to > guard the > >> rest of the system against failure (think about the timeout > associated with > >> trying to connect to a gone-down remote system). The suspension > window is > >> increased after each unsuccessful retry in sequence (up to some > upper limit). > >> Think of it as graceful degradation of some system components in > order to > >> preserve overall health. > >> > >> Rainer > >>> > >>> David Lang > >>> > >>> > On Tue, Nov 23, 2010 at 8:20 PM, ? wrote: > >>> >> look at ActionResumeInterval, it looks like what you are > needing. > >>> >> > >>> >> David Lang > >>> >> > >>> >> On Tue, 23 Nov 2010, Sivan Greenberg wrote: > >>> >> > >>> >>> Hi all, > >>> >>> > >>> >>> I have two rsyslog nodes set up as follows: > >>> >>> > >>> >>> CLIENT: > >>> >>> > >>> >>> > >>> >>> $ModLoad omrelp > >>> >>> $ModLoad imuxsock > >>> >>> > >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% CLIENT157 > >>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" > >>> >>> $WorkDirectory /root/rsyslog/spool > >>> >>> $MainMsgQueueSaveOnShutDown on > >>> >>> $MainMsgQueueType LinkedList > >>> >>> $MainMsgQueueFileName mainq > >>> >>> $MainMsgQueueSize 5 > >>> >>> $MainMsgQueueCheckpointInterval 1 > >>> >>> $MainMsgQueueHighWatermark 4 > >>> >>> $MainMsgQueueLowWatermark ?2 > >>> >>> $MainMsgQueueMaxDiskSpace 1g > >>> >>> > >>> >>> $WorkDirectory /root/rsyslog/spool > >>> >>> $ActionQueueType Disk # making the queue pure disk type > >>> >>> $ActionQueueSyncQueueFiles on > >>> >>> $ActionQueueFileName forwq > >>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure > >>> >>> :msg, contains, "lead" :omrelp:10.200.10.181:10514;myhostid > >>> >>> > >>> >>> > >>> >>> SERVER: > >>> >>> $ModLoad omrelp > >>> >>> $ModLoad imuxsock > >>> >>> $ModLoad imrelp > >>> >>> $ModLoad ommysql > >>> >>> > >>> >>> $InputRELPServerRun 10514 > >>> >>> > >>> >>> $template myhostid,"%TIMESTAMP:::date-rfc3339% > ReplicaFrontEnd109 > >>> >>> %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" > >>> >>> $WorkDirectory /root/rsyslog/spool > >>> >>> $MainMsgQueueSaveOnShutDown on > >>> >>> $MainMsgQueueType LinkedList > >>> >>> $MainMsgQueueFileName mainq > >>> >>> $MainMsgQueueSize 5 > >>> >>> $MainMsgQueueCheckpointInterval 1 > >>> >>> $MainMsgQueueHighWatermark 4 > >>> >>> $MainMsgQueueLowWatermark ?2 > >>> >>> $MainMsgQueueMaxDiskSpace 1g > >>> >>> > >>> >>> $WorkDirectory /root/rsyslog/spool > >>> >>> $ActionQueueType Disk # making the queue pure disk type > >>> >>> $ActionQueueSyncQueueFiles on > >>> >>> $ActionQueueFileName forwq > >>> >>> $ActionResumeRetryCount -1 # infinite retires on insert failure > >>> >>> :msg, contains, "lead" > :ommysql:127.0.0.1,Syslog,rsyslog,rsyslog > >>> >>> :msg, contains, "lead" /var/log/leads.log > >>> >>> > >>> >>> > >>> >>> ------------- > >>> >>> > >>> >>> Message are delivered fine in normal case where server is > always > >>> up, > >>> >>> however, rsyslog according to the configuration should support > >>> >>> resending messages in the disk queue if they have been stuck > there > >>> due > >>> >>> to the server being down. So when I intentionally stop the > server, > >>> and > >>> >>> bring it up again ($ reboot) messages waiting in the client's > disk > >>> >>> queue seem to never get delivered to the server. Also I > confirmed > >>> that > >>> >>> messages are still in the disk queue at the client side and > rsyslog > >>> >>> never seems to try and send them again after it lost connection > to > >>> the > >>> >>> server; > >>> >>> 1) Is there a way to set the interval by which rsyslog retries? > >>> >>> 2) If not how to fix this? maybe upgrading the 5.6.0 can solve > >>> this? > >>> >>> > >>> >>> Many thanks, > >>> >>> > >>> >>> -Sivan > >>> >>> _______________________________________________ > >>> >>> rsyslog mailing list > >>> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> >>> http://www.rsyslog.com > >>> >>> > >>> >> _______________________________________________ > >>> >> rsyslog mailing list > >>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> >> http://www.rsyslog.com > >>> >> > >>> > _______________________________________________ > >>> > rsyslog mailing list > >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > http://www.rsyslog.com > >>> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From sivan at omniqueue.com Wed Nov 24 14:14:06 2010 From: sivan at omniqueue.com (Sivan Greenberg) Date: Wed, 24 Nov 2010 15:14:06 +0200 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD78F@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD78F@GRFEXC.intern.adiscon.com> Message-ID: On Wed, Nov 24, 2010 at 3:01 PM, Rainer Gerhards wrote: > > Which version do you use? If it is a current one, we need a debug log to see > what is going on. 4.6.4 -Sivan From rgerhards at hq.adiscon.com Wed Nov 24 14:23:38 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Nov 2010 14:23:38 +0100 Subject: [rsyslog] messages stuck on a client, using RELP between client and server, after a restart of the server. References: <9B6E2A8877C38245BFB15CC491A11DA71DD78A@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD78F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD791@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sivan Greenberg > Sent: Wednesday, November 24, 2010 2:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] messages stuck on a client, using RELP between > client and server, after a restart of the server. > > On Wed, Nov 24, 2010 at 3:01 PM, Rainer Gerhards > wrote: > > > > Which version do you use? If it is a current one, we need a debug log > to see > > what is going on. > > 4.6.4 OK, that calls for a debug log. I need the log during the period from where the target system shuts down to when it is up again and maybe three minutes into that time (without any inbound traffic). This will show how the internal state advances. Rainer From friedl at hq.adiscon.com Wed Nov 24 16:30:42 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 24 Nov 2010 16:30:42 +0100 Subject: [rsyslog] rsyslog 3.22.3 (v3-stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD794@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 3.22.3, a member of the v3-stable branch. This update addresses the TLS bug, that has been bothering a lot of users lately. It stops rsyslog from looping, thus disabling functionality and bearing the risk of unresponsiveness of the whole system. See ChangeLog for more details. ChangeLog: http://www.rsyslog.com/changelog-for-3-22-3-v3-stable/ Download: http://www.rsyslog.com/rsyslog-3-22-3-v3-stable/ As always, feedback is appreciated. Best regards, Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Wed Nov 24 16:31:44 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 24 Nov 2010 16:31:44 +0100 Subject: [rsyslog] rsyslog 4.6.5 (v4-stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD795@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 4.6.5, a member of the v4-stable branch. This update addresses the TLS bug, that has been bothering a lot of users lately and a few other issues. It stops rsyslog from looping, thus disabling functionality and bearing the risk of unresponsiveness of the whole system. More details about this issue can be found here: http://bugzilla.adiscon.com/show_bug.cgi?id=194 See ChangeLog for more details. ChangeLog: http://www.rsyslog.com/changelog-for-4-6-5-v4-stable/ Download: http://www.rsyslog.com/rsyslog-4-6-5-v4-stable/ As always, feedback is appreciated. Best regards, Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Wed Nov 24 16:45:51 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 24 Nov 2010 16:45:51 +0100 Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 5.6.1, the new v5-stable. This update addresses the TLS bug, that has been bothering a lot of users lately. It stops rsyslog from looping, thus disabling functionality and bearing the risk of unresponsiveness of the whole system. More details about this issue can be found here: http://bugzilla.adiscon.com/show_bug.cgi?id=194 Other issues have been fixed for imptcp, failing testbench, segfault on empty templates and failed compile. For more detailed information, please review the ChangeLog. We strongly recommend to use this release if you are using (or planning to use) syslog with TLS enabled. ChangeLog: http://www.rsyslog.com/changelog-for-5-6-1-v5-stable/ Download: http://www.rsyslog.com/rsyslog-5-6-1-v5-stable/ As always, feedback is appreciated. Best regards, Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From buraphalinuxserver at gmail.com Wed Nov 24 16:58:21 2010 From: buraphalinuxserver at gmail.com (Somchai Smythe) Date: Wed, 24 Nov 2010 22:58:21 +0700 Subject: [rsyslog] rsyslog 4.6.5 (v4-stable) released In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD795@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD795@GRFEXC.intern.adiscon.com> Message-ID: My build won't complete (linux with gcc 4.4.5 and kernel 2.6.36.1 and gnutls-2.10.3): ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/man --infodir=/usr/info --enable-relp --enable-mail --enable-gnutls --build=i686-pc-linux-gnu .... /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -export-dynamic -o rsyslogd rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o -lz -lpthread ../runtime/librsyslog.la libtool: link: gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o rsyslogd rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o -Wl,--export-dynamic -lz -lpthread ../runtime/.libs/librsyslog.a -ldl -lrt ../runtime/.libs/librsyslog.a(librsyslog_la-msg.o): In function `msgDestruct': /tmp/rsyslog-4.6.5/runtime/msg.c:803: undefined reference to `ATOMIC_INC_AND_FETCH' ../runtime/.libs/librsyslog.a(librsyslog_la-wtp.o): In function `wtpProcessThrdChanges': /tmp/rsyslog-4.6.5/runtime/wtp.c:219: undefined reference to `ATOMIC_STORE_0_TO_INT' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiSetState': /tmp/rsyslog-4.6.5/runtime/wti.c:149: undefined reference to `ATOMIC_CAS_VAL' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiWorkerCancelCleanup': /tmp/rsyslog-4.6.5/runtime/wti.c:330: undefined reference to `ATOMIC_STORE_1_TO_INT' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiWorker': /tmp/rsyslog-4.6.5/runtime/wti.c:417: undefined reference to `ATOMIC_STORE_1_TO_INT' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiCancelThrd': /tmp/rsyslog-4.6.5/runtime/wti.c:181: undefined reference to `ATOMIC_STORE_1_TO_INT' collect2: ld returned 1 exit status make[2]: *** [rsyslogd] Error 1 make[2]: Leaving directory `/tmp/rsyslog-4.6.5/tools' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/rsyslog-4.6.5' make: *** [all] Error 2 On 11/24/10, Florian Riedl wrote: > Hi all, > > We have just released rsyslog 4.6.5, a member of the v4-stable branch. > > This update addresses the TLS bug, that has been bothering a lot of users > lately and a few other issues. It stops rsyslog from looping, thus disabling > functionality and bearing the risk of unresponsiveness of the whole system. > More details about this issue can be found here: > > http://bugzilla.adiscon.com/show_bug.cgi?id=194 > > > See ChangeLog for more details. > > ChangeLog: > > http://www.rsyslog.com/changelog-for-4-6-5-v4-stable/ > > > Download: > > http://www.rsyslog.com/rsyslog-4-6-5-v4-stable/ > > > As always, feedback is appreciated. > > Best regards, > Florian Riedl > > -- > > Support > > ======= > > > > Improving rsyslog is costly, but you can help! We are looking for > organizations that find rsyslog useful and wish to contribute back. You can > contribute by reporting bugs, improve the software, or donate money or > equipment. > > > > Commercial support contracts for rsyslog are available, and they help > finance > continued maintenance. Adiscon GmbH, a privately held German company, is > currently funding rsyslog development. We are always looking for interesting > development projects. For details on how to help, please see > http://www.rsyslog.com/doc-how2help.html > . > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From friedl at hq.adiscon.com Wed Nov 24 17:01:03 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 24 Nov 2010 17:01:03 +0100 Subject: [rsyslog] rsyslog 4.6.5 (v4-stable) released References: <9B6E2A8877C38245BFB15CC491A11DA71DD795@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD797@GRFEXC.intern.adiscon.com> Which was the last version, where this worked? And, which distribution (and version) do you use? 32bit or 64bit? Florian -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Somchai Smythe Sent: Mittwoch, 24. November 2010 16:58 To: rsyslog-users Subject: Re: [rsyslog] rsyslog 4.6.5 (v4-stable) released My build won't complete (linux with gcc 4.4.5 and kernel 2.6.36.1 and gnutls-2.10.3): ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/man --infodir=/usr/info --enable-relp --enable-mail --enable-gnutls --build=i686-pc-linux-gnu .... /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -export-dynamic -o rsyslogd rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o -lz -lpthread ../runtime/librsyslog.la libtool: link: gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o rsyslogd rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o -Wl,--export-dynamic -lz -lpthread ../runtime/.libs/librsyslog.a -ldl -lrt ../runtime/.libs/librsyslog.a(librsyslog_la-msg.o): In function `msgDestruct': /tmp/rsyslog-4.6.5/runtime/msg.c:803: undefined reference to `ATOMIC_INC_AND_FETCH' ../runtime/.libs/librsyslog.a(librsyslog_la-wtp.o): In function `wtpProcessThrdChanges': /tmp/rsyslog-4.6.5/runtime/wtp.c:219: undefined reference to `ATOMIC_STORE_0_TO_INT' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiSetState': /tmp/rsyslog-4.6.5/runtime/wti.c:149: undefined reference to `ATOMIC_CAS_VAL' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiWorkerCancelCleanup': /tmp/rsyslog-4.6.5/runtime/wti.c:330: undefined reference to `ATOMIC_STORE_1_TO_INT' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiWorker': /tmp/rsyslog-4.6.5/runtime/wti.c:417: undefined reference to `ATOMIC_STORE_1_TO_INT' ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiCancelThrd': /tmp/rsyslog-4.6.5/runtime/wti.c:181: undefined reference to `ATOMIC_STORE_1_TO_INT' collect2: ld returned 1 exit status make[2]: *** [rsyslogd] Error 1 make[2]: Leaving directory `/tmp/rsyslog-4.6.5/tools' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/rsyslog-4.6.5' make: *** [all] Error 2 On 11/24/10, Florian Riedl wrote: > Hi all, > > We have just released rsyslog 4.6.5, a member of the v4-stable branch. > > This update addresses the TLS bug, that has been bothering a lot of users > lately and a few other issues. It stops rsyslog from looping, thus disabling > functionality and bearing the risk of unresponsiveness of the whole system. > More details about this issue can be found here: > > http://bugzilla.adiscon.com/show_bug.cgi?id=194 > > > See ChangeLog for more details. > > ChangeLog: > > http://www.rsyslog.com/changelog-for-4-6-5-v4-stable/ > > > Download: > > http://www.rsyslog.com/rsyslog-4-6-5-v4-stable/ > > > As always, feedback is appreciated. > > Best regards, > Florian Riedl > > -- > > Support > > ======= > > > > Improving rsyslog is costly, but you can help! We are looking for > organizations that find rsyslog useful and wish to contribute back. You can > contribute by reporting bugs, improve the software, or donate money or > equipment. > > > > Commercial support contracts for rsyslog are available, and they help > finance > continued maintenance. Adiscon GmbH, a privately held German company, is > currently funding rsyslog development. We are always looking for interesting > development projects. For details on how to help, please see > http://www.rsyslog.com/doc-how2help.html > . > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From buraphalinuxserver at gmail.com Wed Nov 24 17:26:36 2010 From: buraphalinuxserver at gmail.com (Somchai Smythe) Date: Wed, 24 Nov 2010 23:26:36 +0700 Subject: [rsyslog] rsyslog 4.6.5 (v4-stable) released In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD797@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD795@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DD797@GRFEXC.intern.adiscon.com> Message-ID: Hello, 64bit kernel, 32bit userland, core2 CPU, compiled from source personal system 2.6.4 worked, but after I upgraded my compiler toolchain to 4.4.5 this was the first time I tried to build rsyslog. I build other stuff daily and had no issues with the toolchain. 2.6.4 fails with the same error now. I searched on the net and learned many people have this compile error on rsyslog with various versions. rsyslog requires me to set -march and has no graceful message if I do not. I had to modify my configure to be: ./configure \ --prefix=/usr \ --sysconfdir=/etc \ --mandir=/usr/man \ --infodir=/usr/info \ --enable-relp \ --enable-mail \ --enable-gnutls \ --build=i686-pc-linux-gnu \ CFLAGS="${CFLAGS:=-O2 -march=i686 -mtune=core2 -fno-strict-aliasing -fgnu89-inline}" and by adding that last line things work. If the configure fails the test for the features you need, configure should exit with a non-zero code and not generate the Makefile, in my opinion. Maybe configure forgot to test for this mysterious atomic bitop stuff? Anyway, I have a solution now, and I reply to the list so other people who hit this will have the answer. JGH On 11/24/10, Florian Riedl wrote: > Which was the last version, where this worked? And, which distribution (and > version) do you use? 32bit or 64bit? > > Florian > > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Somchai Smythe > Sent: Mittwoch, 24. November 2010 16:58 > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 4.6.5 (v4-stable) released > > My build won't complete (linux with gcc 4.4.5 and kernel 2.6.36.1 and > gnutls-2.10.3): > > ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/man > --infodir=/usr/info --enable-relp --enable-mail --enable-gnutls > --build=i686-pc-linux-gnu > > .... > > > /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > -Wmissing-format-attribute -g -export-dynamic -o rsyslogd > rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o > rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o > rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o -lz > -lpthread ../runtime/librsyslog.la > libtool: link: gcc -g -O2 -W -Wall -Wformat-security -Wshadow > -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o rsyslogd > rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o > rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o > rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o > -Wl,--export-dynamic -lz -lpthread ../runtime/.libs/librsyslog.a -ldl > -lrt > ../runtime/.libs/librsyslog.a(librsyslog_la-msg.o): In function > `msgDestruct': > /tmp/rsyslog-4.6.5/runtime/msg.c:803: undefined reference to > `ATOMIC_INC_AND_FETCH' > ../runtime/.libs/librsyslog.a(librsyslog_la-wtp.o): In function > `wtpProcessThrdChanges': > /tmp/rsyslog-4.6.5/runtime/wtp.c:219: undefined reference to > `ATOMIC_STORE_0_TO_INT' > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > `wtiSetState': > /tmp/rsyslog-4.6.5/runtime/wti.c:149: undefined reference to > `ATOMIC_CAS_VAL' > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > `wtiWorkerCancelCleanup': > /tmp/rsyslog-4.6.5/runtime/wti.c:330: undefined reference to > `ATOMIC_STORE_1_TO_INT' > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function `wtiWorker': > /tmp/rsyslog-4.6.5/runtime/wti.c:417: undefined reference to > `ATOMIC_STORE_1_TO_INT' > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > `wtiCancelThrd': > /tmp/rsyslog-4.6.5/runtime/wti.c:181: undefined reference to > `ATOMIC_STORE_1_TO_INT' > collect2: ld returned 1 exit status > make[2]: *** [rsyslogd] Error 1 > make[2]: Leaving directory `/tmp/rsyslog-4.6.5/tools' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/tmp/rsyslog-4.6.5' > make: *** [all] Error 2 > > > On 11/24/10, Florian Riedl wrote: >> Hi all, >> >> We have just released rsyslog 4.6.5, a member of the v4-stable branch. >> >> This update addresses the TLS bug, that has been bothering a lot of users >> lately and a few other issues. It stops rsyslog from looping, thus > disabling >> functionality and bearing the risk of unresponsiveness of the whole >> system. >> More details about this issue can be found here: >> >> http://bugzilla.adiscon.com/show_bug.cgi?id=194 >> >> >> See ChangeLog for more details. >> >> ChangeLog: >> >> http://www.rsyslog.com/changelog-for-4-6-5-v4-stable/ >> >> >> Download: >> >> http://www.rsyslog.com/rsyslog-4-6-5-v4-stable/ >> >> >> As always, feedback is appreciated. >> >> Best regards, >> Florian Riedl >> >> -- >> >> Support >> >> ======= >> >> >> >> Improving rsyslog is costly, but you can help! We are looking for >> organizations that find rsyslog useful and wish to contribute back. You > can >> contribute by reporting bugs, improve the software, or donate money or >> equipment. >> >> >> >> Commercial support contracts for rsyslog are available, and they help >> finance >> continued maintenance. Adiscon GmbH, a privately held German company, is >> currently funding rsyslog development. We are always looking for > interesting >> development projects. For details on how to help, please see >> http://www.rsyslog.com/doc-how2help.html >> . >> >> >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Nov 24 17:44:03 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Nov 2010 17:44:03 +0100 Subject: [rsyslog] rsyslog 4.6.5 (v4-stable) released References: <9B6E2A8877C38245BFB15CC491A11DA71DD795@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DD797@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD799@GRFEXC.intern.adiscon.com> The problem is rooted in this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=202 Which I do not know how to fix, "thanks" to me too-weak autoconf knowledge. Help would be appriciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Somchai Smythe > Sent: Wednesday, November 24, 2010 5:27 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 4.6.5 (v4-stable) released > > Hello, > > 64bit kernel, 32bit userland, core2 CPU, compiled from source personal > system > > 2.6.4 worked, but after I upgraded my compiler toolchain to 4.4.5 this > was the first time I tried to build rsyslog. I build other stuff > daily and had no issues with the toolchain. 2.6.4 fails with the same > error now. I searched on the net and learned many people have this > compile error on rsyslog with various versions. > > rsyslog requires me to set -march and has no graceful message if I do > not. I had to modify my configure to be: > > ./configure \ > --prefix=/usr \ > --sysconfdir=/etc \ > --mandir=/usr/man \ > --infodir=/usr/info \ > --enable-relp \ > --enable-mail \ > --enable-gnutls \ > --build=i686-pc-linux-gnu \ > CFLAGS="${CFLAGS:=-O2 -march=i686 -mtune=core2 -fno-strict-aliasing > -fgnu89-inline}" > > and by adding that last line things work. If the configure fails the > test for the features you need, configure should exit with a non-zero > code and not generate the Makefile, in my opinion. Maybe configure > forgot to test for this mysterious atomic bitop stuff? Anyway, I have > a solution now, and I reply to the list so other people who hit this > will have the answer. > > JGH > > > On 11/24/10, Florian Riedl wrote: > > Which was the last version, where this worked? And, which > distribution (and > > version) do you use? 32bit or 64bit? > > > > Florian > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Somchai > Smythe > > Sent: Mittwoch, 24. November 2010 16:58 > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog 4.6.5 (v4-stable) released > > > > My build won't complete (linux with gcc 4.4.5 and kernel 2.6.36.1 and > > gnutls-2.10.3): > > > > ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/man > > --infodir=/usr/info --enable-relp --enable-mail --enable-gnutls > > --build=i686-pc-linux-gnu > > > > .... > > > > > > /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > > -Wmissing-format-attribute -g -export-dynamic -o rsyslogd > > rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o > > rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o > > rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o -lz > > -lpthread ../runtime/librsyslog.la > > libtool: link: gcc -g -O2 -W -Wall -Wformat-security -Wshadow > > -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o > rsyslogd > > rsyslogd-syslogd.o rsyslogd-omshell.o rsyslogd-omusrmsg.o > > rsyslogd-omfwd.o rsyslogd-omfile.o rsyslogd-ompipe.o > > rsyslogd-omdiscard.o rsyslogd-iminternal.o rsyslogd-pidfile.o > > -Wl,--export-dynamic -lz -lpthread ../runtime/.libs/librsyslog.a - > ldl > > -lrt > > ../runtime/.libs/librsyslog.a(librsyslog_la-msg.o): In function > > `msgDestruct': > > /tmp/rsyslog-4.6.5/runtime/msg.c:803: undefined reference to > > `ATOMIC_INC_AND_FETCH' > > ../runtime/.libs/librsyslog.a(librsyslog_la-wtp.o): In function > > `wtpProcessThrdChanges': > > /tmp/rsyslog-4.6.5/runtime/wtp.c:219: undefined reference to > > `ATOMIC_STORE_0_TO_INT' > > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > > `wtiSetState': > > /tmp/rsyslog-4.6.5/runtime/wti.c:149: undefined reference to > > `ATOMIC_CAS_VAL' > > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > > `wtiWorkerCancelCleanup': > > /tmp/rsyslog-4.6.5/runtime/wti.c:330: undefined reference to > > `ATOMIC_STORE_1_TO_INT' > > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > `wtiWorker': > > /tmp/rsyslog-4.6.5/runtime/wti.c:417: undefined reference to > > `ATOMIC_STORE_1_TO_INT' > > ../runtime/.libs/librsyslog.a(librsyslog_la-wti.o): In function > > `wtiCancelThrd': > > /tmp/rsyslog-4.6.5/runtime/wti.c:181: undefined reference to > > `ATOMIC_STORE_1_TO_INT' > > collect2: ld returned 1 exit status > > make[2]: *** [rsyslogd] Error 1 > > make[2]: Leaving directory `/tmp/rsyslog-4.6.5/tools' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/tmp/rsyslog-4.6.5' > > make: *** [all] Error 2 > > > > > > On 11/24/10, Florian Riedl wrote: > >> Hi all, > >> > >> We have just released rsyslog 4.6.5, a member of the v4-stable > branch. > >> > >> This update addresses the TLS bug, that has been bothering a lot of > users > >> lately and a few other issues. It stops rsyslog from looping, thus > > disabling > >> functionality and bearing the risk of unresponsiveness of the whole > >> system. > >> More details about this issue can be found here: > >> > >> http://bugzilla.adiscon.com/show_bug.cgi?id=194 > >> > >> > >> See ChangeLog for more details. > >> > >> ChangeLog: > >> > >> http://www.rsyslog.com/changelog-for-4-6-5-v4-stable/ > >> > >> > >> Download: > >> > >> http://www.rsyslog.com/rsyslog-4-6-5-v4-stable/ > >> > >> > >> As always, feedback is appreciated. > >> > >> Best regards, > >> Florian Riedl > >> > >> -- > >> > >> Support > >> > >> ======= > >> > >> > >> > >> Improving rsyslog is costly, but you can help! We are looking for > >> organizations that find rsyslog useful and wish to contribute back. > You > > can > >> contribute by reporting bugs, improve the software, or donate money > or > >> equipment. > >> > >> > >> > >> Commercial support contracts for rsyslog are available, and they > help > >> finance > >> continued maintenance. Adiscon GmbH, a privately held German > company, is > >> currently funding rsyslog development. We are always looking for > > interesting > >> development projects. For details on how to help, please see > >> http://www.rsyslog.com/doc-how2help.html > >> . > >> > >> > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Wed Nov 24 22:25:21 2010 From: david at lang.hm (david at lang.hm) Date: Wed, 24 Nov 2010 13:25:21 -0800 (PST) Subject: [rsyslog] hostname / fromhost showing only node In-Reply-To: References: Message-ID: On Wed, 24 Nov 2010, Monty wrote: > Hi, > > I'm trying to filter based on the host which is sending the syslog > data. I've tried this based around hostname and fromhost, but both > only contain the node name, not the full hostname. > > I suspected this could be because the rsyslog server is running on the > same domain as the host doing the reporting, but my /etc/resolv.conf > does not have any reference to the domain, so I cannot see why rsyslog > would shorten the full hostname to just the node name. Resolving the > IP address from the command line returns the full hostname. I've seen > messages from Rainer saying that rsyslog uses the systems resolving > capacity, so I can't see why it should be any different. hostname would be whatever is put in the log by the sending server. fromhost should be a reverse lookup of the IP address you received the log from (note that if this is relayed through another host, you get the name/IP of the relay, not the sender) if you are only getting a short name, check your /etc/hosts file to see if you have it listed there with a short name first (name resolution will use /etc/hosts before going to DNS, and a name lookup into /etc/hosts will return the first name on the line) David Lang From david at lang.hm Wed Nov 24 23:37:35 2010 From: david at lang.hm (david at lang.hm) Date: Wed, 24 Nov 2010 14:37:35 -0800 (PST) Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> Message-ID: does this include the imfile fix that was in the 4.x release? David Lang On Wed, 24 Nov 2010, Florian Riedl wrote: > Date: Wed, 24 Nov 2010 16:45:51 +0100 > From: Florian Riedl > Reply-To: rsyslog-users > To: rsyslog-users > Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released > > Hi all, > > We have just released rsyslog 5.6.1, the new v5-stable. > > This update addresses the TLS bug, that has been bothering a lot of users > lately. It stops rsyslog from looping, thus disabling functionality and > bearing the risk of unresponsiveness of the whole system. More details about > this issue can be found here: > > http://bugzilla.adiscon.com/show_bug.cgi?id=194 > > > Other issues have been fixed for imptcp, failing testbench, segfault on empty > templates and failed compile. For more detailed information, please review > the ChangeLog. > > We strongly recommend to use this release if you are using (or planning to > use) syslog with TLS enabled. > > ChangeLog: > > http://www.rsyslog.com/changelog-for-5-6-1-v5-stable/ > > > Download: > > http://www.rsyslog.com/rsyslog-5-6-1-v5-stable/ > > > As always, feedback is appreciated. > > Best regards, > Florian Riedl > > > > > > -- > > Support > > ======= > > > > Improving rsyslog is costly, but you can help! We are looking for > organizations that find rsyslog useful and wish to contribute back. You can > contribute by reporting bugs, improve the software, or donate money or > equipment. > > > > Commercial support contracts for rsyslog are available, and they help finance > continued maintenance. Adiscon GmbH, a privately held German company, is > currently funding rsyslog development. We are always looking for interesting > development projects. For details on how to help, please see > http://www.rsyslog.com/doc-how2help.html > . > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Thu Nov 25 07:28:48 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 25 Nov 2010 07:28:48 +0100 Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released References: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD79C@GRFEXC.intern.adiscon.com> Good point. I have to admit at least part of it is not included. I was so obsessed with finally fixing the TLS issue that I forgot to merge that one :( I'll update the -beta and -devel versions today and will see that I merge the right things into it now. Sorry for that... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, November 24, 2010 11:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 5.6.1 (v5-stable) released > > does this include the imfile fix that was in the 4.x release? > > David Lang > > On Wed, 24 Nov 2010, Florian Riedl wrote: > > > Date: Wed, 24 Nov 2010 16:45:51 +0100 > > From: Florian Riedl > > Reply-To: rsyslog-users > > To: rsyslog-users > > Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released > > > > Hi all, > > > > We have just released rsyslog 5.6.1, the new v5-stable. > > > > This update addresses the TLS bug, that has been bothering a lot of > users > > lately. It stops rsyslog from looping, thus disabling functionality > and > > bearing the risk of unresponsiveness of the whole system. More > details about > > this issue can be found here: > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=194 > > > > > > Other issues have been fixed for imptcp, failing testbench, segfault > on empty > > templates and failed compile. For more detailed information, please > review > > the ChangeLog. > > > > We strongly recommend to use this release if you are using (or > planning to > > use) syslog with TLS enabled. > > > > ChangeLog: > > > > http://www.rsyslog.com/changelog-for-5-6-1-v5-stable/ > > > > > > Download: > > > > http://www.rsyslog.com/rsyslog-5-6-1-v5-stable/ > > > > > > As always, feedback is appreciated. > > > > Best regards, > > Florian Riedl > > > > > > > > > > > > -- > > > > Support > > > > ======= > > > > > > > > Improving rsyslog is costly, but you can help! We are looking for > > organizations that find rsyslog useful and wish to contribute back. > You can > > contribute by reporting bugs, improve the software, or donate money > or > > equipment. > > > > > > > > Commercial support contracts for rsyslog are available, and they help > finance > > continued maintenance. Adiscon GmbH, a privately held German > company, is > > currently funding rsyslog development. We are always looking for > interesting > > development projects. For details on how to help, please see > > http://www.rsyslog.com/doc-how2help.html > > . > > > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From sod.thiago at gmail.com Thu Nov 25 14:10:44 2010 From: sod.thiago at gmail.com (Thiago Anderson) Date: Thu, 25 Nov 2010 11:10:44 -0200 Subject: [rsyslog] ::: Rsyslog 5.6.0 + OmOracle Message-ID: Hello everyone, I am trying to configure rsyslog to store data in Oracle and some are experiencing problems when trying to climb the service. Some information from the OS (this machine is a virtual machine) : [root at clockingit sbin]# uname -a Linux clockingit 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:40 EST 2010 i686 i686 i386 GNU/Linux [root at clockingit sbin]# cat /etc/redhat-release CentOS release 5.5 (Final) VMtools installed. The build of rsyslog was made as follows: 1) Installed the packages from Oracle: rpm -hiv oracle-instantclient-basic-10.2.0.4-1.i386.rpm rpm -hiv oracle-instantclient-devel-10.2.0.4-1.i386.rpm rpm -hiv oracle-instantclient-odbc-10.2.0.4-1.i386.rpm rpm -hiv oracle-instantclient-sqlplus-10.2.0.4-1.i386.rpm 2) Installed other packages: yum install mysql-devel libdbi libdbi-devel gnutls-devel net-snmp-libs net-snmp-devel librelp-devel automake 3) Installed liblogging wget http://download.rsyslog.com/liblogging/liblogging-0.7.1.tar.gz tar zxpvf liblogging-0.7.1.tar.gz cd liblogging ./configure ; make ; make install 4) Compile Syslog wget http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.6.0.tar.gz tar zxpvf rsyslog-5.6.0.tar.gz cd rsyslog-5.6.0 LIBLOGGING_LIBS=/usr/local/lib LIBLOGGING_CFLAGS=-I/usr/local/include ./configure --enable-rtinst --enable-oracle --enable-mysql --enable-gssapi-krb5 --enable-debug --enable-memcheck --enable-libdbi --enable-mail --enable-relp --enable-rfc3195 --enable-imfile --enable-imtemplate --enable-omtemplate --enable-omuxsock --enable-snmp --enable-libdbi --prefix=/usr/local/rsyslog make make install 5) Created /etc/ld.so.conf.d/oracle.conf, with the following settings: /usr/lib/oracle/10.2.0.4/client/lib 6) A few more additional procedures: mkdir /usr/local/rsyslog/etc cp /usr/local/src/rsyslog-5.6.0/rsyslog.conf /usr/local/rsyslog/etc/ ln -s /usr/local/rsyslog/etc/rsyslog.conf /etc/rsyslog.conf /etc/init.d/syslog stop chkconfig syslog off 7) Making the necessary settings in /etc/profile export NLS_LANG="AMERICAN_AMERICA.WE8ISO8859P1" export NLS_DATE_FORMAT="YYYY_MM_DD HH24:MI:SS" export NLS_DATE_FORMAT="YYYY_MM_DD HH24:MI:SS" export PATH=/usr/lib/oracle/10.2.0.4/client:/usr/local/bin:$PATH export ORACLE_SID=RECPOC export TNS_ADMIN=/usr/lib/oracle/10.2.0.4/client export LD_LIBRARY_PATH="/usr/lib/oracle/10.2.0.4/client/lib" 8) Making the necessary settings in /usr/local/rsyslog/etc/rsyslog.conf .The rsyslog stunnel receives the information, some settings were made with mysql and they are functioning normally, but this test server, not working with Oracle. The settings for the connection with the oracle were as follows(full file - rsyslog.conf): ------------------ cat /etc/rsyslog.conf ----------------- [root at clockingit etc]# cat rsyslog.conf # if you experience problems, check # http://www.rsyslog.com/troubleshoot for assistance # rsyslog v3: load input modules # If you do not load inputs, nothing happens! # You may need to set the module load path if modules are not found. $ModLoad immark # provides --MARK-- message capability $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # kernel logging (formerly provided by rklogd) $ModLoad imtcp $InputTCPServerRun 61514 $AllowedSender TCP, 127.0.0.1 $ModLoad omoracle $OmoracleDBUser syslog $OmoracleDBPassword syslog $OmoracleDB RECPOC $OmoracleBatchSize 1 $OmoracleBatchItemSize 4096 $OmoracleStatementTemplate OmoracleStatement $template OmoracleStatement,"insert into SystemEvents(fromhost,message) values (:host,:message)" $template RsysLog,"%hostname%%msg%" *.* :omoracle:;RsysLog # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none -/var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* -/var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit -/var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log # Remote Logging (we use TCP for reliable delivery) # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. #$WorkDirectory /rsyslog/spool # where to place spool files #$ActionQueueFileName uniqName # unique name prefix for spool files #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown #$ActionQueueType LinkedList # run asynchronously #$ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional #*.* @@remote-host:514 # ######### Receiving Messages from Remote Hosts ########## # TCP Syslog Server: # provides TCP syslog reception and GSS-API (if compiled to support it) #$ModLoad imtcp.so # load module #$InputTCPServerRun 514 # start up TCP listener at port 514 # UDP Syslog Server: #$ModLoad imudp.so # provides UDP syslog reception #$UDPServerRun 514 # start a UDP syslog server at standard port 514 ----------------- EOF /etc/rsyslog ----------------- The problem: When I remove the module configuration of Oracle (omoracle), rsyslog up the service normally, but with the same configuration omoracle not rise. The start comand: /usr/local/rsyslog/sbin/rsyslogd -r -d ( debug ) Below is the debug: 9386.459460668:b7f776c0: rsyslogd 5.6.0 startup, compatibility mode 0, module path '', cwd:/usr/local/rsyslog/sbin 9386.460705992:b7f776c0: caller requested object 'net', not found (iRet -3003) 9386.460834286:b7f776c0: Requested to load module 'lmnet' 9386.460900782:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/lmnet.so' 9386.461267774:b7f776c0: module of type 2 being loaded. 9386.461342025:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.461402825:b7f776c0: source file conf.c requested reference for module 'lmnet', reference count now 1 9386.461632584:b7f776c0: rsyslog runtime initialized, version 5.6.0, current users 1 9386.461978209:b7f776c0: source file syslogd.c requested reference for module 'lmnet', reference count now 2 9386.464242550:b7f776c0: omfile: using transactional output interface. 9386.464437929:b7f776c0: module of type 1 being loaded. 9386.464540645:b7f776c0: module of type 1 being loaded. 9386.464605885:b7f776c0: entry point 'beginTransaction' not present in module 9386.464693632:b7f776c0: entry point 'endTransaction' not present in module 9386.464779977:b7f776c0: source file omfwd.c requested reference for module 'lmnet', reference count now 3 9386.464932176:b7f776c0: module of type 1 being loaded. 9386.464991435:b7f776c0: entry point 'doHUP' not present in module 9386.465044665:b7f776c0: entry point 'beginTransaction' not present in module 9386.465102420:b7f776c0: entry point 'endTransaction' not present in module 9386.465162298:b7f776c0: module of type 1 being loaded. 9386.465218887:b7f776c0: entry point 'doHUP' not present in module 9386.465273380:b7f776c0: entry point 'beginTransaction' not present in module 9386.465337106:b7f776c0: entry point 'endTransaction' not present in module 9386.465413925:b7f776c0: module of type 1 being loaded. 9386.465472438:b7f776c0: entry point 'doHUP' not present in module 9386.465526283:b7f776c0: entry point 'beginTransaction' not present in module 9386.465585290:b7f776c0: entry point 'endTransaction' not present in module 9386.465644377:b7f776c0: module of type 1 being loaded. 9386.465718748:b7f776c0: entry point 'doHUP' not present in module 9386.465772221:b7f776c0: entry point 'beginTransaction' not present in module 9386.465831964:b7f776c0: entry point 'endTransaction' not present in module 9386.465925594:b7f776c0: rfc5424 parser init called 9386.465992447:b7f776c0: GetParserName addr 0x805b3b0 9386.466047803:b7f776c0: module of type 3 being loaded. 9386.466142637:b7f776c0: Parser 'rsyslog.rfc5424' added to list of available parsers. 9386.466227575:b7f776c0: rfc3164 parser init called 9386.466282578:b7f776c0: module of type 3 being loaded. 9386.466348328:b7f776c0: Parser 'rsyslog.rfc3164' added to list of available parsers. 9386.466405679:b7f776c0: Parser 'rsyslog.rfc5424' added to default parser set. 9386.466459692:b7f776c0: Parser 'rsyslog.rfc3164' added to default parser set. 9386.466514601:b7f776c0: rsyslog standard file format strgen init called, compiled with version 5.6.0 9386.466571888:b7f776c0: module of type 4 being loaded. 9386.466638748:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.466704220:b7f776c0: Strgen 'RSYSLOG_FileFormat' added to list of available strgens. 9386.466775892:b7f776c0: traditional file format strgen init called, compiled with version 5.6.0 9386.466866308:b7f776c0: module of type 4 being loaded. 9386.466921818:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.466987684:b7f776c0: Strgen 'RSYSLOG_TraditionalFileFormat' added to list of available strgens. 9386.467047889:b7f776c0: rsyslog standard (network) forward format strgen init called, compiled with version 5.6.0 9386.467111051:b7f776c0: module of type 4 being loaded. 9386.467164644:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.467220176:b7f776c0: Strgen 'RSYSLOG_ForwardFormat' added to list of available strgens. 9386.467279229:b7f776c0: rsyslog traditional (network) forward format strgen init called, compiled with version 5.6.0 9386.467336561:b7f776c0: module of type 4 being loaded. 9386.467454640:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.467531246:b7f776c0: Strgen 'RSYSLOG_TraditionalForwardFormat' added to list of available strgens. 9386.467949432:b7f776c0: Called LogError, msg: WARNING: rsyslogd is running in compatibility mode. Automatically gene rated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c4 as the first rsyslogd option. 9386.468402036:b7f776c0: Checking pidfile. 9386.473203091:b7f776c0: Writing pidfile /var/run/rsyslogd.pid. 9386.475178441:b7f776c0: template bound to strgen 'RSYSLOG_FileFormat' 9386.475244795:b7f776c0: template bound to strgen 'RSYSLOG_TraditionalFileFormat' 9386.475376888:b7f776c0: template bound to strgen 'RSYSLOG_ForwardFormat' 9386.475472607:b7f776c0: template bound to strgen 'RSYSLOG_TraditionalForwardFormat' 9386.475695660:b7f776c0: rsyslog 5.6.0 - called init() 9386.476121338:b7f776c0: cfline: '$ModLoad immark # provides --MARK-- message capability' 9386.476427233:b7f776c0: Requested to load module 'immark' 9386.476592864:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/immark.so' 9386.477409072:b7f776c0: module of type 0 being loaded. 9386.477513727:b7f776c0: cfline: '$ModLoad imuxsock # provides support for local system logging (e.g. via logger comm and)' 9386.477584738:b7f776c0: Requested to load module 'imuxsock' 9386.477654122:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/imuxsock.so' 9386.478017990:b7f776c0: imuxsock version 5.6.0 initializing 9386.478151092:b7f776c0: module of type 0 being loaded. 9386.478219197:b7f776c0: cfline: '$ModLoad imklog # kernel logging (formerly provided by rklogd)' 9386.478283473:b7f776c0: Requested to load module 'imklog' 9386.478361324:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/imklog.so' 9386.478843620:b7f776c0: module of type 0 being loaded. 9386.478915168:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.478979414:b7f776c0: cfline: '$ModLoad imtcp' 9386.479061825:b7f776c0: Requested to load module 'imtcp' 9386.479132896:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/imtcp.so' 9386.479505382:b7f776c0: source file imtcp.c requested reference for module 'lmnet', reference count now 4 9386.479616224:b7f776c0: caller requested object 'netstrm', not found (iRet -3003) 9386.479734450:b7f776c0: Requested to load module 'lmnetstrms' 9386.479795370:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/lmnetstrms.so' 9386.480155046:b7f776c0: doing nsselClassInit 9386.480224610:b7f776c0: doing nspollClassInit 9386.480336922:b7f776c0: module of type 2 being loaded. 9386.480396020:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.480468134:b7f776c0: source file imtcp.c requested reference for module 'lmnetstrms', reference count now 1 9386.480533771:b7f776c0: caller requested object 'tcps_sess', not found (iRet -3003) 9386.480593409:b7f776c0: Requested to load module 'lmtcpsrv' 9386.480647962:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/lmtcpsrv.so' 9386.480910820:b7f776c0: source file tcps_sess.c requested reference for module 'lmnetstrms', reference count now 2 9386.481017830:b7f776c0: source file tcpsrv.c requested reference for module 'lmnet', reference count now 5 9386.481106586:b7f776c0: source file tcpsrv.c requested reference for module 'lmnetstrms', reference count now 3 9386.481279537:b7f776c0: module of type 2 being loaded. 9386.481355190:b7f776c0: entry point 'isCompatibleWithFeature' not present in module 9386.481413452:b7f776c0: source file imtcp.c requested reference for module 'lmtcpsrv', reference count now 1 9386.481486050:b7f776c0: source file imtcp.c requested reference for module 'lmtcpsrv', reference count now 2 9386.481660287:b7f776c0: module of type 0 being loaded. 9386.481731246:b7f776c0: cfline: '$InputTCPServerRun 61514' 9386.481917562:b7f776c0: cfline: '$AllowedSender TCP, 127.0.0.1' 9386.482529296:b7f776c0: cfline: '$ModLoad omoracle' 9386.482600735:b7f776c0: Requested to load module 'omoracle' 9386.482701306:b7f776c0: loading module '/usr/local/rsyslog/lib/rsyslog/omoracle.so' 9386.522362483:b7f776c0: module of type 1 being loaded. 9386.522478241:b7f776c0: entry point 'doHUP' not present in module 9386.522537748:b7f776c0: entry point 'beginTransaction' not present in module 9386.522595530:b7f776c0: entry point 'endTransaction' not present in module 9386.522740094:b7f776c0: cfline: '$OmoracleDBUser syslog' 9386.522820278:b7f776c0: cfline: '$OmoracleDBPassword syslog' 9386.522883951:b7f776c0: cfline: '$OmoracleDB RECPOC' 9386.522954785:b7f776c0: cfline: '$OmoracleBatchSize 1' 9386.523071781:b7f776c0: cfline: '$OmoracleBatchItemSize 4096' 9386.523137524:b7f776c0: cfline: '$OmoracleStatementTemplate OmoracleStatement' 9386.523209960:b7f776c0: cfline: '$template OmoracleStatement,"insert into SystemEvents(fromhost,message) values (:ho st,:message)"' 9386.523282212:b7f776c0: cfline: '$template RsysLog,"%hostname%%msg%"' 9386.523348619:b7f776c0: cfline: '*.* :omoracle:;RsysLog' 9386.523465941:b7f776c0: - traditional PRI filter 9386.523524791:b7f776c0: symbolic name: * ==> 255 9386.523626345:b7f776c0: tried selector action for builtin-file: -2001 9386.523709307:b7f776c0: tried selector action for builtin-pipe: -2001 9386.523788816:b7f776c0: tried selector action for builtin-fwd: -2001 9386.523862223:b7f776c0: tried selector action for builtin-shell: -2001 9386.523940483:b7f776c0: tried selector action for builtin-discard: -2001 9386.523998958:b7f776c0: tried selector action for builtin-usrmsg: -2001 9386.529029280:b7f776c0: omoracle will run stored statement: insert into SystemEvents(fromhost,message) values (:host ,:message) 9386.529107502:b7f776c0: omoracle statement has 2 parameters 9386.565579406:b7f776c0: omoracle module got all its resources allocated and connected to the DB 9386.565691677:b7f776c0: tried selector action for omoracle: 0 9386.565837761:b7f776c0: Module omoracle processed this config line. 9386.565971290:b7f776c0: template: 'RsysLog' assigned 9386.566026080:b7f776c0: module is incompatible with RepeatedMsgReduction - turned off 9386.566212920:b7f776c0: action 1 queue: save on shutdown 1, max disk space allowed 0 9386.566355659:b7f776c0: action 1 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.566438689:b7f776c0: Action 0x91cea00: queue 0x91cefe8 created 9386.566508579:b7f776c0: module is incompatible with RepeatedMsgReduction - turned off 9386.566604932:b7f776c0: cfline: '*.info;mail.none;authpriv.none;cron.none -/var/log/messages' 9386.566683139:b7f776c0: selector line successfully processed 9386.566744415:b7f776c0: - traditional PRI filter 9386.566794405:b7f776c0: symbolic name: info ==> 6 9386.566868221:b7f776c0: symbolic name: none ==> 16 9386.567036976:b7f776c0: symbolic name: mail ==> 16 9386.567103787:b7f776c0: symbolic name: none ==> 16 9386.567184987:b7f776c0: symbolic name: authpriv ==> 80 9386.567251678:b7f776c0: symbolic name: none ==> 16 9386.567317016:b7f776c0: symbolic name: cron ==> 72 9386.567565043:b7f776c0: file stream messages params: flush interval 0, async write 0 9386.567623485:b7f776c0: tried selector action for builtin-file: 0 9386.567673051:b7f776c0: Module builtin-file processed this config line. 9386.567733697:b7f776c0: template: 'RSYSLOG_FileFormat' assigned 9386.567807817:b7f776c0: action 2 queue: save on shutdown 1, max disk space allowed 0 9386.567878814:b7f776c0: action 2 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.567978317:b7f776c0: Action 0x91d0158: queue 0x91d0238 created 9386.568035641:b7f776c0: cfline: 'authpriv.* /var/log/secure' 9386.568092095:b7f776c0: selector line successfully processed 9386.568143078:b7f776c0: - traditional PRI filter 9386.568208048:b7f776c0: symbolic name: * ==> 255 9386.568274084:b7f776c0: symbolic name: authpriv ==> 80 9386.568367669:b7f776c0: file stream secure params: flush interval 0, async write 0 9386.568474510:b7f776c0: tried selector action for builtin-file: 0 9386.568523708:b7f776c0: Module builtin-file processed this config line. 9386.568577440:b7f776c0: template: 'RSYSLOG_FileFormat' assigned 9386.568631236:b7f776c0: action 3 queue: save on shutdown 1, max disk space allowed 0 9386.568699026:b7f776c0: action 3 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.568774244:b7f776c0: Action 0x91d0c38: queue 0x91d0ce8 created 9386.568829525:b7f776c0: cfline: 'mail.* -/var/log/maillog' 9386.568883905:b7f776c0: selector line successfully processed 9386.568934682:b7f776c0: - traditional PRI filter 9386.569013545:b7f776c0: symbolic name: * ==> 255 9386.569078751:b7f776c0: symbolic name: mail ==> 16 9386.569160052:b7f776c0: file stream maillog params: flush interval 0, async write 0 9386.569223931:b7f776c0: tried selector action for builtin-file: 0 9386.569274929:b7f776c0: Module builtin-file processed this config line. 9386.569327821:b7f776c0: template: 'RSYSLOG_FileFormat' assigned 9386.569381624:b7f776c0: action 4 queue: save on shutdown 1, max disk space allowed 0 9386.569447802:b7f776c0: action 4 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.569525915:b7f776c0: Action 0x91d16e8: queue 0x91d1798 created 9386.569580318:b7f776c0: cfline: 'cron.* -/var/log/cron' 9386.569634905:b7f776c0: selector line successfully processed 9386.569686529:b7f776c0: - traditional PRI filter 9386.569742387:b7f776c0: symbolic name: * ==> 255 9386.569808861:b7f776c0: symbolic name: cron ==> 72 9386.569888445:b7f776c0: file stream cron params: flush interval 0, async write 0 9386.569940058:b7f776c0: tried selector action for builtin-file: 0 9386.570037472:b7f776c0: Module builtin-file processed this config line. 9386.570093266:b7f776c0: template: 'RSYSLOG_FileFormat' assigned 9386.570146526:b7f776c0: action 5 queue: save on shutdown 1, max disk space allowed 0 9386.570223255:b7f776c0: action 5 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.570297851:b7f776c0: Action 0x91d2168: queue 0x91d2248 created 9386.570352854:b7f776c0: cfline: '*.emerg *' 9386.570406496:b7f776c0: selector line successfully processed 9386.570457446:b7f776c0: - traditional PRI filter 9386.570514684:b7f776c0: symbolic name: emerg ==> 0 9386.570581375:b7f776c0: tried selector action for builtin-file: -2001 9386.570635842:b7f776c0: tried selector action for builtin-pipe: -2001 9386.570692727:b7f776c0: tried selector action for builtin-fwd: -2001 9386.570746459:b7f776c0: tried selector action for builtin-shell: -2001 9386.570806708:b7f776c0: tried selector action for builtin-discard: -2001 9386.570858078:b7f776c0: write-alltried selector action for builtin-usrmsg: 0 9386.570926351:b7f776c0: Module builtin-usrmsg processed this config line. 9386.570979648:b7f776c0: template: ' WallFmt' assigned 9386.571063544:b7f776c0: action 6 queue: save on shutdown 1, max disk space allowed 0 9386.571132129:b7f776c0: action 6 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.571199807:b7f776c0: Action 0x91d26e0: queue 0x91d2868 created 9386.571299133:b7f776c0: cfline: 'uucp,news.crit -/var/log/spooler' 9386.571353585:b7f776c0: selector line successfully processed 9386.571404951:b7f776c0: - traditional PRI filter 9386.571454614:b7f776c0: symbolic name: crit ==> 2 9386.571524253:b7f776c0: symbolic name: uucp ==> 64 9386.571591900:b7f776c0: symbolic name: news ==> 56 9386.571672245:b7f776c0: file stream spooler params: flush interval 0, async write 0 9386.571751907:b7f776c0: tried selector action for builtin-file: 0 9386.571809509:b7f776c0: Module builtin-file processed this config line. 9386.571862940:b7f776c0: template: 'RSYSLOG_FileFormat' assigned 9386.571916845:b7f776c0: action 7 queue: save on shutdown 1, max disk space allowed 0 9386.571984597:b7f776c0: action 7 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.572080343:b7f776c0: Action 0x91d32a0: queue 0x91d3350 created 9386.572135590:b7f776c0: cfline: 'local7.* /var/log/boot.log' 9386.572189449:b7f776c0: selector line successfully processed 9386.572250062:b7f776c0: - traditional PRI filter 9386.572304671:b7f776c0: symbolic name: * ==> 255 9386.572371007:b7f776c0: symbolic name: local7 ==> 184 9386.572452469:b7f776c0: file stream boot.log params: flush interval 0, async write 0 9386.572504247:b7f776c0: tried selector action for builtin-file: 0 9386.572560229:b7f776c0: Module builtin-file processed this config line. 9386.572613979:b7f776c0: template: 'RSYSLOG_FileFormat' assigned 9386.572666744:b7f776c0: action 8 queue: save on shutdown 1, max disk space allowed 0 9386.572734016:b7f776c0: action 8 queue: type 3, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 970, light delay 700, deq batch size 16 starting 9386.572807341:b7f776c0: Action 0x91d3cf8: queue 0x91d3da8 created 9386.572882916:b7f776c0: selector line successfully processed 9386.573017334:b7f776c0: Called LogError, msg: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark 9386.573195235:b7f776c0: Requested to load module 'immark' 9386.573279933:b7f776c0: Module 'immark' already loaded 9386.573347258:b7f776c0: Called LogError, msg: Warning: backward compatibility layer added to following directive to rsyslog.conf: MarkMessagePeriod 1200 9386.573479519:b7f776c0: Called LogError, msg: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad imuxsock 9386.573553242:b7f776c0: Requested to load module 'imuxsock' 9386.573610487:b7f776c0: Module 'imuxsock' already loaded 9386.573941286:b7f776c0: main Q: is NOT disk-assisted 9386.574012046:b7f776c0: main Q: type 0, enq-only 0, disk assisted 0, maxFileSz 1048576, lqsize 0, pqsize 0, child 0, full delay 9700, light delay 7000, deq batch size 32 starting 9386.574179252:b7f776c0: main Q:Reg: finalizing construction of worker thread pool 9386.574255690:b7f776c0: main Q:Reg/w0: finalizing construction of worker instance data 9386.574341877:b7f776c0: main Q: queue finished initialization 9386.574431360:b7f776c0: Main processing queue is initialized and running 9386.574633706:b7f776c0: Opened UNIX socket '/dev/log' (fd 0). 9386.574792483:b7f776c0: main Q: entry added, size now log 1, phys 1 entries 9386.574870990:b7f776c0: main Q:Reg: high activity - starting 1 additional worker thread(s). 9386.575123214:b7f776c0: main Q:Reg: started with state 0, num workers now 1 9386.575186646:b7f776c0: main Q: EnqueueMsg advised worker start 9386.575337849:b7f776c0: Allowed TCP Senders: 9386.575456832:b7f776c0: 127.0.0.1/32 9386.575524971:b7f776c0: All Rulesets: 9386.575581144:b7f776c0: ruleset 0x91841c0: rsyslog ruleset RSYSLOG_DefaultRuleset: 9386.575655923:b7f776c0: rule 0x918c598: rsyslog rule: 9386.575721224:b7f776c0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF Actions: 9386.576187155:b7f776c0: omoracle: Instance data: 0x918d088 9386.576256366:b7f776c0: RepeatedMsgReduction: 0 9386.576305853:b7f776c0: Resume Interval: 30 9386.576354837:b7f776c0: State: rdy 9386.576455608:b7f776c0: Exec only when previous is suspended: 0 9386.576514919:b7f776c0: submission mode: fast, but supports partial mark messages 9386.576570590:b7f776c0: 9386.576621952:b7f776c0: 9386.576676587:b7f776c0: 9386.576727105:b7f776c0: rule 0x91cf4f8: rsyslog rule: 9386.576792642:b7f776c0: 7F 7F X 7F 7F 7F 7F 7F 7F X X 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F Actions: 9386.577304041:b7f776c0: builtin-file: /var/log/messages 9386.577372229:b7f776c0: template='/var/log/messages' 9386.577428844:b7f776c0: use async writer=0 9386.577656037:b7f776c0: flush on TX end=1 9386.577714610:b7f776c0: flush interval=1 9386.577764423:b7f776c0: file cache size=10 9386.577814431:b7f776c0: create directories: yes 9386.577863828:b7f776c0: file owner 0, group 0 9386.577913255:b7f776c0: force chown() for all files: no 9386.577970313:b7f776c0: directory owner 0, group 0 9386.578020423:b7f776c0: dir create mode 0700, file create mode 0644 9386.578069711:b7f776c0: fail if owner/group can not be set: no 9386.578119048:b7f776c0: Instance data: 0x91cf5f8 9386.578192988:b7f776c0: RepeatedMsgReduction: 0 9386.578243769:b7f776c0: Resume Interval: 30 9386.578327323:b7f776c0: State: rdy 9386.578378738:b7f776c0: Exec only when previous is suspended: 0 9386.578432511:b7f776c0: submission mode: fast, but supports partial mark messages 9386.578484454:b7f776c0: 9386.578531920:b7f776c0: 9386.578580083:b7f776c0: 9386.578641018:b7f776c0: rule 0x91d0518: rsyslog rule: 9386.578718067:b7f776c0: X X X X X X X X X X FF X X X X X X X X X X X X X X Actions: 9386.579149354:b7f776c0: builtin-file: /var/log/secure 9386.579245770:b7f776c0: template='/var/log/secure' 9386.579295227:b7f776c0: use async writer=0 9386.579344444:b7f776c0: flush on TX end=1 9386.579393639:b7f776c0: flush interval=1 9386.579441937:b7f776c0: file cache size=10 9386.579497476:b7f776c0: create directories: yes 9386.579546705:b7f776c0: file owner 0, group 0 9386.579596098:b7f776c0: force chown() for all files: no 9386.579658743:b7f776c0: directory owner 0, group 0 9386.579715869:b7f776c0: dir create mode 0700, file create mode 0644 9386.579769859:b7f776c0: fail if owner/group can not be set: no 9386.579821030:b7f776c0: Instance data: 0x91d0578 9386.579870952:b7f776c0: RepeatedMsgReduction: 0 9386.579921163:b7f776c0: Resume Interval: 30 9386.579985567:b7f776c0: State: rdy 9386.580036430:b7f776c0: Exec only when previous is suspended: 0 9386.580097515:b7f776c0: submission mode: fast, but supports partial mark messages 9386.580169464:b7f776c0: 9386.580262910:b7f776c0: 9386.580315948:b7f776c0: 9386.580406245:b7f776c0: rule 0x91d0fc8: rsyslog rule: 9386.580492312:b7f776c0: X X FF X X X 9386.580768711:1959b90: we deleted 0 objects and enqueued 0 objects 9386.580890979:1959b90: delete batch from store, new sizes: log 1, phys 1 9386.580988460:b7f776c0: X 9386.581127869:1959b90: ZZZ: processBatch: batch of 1 elements must be processed 9386.581222460:1959b90: Processing next rule 9386.581304162:1959b90: testing filter, f_pmask 255 9386.581486881:1959b90: Processing next action 9386.581573503:1959b90: Called action(NotAllMark), logging to omoracle 9386.581649007:1959b90: Called action(Batch), logging to omoracle 9386.581763100:b7f776c0: X X X X X X X X X X X X X X X X 9386.582340610:1959b90: Action 0x91cea00 transitioned to state: itx 9386.582412491:1959b90: entering actionCalldoAction(), state: itx 9386.582506631:1959b90: batch[0][0]=clockingit 9386.582591337:1959b90: batch[1][0]=imklog 5.6.0, log source = /proc/kmsg started. 9386.582671588:1959b90: Action 0x91cea00 transitioned to state: rdy 9386.582766553:1959b90: action call returned 0 9386.582845848:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.582916863:1959b90: Processing next rule 9386.582992884:1959b90: testing filter, f_pmask 127 9386.583064964:1959b90: Processing next action 9386.583143186:1959b90: Called action(NotAllMark), logging to builtin-file 9386.583214358:1959b90: Called action(Batch), logging to builtin-file 9386.583312598:b7f776c0: X X Actions: 9386.583473988:1959b90: Action 0x91d0158 transitioned to state: itx 9386.583562230:1959b90: entering actionCalldoAction(), state: itx 9386.583639511:1959b90: file to log to: /var/log/messages 9386.583727524:b7f776c0: builtin-file: /var/log/maillog 9386.583841434:b7f776c0: template='/var/log/maillog' 9386.583941326:b7f776c0: use async writer=0 9386.584008648:b7f776c0: flush on TX end=1 9386.584077795:b7f776c0: flush interval=1 9386.584142349:b7f776c0: file cache size=10 9386.584207507:b7f776c0: create directories: yes 9386.584290397:b7f776c0: file owner 0, group 0 9386.584361424:b7f776c0: force chown() for all files: no 9386.584426251:b7f776c0: directory owner 0, group 0 9386.584490888:b7f776c0: dir create mode 0700, file create mode 0644 9386.584555670:b7f776c0: fail if owner/group can not be set: no 9386.584628568:b7f776c0: Instance data: 0x91d1028 9386.584693651:b7f776c0: RepeatedMsgReduction: 0 9386.584761043:b7f776c0: Resume Interval: 30 9386.584825980:b7f776c0: State: rdy 9386.584893946:b7f776c0: Exec only when previous is suspended: 0 9386.584960206:b7f776c0: submission mode: fast, but supports partial mark messages 9386.585025285:b7f776c0: 9386.585093442:b7f776c0: 9386.585157565:b7f776c0: 9386.585228134:b7f776c0: rule 0x91d1a28: rsyslog rule: 9386.585332133:b7f776c0: X X 9386.585451971:1959b90: write to stream, pData->pStrm 0x91cf948, lenBuf 99 9386.585550011:b7f776c0: X X X X 9386.585730586:1959b90: action call returned -2121 9386.585828334:b7f776c0: X X 9386.585957254:1959b90: strm 0x91cf948: file -1(messages) flush, buflen 99 9386.586073961:b7f776c0: X 9386.586220078:1959b90: file '/var/log/messages' opened as #6 with mode 420 9386.586327415:b7f776c0: FF X 9386.586458548:1959b90: strm 0x91cf948: opened file '/var/log/messages' for WRITE as 6 9386.586568572:b7f776c0: X X 9386.586696030:1959b90: strm 0x91cf948: file 6 write wrote 99 bytes 9386.586804728:1959b90: Action 0x91d0158 transitioned to state: rdy 9386.586883625:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.586955274:1959b90: Processing next rule 9386.587026157:1959b90: testing filter, f_pmask 0 9386.587104350:1959b90: Processing next action 9386.587177146:1959b90: Called action(NotAllMark), logging to builtin-file 9386.587248165:1959b90: Called action(Batch), logging to builtin-file 9386.587331310:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.587402497:1959b90: Processing next rule 9386.587472399:1959b90: testing filter, f_pmask 0 9386.587546376:1959b90: Processing next action 9386.587667695:1959b90: Called action(NotAllMark), logging to builtin-file 9386.587738388:1959b90: Called action(Batch), logging to builtin-file 9386.587826412:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.587902684:1959b90: Processing next rule 9386.587973092:1959b90: testing filter, f_pmask 0 9386.588043439:1959b90: Processing next action 9386.588122494:1959b90: Called action(NotAllMark), logging to builtin-file 9386.588194582:1959b90: Called action(Batch), logging to builtin-file 9386.588270494:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.588346575:1959b90: Processing next rule 9386.588417567:1959b90: testing filter, f_pmask 1 9386.588487146:1959b90: Processing next action 9386.588557925:1959b90: Called action(NotAllMark), logging to builtin-usrmsg 9386.588635067:1959b90: Called action(Batch), logging to builtin-usrmsg 9386.588711403:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.588782846:1959b90: Processing next rule 9386.588868106:1959b90: testing filter, f_pmask 0 9386.588939080:1959b90: Processing next action 9386.589008314:1959b90: Called action(NotAllMark), logging to builtin-file 9386.589078624:1959b90: Called action(Batch), logging to builtin-file 9386.589162482:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.589233257:1959b90: Processing next rule 9386.589305922:1959b90: testing filter, f_pmask 0 9386.589377897:1959b90: Processing next action 9386.589449538:1959b90: Called action(NotAllMark), logging to builtin-file 9386.589518892:1959b90: Called action(Batch), logging to builtin-file 9386.589594587:1959b90: ruleset: get iRet 0 from rule.ProcessMsg() 9386.589672396:1959b90: ruleset.ProcessMsg() returns 0 9386.589745185:1959b90: regular consumer finished, iret=0, szlog 0 sz phys 1 9386.589836925:b7f776c0: X X 9386.589951307:1959b90: queue.c:1674:ConsumerReg:invocation 1: WARNING: mutex still owned by us as we exit function, mutex: 0x91d35a8[0/1959b90] 9386.590105461:b7f776c0: X X X X X X X X X 9386.590446496:1959b90: we deleted 1 objects and enqueued 0 objects 9386.590531446:1959b90: delete batch from store, new sizes: log 0, phys 0 9386.590607013:1959b90: regular consumer finished, iret=4, szlog 0 sz phys 0 9386.590695559:1959b90: main Q:Reg/w0: worker IDLE, waiting for work. 9386.590783321:b7f776c0: X Actions: 9386.590866924:b7f776c0: builtin-file: /var/log/cron 9386.590933383:b7f776c0: template='/var/log/cron' 9386.590990384:b7f776c0: use async writer=0 9386.591039230:b7f776c0: flush on TX end=1 9386.591088260:b7f776c0: flush interval=1 9386.591137165:b7f776c0: file cache size=10 9386.591186217:b7f776c0: create directories: yes 9386.591241176:b7f776c0: file owner 0, group 0 9386.591291487:b7f776c0: force chown() for all files: no 9386.591340532:b7f776c0: directory owner 0, group 0 9386.591390233:b7f776c0: dir create mode 0700, file create mode 0644 9386.591458750:b7f776c0: fail if owner/group can not be set: no 9386.591520971:b7f776c0: Instance data: 0x91d1a88 9386.591570680:b7f776c0: RepeatedMsgReduction: 0 9386.591620264:b7f776c0: Resume Interval: 30 9386.591669451:b7f776c0: State: rdy 9386.591718957:b7f776c0: Exec only when previous is suspended: 0 9386.591774519:b7f776c0: submission mode: fast, but supports partial mark messages 9386.591823792:b7f776c0: 9386.591880745:b7f776c0: 9386.591928769:b7f776c0: 9386.591979891:b7f776c0: rule 0x91d2518: rsyslog rule: 9386.592052736:b7f776c0: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Actions: 9386.592508495:b7f776c0: builtin-usrmsg: Instance data: 0x91d2610 9386.592585828:b7f776c0: RepeatedMsgReduction: 0 9386.592635319:b7f776c0: Resume Interval: 30 9386.592684232:b7f776c0: State: rdy 9386.592733753:b7f776c0: Exec only when previous is suspended: 0 9386.592789806:b7f776c0: submission mode: fast, but supports partial mark messages 9386.592839042:b7f776c0: 9386.592894742:b7f776c0: 9386.592943123:b7f776c0: 9386.592993281:b7f776c0: rule 0x91d2b68: rsyslog rule: 9386.593064555:b7f776c0: X X X X X X X 7 7 X X X X X X X X X X X X X X X X Actions: 9386.593521589:b7f776c0: builtin-file: /var/log/spooler 9386.593596470:b7f776c0: template='/var/log/spooler' 9386.593647142:b7f776c0: use async writer=0 9386.593696404:b7f776c0: flush on TX end=1 9386.593745118:b7f776c0: flush interval=1 9386.593799349:b7f776c0: file cache size=10 9386.593850144:b7f776c0: create directories: yes 9386.593908177:b7f776c0: file owner 0, group 0 9386.593957316:b7f776c0: force chown() for all files: no 9386.594006559:b7f776c0: directory owner 0, group 0 9386.594061723:b7f776c0: dir create mode 0700, file create mode 0644 9386.594111859:b7f776c0: fail if owner/group can not be set: no 9386.594161451:b7f776c0: Instance data: 0x91d2bc8 9386.594211005:b7f776c0: RepeatedMsgReduction: 0 9386.594259630:b7f776c0: Resume Interval: 30 9386.594313946:b7f776c0: State: rdy 9386.594363040:b7f776c0: Exec only when previous is suspended: 0 9386.594412515:b7f776c0: submission mode: fast, but supports partial mark messages 9386.594461729:b7f776c0: 9386.594526860:b7f776c0: 9386.594583033:b7f776c0: 9386.594634219:b7f776c0: rule 0x91d35e8: rsyslog rule: 9386.594699418:b7f776c0: X X X X X X X X X X X X X X X X X X X X X X X FF X Actions: 9386.595146447:b7f776c0: builtin-file: /var/log/boot.log 9386.595213116:b7f776c0: template='/var/log/boot.log' 9386.595262813:b7f776c0: use async writer=0 9386.595311779:b7f776c0: flush on TX end=1 9386.595372470:b7f776c0: flush interval=1 9386.595422081:b7f776c0: file cache size=10 9386.595470893:b7f776c0: create directories: yes 9386.595537374:b7f776c0: file owner 0, group 0 9386.595592002:b7f776c0: force chown() for all files: no 9386.595642610:b7f776c0: directory owner 0, group 0 9386.595692337:b7f776c0: dir create mode 0700, file create mode 0644 9386.595741097:b7f776c0: fail if owner/group can not be set: no 9386.595820133:b7f776c0: Instance data: 0x91d3648 9386.595877738:b7f776c0: RepeatedMsgReduction: 0 9386.595935816:b7f776c0: Resume Interval: 30 9386.595985659:b7f776c0: State: rdy 9386.596035007:b7f776c0: Exec only when previous is suspended: 0 9386.596084048:b7f776c0: submission mode: fast, but supports partial mark messages 9386.596139621:b7f776c0: 9386.596187634:b7f776c0: 9386.596235824:b7f776c0: 9386.596284621:b7f776c0: End of Rulesets. 9386.596334670:b7f776c0: 9386.596392973:b7f776c0: Template: Name='RSYSLOG_DebugFormat' 9386.596457516:b7f776c0: Entry(917fc38): type 1, (CONSTANT), value: 'Debug line with all properties: FROMHOST: '' 9386.596560664:b7f776c0: Entry(917fd40): type 2, (FIELD), value: '7' 9386.596686637:b7f776c0: Entry(917fed8): type 1, (CONSTANT), value: '', fromhost-ip: '' 9386.596768590:b7f776c0: Entry(917ff38): type 2, (FIELD), value: '8' 9386.596855186:b7f776c0: Entry(9180020): type 1, (CONSTANT), value: '', HOSTNAME: '' 9386.596953380:b7f776c0: Entry(9180080): type 2, (FIELD), value: '3' 9386.597034628:b7f776c0: Entry(9180168): type 1, (CONSTANT), value: '', PRI: ' 9386.597115573:b7f776c0: Entry(91801c8): type 2, (FIELD), value: '9' 9386.597204152:b7f776c0: Entry(91802b0): type 1, (CONSTANT), value: ', syslogtag '' 9386.597285835:b7f776c0: Entry(9180310): type 2, (FIELD), value: '4' 9386.597367324:b7f776c0: Entry(91803f8): type 1, (CONSTANT), value: '', programname: '' 9386.597455142:b7f776c0: Entry(9180458): type 2, (FIELD), value: '17' 9386.597536112:b7f776c0: Entry(9180540): type 1, (CONSTANT), value: '', APP-NAME: '' 9386.597642039:b7f776c0: Entry(91805a0): type 2, (FIELD), value: '20' 9386.597724795:b7f776c0: Entry(9180688): type 1, (CONSTANT), value: '', PROCID: '' 9386.597805994:b7f776c0: Entry(91806e8): type 2, (FIELD), value: '21' 9386.597892714:b7f776c0: Entry(91807d0): type 1, (CONSTANT), value: '', MSGID: '' 9386.597984679:b7f776c0: Entry(9180830): type 2, (FIELD), value: '22' 9386.598066130:b7f776c0: Entry(9180918): type 1, (CONSTANT), value: '', TIMESTAMP: '' 9386.598152298:b7f776c0: Entry(9180978): type 2, (FIELD), value: '2' 9386.598234836:b7f776c0: Entry(9180a60): type 1, (CONSTANT), value: '', STRUCTURED-DATA: '' 9386.598316505:b7f776c0: Entry(9180ac0): type 2, (FIELD), value: '19' 9386.598401653:b7f776c0: Entry(9180bc0): type 1, (CONSTANT), value: '', msg: '' 9386.598485879:b7f776c0: Entry(9180c20): type 2, (FIELD), value: '1' 9386.598584234:b7f776c0: Entry(9180d08): type 1, (CONSTANT), value: '' escaped msg: '' 9386.598675352:b7f776c0: Entry(9180d68): type 2, (FIELD), value: '1' [drop control-characters] 9386.598773692:b7f776c0: Entry(9180e50): type 1, (CONSTANT), value: '' inputname: ' 9386.598855252:b7f776c0: Entry(9180eb0): type 2, (FIELD), value: '6' 9386.598949520:b7f776c0: Entry(9180f98): type 1, (CONSTANT), value: ' rawmsg: '' 9386.599030884:b7f776c0: Entry(9180ff8): type 2, (FIELD), value: '5' 9386.599151353:b7f776c0: Entry(91810e0): type 1, (CONSTANT), value: '' ' 9386.599239902:b7f776c0: Template: Name='RSYSLOG_SyslogProtocol23Format' 9386.599304966:b7f776c0: Entry(9181218): type 1, (CONSTANT), value: '<' 9386.599438067:b7f776c0: Entry(9181278): type 2, (FIELD), value: '9' 9386.599520925:b7f776c0: Entry(9181360): type 1, (CONSTANT), value: '>1 ' 9386.599621233:b7f776c0: Entry(91813c0): type 2, (FIELD), value: '2' [Format as RFC3339-Date] 9386.599727872:b7f776c0: Entry(91814a8): type 1, (CONSTANT), value: ' ' 9386.599809458:b7f776c0: Entry(9181508): type 2, (FIELD), value: '3' 9386.599891715:b7f776c0: Entry(91815f0): type 1, (CONSTANT), value: ' ' 9386.599985547:b7f776c0: Entry(9181650): type 2, (FIELD), value: '20' 9386.600067456:b7f776c0: Entry(9181738): type 1, (CONSTANT), value: ' ' 9386.600148674:b7f776c0: Entry(9181798): type 2, (FIELD), value: '21' 9386.600236717:b7f776c0: Entry(9181880): type 1, (CONSTANT), value: ' ' 9386.600318104:b7f776c0: Entry(91818e0): type 2, (FIELD), value: '22' 9386.600399525:b7f776c0: Entry(91819c8): type 1, (CONSTANT), value: ' ' 9386.600518407:b7f776c0: Entry(9181a28): type 2, (FIELD), value: '19' 9386.600600038:b7f776c0: Entry(9181b10): type 1, (CONSTANT), value: ' ' 9386.600708248:b7f776c0: Entry(9181b70): type 2, (FIELD), value: '1' 9386.600792474:b7f776c0: Entry(9181c58): type 1, (CONSTANT), value: ' ' 9386.600873523:b7f776c0: Template: Name='RSYSLOG_FileFormat' 9386.600938306:b7f776c0: Template: Name='RSYSLOG_TraditionalFileFormat' 9386.601015512:b7f776c0: Template: Name=' WallFmt' 9386.601080040:b7f776c0: Entry(9181f20): type 1, (CONSTANT), value: ' Message from syslogd@' 9386.601160999:b7f776c0: Entry(9181f80): type 2, (FIELD), value: '3' 9386.601248487:b7f776c0: Entry(9182078): type 1, (CONSTANT), value: ' at ' 9386.601329432:b7f776c0: Entry(91820d8): type 2, (FIELD), value: '16' 9386.601410305:b7f776c0: Entry(91821d8): type 1, (CONSTANT), value: ' ... ' 9386.601497640:b7f776c0: Entry(9182238): type 2, (FIELD), value: '4' 9386.601578367:b7f776c0: Entry(9182298): type 2, (FIELD), value: '1' 9386.601676325:b7f776c0: Entry(9182380): type 1, (CONSTANT), value: ' ' 9386.601765587:b7f776c0: Template: Name='RSYSLOG_ForwardFormat' 9386.601829777:b7f776c0: Template: Name='RSYSLOG_TraditionalForwardFormat' 9386.601926872:b7f776c0: Template: Name=' StdUsrMsgFmt' 9386.602018523:b7f776c0: Entry(9182528): type 1, (CONSTANT), value: ' ' 9386.602131337:b7f776c0: Entry(9182588): type 2, (FIELD), value: '4' 9386.602213939:b7f776c0: Entry(91825e8): type 2, (FIELD), value: '1' 9386.602301465:b7f776c0: Entry(91826d0): type 1, (CONSTANT), value: ' ' 9386.602382177:b7f776c0: Template: Name=' StdDBFmt' [SQL-Format (MySQL)] 9386.602463448:b7f776c0: Entry(91828e0): type 1, (CONSTANT), value: 'insert into SystemEvents (Message, Facili ty, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('' 9386.602556899:b7f776c0: Entry(9182940): type 2, (FIELD), value: '1' 9386.602638586:b7f776c0: Entry(9182a38): type 1, (CONSTANT), value: '', ' 9386.602737668:b7f776c0: Entry(9182a98): type 2, (FIELD), value: '12' 9386.602834613:b7f776c0: Entry(9182b98): type 1, (CONSTANT), value: ', '' 9386.602916728:b7f776c0: Entry(9182bf8): type 2, (FIELD), value: '3' 9386.603013485:b7f776c0: Entry(9182ce0): type 1, (CONSTANT), value: '', ' 9386.603096657:b7f776c0: Entry(9182d40): type 2, (FIELD), value: '14' 9386.603178052:b7f776c0: Entry(9182e28): type 1, (CONSTANT), value: ', '' 9386.603258738:b7f776c0: Entry(9182e88): type 2, (FIELD), value: '2' [Format as MySQL-Date] 9386.603362868:b7f776c0: Entry(9182f70): type 1, (CONSTANT), value: '', '' 9386.603444289:b7f776c0: Entry(9182fd0): type 2, (FIELD), value: '16' [Format as MySQL-Date] 9386.603549240:b7f776c0: Entry(91830b8): type 1, (CONSTANT), value: '', ' 9386.603630260:b7f776c0: Entry(9183118): type 2, (FIELD), value: '11' 9386.603732661:b7f776c0: Entry(9183200): type 1, (CONSTANT), value: ', '' 9386.603821514:b7f776c0: Entry(9183260): type 2, (FIELD), value: '4' 9386.603903771:b7f776c0: Entry(9183348): type 1, (CONSTANT), value: '')' 9386.603984832:b7f776c0: Template: Name=' StdPgSQLFmt' [SQL-Format (standard SQL)] 9386.604080945:b7f776c0: Entry(9183558): type 1, (CONSTANT), value: 'insert into SystemEvents (Message, Facili ty, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('' 9386.604166175:b7f776c0: Entry(91835b8): type 2, (FIELD), value: '1' 9386.604247195:b7f776c0: Entry(91836a0): type 1, (CONSTANT), value: '', ' 9386.604334709:b7f776c0: Entry(9183700): type 2, (FIELD), value: '12' 9386.604427219:b7f776c0: Entry(9183800): type 1, (CONSTANT), value: ', '' 9386.604509112:b7f776c0: Entry(9183860): type 2, (FIELD), value: '3' 9386.604596649:b7f776c0: Entry(9183948): type 1, (CONSTANT), value: '', ' 9386.604677151:b7f776c0: Entry(91839a8): type 2, (FIELD), value: '14' 9386.604832054:b7f776c0: Entry(9183a90): type 1, (CONSTANT), value: ', '' 9386.604913479:b7f776c0: Entry(9183af0): type 2, (FIELD), value: '2' [Format as PgSQL-Date] 9386.605022139:b7f776c0: Entry(9183bd8): type 1, (CONSTANT), value: '', '' 9386.605144275:b7f776c0: Entry(9183c38): type 2, (FIELD), value: '16' [Format as PgSQL-Date] 9386.605242132:b7f776c0: Entry(9183d20): type 1, (CONSTANT), value: '', ' 9386.605329928:b7f776c0: Entry(9183d80): type 2, (FIELD), value: '11' 9386.605412260:b7f776c0: Entry(9183e68): type 1, (CONSTANT), value: ', '' 9386.605493887:b7f776c0: Entry(9183ec8): type 2, (FIELD), value: '4' 9386.605582387:b7f776c0: Entry(9183fb0): type 1, (CONSTANT), value: '')' 9386.605663797:b7f776c0: Template: Name='RSYSLOG_omudpspoofDfltSourceTpl' 9386.605728741:b7f776c0: Entry(9184060): type 2, (FIELD), value: '8' 9386.605834367:b7f776c0: Template: Name='OmoracleStatement' 9386.605901070:b7f776c0: Entry(918c310): type 1, (CONSTANT), value: 'insert into SystemEvents(fromhost,message ) values (:host,:message)' 9386.605986151:b7f776c0: Template: Name='RsysLog' 9386.606060477:b7f776c0: Entry(918c370): type 2, (FIELD), value: '3' 9386.606148891:b7f776c0: Entry(918c3d0): type 2, (FIELD), value: '1' 9386.606230743:b7f776c0: Loaded Module: Name='lmnet', IFVersion=5, type=library module. 9386.606327347:b7f776c0: Entry points: 9386.606382992:b7f776c0: queryEtryPt: 0xdbc3d0 9386.606432074:b7f776c0: dbgPrintInstInfo: 0x0 9386.606481553:b7f776c0: freeInstance: 0x0 9386.606530373:b7f776c0: 9386.606578468:b7f776c0: Loaded Module: Name='builtin-file', IFVersion=5, type=output module. 9386.606680705:b7f776c0: Entry points: 9386.606730038:b7f776c0: queryEtryPt: 0x8058650 9386.606800130:b7f776c0: dbgPrintInstInfo: 0x8057de0 9386.606887978:b7f776c0: freeInstance: 0x8059060 9386.606940537:b7f776c0: Output Module Entry Points: 9386.606989086:b7f776c0: doAction: 0x80599f0 9386.607038337:b7f776c0: parseSelectorAct: 0x80591d0 9386.607096809:b7f776c0: tryResume: 0x8057d00 9386.607152048:b7f776c0: doHUP: 0x805a070 9386.607201152:b7f776c0: BeginTransaction: 0x8057cb0 9386.607250144:b7f776c0: EndTransaction: 0x8057c30 9386.607298919:b7f776c0: 9386.607347427:b7f776c0: Loaded Module: Name='builtin-pipe', IFVersion=5, type=output module. 9386.607449266:b7f776c0: Entry points: 9386.607499079:b7f776c0: queryEtryPt: 0x805a410 9386.607548326:b7f776c0: dbgPrintInstInfo: 0x805a600 9386.607597453:b7f776c0: freeInstance: 0x805a710 9386.607653206:b7f776c0: Output Module Entry Points: 9386.607701748:b7f776c0: doAction: 0x805aab0 9386.607750099:b7f776c0: parseSelectorAct: 0x805a790 9386.607816378:b7f776c0: tryResume: 0x805a330 9386.607865348:b7f776c0: doHUP: 0x805a690 9386.607920628:b7f776c0: BeginTransaction: 0x0 9386.607969759:b7f776c0: EndTransaction: 0x0 9386.608018432:b7f776c0: 9386.608076705:b7f776c0: Loaded Module: Name='builtin-fwd', IFVersion=5, type=output module. 9386.608179440:b7f776c0: Entry points: 9386.608228822:b7f776c0: queryEtryPt: 0x8056440 9386.608277856:b7f776c0: dbgPrintInstInfo: 0x8056610 9386.608326671:b7f776c0: freeInstance: 0x8056a70 9386.608375408:b7f776c0: Output Module Entry Points: 9386.608429796:b7f776c0: doAction: 0x8057770 9386.608478578:b7f776c0: parseSelectorAct: 0x8056b80 9386.608527057:b7f776c0: tryResume: 0x80576e0 9386.608575464:b7f776c0: doHUP: 0x0 9386.608623923:b7f776c0: BeginTransaction: 0x0 9386.608679500:b7f776c0: EndTransaction: 0x0 9386.608728443:b7f776c0: 9386.608776415:b7f776c0: Loaded Module: Name='builtin-shell', IFVersion=5, type=output module. 9386.608904271:b7f776c0: Entry points: 9386.608955261:b7f776c0: queryEtryPt: 0x8054e40 9386.609004309:b7f776c0: dbgPrintInstInfo: 0x80552c0 9386.609053129:b7f776c0: freeInstance: 0x8055010 9386.609111803:b7f776c0: Output Module Entry Points: 9386.609166900:b7f776c0: doAction: 0x8055330 9386.609216645:b7f776c0: parseSelectorAct: 0x8055080 9386.609265776:b7f776c0: tryResume: 0x8054d60 9386.609314843:b7f776c0: doHUP: 0x0 9386.609363483:b7f776c0: BeginTransaction: 0x0 9386.609418692:b7f776c0: EndTransaction: 0x0 9386.609493697:b7f776c0: 9386.609542426:b7f776c0: Loaded Module: Name='builtin-discard', IFVersion=5, type=output module. 9386.609638081:b7f776c0: Entry points: 9386.609693208:b7f776c0: queryEtryPt: 0x805afa0 9386.609742508:b7f776c0: dbgPrintInstInfo: 0x805af50 9386.609791440:b7f776c0: freeInstance: 0x805b1d0 9386.609859709:b7f776c0: Output Module Entry Points: 9386.609910141:b7f776c0: doAction: 0x805b170 9386.609966044:b7f776c0: parseSelectorAct: 0x805b240 9386.610053851:b7f776c0: tryResume: 0x805ae80 9386.610112679:b7f776c0: doHUP: 0x0 9386.610161731:b7f776c0: BeginTransaction: 0x0 9386.610216715:b7f776c0: EndTransaction: 0x0 9386.610266120:b7f776c0: 9386.610314902:b7f776c0: Loaded Module: Name='builtin-usrmsg', IFVersion=5, type=output module. 9386.610411063:b7f776c0: Entry points: 9386.610467379:b7f776c0: queryEtryPt: 0x8055670 9386.610517020:b7f776c0: dbgPrintInstInfo: 0x8055840 9386.610566507:b7f776c0: freeInstance: 0x80558c0 9386.610615855:b7f776c0: Output Module Entry Points: 9386.610665364:b7f776c0: doAction: 0x8055c70 9386.610720844:b7f776c0: parseSelectorAct: 0x8055930 9386.610770226:b7f776c0: tryResume: 0x8055590 9386.610819413:b7f776c0: doHUP: 0x0 9386.610886239:b7f776c0: BeginTransaction: 0x0 9386.610936142:b7f776c0: EndTransaction: 0x0 9386.610991588:b7f776c0: 9386.611039800:b7f776c0: Loaded Module: Name='builtin-pmrfc5424', IFVersion=5, type=parser module. 9386.611146439:b7f776c0: Entry points: 9386.611196215:b7f776c0: queryEtryPt: 0x805b810 9386.611251345:b7f776c0: dbgPrintInstInfo: 0x0 9386.611300333:b7f776c0: freeInstance: 0x0 9386.611348703:b7f776c0: Parser Module Entry Points 9386.611397417:b7f776c0: parse: 0x805b980 9386.611446376:b7f776c0: 9386.611500557:b7f776c0: Loaded Module: Name='builtin-pmrfc3164', IFVersion=5, type=parser module. 9386.611595395:b7f776c0: Entry points: 9386.611644541:b7f776c0: queryEtryPt: 0x805c050 9386.611693721:b7f776c0: dbgPrintInstInfo: 0x0 9386.611748300:b7f776c0: freeInstance: 0x0 9386.611797277:b7f776c0: Parser Module Entry Points 9386.611846479:b7f776c0: parse: 0x805c1c0 9386.611917692:b7f776c0: 9386.611972497:b7f776c0: Loaded Module: Name='builtin-smfile', IFVersion=5, type=strgen module. 9386.612070852:b7f776c0: Entry points: 9386.612129680:b7f776c0: queryEtryPt: 0x805cc90 9386.612179298:b7f776c0: dbgPrintInstInfo: 0x0 9386.612233000:b7f776c0: freeInstance: 0x0 9386.612282993:b7f776c0: Strgen Module Entry Points 9386.612332304:b7f776c0: strgen: 0x805cde0 9386.612381071:b7f776c0: 9386.612428593:b7f776c0: Loaded Module: Name='builtin-smtradfile', IFVersion=5, type=strgen module. 9386.612531073:b7f776c0: Entry points: 9386.612579863:b7f776c0: queryEtryPt: 0x805c7a0 9386.612629196:b7f776c0: dbgPrintInstInfo: 0x0 9386.612678135:b7f776c0: freeInstance: 0x0 9386.612726959:b7f776c0: Strgen Module Entry Points 9386.612819442:b7f776c0: strgen: 0x805c8f0 9386.612870339:b7f776c0: 9386.612936985:b7f776c0: Loaded Module: Name='builtin-smfwd', IFVersion=5, type=strgen module. 9386.613041464:b7f776c0: Entry points: 9386.613091127:b7f776c0: queryEtryPt: 0x805d180 9386.613150098:b7f776c0: dbgPrintInstInfo: 0x0 9386.613199555:b7f776c0: freeInstance: 0x0 9386.613251925:b7f776c0: Strgen Module Entry Points 9386.613302511:b7f776c0: strgen: 0x805d2d0 9386.613350918:b7f776c0: 9386.613406600:b7f776c0: Loaded Module: Name='builtin-smtradfwd', IFVersion=5, type=strgen module. 9386.613507051:b7f776c0: Entry points: 9386.613557753:b7f776c0: queryEtryPt: 0x805d6b0 9386.613606374:b7f776c0: dbgPrintInstInfo: 0x0 9386.613655325:b7f776c0: freeInstance: 0x0 9386.613703949:b7f776c0: Strgen Module Entry Points 9386.613752990:b7f776c0: strgen: 0x805d800 9386.613807648:b7f776c0: 9386.613855650:b7f776c0: Loaded Module: Name='immark', IFVersion=5, type=input module. 9386.613968951:b7f776c0: Entry points: 9386.614064003:b7f776c0: queryEtryPt: 0xf57980 9386.614113029:b7f776c0: dbgPrintInstInfo: 0x0 9386.614170986:b7f776c0: freeInstance: 0x0 9386.614219708:b7f776c0: Input Module Entry Points 9386.614272169:b7f776c0: runInput: 0xf57b40 9386.614323550:b7f776c0: willRun: 0xf57880 9386.614372984:b7f776c0: afterRun: 0xf57820 9386.614421545:b7f776c0: 9386.614469360:b7f776c0: Loaded Module: Name='imuxsock', IFVersion=5, type=input module. 9386.614572687:b7f776c0: Entry points: 9386.614621387:b7f776c0: queryEtryPt: 0xf09c00 9386.614670394:b7f776c0: dbgPrintInstInfo: 0x0 9386.614718984:b7f776c0: freeInstance: 0x0 9386.614767373:b7f776c0: Input Module Entry Points 9386.614822076:b7f776c0: runInput: 0xf0a2f0 9386.614870966:b7f776c0: willRun: 0xf0a000 9386.614919118:b7f776c0: afterRun: 0xf09ed0 9386.614990505:b7f776c0: 9386.615044525:b7f776c0: Loaded Module: Name='imklog', IFVersion=5, type=input module. 9386.615141474:b7f776c0: Entry points: 9386.615198851:b7f776c0: queryEtryPt: 0x13a8d0 9386.615247423:b7f776c0: dbgPrintInstInfo: 0x0 9386.615300034:b7f776c0: freeInstance: 0x0 9386.615350777:b7f776c0: Input Module Entry Points 9386.615399968:b7f776c0: runInput: 0x13abf0 9386.615448506:b7f776c0: willRun: 0x13ab20 9386.615497832:b7f776c0: afterRun: 0x13aa70 9386.615546505:b7f776c0: 9386.615601002:b7f776c0: Loaded Module: Name='lmnetstrms', IFVersion=5, type=library module. 9386.615696323:b7f776c0: Entry points: 9386.615744524:b7f776c0: queryEtryPt: 0x7c2e40 9386.615832196:b7f776c0: dbgPrintInstInfo: 0x0 9386.615883108:b7f776c0: freeInstance: 0x0 9386.615932778:b7f776c0: 9386.615998971:b7f776c0: Loaded Module: Name='lmtcpsrv', IFVersion=5, type=library module. 9386.616105501:b7f776c0: Entry points: 9386.616154699:b7f776c0: queryEtryPt: 0xe12000 9386.616213752:b7f776c0: dbgPrintInstInfo: 0x0 9386.616262733:b7f776c0: freeInstance: 0x0 9386.616311234:b7f776c0: 9386.616368427:b7f776c0: Loaded Module: Name='imtcp', IFVersion=5, type=input module. 9386.616464734:b7f776c0: Entry points: 9386.616513711:b7f776c0: queryEtryPt: 0x18b3c0 9386.616561852:b7f776c0: dbgPrintInstInfo: 0x0 9386.616617556:b7f776c0: freeInstance: 0x0 9386.616666323:b7f776c0: Input Module Entry Points 9386.616715683:b7f776c0: runInput: 0x18b6a0 9386.616764191:b7f776c0: willRun: 0x18adb0 9386.616813457:b7f776c0: afterRun: 0x18ad30 9386.616868632:b7f776c0: 9386.616917219:b7f776c0: Loaded Module: Name='omoracle', IFVersion=5, type=output module. 9386.617029393:b7f776c0: Entry points: 9386.617079394:b7f776c0: queryEtryPt: 0x111f20 9386.617170793:b7f776c0: dbgPrintInstInfo: 0x111df0 9386.617229089:b7f776c0: freeInstance: 0x112ad0 9386.617278006:b7f776c0: Output Module Entry Points: 9386.617327710:b7f776c0: doAction: 0x1134a0 9386.617382680:b7f776c0: parseSelectorAct: 0x112c70 9386.617431998:b7f776c0: tryResume: 0x112550 9386.617481046:b7f776c0: doHUP: 0x0 9386.617530289:b7f776c0: BeginTransaction: 0x0 9386.617579484:b7f776c0: EndTransaction: 0x0 9386.617634731:b7f776c0: 9386.617686528:b7f776c0: Sytem Line Configuration Commands: 9386.617737852:b7f776c0: Command 'workdirectory': 9386.617788063:b7f776c0: type : 11 9386.617837418:b7f776c0: pData: 0x80c398c 9386.617893715:b7f776c0: Hdlr : 0x0 9386.617943277:b7f776c0: Owner: 0x0 9386.617992584:b7f776c0: 9386.618065324:b7f776c0: Command 'dropmsgswithmaliciousdnsptrrecords': 9386.618122779:b7f776c0: type : 4 9386.618173061:b7f776c0: pData: 0x80c39a0 9386.618230539:b7f776c0: Hdlr : 0x0 9386.618280060:b7f776c0: Owner: 0x0 9386.618328486:b7f776c0: 9386.618382761:b7f776c0: Command 'defaultnetstreamdriver': 9386.618433062:b7f776c0: type : 11 9386.618482647:b7f776c0: pData: 0x80c397c 9386.618531852:b7f776c0: Hdlr : 0x0 9386.618580394:b7f776c0: Owner: 0x0 9386.618635097:b7f776c0: 9386.618713650:b7f776c0: Command 'defaultnetstreamdrivercafile': 9386.618763650:b7f776c0: type : 11 9386.618811870:b7f776c0: pData: 0x80c3980 9386.618860888:b7f776c0: Hdlr : 0x0 9386.618917211:b7f776c0: Owner: 0x0 9386.618965836:b7f776c0: 9386.619014558:b7f776c0: Command 'defaultnetstreamdriverkeyfile': 9386.619081744:b7f776c0: type : 11 9386.619130928:b7f776c0: pData: 0x80c3984 9386.619187776:b7f776c0: Hdlr : 0x0 9386.619245141:b7f776c0: Owner: 0x0 9386.619294242:b7f776c0: 9386.619343129:b7f776c0: Command 'defaultnetstreamdrivercertfile': 9386.619396501:b7f776c0: type : 11 9386.619447885:b7f776c0: pData: 0x80c3988 9386.619496405:b7f776c0: Hdlr : 0x0 9386.619545228:b7f776c0: Owner: 0x0 9386.619593882:b7f776c0: 9386.619641888:b7f776c0: Command 'optimizeforuniprocessor': 9386.619697431:b7f776c0: type : 4 9386.619746108:b7f776c0: pData: 0x80c1690 9386.619795100:b7f776c0: Hdlr : 0x0 9386.619843998:b7f776c0: Owner: 0x0 9386.619892713:b7f776c0: 9386.619948537:b7f776c0: Command 'preservefqdn': 9386.619998197:b7f776c0: type : 4 9386.620046709:b7f776c0: pData: 0x80c39a4 9386.620112522:b7f776c0: Hdlr : 0x0 9386.620167769:b7f776c0: Owner: 0x0 9386.620218100:b7f776c0: 9386.620277085:b7f776c0: Command 'resetconfigvariables': 9386.620326977:b7f776c0: type : 1 9386.620375721:b7f776c0: pData: 0x0 9386.620431778:b7f776c0: Hdlr : 0x805eb60 9386.620481520:b7f776c0: Owner: 0x0 9386.620530370:b7f776c0: 9386.620579144:b7f776c0: type : 1 9386.620628766:b7f776c0: pData: 0x0 9386.620684076:b7f776c0: Hdlr : 0x80622c0 9386.620733410:b7f776c0: Owner: 0x0 9386.620782844:b7f776c0: 9386.620831082:b7f776c0: type : 1 9386.620880412:b7f776c0: pData: 0x0 9386.620934470:b7f776c0: Hdlr : 0x80a0500 9386.620985836:b7f776c0: Owner: 0x0 9386.621034426:b7f776c0: 9386.621082458:b7f776c0: type : 1 9386.621148565:b7f776c0: pData: 0x0 9386.621205728:b7f776c0: Hdlr : 0x8058f90 9386.621263730:b7f776c0: Owner: 0x8059930 9386.621312512:b7f776c0: 9386.621361036:b7f776c0: type : 1 9386.621410583:b7f776c0: pData: 0x0 9386.621465942:b7f776c0: Hdlr : 0x8056870 9386.621514908:b7f776c0: Owner: 0x80568b0 9386.621563817:b7f776c0: 9386.621611755:b7f776c0: type : 1 9386.621661010:b7f776c0: pData: 0x0 9386.621717100:b7f776c0: Hdlr : 0x804f710 9386.621766317:b7f776c0: Owner: 0x0 9386.621815032:b7f776c0: 9386.621863645:b7f776c0: type : 1 9386.621912243:b7f776c0: pData: 0x0 9386.621975714:b7f776c0: Hdlr : 0xf575a0 9386.622025996:b7f776c0: Owner: 0xf577c0 9386.622074778:b7f776c0: 9386.622138075:b7f776c0: type : 1 9386.622189407:b7f776c0: pData: 0x0 9386.622245892:b7f776c0: Hdlr : 0xf09e40 9386.622304243:b7f776c0: Owner: 0xf09a40 9386.622360934:b7f776c0: 9386.622409761:b7f776c0: type : 1 9386.622459147:b7f776c0: pData: 0x0 9386.622513940:b7f776c0: Hdlr : 0x13ac80 9386.622563490:b7f776c0: Owner: 0x13ad10 9386.622611762:b7f776c0: 9386.622660469:b7f776c0: type : 1 9386.622710076:b7f776c0: pData: 0x0 9386.622765244:b7f776c0: Hdlr : 0x18b740 9386.622814375:b7f776c0: Owner: 0x18ab20 9386.622863289:b7f776c0: 9386.622911917:b7f776c0: type : 1 9386.622960909:b7f776c0: pData: 0x0 9386.623018552:b7f776c0: Hdlr : 0x113600 9386.623069381:b7f776c0: Owner: 0x111e50 9386.623120008:b7f776c0: 9386.623188124:b7f776c0: Command 'rulesetparser': 9386.623245737:b7f776c0: type : 11 9386.623305787:b7f776c0: pData: 0x0 9386.623354993:b7f776c0: Hdlr : 0x809a0f0 9386.623403726:b7f776c0: Owner: 0x0 9386.623452921:b7f776c0: 9386.623508014:b7f776c0: Command 'rulesetcreatemainqueue': 9386.623558026:b7f776c0: type : 4 9386.623606699:b7f776c0: pData: 0x0 9386.623656231:b7f776c0: Hdlr : 0x8099ea0 9386.623705572:b7f776c0: Owner: 0x0 9386.623760744:b7f776c0: 9386.623810039:b7f776c0: Command 'controlcharacterescapeprefix': 9386.623860130:b7f776c0: type : 8 9386.623908762:b7f776c0: pData: 0x80c1a48 9386.623957945:b7f776c0: Hdlr : 0x0 9386.624014167:b7f776c0: Owner: 0x0 9386.624063587:b7f776c0: 9386.624112616:b7f776c0: Command 'droptrailinglfonreception': 9386.624210364:b7f776c0: type : 4 9386.624268048:b7f776c0: pData: 0x80c1a4c 9386.624326625:b7f776c0: Hdlr : 0x0 9386.624375992:b7f776c0: Owner: 0x0 9386.624424691:b7f776c0: 9386.624472686:b7f776c0: Command 'escapecontrolcharactersonreceive': 9386.624528420:b7f776c0: type : 4 9386.624577412:b7f776c0: pData: 0x80c1a50 9386.624626389:b7f776c0: Hdlr : 0x0 9386.624674942:b7f776c0: Owner: 0x0 9386.624723229:b7f776c0: 9386.624777374:b7f776c0: Command 'escape8bitcharactersonreceive': 9386.624827408:b7f776c0: type : 4 9386.624876505:b7f776c0: pData: 0x80c3a58 9386.624925899:b7f776c0: Hdlr : 0x0 9386.624974501:b7f776c0: Owner: 0x0 9386.625028585:b7f776c0: 9386.625079399:b7f776c0: Command 'escapecontrolcharactertab': 9386.625128691:b7f776c0: type : 4 9386.625177323:b7f776c0: pData: 0x80c1a54 9386.625244554:b7f776c0: Hdlr : 0x0 9386.625300885:b7f776c0: Owner: 0x0 9386.625358149:b7f776c0: 9386.625406882:b7f776c0: Command 'actionname': 9386.625456407:b7f776c0: type : 11 9386.625505144:b7f776c0: pData: 0x80c4aa8 9386.625560357:b7f776c0: Hdlr : 0x0 9386.625609503:b7f776c0: Owner: 0x0 9386.625658592:b7f776c0: 9386.625706932:b7f776c0: Command 'actionqueuefilename': 9386.625879425:b7f776c0: type : 11 9386.625936232:b7f776c0: pData: 0x80c49f4 9386.625994066:b7f776c0: Hdlr : 0x0 9386.626044089:b7f776c0: Owner: 0x0 9386.626093906:b7f776c0: 9386.626143082:b7f776c0: Command 'actionqueuesize': 9386.626193353:b7f776c0: type : 6 9386.626270555:b7f776c0: pData: 0x80c3400 9386.626320196:b7f776c0: Hdlr : 0x0 9386.626377370:b7f776c0: Owner: 0x0 9386.626426013:b7f776c0: 9386.626479096:b7f776c0: Command 'actionwriteallmarkmessages': 9386.626529922:b7f776c0: type : 4 9386.626578817:b7f776c0: pData: 0x80c49f8 9386.626628131:b7f776c0: Hdlr : 0x0 9386.626676902:b7f776c0: Owner: 0x0 9386.626735137:b7f776c0: 9386.626786987:b7f776c0: Command 'actionqueuedequeuebatchsize': 9386.626836680:b7f776c0: type : 6 9386.626886381:b7f776c0: pData: 0x80c3404 9386.626935673:b7f776c0: Hdlr : 0x0 9386.626990087:b7f776c0: Owner: 0x0 9386.627041243:b7f776c0: 9386.627089732:b7f776c0: Command 'actionqueuemaxdiskspace': 9386.627139797:b7f776c0: type : 7 9386.627188811:b7f776c0: pData: 0x80c4a00 9386.627242667:b7f776c0: Hdlr : 0x0 9386.627293642:b7f776c0: Owner: 0x0 9386.627342222:b7f776c0: 9386.627390580:b7f776c0: Command 'actionqueuehighwatermark': 9386.627439846:b7f776c0: type : 6 9386.627488774:b7f776c0: pData: 0x80c3408 9386.627543207:b7f776c0: Hdlr : 0x0 9386.627592387:b7f776c0: Owner: 0x0 9386.627641206:b7f776c0: 9386.627689159:b7f776c0: Command 'actionqueuelowwatermark': 9386.627738496:b7f776c0: type : 6 9386.628055023:b7f776c0: pData: 0x80c340c 9386.628106707:b7f776c0: Hdlr : 0x0 9386.628155834:b7f776c0: Owner: 0x0 9386.628212149:b7f776c0: 9386.628264659:b7f776c0: Command 'actionqueuediscardmark': 9386.628314802:b7f776c0: type : 6 9386.628363903:b7f776c0: pData: 0x80c3410 9386.628413322:b7f776c0: Hdlr : 0x0 9386.628468355:b7f776c0: Owner: 0x0 9386.628518277:b7f776c0: 9386.628567160:b7f776c0: Command 'actionqueuediscardseverity': 9386.628616377:b7f776c0: type : 6 9386.628665107:b7f776c0: pData: 0x80c3414 9386.628719994:b7f776c0: Hdlr : 0x0 9386.628770122:b7f776c0: Owner: 0x0 9386.628818904:b7f776c0: 9386.628867885:b7f776c0: Command 'actionqueuecheckpointinterval': 9386.628988496:b7f776c0: type : 6 9386.629038335:b7f776c0: pData: 0x80c4a08 9386.629087466:b7f776c0: Hdlr : 0x0 9386.629136402:b7f776c0: Owner: 0x0 9386.629184940:b7f776c0: 9386.629240266:b7f776c0: Command 'actionqueuesyncqueuefiles': 9386.629290544:b7f776c0: type : 4 9386.629340477:b7f776c0: pData: 0x80c4a0c 9386.629389953:b7f776c0: Hdlr : 0x0 9386.629439425:b7f776c0: Owner: 0x0 9386.629494840:b7f776c0: 9386.629544125:b7f776c0: Command 'actionqueuetype': 9386.629594744:b7f776c0: type : 11 9386.629643905:b7f776c0: pData: 0x0 9386.629693415:b7f776c0: Hdlr : 0x80a0f30 9386.629750203:b7f776c0: Owner: 0x0 9386.629799618:b7f776c0: 9386.629848757:b7f776c0: Command 'actionqueueworkerthreads': 9386.629969873:b7f776c0: type : 6 9386.630027490:b7f776c0: pData: 0x80c3418 9386.630076924:b7f776c0: Hdlr : 0x0 9386.630126018:b7f776c0: Owner: 0x0 9386.630175201:b7f776c0: 9386.630223852:b7f776c0: Command 'actionqueuetimeoutshutdown': 9386.630280778:b7f776c0: type : 6 9386.630330089:b7f776c0: pData: 0x80c4a10 9386.630378909:b7f776c0: Hdlr : 0x0 9386.630428452:b7f776c0: Owner: 0x0 9386.630477140:b7f776c0: 9386.630534191:b7f776c0: Command 'actionqueuetimeoutactioncompletion': 9386.630584304:b7f776c0: type : 6 9386.630633637:b7f776c0: pData: 0x80c341c 9386.630683285:b7f776c0: Hdlr : 0x0 9386.630732266:b7f776c0: Owner: 0x0 9386.630788357:b7f776c0: 9386.630837439:b7f776c0: Command 'actionqueuetimeoutenqueue': 9386.630887151:b7f776c0: type : 6 9386.630966247:b7f776c0: pData: 0x80c3420 9386.631020774:b7f776c0: Hdlr : 0x0 9386.631071626:b7f776c0: Owner: 0x0 9386.631120726:b7f776c0: 9386.631169715:b7f776c0: Command 'actionqueueworkertimeoutthreadshutdown': 9386.631219764:b7f776c0: type : 6 9386.631272911:b7f776c0: pData: 0x80c3424 9386.631325083:b7f776c0: Hdlr : 0x0 9386.631374277:b7f776c0: Owner: 0x0 9386.631423116:b7f776c0: 9386.631472205:b7f776c0: Command 'actionqueueworkerthreadminimummessages': 9386.631521958:b7f776c0: type : 6 9386.631577291:b7f776c0: pData: 0x80c3428 9386.631626272:b7f776c0: Hdlr : 0x0 9386.631685996:b7f776c0: Owner: 0x0 9386.631734722:b7f776c0: 9386.631787685:b7f776c0: Command 'actionqueuemaxfilesize': 9386.631839096:b7f776c0: type : 7 9386.631888549:b7f776c0: pData: 0x80c3430 9386.631965564:b7f776c0: Hdlr : 0x0 9386.632016258:b7f776c0: Owner: 0x0 9386.632071891:b7f776c0: 9386.632120418:b7f776c0: Command 'actionqueuesaveonshutdown': 9386.632170363:b7f776c0: type : 4 9386.632219850:b7f776c0: pData: 0x80c3438 9386.632269562:b7f776c0: Hdlr : 0x0 9386.632449315:b7f776c0: Owner: 0x0 9386.632499335:b7f776c0: 9386.632552654:b7f776c0: Command 'actionqueuedequeueslowdown': 9386.632605118:b7f776c0: type : 6 9386.632654279:b7f776c0: pData: 0x80c4a14 9386.632703492:b7f776c0: Hdlr : 0x0 9386.632752634:b7f776c0: Owner: 0x0 9386.632801619:b7f776c0: 9386.632858219:b7f776c0: Command 'actionqueuedequeuetimebegin': 9386.632908512:b7f776c0: type : 6 9386.632987064:b7f776c0: pData: 0x80c4a18 9386.633038126:b7f776c0: Hdlr : 0x0 9386.633093466:b7f776c0: Owner: 0x0 9386.633142323:b7f776c0: 9386.633190963:b7f776c0: Command 'actionqueuedequeuetimeend': 9386.633240195:b7f776c0: type : 6 9386.633289265:b7f776c0: pData: 0x80c343c 9386.633344921:b7f776c0: Hdlr : 0x0 9386.633395165:b7f776c0: Owner: 0x0 9386.633443955:b7f776c0: 9386.633492692:b7f776c0: Command 'actionexeconlyeverynthtime': 9386.633542816:b7f776c0: type : 6 9386.633598603:b7f776c0: pData: 0x80c4a1c 9386.633648450:b7f776c0: Hdlr : 0x0 9386.633697427:b7f776c0: Owner: 0x0 9386.633746430:b7f776c0: 9386.633795448:b7f776c0: Command 'actionexeconlyeverynthtimetimeout': 9386.633853245:b7f776c0: type : 6 9386.633902215:b7f776c0: pData: 0x80c4a20 9386.633951548:b7f776c0: Hdlr : 0x0 9386.634030648:b7f776c0: Owner: 0x0 9386.634079617:b7f776c0: 9386.634134414:b7f776c0: Command 'actionexeconlyonceeveryinterval': 9386.634184163:b7f776c0: type : 6 9386.634233512:b7f776c0: pData: 0x80c4a24 9386.634282961:b7f776c0: Hdlr : 0x0 9386.634331736:b7f776c0: Owner: 0x0 9386.634388242:b7f776c0: 9386.634437152:b7f776c0: Command 'repeatedmsgcontainsoriginalmsg': 9386.634486444:b7f776c0: type : 4 9386.634535593:b7f776c0: pData: 0x80c4a28 9386.634584267:b7f776c0: Hdlr : 0x0 9386.634639810:b7f776c0: Owner: 0x0 9386.634688584:b7f776c0: 9386.634737445:b7f776c0: Command 'dynafilecachesize': 9386.634787509:b7f776c0: type : 6 9386.634836456:b7f776c0: pData: 0x0 9386.634893717:b7f776c0: Hdlr : 0x80589c0 9386.634942982:b7f776c0: Owner: 0x8059930 9386.635023635:b7f776c0: 9386.635073350:b7f776c0: Command 'omfileziplevel': 9386.635129711:b7f776c0: type : 6 9386.635178718:b7f776c0: pData: 0x80c37b0 9386.635258991:b7f776c0: Hdlr : 0x0 9386.635308699:b7f776c0: Owner: 0x8059930 9386.635357485:b7f776c0: 9386.635413242:b7f776c0: Command 'omfileflushinterval': 9386.635462864:b7f776c0: type : 6 9386.635512490:b7f776c0: pData: 0x80c1010 9386.635561785:b7f776c0: Hdlr : 0x0 9386.635610867:b7f776c0: Owner: 0x8059930 9386.635678886:b7f776c0: 9386.635727889:b7f776c0: Command 'omfileasyncwriting': 9386.635778066:b7f776c0: type : 4 9386.635827610:b7f776c0: pData: 0x80c37b4 9386.635882740:b7f776c0: Hdlr : 0x0 9386.635934507:b7f776c0: Owner: 0x8059930 9386.635983244:b7f776c0: 9386.636073834:b7f776c0: Command 'omfileflushontxend': 9386.636124408:b7f776c0: type : 4 9386.636180315:b7f776c0: pData: 0x80c1014 9386.636229757:b7f776c0: Hdlr : 0x0 9386.636279000:b7f776c0: Owner: 0x8059930 9386.636327910:b7f776c0: 9386.636376410:b7f776c0: Command 'omfileiobuffersize': 9386.636433682:b7f776c0: type : 7 9386.636483533:b7f776c0: pData: 0x80c1018 9386.636533740:b7f776c0: Hdlr : 0x0 9386.636583118:b7f776c0: Owner: 0x8059930 9386.636632103:b7f776c0: 9386.636688321:b7f776c0: Command 'dirowner': 9386.636738693:b7f776c0: type : 2 9386.636943810:b7f776c0: pData: 0x80c3848 9386.637056625:b7f776c0: Hdlr : 0x0 9386.637111230:b7f776c0: Owner: 0x8059930 9386.637160942:b7f776c0: 9386.637212079:b7f776c0: Command 'dirgroup': 9386.637269332:b7f776c0: type : 3 9386.637319787:b7f776c0: pData: 0x80c384c 9386.637369424:b7f776c0: Hdlr : 0x0 9386.637418967:b7f776c0: Owner: 0x8059930 9386.637468251:b7f776c0: 9386.637525718:b7f776c0: Command 'fileowner': 9386.637576218:b7f776c0: type : 2 9386.637625716:b7f776c0: pData: 0x80c3840 9386.637675383:b7f776c0: Hdlr : 0x0 9386.637724641:b7f776c0: Owner: 0x8059930 9386.637780387:b7f776c0: 9386.637829322:b7f776c0: Command 'filegroup': 9386.637879297:b7f776c0: type : 3 9386.637929114:b7f776c0: pData: 0x80c3844 9386.637978226:b7f776c0: Hdlr : 0x0 9386.638035179:b7f776c0: Owner: 0x8059930 9386.638116551:b7f776c0: 9386.638166001:b7f776c0: Command 'dircreatemode': 9386.638216110:b7f776c0: type : 5 9386.638272216:b7f776c0: pData: 0x80c1020 9386.638321556:b7f776c0: Hdlr : 0x0 9386.638371220:b7f776c0: Owner: 0x8059930 9386.638420433:b7f776c0: 9386.638469530:b7f776c0: Command 'filecreatemode': 9386.638526618:b7f776c0: type : 5 9386.638576240:b7f776c0: pData: 0x80c1024 9386.638625457:b7f776c0: Hdlr : 0x0 9386.638675011:b7f776c0: Owner: 0x8059930 9386.638723947:b7f776c0: 9386.638785321:b7f776c0: Command 'createdirs': 9386.638835363:b7f776c0: type : 4 9386.638884741:b7f776c0: pData: 0x80c1028 9386.638934528:b7f776c0: Hdlr : 0x0 9386.638983948:b7f776c0: Owner: 0x8059930 9386.639040237:b7f776c0: 9386.639113944:b7f776c0: Command 'failonchownfailure': 9386.639164905:b7f776c0: type : 4 9386.639214800:b7f776c0: pData: 0x80c383c 9386.639263935:b7f776c0: Hdlr : 0x0 9386.639319425:b7f776c0: Owner: 0x8059930 9386.639368421:b7f776c0: 9386.639417443:b7f776c0: Command 'omfileForceChown': 9386.639467279:b7f776c0: type : 4 9386.639516889:b7f776c0: pData: 0x80c37b8 9386.639698518:b7f776c0: Hdlr : 0x0 9386.639749062:b7f776c0: Owner: 0x8059930 9386.639807020:b7f776c0: 9386.639856016:b7f776c0: Command 'actionfileenablesync': 9386.639905908:b7f776c0: type : 4 9386.639955444:b7f776c0: pData: 0x80c37bc 9386.640004544:b7f776c0: Hdlr : 0x0 9386.640061145:b7f776c0: Owner: 0x8059930 9386.640139134:b7f776c0: 9386.640188565:b7f776c0: Command 'actionfiledefaulttemplate': 9386.640238390:b7f776c0: type : 11 9386.640288079:b7f776c0: pData: 0x80c37ac 9386.640343453:b7f776c0: Hdlr : 0x0 9386.640392768:b7f776c0: Owner: 0x0 9386.640441898:b7f776c0: 9386.640491040:b7f776c0: Command 'actionforwarddefaulttemplate': 9386.640541581:b7f776c0: type : 11 9386.640597987:b7f776c0: pData: 0x80c3740 9386.640647451:b7f776c0: Hdlr : 0x0 9386.640696484:b7f776c0: Owner: 0x0 9386.640745525:b7f776c0: 9386.640794821:b7f776c0: Command 'actionsendtcprebindinterval': 9386.640852055:b7f776c0: type : 6 9386.640901205:b7f776c0: pData: 0x80c3744 9386.640984879:b7f776c0: Hdlr : 0x0 9386.641033939:b7f776c0: Owner: 0x0 9386.641116214:b7f776c0: 9386.641167970:b7f776c0: Command 'actionsendudprebindinterval': 9386.641217723:b7f776c0: type : 6 9386.641267360:b7f776c0: pData: 0x80c3748 9386.641320897:b7f776c0: Hdlr : 0x0 9386.641373114:b7f776c0: Owner: 0x0 9386.641422597:b7f776c0: 9386.641471338:b7f776c0: Command 'actionsendstreamdriver': 9386.641521346:b7f776c0: type : 11 9386.641570436:b7f776c0: pData: 0x80c374c 9386.641626448:b7f776c0: Hdlr : 0x0 9386.641675346:b7f776c0: Owner: 0x0 9386.641723948:b7f776c0: 9386.641772647:b7f776c0: Command 'actionsendstreamdrivermode': 9386.641973352:b7f776c0: type : 6 9386.642023701:b7f776c0: pData: 0x80c3750 9386.642073638:b7f776c0: Hdlr : 0x0 9386.642154042:b7f776c0: Owner: 0x0 9386.642203597:b7f776c0: 9386.642252922:b7f776c0: Command 'actionsendstreamdriverauthmode': 9386.642303227:b7f776c0: type : 11 9386.642358886:b7f776c0: pData: 0x80c3754 9386.642409063:b7f776c0: Hdlr : 0x0 9386.642458404:b7f776c0: Owner: 0x0 9386.642507625:b7f776c0: 9386.642557037:b7f776c0: Command 'actionsendstreamdriverpermittedpeer': 9386.642614488:b7f776c0: type : 11 9386.642664553:b7f776c0: pData: 0x0 9386.642714182:b7f776c0: Hdlr : 0x80569f0 9386.642763812:b7f776c0: Owner: 0x0 9386.642813066:b7f776c0: 9386.642868789:b7f776c0: Command 'actionsendresendlastmsgonreconnect': 9386.642919094:b7f776c0: type : 4 9386.642968326:b7f776c0: pData: 0x80c3758 9386.643017891:b7f776c0: Hdlr : 0x0 9386.643067315:b7f776c0: Owner: 0x0 9386.643143418:b7f776c0: 9386.643195943:b7f776c0: Command 'logrsyslogstatusmessages': 9386.643246033:b7f776c0: type : 4 9386.643295307:b7f776c0: pData: 0x80c0644 9386.643344044:b7f776c0: Hdlr : 0x0 9386.643400040:b7f776c0: Owner: 0x0 9386.643448920:b7f776c0: 9386.643497721:b7f776c0: Command 'actionresumeretrycount': 9386.643547976:b7f776c0: type : 6 9386.643597910:b7f776c0: pData: 0x80c49f0 9386.643653745:b7f776c0: Hdlr : 0x0 9386.643702988:b7f776c0: Owner: 0x0 9386.643752494:b7f776c0: 9386.643801205:b7f776c0: Command 'defaultruleset': 9386.643851363:b7f776c0: type : 11 9386.643907964:b7f776c0: pData: 0x0 9386.643957492:b7f776c0: Hdlr : 0x804f930 9386.644006915:b7f776c0: Owner: 0x0 9386.644055532:b7f776c0: 9386.644104239:b7f776c0: Command 'ruleset': 9386.644184835:b7f776c0: type : 11 9386.644235267:b7f776c0: pData: 0x0 9386.644283977:b7f776c0: Hdlr : 0x804f860 9386.644332636:b7f776c0: Owner: 0x0 9386.644381245:b7f776c0: 9386.644435622:b7f776c0: Command 'sleep': 9386.644485533:b7f776c0: type : 6 9386.644535035:b7f776c0: pData: 0x0 9386.644583820:b7f776c0: Hdlr : 0x804fde0 9386.644632490:b7f776c0: Owner: 0x0 9386.644688438:b7f776c0: 9386.644737284:b7f776c0: Command 'mainmsgqueuefilename': 9386.644787746:b7f776c0: type : 11 9386.644837679:b7f776c0: pData: 0x80c3634 9386.644887222:b7f776c0: Hdlr : 0x0 9386.644942491:b7f776c0: Owner: 0x0 9386.644991247:b7f776c0: 9386.645039875:b7f776c0: Command 'mainmsgqueuesize': 9386.645089636:b7f776c0: type : 6 9386.645138932:b7f776c0: pData: 0x80c0648 9386.645214447:b7f776c0: Hdlr : 0x0 9386.645263949:b7f776c0: Owner: 0x0 9386.645312682:b7f776c0: 9386.645361561:b7f776c0: Command 'mainmsgqueuehighwatermark': 9386.645410947:b7f776c0: type : 6 9386.645465470:b7f776c0: pData: 0x80c064c 9386.645514826:b7f776c0: Hdlr : 0x0 9386.645564039:b7f776c0: Owner: 0x0 9386.645613192:b7f776c0: 9386.645662451:b7f776c0: Command 'mainmsgqueuelowwatermark': 9386.645718624:b7f776c0: type : 6 9386.645768054:b7f776c0: pData: 0x80c0650 9386.645816904:b7f776c0: Hdlr : 0x0 9386.645866571:b7f776c0: Owner: 0x0 9386.645915443:b7f776c0: 9386.645971346:b7f776c0: Command 'mainmsgqueuediscardmark': 9386.646021556:b7f776c0: type : 6 9386.646070683:b7f776c0: pData: 0x80c0654 9386.646120118:b7f776c0: Hdlr : 0x0 9386.646197249:b7f776c0: Owner: 0x0 9386.646248183:b7f776c0: 9386.646297074:b7f776c0: Command 'mainmsgqueuediscardseverity': 9386.646346955:b7f776c0: type : 10 9386.646428312:b7f776c0: pData: 0x80c0658 9386.646485006:b7f776c0: Hdlr : 0x0 9386.646534171:b7f776c0: Owner: 0x0 9386.646583238:b7f776c0: 9386.646632162:b7f776c0: Command 'mainmsgqueuecheckpointinterval': 9386.646685290:b7f776c0: type : 6 9386.646740668:b7f776c0: pData: 0x80c3638 9386.646789754:b7f776c0: Hdlr : 0x0 9386.646838278:b7f776c0: Owner: 0x0 9386.646886673:b7f776c0: 9386.646935152:b7f776c0: Command 'mainmsgqueuesyncqueuefiles': 9386.646992168:b7f776c0: type : 4 9386.647041179:b7f776c0: pData: 0x80c363c 9386.647090355:b7f776c0: Hdlr : 0x0 9386.647139006:b7f776c0: Owner: 0x0 9386.647224251:b7f776c0: 9386.647273914:b7f776c0: Command 'mainmsgqueuetype': 9386.647323652:b7f776c0: type : 11 9386.647372438:b7f776c0: pData: 0x0 9386.647421310:b7f776c0: Hdlr : 0x804fc60 9386.647476913:b7f776c0: Owner: 0x0 9386.647526190:b7f776c0: 9386.647574931:b7f776c0: Command 'mainmsgqueueworkerthreads': 9386.647625138:b7f776c0: type : 6 9386.647674287:b7f776c0: pData: 0x80c065c 9386.647731252:b7f776c0: Hdlr : 0x0 9386.647781099:b7f776c0: Owner: 0x0 9386.647829749:b7f776c0: 9386.647878261:b7f776c0: Command 'mainmsgqueuetimeoutshutdown': 9386.647927576:b7f776c0: type : 6 9386.647982054:b7f776c0: pData: 0x80c0660 9386.648033037:b7f776c0: Hdlr : 0x0 9386.648082160:b7f776c0: Owner: 0x0 9386.648131227:b7f776c0: 9386.648180482:b7f776c0: Command 'mainmsgqueuetimeoutactioncompletion': 9386.648259919:b7f776c0: type : 6 9386.648310133:b7f776c0: pData: 0x80c0664 9386.648359722:b7f776c0: Hdlr : 0x0 9386.648408755:b7f776c0: Owner: 0x0 9386.648457867:b7f776c0: 9386.648513237:b7f776c0: Command 'mainmsgqueuetimeoutenqueue': 9386.648563834:b7f776c0: type : 6 9386.648613430:b7f776c0: pData: 0x80c0668 9386.648662632:b7f776c0: Hdlr : 0x0 9386.648712130:b7f776c0: Owner: 0x0 9386.648768101:b7f776c0: 9386.648816920:b7f776c0: Command 'mainmsgqueueworkertimeoutthreadshutdown': 9386.648866827:b7f776c0: type : 6 9386.648915954:b7f776c0: pData: 0x80c066c 9386.648965497:b7f776c0: Hdlr : 0x0 9386.649022927:b7f776c0: Owner: 0x0 9386.649072327:b7f776c0: 9386.649121814:b7f776c0: Command 'mainmsgqueuedequeueslowdown': 9386.649172509:b7f776c0: type : 6 9386.649222168:b7f776c0: pData: 0x80c3640 9386.649302262:b7f776c0: Hdlr : 0x0 9386.649351295:b7f776c0: Owner: 0x0 9386.649400849:b7f776c0: 9386.649451068:b7f776c0: Command 'mainmsgqueueworkerthreadminimummessages': 9386.649500907:b7f776c0: type : 6 9386.649556424:b7f776c0: pData: 0x80c0670 9386.649606057:b7f776c0: Hdlr : 0x0 9386.649654869:b7f776c0: Owner: 0x0 9386.649703767:b7f776c0: 9386.649752846:b7f776c0: Command 'mainmsgqueuemaxfilesize': 9386.649808929:b7f776c0: type : 7 9386.649858232:b7f776c0: pData: 0x80c0678 9386.649907123:b7f776c0: Hdlr : 0x0 9386.649956231:b7f776c0: Owner: 0x0 9386.650005350:b7f776c0: 9386.650061171:b7f776c0: Command 'mainmsgqueuedequeuebatchsize': 9386.650111175:b7f776c0: type : 7 9386.650168042:b7f776c0: pData: 0x80c0680 9386.650216760:b7f776c0: Hdlr : 0x0 9386.650297585:b7f776c0: Owner: 0x0 9386.650347518:b7f776c0: 9386.650396071:b7f776c0: Command 'mainmsgqueuemaxdiskspace': 9386.650446346:b7f776c0: type : 7 9386.650495319:b7f776c0: pData: 0x80c3648 9386.650551383:b7f776c0: Hdlr : 0x0 9386.650600863:b7f776c0: Owner: 0x0 9386.650649701:b7f776c0: 9386.650698566:b7f776c0: Command 'mainmsgqueuesaveonshutdown': 9386.650748536:b7f776c0: type : 4 9386.650803641:b7f776c0: pData: 0x80c0688 9386.650853885:b7f776c0: Hdlr : 0x0 9386.650903023:b7f776c0: Owner: 0x0 9386.650952387:b7f776c0: 9386.651001401:b7f776c0: Command 'mainmsgqueuedequeuetimebegin':9386.651058272:b7f776c0: type : 6 9386.651108647:b7f776c0: pData: 0x80c3650 9386.651158164:b7f776c0: Hdlr : 0x0 9386.651207108:b7f776c0: Owner: 0x0 9386.651266468:b7f776c0: 9386.651354751:b7f776c0: Command 'mainmsgqueuedequeuetimeend': 9386.651406169:b7f776c0: type : 6 9386.651476835:b7f776c0: pData: 0x80c068c 9386.651527931:b7f776c0: Hdlr : 0x0 9386.651584438:b7f776c0: Owner: 0x0 9386.651662120:b7f776c0: 9386.651711382:b7f776c0: Command 'abortonuncleanconfig': 9386.651761480:b7f776c0: type : 4 9386.651810967:b7f776c0: pData: 0x80c3600 9386.651866529:b7f776c0: Hdlr : 0x0 9386.651915888:b7f776c0: Owner: 0x0 9386.651964914:b7f776c0: 9386.652013287:b7f776c0: Command 'repeatedmsgreduction': 9386.652063393:b7f776c0: type : 4 9386.652119483:b7f776c0: pData: 0x80c4b6c 9386.652168565:b7f776c0: Hdlr : 0x0 9386.652217895:b7f776c0: Owner: 0x0 9386.652266999:b7f776c0: 9386.652315414:b7f776c0: Command 'actionexeconlywhenpreviousissuspended': 9386.652372153:b7f776c0: type : 4 9386.652422195:b7f776c0: pData: 0x80c4b68 9386.652497635:b7f776c0: Hdlr : 0x0 9386.652547291:b7f776c0: Owner: 0x0 9386.652603325:b7f776c0: 9386.652652197:b7f776c0: Command 'actionresumeinterval': 9386.652702441:b7f776c0: type : 6 9386.652752063:b7f776c0: pData: 0x0 9386.652800905:b7f776c0: Hdlr : 0x804fc50 9386.652856486:b7f776c0: Owner: 0x0 9386.652906014:b7f776c0: 9386.652954492:b7f776c0: Command 'template': 9386.653004729:b7f776c0: type : 1 9386.653053961:b7f776c0: pData: 0x0 9386.653109654:b7f776c0: Hdlr : 0x80618e0 9386.653159381:b7f776c0: Owner: 0x0 9386.653208429:b7f776c0: 9386.653257519:b7f776c0: Command 'outchannel': 9386.653307047:b7f776c0: type : 1 9386.653361986:b7f776c0: pData: 0x1 9386.653412471:b7f776c0: Hdlr : 0x80618e0 9386.653461650:b7f776c0: Owner: 0x0 9386.653536232:b7f776c0: 9386.653585704:b7f776c0: Command 'allowedsender': 9386.653642011:b7f776c0: type : 1 9386.653690958:b7f776c0: pData: 0x2 9386.653740243:b7f776c0: Hdlr : 0x80618e0 9386.653789595:b7f776c0: Owner: 0x0 9386.653838291:b7f776c0: 9386.653893545:b7f776c0: Command 'modload': 9386.653942702:b7f776c0: type : 1 9386.653992016:b7f776c0: pData: 0x0 9386.654041196:b7f776c0: Hdlr : 0x8060f00 9386.654090953:b7f776c0: Owner: 0x0 9386.654146608:b7f776c0: 9386.654195375:b7f776c0: Command 'includeconfig': 9386.654245477:b7f776c0: type : 1 9386.654295021:b7f776c0: pData: 0x0 9386.654344080:b7f776c0: Hdlr : 0x8061a80 9386.654399844:b7f776c0: Owner: 0x0 9386.654449403:b7f776c0: 9386.654521003:b7f776c0: Command 'umask': 9386.654572758:b7f776c0: type : 5 9386.654622523:b7f776c0: pData: 0x0 9386.654677312:b7f776c0: Hdlr : 0x804fc10 9386.654726079:b7f776c0: Owner: 0x0 9386.654775558:b7f776c0: 9386.654824202:b7f776c0: Command 'maxopenfiles': 9386.654874907:b7f776c0: type : 6 9386.654931245:b7f776c0: pData: 0x0 9386.654980474:b7f776c0: Hdlr : 0x804fb10 9386.655029496:b7f776c0: Owner: 0x0 9386.655079136:b7f776c0: 9386.655128166:b7f776c0: Command 'debugprinttemplatelist': 9386.655185404:b7f776c0: type : 4 9386.655234542:b7f776c0: pData: 0x80c0690 9386.655284059:b7f776c0: Hdlr : 0x0 9386.655333325:b7f776c0: Owner: 0x0 9386.655382707:b7f776c0: 9386.655438921:b7f776c0: Command 'debugprintmodulelist': 9386.655489045:b7f776c0: type : 4 9386.655562599:b7f776c0: pData: 0x80c0694 9386.655612337:b7f776c0: Hdlr : 0x0 9386.655667213:b7f776c0: Owner: 0x0 9386.655717045:b7f776c0: 9386.655766446:b7f776c0: Command 'debugprintcfsyslinehandlerlist': 9386.655816934:b7f776c0: type : 4 9386.655866462:b7f776c0: pData: 0x80c0698 9386.655921792:b7f776c0: Hdlr : 0x0 9386.655971683:b7f776c0: Owner: 0x0 9386.656020619:b7f776c0: 9386.656069735:b7f776c0: Command 'moddir': 9386.656119140:b7f776c0: type : 11 9386.656174473:b7f776c0: pData: 0x80c4368 9386.656225122:b7f776c0: Hdlr : 0x0 9386.656274182:b7f776c0: Owner: 0x0 9386.656323024:b7f776c0: 9386.656371157:b7f776c0: Command 'generateconfiggraph': 9386.656424330:b7f776c0: type : 11 9386.656476509:b7f776c0: pData: 0x80c3654 9386.656549515:b7f776c0: Hdlr : 0x0 9386.656599257:b7f776c0: Owner: 0x0 9386.656648410:b7f776c0: 9386.656703271:b7f776c0: Command 'errormessagestostderr': 9386.656753346:b7f776c0: type : 4 9386.656803152:b7f776c0: pData: 0x80c069c 9386.656852489:b7f776c0: Hdlr : 0x0 9386.656901871:b7f776c0: Owner: 0x0 9386.656958097:b7f776c0: 9386.657006965:b7f776c0: Command 'maxmessagesize': 9386.657084587:b7f776c0: type : 7 9386.657134393:b7f776c0: pData: 0x0 9386.657184199:b7f776c0: Hdlr : 0x804f640 9386.657241226:b7f776c0: Owner: 0x0 9386.657290605:b7f776c0: 9386.657339769:b7f776c0: Command 'privdroptouser': 9386.657390336:b7f776c0: type : 2 9386.657439898:b7f776c0: pData: 0x80c3658 9386.657495426:b7f776c0: Hdlr : 0x0 9386.657577638:b7f776c0: Owner: 0x0 9386.657627961:b7f776c0: 9386.657677287:b7f776c0: Command 'privdroptouserid': 9386.657733396:b7f776c0: type : 6 9386.657783461:b7f776c0: pData: 0x80c3658 9386.657832948:b7f776c0: Hdlr : 0x0 9386.657882397:b7f776c0: Owner: 0x0 9386.657931918:b7f776c0: 9386.657988429:b7f776c0: Command 'privdroptogroup': 9386.658039007:b7f776c0: type : 3 9386.658088119:b7f776c0: pData: 0x80c365c 9386.658137692:b7f776c0: Hdlr : 0x0 9386.658186849:b7f776c0: Owner: 0x0 9386.658242354:b7f776c0: 9386.658291118:b7f776c0: Command 'privdroptogroupid': 9386.658340590:b7f776c0: type : 3 9386.658390219:b7f776c0: pData: 0x80c365c 9386.658439590:b7f776c0: Hdlr : 0x0 9386.658495069:b7f776c0: Owner: 0x0 9386.658572692:b7f776c0: 9386.658623892:b7f776c0: Command 'markmessageperiod': 9386.658674122:b7f776c0: type : 6 9386.658723628:b7f776c0: pData: 0xf5822c 9386.658778949:b7f776c0: Hdlr : 0x0 9386.658827998:b7f776c0: Owner: 0xf577c0 9386.658877125:b7f776c0: 9386.658926083:b7f776c0: Command 'omitlocallogging': 9386.658975986:b7f776c0: type : 4 9386.659032718:b7f776c0: pData: 0xf0c8b0 9386.659089401:b7f776c0: Hdlr : 0x0 9386.659138682:b7f776c0: Owner: 0xf09a40 9386.659187899:b7f776c0: 9386.659241248:b7f776c0: Command 'inputunixlistensocketignoremsgtimestamp': 9386.659293757:b7f776c0: type : 4 9386.659343293:b7f776c0: pData: 0xf0c750 9386.659392983:b7f776c0: Hdlr : 0x0 9386.659441948:b7f776c0: Owner: 0xf09a40 9386.659490667:b7f776c0: 9386.659544927:b7f776c0: Command 'systemlogsocketname': 9386.659633746:b7f776c0: type : 11 9386.659683518:b7f776c0: pData: 0xf0c8b4 9386.659732941:b7f776c0: Hdlr : 0x0 9386.659788739:b7f776c0: Owner: 0xf09a40 9386.659838264:b7f776c0: 9386.659887833:b7f776c0: Command 'inputunixlistensockethostname': 9386.659938014:b7f776c0: type : 11 9386.659987366:b7f776c0: pData: 0xf0c8b8 9386.660044146:b7f776c0: Hdlr : 0x0 9386.660095103:b7f776c0: Owner: 0xf09a40 9386.660143566:b7f776c0: 9386.660192885:b7f776c0: Command 'inputunixlistensocketflowcontrol': 9386.660242919:b7f776c0: type : 4 9386.660299235:b7f776c0: pData: 0xf0c8bc 9386.660348643:b7f776c0: Hdlr : 0x0 9386.660398096:b7f776c0: Owner: 0xf09a40 9386.660446548:b7f776c0: 9386.660495698:b7f776c0: Command 'inputunixlistensocketcreatepath': 9386.660555073:b7f776c0: type : 4 9386.660631859:b7f776c0: pData: 0xf0c8c0 9386.660680780:b7f776c0: Hdlr : 0x0 9386.660729359:b7f776c0: Owner: 0xf09a40 9386.660782784:b7f776c0: 9386.660832447:b7f776c0: Command 'addunixlistensocket': 9386.660882114:b7f776c0: type : 11 9386.660931560:b7f776c0: pData: 0x0 9386.660980702:b7f776c0: Hdlr : 0xf0adf0 9386.661033410:b7f776c0: Owner: 0xf09a40 9386.661085188:b7f776c0: 9386.661133644:b7f776c0: Command 'systemlogsocketignoremsgtimestamp': 9386.661182643:b7f776c0: type : 4 9386.661231200:b7f776c0: pData: 0x0 9386.661280522:b7f776c0: Hdlr : 0xf09b90 9386.661336174:b7f776c0: Owner: 0xf09a40 9386.661384892:b7f776c0: 9386.661433464:b7f776c0: Command 'systemlogsocketflowcontrol': 9386.661483337:b7f776c0: type : 4 9386.661532573:b7f776c0: pData: 0x0 9386.661610825:b7f776c0: Hdlr : 0xf09b20 9386.661662131:b7f776c0: Owner: 0xf09a40 9386.661710527:b7f776c0: 9386.661759556:b7f776c0: Command 'debugprintkernelsymbols': 9386.661815857:b7f776c0: type : 4 9386.661864972:b7f776c0: pData: 0x13e428 9386.661913638:b7f776c0: Hdlr : 0x0 9386.661962491:b7f776c0: Owner: 0x13ad10 9386.662011780:b7f776c0: 9386.662066378:b7f776c0: Command 'klogpath': 9386.662117200:b7f776c0: type : 11 9386.662165772:b7f776c0: pData: 0x13e43c 9386.662215090:b7f776c0: Hdlr : 0x0 9386.662263621:b7f776c0: Owner: 0x13ad10 9386.662317034:b7f776c0: 9386.662395593:b7f776c0: Command 'klogsymbollookup': 9386.662446085:b7f776c0: type : 4 9386.662494984:b7f776c0: pData: 0x13e434 9386.662544069:b7f776c0: Hdlr : 0x0 9386.662622626:b7f776c0: Owner: 0x13ad10 9386.662672765:b7f776c0: 9386.662721168:b7f776c0: Command 'klogsymbolstwice': 9386.662771255:b7f776c0: type : 4 9386.662820154:b7f776c0: pData: 0x13e42c 9386.662875967:b7f776c0: Hdlr : 0x0 9386.662924351:b7f776c0: Owner: 0x13ad10 9386.662972634:b7f776c0: 9386.663020936:b7f776c0: Command 'klogusesyscallinterface': 9386.663070941:b7f776c0: type : 4 9386.663126701:b7f776c0: pData: 0x13e430 9386.663175333:b7f776c0: Hdlr : 0x0 9386.663224123:b7f776c0: Owner: 0x13ad10 9386.663273209:b7f776c0: 9386.663321207:b7f776c0: Command 'klogpermitnonkernelfacility': 9386.663378209:b7f776c0: type : 4 9386.663427396:b7f776c0: pData: 0x13e438 9386.663476309:b7f776c0: Hdlr : 0x0 9386.663525432:b7f776c0: Owner: 0x13ad10 9386.663573536:b7f776c0: 9386.663656467:b7f776c0: Command 'klogconsoleloglevel': 9386.663706903:b7f776c0: type : 6 9386.663755497:b7f776c0: pData: 0x13e240 9386.663804489:b7f776c0: Hdlr : 0x0 9386.663859054:b7f776c0: Owner: 0x13ad10 9386.663908863:b7f776c0: 9386.663957405:b7f776c0: Command 'kloginternalmsgfacility': 9386.664006675:b7f776c0: type : 9 9386.664056105:b7f776c0: pData: 0x13f9ec 9386.664110313:b7f776c0: Hdlr : 0x0 9386.664160970:b7f776c0: Owner: 0x13ad10 9386.664209557:b7f776c0: 9386.664258223:b7f776c0: Command 'inputtcpserverrun': 9386.664307781:b7f776c0: type : 11 9386.664355817:b7f776c0: pData: 0x0 9386.664411724:b7f776c0: Hdlr : 0x18ae40 9386.664460562:b7f776c0: Owner: 0x18ab20 9386.664508703:b7f776c0: 9386.664557031:b7f776c0: Command 'inputtcpmaxsessions': 9386.664606492:b7f776c0: type : 6 9386.664688944:b7f776c0: pData: 0x18c388 9386.664738517:b7f776c0: Hdlr : 0x0 9386.664787498:b7f776c0: Owner: 0x18ab20 9386.664836554:b7f776c0: 9386.664891710:b7f776c0: Command 'inputtcpmaxlisteners': 9386.664941576:b7f776c0: type : 6 9386.664990448:b7f776c0: pData: 0x18c38c 9386.665039609:b7f776c0: Hdlr : 0x0 9386.665088710:b7f776c0: Owner: 0x18ab20 9386.665145576:b7f776c0: 9386.665194520:b7f776c0: Command 'inputtcpservernotifyonconnectionclose': 9386.665243602:b7f776c0: type : 4 9386.665292740:b7f776c0: pData: 0x18c3a0 9386.665341721:b7f776c0: Hdlr : 0x0 9386.665397736:b7f776c0: Owner: 0x18ab20 9386.665446331:b7f776c0: 9386.665495375:b7f776c0: Command 'inputtcpserverstreamdrivermode': 9386.665545575:b7f776c0: type : 6 9386.665594792:b7f776c0: pData: 0x18c3a4 9386.665676467:b7f776c0: Hdlr : 0x0 9386.665725902:b7f776c0: Owner: 0x18ab20 9386.665774373:b7f776c0: 9386.665822948:b7f776c0: Command 'inputtcpserverstreamdriverauthmode': 9386.665872683:b7f776c0: type : 11 9386.665928751:b7f776c0: pData: 0x18c3a8 9386.665977195:b7f776c0: Hdlr : 0x0 9386.666026555:b7f776c0: Owner: 0x18ab20 9386.666075434:b7f776c0: 9386.666124419:b7f776c0: Command 'inputtcpserverstreamdriverpermittedpeer': 9386.666180671:b7f776c0: type : 11 9386.666230368:b7f776c0: pData: 0x0 9386.666279165:b7f776c0: Hdlr : 0x18b8c0 9386.666327932:b7f776c0: Owner: 0x18ab20 9386.666377445:b7f776c0: 9386.666433532:b7f776c0: Command 'inputtcpserveraddtlframedelimiter': 9386.666483285:b7f776c0: type : 6 9386.666532970:b7f776c0: pData: 0x18c390 9386.666582225:b7f776c0: Hdlr : 0x0 9386.666631397:b7f776c0: Owner: 0x18ab20 9386.666712949:b7f776c0: 9386.666762020:b7f776c0: Command 'inputtcpserverdisablelfdelimiter': 9386.666811642:b7f776c0: type : 4 9386.666861050:b7f776c0: pData: 0x18c3ac 9386.666909982:b7f776c0: Hdlr : 0x0 9386.667113435:b7f776c0: Owner: 0x18ab20 9386.667169271:b7f776c0: 9386.667221551:b7f776c0: Command 'inputtcpserverinputname': 9386.667271841:b7f776c0: type : 11 9386.667321369:b7f776c0: pData: 0x18c3b0 9386.667370920:b7f776c0: Hdlr : 0x0 9386.667425619:b7f776c0: Owner: 0x18ab20 9386.667476685:b7f776c0: 9386.667525748:b7f776c0: Command 'inputtcpserverbindruleset': 9386.667575625:b7f776c0: type : 11 9386.667652351:b7f776c0: pData: 0x0 9386.667716148:b7f776c0: Hdlr : 0x18b7d0 9386.667766283:b7f776c0: Owner: 0x18ab20 9386.667815324:b7f776c0: 9386.667864365:b7f776c0: Command 'omoracledbuser': 9386.667914688:b7f776c0: type : 11 9386.667970557:b7f776c0: pData: 0x114f9c 9386.668020273:b7f776c0: Hdlr : 0x0 9386.668069884:b7f776c0: Owner: 0x111e50 9386.668119828:b7f776c0: 9386.668168516:b7f776c0: Command 'omoracledbpassword': 9386.668225623:b7f776c0: type : 11 9386.668274701:b7f776c0: pData: 0x114fa0 9386.668324593:b7f776c0: Hdlr : 0x0 9386.668373709:b7f776c0: Owner: 0x111e50 9386.668422450:b7f776c0: 9386.668478113:b7f776c0: Command 'omoracledb': 9386.668528241:b7f776c0: type : 11 9386.668577983:b7f776c0: pData: 0x114f98 9386.668626881:b7f776c0: Hdlr : 0x0 9386.668676695:b7f776c0: Owner: 0x111e50 9386.668765529:b7f776c0: 9386.668814851:b7f776c0: Command 'omoraclebatchsize': 9386.668864473:b7f776c0: type : 6 9386.668914249:b7f776c0: pData: 0x114fa4 9386.668969087:b7f776c0: Hdlr : 0x0 9386.669019148:b7f776c0: Owner: 0x111e50 9386.669068342:b7f776c0: 9386.669117623:b7f776c0: Command 'omoraclestatementtemplate': 9386.669167684:b7f776c0: type : 11 9386.669223095:b7f776c0: pData: 0x114fac 9386.669273782:b7f776c0: Hdlr : 0x0 9386.669322707:b7f776c0: Owner: 0x111e50 9386.669371268:b7f776c0: 9386.669419723:b7f776c0: Command 'omoraclebatchitemsize': 9386.669475371:b7f776c0: type : 6 9386.669526580:b7f776c0: pData: 0x114fa8 9386.669575065:b7f776c0: Hdlr : 0x0 9386.669624669:b7f776c0: Owner: 0x111e50 9386.669673698:b7f776c0: 9386.669725997:b7f776c0: 9386.669777701:b7f776c0: Messages with malicious PTR DNS Records are not dropped. 9386.669826899:b7f776c0: Main queue size 10000 messages. 9386.669876465:b7f776c0: Main queue worker threads: 1, wThread shutdown: 60000, Perists every 0 updates. 9386.669929949:b7f776c0: Main queue timeouts: shutdown: 1500, action completion shutdown: 1000, enq: 2000 9386.669989463:b7f776c0: Main queue watermarks: high: 8000, low: 2000, discard: 9800, discard-severity: 8 9386.670043877:b7f776c0: Main queue save on shutdown 1, max disk space allowed 0 9386.670093207:b7f776c0: Work Directory: ''. 9386.670147970:b7f776c0: started. 9386.670265052:b7f776c0: main Q: entry added, size now log 1, phys 1 entries 9386.670351408:b7f776c0: main Q: EnqueueMsg advised worker start 9386.670416933:b7f776c0: Debugging enabled, SIGUSR1 to turn off debugging. 9386.673184474:1959b90: wti 0x91de9c0: worker awoke from idle processing 9386.673281539:1959b90: wti.c:323:wtiWorker: mutex 0x91d35a8 UNlocked [but we did not yet know this mutex!] 9386.673368978:1959b90: we deleted 0 objects and enqueued 0 objects 9386.673425984:1959b90: delete batch from store, new sizes: log 1, phys 1 9386.673493545:1959b90: ZZZ: processBatch: batch of 1 elements must be processed 9386.673553430:1959b90: Processing next rule 9386.673643959:1959b90: testing filter, f_pmask 255 9386.673723389:1959b90: Processing next action 9386.673806055:1959b90: Called action(NotAllMark), logging to omoracle 9386.673900896:1959b90: Called action(Batch), logging to omoracle 9386.673992772:1959b90: Action 0x91cea00 transitioned to state: itx 9386.674069891:1959b90: entering actionCalldoAction(), state: itx 9386.674159772:1959b90: omoracle batch size limit hit, sending into DB 9386.674955647:1959b90: omoracle bound line 0, length 11: clockingit 9386.675032894:1959b90: omoracle bound line 0, length 47: imklog 5.6.0, log source = /proc/kmsg started. 9386.675334777:b7f776c0: initialization completed, transitioning to regular run mode [root at clockingit sbin]# ps ax | grep rsys [root at clockingit sbin]# Well folks this is it, anyone have any idea how to solve the problem? Thanks for help. NOTE: sqlplus works normally: [root at clockingit etc]# sqlplus syslog at RECPOC/syslog SQL*Plus: Release 10.2.0.4.0 - Production on Thu Nov 25 11:05:07 2010 Copyright (c) 1982, 2007, Oracle. All Rights Reserved. Error accessing PRODUCT_USER_PROFILE Warning: Product user profile information not loaded! You may need to run PUPBLD.SQL as SYSTEM Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> desc SystemEvents; Name Null? Type ----------------------------------------- -------- ---------------------------- ID NOT NULL NUMBER(10) CUSTOMERID NUMBER(20) RECEIVEDAT TIMESTAMP(6) DEVICEREPORTEDTIME TIMESTAMP(6) FACILITY VARCHAR2(100) PRIORITY VARCHAR2(100) FROMHOST VARCHAR2(60) MESSAGE VARCHAR2(3999) NTSEVERITY VARCHAR2(100) IMPORTANCE NUMBER(11) EVENTSOURCE VARCHAR2(60) EVENTUSER VARCHAR2(60) EVENTCATEGORY NUMBER(11) EVENTID NUMBER(11) EVENTBINARYDATA VARCHAR2(60) MAXAVAILABLE NUMBER(11) CURRUSAGE NUMBER(11) MINUSAGE NUMBER(11) MAXUSAGE NUMBER(11) INFOUNITID NUMBER(11) SYSLOGTAG VARCHAR2(60) EVENTLOGTYPE VARCHAR2(60) GENERICFILENAME VARCHAR2(60) SYSTEMID NUMBER(11) CHECKSUM NOT NULL NUMBER(11) SQL> insert into systemevents (fromhost,message) values ('clockingit','test'); 1 row created. thanks for help, Thiago. From unclemonty at gmail.com Thu Nov 25 14:19:36 2010 From: unclemonty at gmail.com (Monty) Date: Thu, 25 Nov 2010 13:19:36 +0000 Subject: [rsyslog] hostname / fromhost showing only node In-Reply-To: References: Message-ID: On Wed, Nov 24, 2010 at 9:25 PM, wrote: > > hostname would be whatever is put in the log by the sending server. > > fromhost should be a reverse lookup of the IP address you received the log > from (note that if this is relayed through another host, you get the name/IP > of the relay, not the sender) > > if you are only getting a short name, check your /etc/hosts file to see if > you have it listed there with a short name first (name resolution will use > /etc/hosts before going to DNS, and a name lookup into /etc/hosts will > return the first name on the line) > After much searching I discovered that I needed to set the following to avoid rsyslog making the names short to be compatible with sysklog: $PreserveFQDN on After setting this, I now have the full hostnames. From rgerhards at hq.adiscon.com Thu Nov 25 15:53:39 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 25 Nov 2010 15:53:39 +0100 Subject: [rsyslog] [PATCH] avoid use of non-word size atomics Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7A8@GRFEXC.intern.adiscon.com> Chris, sorry for the late reply, I have now merged the patch to v4 and v5. It will be merged in v6 soon, I am just finishing some things with the older versions so that I do not merge too often. Rainer From friedl at hq.adiscon.com Thu Nov 25 16:46:01 2010 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 25 Nov 2010 16:46:01 +0100 Subject: [rsyslog] rsyslog 4.7.3 (v4-devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7AA@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 4.7.3, a member of the v4-devel branch. 4.7.3 adds some new functions as well as some bugfixes. Now we have omuxsock (permits to write message to local Unix sockets) and imptcp (simplified, Linux-specific and potentielly fast syslog plain tcp input plugin). The fixes take care of the TLS bug that let rsyslog go into a loop, a fix for imfile and more. This release is recommended to all users of syslog via TLS. For more information about this release, please review the ChangeLog. ChangeLog: http://www.rsyslog.com/changelog-for-4-7-3-v4-devel/ Download: http://www.rsyslog.com/rsyslog-4-7-3-v4-devel/ As always, feedback is appreciated. Best regards, Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From mmonitz at gmail.com Sun Nov 28 07:13:40 2010 From: mmonitz at gmail.com (matan monitz) Date: Sun, 28 Nov 2010 08:13:40 +0200 Subject: [rsyslog] website down? Message-ID: hi i'm trying to get 5.6.1 but website seems down any alternate download mirror? From john at feurix.com Sun Nov 28 14:37:36 2010 From: john at feurix.com (John Feuerstein) Date: Sun, 28 Nov 2010 14:37:36 +0100 Subject: [rsyslog] website down? In-Reply-To: References: Message-ID: <4CF25B20.3040202@feurix.com> Hi, > i'm trying to get 5.6.1 but website seems down > any alternate download mirror? it seems like there is no official mirror. I didn't find a single distribution with this version on it's mirrors (remember that 5.6.1 is just a few days old, while you can find 5.6.0 if you want to...). Check out this to get the file md5sum: http://www.google.com/search?q=cache:http://www.rsyslog.com/tag/5-6-1/ > 5.6.1(v5-stable) > md5sum: 315ca33f4c4bb91f9e4b12e5afd8e550 If you trust this, all you have to do now is find someone who already dowloaded it (come on, list...) ;-) I'm sorry I don't have it. Best, John From john at feurix.com Sun Nov 28 14:44:22 2010 From: john at feurix.com (John Feuerstein) Date: Sun, 28 Nov 2010 14:44:22 +0100 Subject: [rsyslog] website down? In-Reply-To: <4CF25B20.3040202@feurix.com> References: <4CF25B20.3040202@feurix.com> Message-ID: <4CF25CB6.70507@feurix.com> Git seems to be up. You can get a snapshot from here (click on the snapshot link, it will create a tar.gz on the fly): http://git.adiscon.com/?p=rsyslog.git;a=commit;h=v5.6.1 Best, John From mmonitz at gmail.com Sun Nov 28 14:56:53 2010 From: mmonitz at gmail.com (matan monitz) Date: Sun, 28 Nov 2010 15:56:53 +0200 Subject: [rsyslog] website down? In-Reply-To: <4CF25CB6.70507@feurix.com> References: <4CF25B20.3040202@feurix.com> <4CF25CB6.70507@feurix.com> Message-ID: thank you guess it's back to reading documentation from google cache On Sun, Nov 28, 2010 at 3:44 PM, John Feuerstein wrote: > Git seems to be up. You can get a snapshot from here (click on the > snapshot link, it will create a tar.gz on the fly): > > http://git.adiscon.com/?p=rsyslog.git;a=commit;h=v5.6.1 > > Best, > John > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Sun Nov 28 16:24:24 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 28 Nov 2010 16:24:24 +0100 Subject: [rsyslog] website down? References: <4CF25B20.3040202@feurix.com> <4CF25CB6.70507@feurix.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7BD@GRFEXC.intern.adiscon.com> Sorry -- I just tried to re-set the server, but this obviously has not worked. Will try to fix this asap... Everything, including doc, is inside the git. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of John Feuerstein > Sent: Sunday, November 28, 2010 2:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] website down? > > Git seems to be up. You can get a snapshot from here (click on the > snapshot link, it will create a tar.gz on the fly): > > http://git.adiscon.com/?p=rsyslog.git;a=commit;h=v5.6.1 > > Best, > John > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From dave at fly.srk.fer.hr Mon Nov 29 12:10:15 2010 From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=) Date: Mon, 29 Nov 2010 12:10:15 +0100 Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> Message-ID: <20101129111015.GA23306@fly.srk.fer.hr> Florian Riedl wrote: > We have just released rsyslog 5.6.1, the new v5-stable. I'm attaching a patch needed to build on Linux systems which lack epoll_create1(). BTW, can you use libevent[1] for things like epoll? It gives you a portable API and supports more systems. [1] http://monkey.org/~provos/libevent/ -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | dave at fly.srk.fer.hr -------------- next part -------------- A non-text attachment was scrubbed... Name: rsyslog-5.6.1-epoll_create1.diff Type: text/x-diff Size: 357 bytes Desc: not available URL: From rgerhards at hq.adiscon.com Mon Nov 29 12:18:34 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 29 Nov 2010 12:18:34 +0100 Subject: [rsyslog] rsyslog 5.6.1 (v5-stable) released References: <9B6E2A8877C38245BFB15CC491A11DA71DD796@GRFEXC.intern.adiscon.com> <20101129111015.GA23306@fly.srk.fer.hr> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7C5@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar > Sent: Monday, November 29, 2010 12:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 5.6.1 (v5-stable) released > > Florian Riedl wrote: > > > We have just released rsyslog 5.6.1, the new v5-stable. > > I'm attaching a patch needed to build on Linux systems which lack > epoll_create1(). Thanks, but it is already fixed in git, together with a couple of other things I'll probably release 5.6.2 tomorrow. > > BTW, can you use libevent[1] for things like epoll? It gives you a > portable API and supports more systems. This is really a good idea, but I am a bit concerned about dragging in the dependency. Because that would mean every system would also need libevent installed, at least when I finally switch everything to that... Rainer > > [1] http://monkey.org/~provos/libevent/ > > -- > .-. .-. Yes, I am an agent of Satan, but my duties are largely > (_ \ / _) ceremonial. > | > | dave at fly.srk.fer.hr From hajo.locke at gmx.de Mon Nov 29 16:19:23 2010 From: hajo.locke at gmx.de (Hajo Locke) Date: Mon, 29 Nov 2010 16:19:23 +0100 Subject: [rsyslog] queuelength of unwritten messages Message-ID: <0129623643304F288F738A7AB0739801@nmm.local> Hello, read article on http://www.rsyslog.com/doc/rsyslog_high_database_rate.html is there a possibility to find out queuelength of unwritten messages to see if it is still growing? i try to use rsyslog for my smtp-cluster but iam unexperienced how fast my system is writing data. if i could see a kind of number of stored messages it would help to assess current situation. btw, is it possible to write with usual syslog to a rsyslogserver? or should rsyslog be used for client and server? Thanks, Hajo From david at lang.hm Mon Nov 29 20:54:11 2010 From: david at lang.hm (david at lang.hm) Date: Mon, 29 Nov 2010 11:54:11 -0800 (PST) Subject: [rsyslog] queuelength of unwritten messages In-Reply-To: <0129623643304F288F738A7AB0739801@nmm.local> References: <0129623643304F288F738A7AB0739801@nmm.local> Message-ID: On Mon, 29 Nov 2010, Hajo Locke wrote: > read article on http://www.rsyslog.com/doc/rsyslog_high_database_rate.html > is there a possibility to find out queuelength of unwritten messages to see > if it is still growing? > i try to use rsyslog for my smtp-cluster but iam unexperienced how fast my > system is writing data. > if i could see a kind of number of stored messages it would help to assess > current situation. historically there has not been a good way to find this. This was asked for a month or two ago, but I'm not sure if it got in or not. One thing you can do is to watch how much ram rsyslog is using, if the queue is continuing to grow, the amount of ram used will grow. However, I do not think it always releases memory when the queue size goes down. > btw, is it possible to write with usual syslog to a rsyslogserver? or should > rsyslog be used for client and server? rsyslog is happy to accept messages from a normal syslog sender. depending on what features you want, there may be advantages to useing rsyslog as the sender as well. 1. software consistancy (you don't have to worry about two different syslog apps) 2. you can use different log formats (high precision timestamps, etc) 3. you can use different transport mechanisms (relp, tcp, encrypted, etc) but in no way do you have to change the senders when you change the receiver. David Lang From rgerhards at hq.adiscon.com Tue Nov 30 10:47:39 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 30 Nov 2010 10:47:39 +0100 Subject: [rsyslog] queuelength of unwritten messages References: <0129623643304F288F738A7AB0739801@nmm.local> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7D0@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Monday, November 29, 2010 8:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] queuelength of unwritten messages > > On Mon, 29 Nov 2010, Hajo Locke wrote: > > > read article on > http://www.rsyslog.com/doc/rsyslog_high_database_rate.html > > is there a possibility to find out queuelength of unwritten messages > to see > > if it is still growing? > > i try to use rsyslog for my smtp-cluster but iam unexperienced how > fast my > > system is writing data. > > if i could see a kind of number of stored messages it would help to > assess > > current situation. > > historically there has not been a good way to find this. This was > asked for a month or two ago, but I'm not sure if it got in or not. I added a stats module a little bit ago: http://www.rsyslog.com/periodic-statistics-on-rsyslog-counters/ I think the counters that were asked for are present (if not, it would probably not be too hard to add them). Rainer From hajo.locke at gmx.de Tue Nov 30 13:23:02 2010 From: hajo.locke at gmx.de (Hajo Locke) Date: Tue, 30 Nov 2010 13:23:02 +0100 Subject: [rsyslog] queuelength of unwritten messages References: <0129623643304F288F738A7AB0739801@nmm.local> <9B6E2A8877C38245BFB15CC491A11DA71DD7D0@GRFEXC.intern.adiscon.com> Message-ID: <8F2BB67B58D8413A8149643D9232FF65@nmm.local> > I added a stats module a little bit ago: > > http://www.rsyslog.com/periodic-statistics-on-rsyslog-counters/ > > I think the counters that were asked for are present (if not, it would > probably not be too hard to add them). great, i will watch development Thanks, Hajo From tbergfeld at hq.adiscon.com Tue Nov 30 15:04:40 2010 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Tue, 30 Nov 2010 15:04:40 +0100 Subject: [rsyslog] rsyslog 5.7.2 (v5-devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7E9@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 5.7.2, the new v5-devel. This update addresses the TLS bug, that has been bothering a lot of users lately. It stops rsyslog from looping, thus disabling functionality and bearing the risk of unresponsiveness of the whole system. More details about this issue can be found here: http://bugzilla.adiscon.com/show_bug.cgi?id=194 There are also some other bugfixes. For more detailed information, please review the ChangeLog. We strongly recommend to use this release if you are using (or planning to use) syslog with TLS enabled. ChangeLog: http://www.rsyslog.com/changelog-for-5-7-2-v5-devel/ Download: http://www.rsyslog.com/rsyslog-5-7-2-v5-devel/ As always, feedback is appreciated. Best regards, Tom Bergfeld -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From tbergfeld at hq.adiscon.com Tue Nov 30 15:05:08 2010 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Tue, 30 Nov 2010 15:05:08 +0100 Subject: [rsyslog] rsyslog 5.6.2 (v5-stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7EA@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 5.6.2, the new v5-stable. This release includes some bug fixes but no new functionality.Please see the ChangeLog for more details. ChangeLog: http://www.rsyslog.com/changelog-for-5-6-2-v5-stable/ Download: http://www.rsyslog.com/rsyslog-5-6-2-v5-stable/ As always, feedback is appreciated. Best regards, Tom Bergfeld -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From tbergfeld at hq.adiscon.com Tue Nov 30 15:05:40 2010 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Tue, 30 Nov 2010 15:05:40 +0100 Subject: [rsyslog] rsyslog 6.1.1 (devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7EB@GRFEXC.intern.adiscon.com> Hi all, We have just released rsyslog 6.1.1, the new v6-devel. This update addresses the TLS bug, that has been bothering a lot of users lately. It stops rsyslog from looping, thus disabling functionality and bearing the risk of unresponsiveness of the whole system. More details about this issue can be found here: http://bugzilla.adiscon.com/show_bug.cgi?id=194 There are also some other bugfixes and the support for omhdfs and systemd is officially added. For more detailed information, please review the ChangeLog. We strongly recommend to use this release if you are using (or planning to use) syslog with TLS enabled. ChangeLog: http://www.rsyslog.com/changelog-for-6-1-1-devel/ Download: http://www.rsyslog.com/rsyslog-6-1-1-devel/ As always, feedback is appreciated. Best regards, Tom Bergfeld -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From jason at jasonantman.com Tue Nov 30 16:35:56 2010 From: jason at jasonantman.com (Jason Antman) Date: Tue, 30 Nov 2010 10:35:56 -0500 Subject: [rsyslog] Configuration graph Message-ID: <4CF519DC.2000701@jasonantman.com> Hi guys, I'm running rsyslogd 5.6.0 with a relatively complex config including two rulesets. I've been having some problems with one if statement never triggering. I attempted to generate a configuration graph following the instructions, but ended up with something that just looks like: inputs | main queue | discard I tried removing one of the rulesets and generating a simpler config, but still no luck. Is this a known issue? Are there any special configure options required to make graph generation work? Does anyone have a working sample graph that they could show me, so I know what it *should* look like? Thanks, Jason PS - the configuration lines I'm having difficulty with: if $msg contains 'logout_suc' and not $msg contains 'role name: Un-registered' \ then :ommysql:localhost,wireless_logs,user,pass;logout & :ommysql:localhost,wireless_logs,user,pass;logout-web From rgerhards at hq.adiscon.com Tue Nov 30 16:41:08 2010 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 30 Nov 2010 16:41:08 +0100 Subject: [rsyslog] Configuration graph References: <4CF519DC.2000701@jasonantman.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD7F0@GRFEXC.intern.adiscon.com> ah, yes, sorry, this is a known issue. We have had some very dramatic changes to the core engine and I needed to disable most part of the configuration graph because it requires a lot of work to make it available again. Sorry for that... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jason Antman > Sent: Tuesday, November 30, 2010 4:36 PM > To: rsyslog-users > Subject: [rsyslog] Configuration graph > > Hi guys, > > I'm running rsyslogd 5.6.0 with a relatively complex config including > two rulesets. I've been having some problems with one if statement > never > triggering. I attempted to generate a configuration graph following the > instructions, but ended up with something that just looks like: > > inputs > | > main queue > | > discard > > I tried removing one of the rulesets and generating a simpler config, > but still no luck. > > Is this a known issue? Are there any special configure options required > to make graph generation work? Does anyone have a working sample graph > that they could show me, so I know what it *should* look like? > > Thanks, > Jason > > PS - the configuration lines I'm having difficulty with: > > if $msg contains 'logout_suc' and not $msg contains 'role name: > Un-registered' \ > then :ommysql:localhost,wireless_logs,user,pass;logout > & :ommysql:localhost,wireless_logs,user,pass;logout-web > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jason at jasonantman.com Tue Nov 30 16:41:51 2010 From: jason at jasonantman.com (Jason Antman) Date: Tue, 30 Nov 2010 10:41:51 -0500 Subject: [rsyslog] Configuration graph In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD7F0@GRFEXC.intern.adiscon.com> References: <4CF519DC.2000701@jasonantman.com> <9B6E2A8877C38245BFB15CC491A11DA71DD7F0@GRFEXC.intern.adiscon.com> Message-ID: <4CF51B3F.1020507@jasonantman.com> No problem, just wanted to make sure it was the code and not me... Thanks, Jason Rainer Gerhards wrote: > ah, yes, sorry, this is a known issue. We have had some very dramatic changes > to the core engine and I needed to disable most part of the configuration > graph because it requires a lot of work to make it available again. > > Sorry for that... > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Jason Antman >> Sent: Tuesday, November 30, 2010 4:36 PM >> To: rsyslog-users >> Subject: [rsyslog] Configuration graph >> >> Hi guys, >> >> I'm running rsyslogd 5.6.0 with a relatively complex config including >> two rulesets. I've been having some problems with one if statement >> never >> triggering. I attempted to generate a configuration graph following the >> instructions, but ended up with something that just looks like: >> >> inputs >> | >> main queue >> | >> discard >> >> I tried removing one of the rulesets and generating a simpler config, >> but still no luck. >> >> Is this a known issue? Are there any special configure options required >> to make graph generation work? Does anyone have a working sample graph >> that they could show me, so I know what it *should* look like? >> >> Thanks, >> Jason >> >> PS - the configuration lines I'm having difficulty with: >> >> if $msg contains 'logout_suc' and not $msg contains 'role name: >> Un-registered' \ >> then :ommysql:localhost,wireless_logs,user,pass;logout >> & :ommysql:localhost,wireless_logs,user,pass;logout-web >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > >