From janfrode at tanso.net Mon Sep 3 10:47:26 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 3 Sep 2007 10:47:26 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> Message-ID: Another one of these, this time with v1.19.3: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xae3fa998 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8057393] rsyslogd[0x804de0a] rsyslogd(llExecFunc+0x3f)[0x805ea3f] rsyslogd[0x804d86a] rsyslogd[0x804d997] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] And I didn't get any core-file, maybe because the v1.19.3 overwrote my "ulimit -c unlimited" change to the initscript... Ooops :-) -jf From rgerhards at hq.adiscon.com Tue Sep 4 17:57:55 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 17:57:55 +0200 Subject: [rsyslog] rsyslog 1.19.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27890E@grfint2.intern.adiscon.com> Hi all, rsyslog 1.19.4 is a bug fixing release. It contains no new features, but stability updates. Most importantly, it addresses some bugs that have lead to program aborts. 1.19.4 is a recommended update for all users. Changelog: http://www.rsyslog.com/Article123.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-56.phtml As always, feedback is very appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Tue Sep 4 17:58:39 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 17:58:39 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> Hi, I have just release 1.19.4 and hope that the fixes also address your problem. I'd appreciate if you could try it out. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Monday, September 03, 2007 10:47 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] v1.19.1 is crashing > > > Another one of these, this time with v1.19.3: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xae3fa998 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x73)[0x8057393] > rsyslogd[0x804de0a] > rsyslogd(llExecFunc+0x3f)[0x805ea3f] > rsyslogd[0x804d86a] > rsyslogd[0x804d997] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > And I didn't get any core-file, maybe because the v1.19.3 overwrote my > "ulimit -c unlimited" change to the initscript... Ooops :-) > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Sep 4 18:58:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 18:58:48 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Hi, I noticed that one problem I received patches for (and that are now included in 1.19.4) is rooted in something that is not fully patched. I think I've found that root cause now (thanks to the info with that patches). The bottom line, however, is that 1.19.4 may still have some stability issues. However, they should surface now only in very obscure cases (but what is obscure...). I'd still appreciate if you could apply 1.19.4 and tell me the outcome. I am now working on fixing the root cause. That might take a short while, as I am thinking about the best *design* to fix the issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, September 04, 2007 5:59 PM > To: rsyslog-users > Subject: Re: [rsyslog] v1.19.1 is crashing > > Hi, > > I have just release 1.19.4 and hope that the fixes also address your > problem. I'd appreciate if you could try it out. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > > Sent: Monday, September 03, 2007 10:47 AM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] v1.19.1 is crashing > > > > > > Another one of these, this time with v1.19.3: > > > > *** glibc detected *** rsyslogd: corrupted double-linked list: > > 0xae3fa998 *** > > ======= Backtrace: ========= > > /lib/libc.so.6[0x4152ce3e] > > /lib/libc.so.6(cfree+0x90)[0x415305d0] > > rsyslogd(MsgDestruct+0x73)[0x8057393] > > rsyslogd[0x804de0a] > > rsyslogd(llExecFunc+0x3f)[0x805ea3f] > > rsyslogd[0x804d86a] > > rsyslogd[0x804d997] > > /lib/libpthread.so.0[0x416112db] > > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > > > And I didn't get any core-file, maybe because the v1.19.3 overwrote > my > > "ulimit -c unlimited" change to the initscript... Ooops :-) > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Wed Sep 5 10:15:26 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 5 Sep 2007 10:15:26 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: On 2007-09-04, Rainer Gerhards wrote: > > I'd still appreciate if you could apply 1.19.4 and tell me the outcome. > I am now working on fixing the root cause. That might take a short > while, as I am thinking about the best *design* to fix the issue. OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you know if it fails again. -jf From theinric at redhat.com Wed Sep 5 11:53:10 2007 From: theinric at redhat.com (theinric at redhat.com) Date: Wed, 05 Sep 2007 11:53:10 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com> Message-ID: <46DE7C86.1040308@redhat.com> You have an error in your config file, but it's probably harmless: %timegenerated::fulltime% The option fulltime doesn't exist and it looks like it never did. Additionally, a colon is missing. I've noticed that this definition is actually present in sample.conf, so you've probably picked it up there. (It looks like it has been there at least from 0.8.1) I've did some testing and it probably doesn't have any impact except for a warning message in debug mode. Jan-Frode Myklebust wrote: > On 2007-08-31, theinric at redhat.com wrote: >> could you please provide some more info on your configuration? >> Configuration file, > > ################################################################################# > $ grep -v ^# /etc/rsyslog.conf|grep -v ^$ > $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.log" > *.* -?DailyPerHostLogs > $template MaillogTemplate,"%timegenerated::fulltime% %HOSTNAME% %syslogtag%: %msg%\n" > $template HourlyMaillog,"/var/log/syslog/maillog/%$YEAR%/%$MONTH%/%$DAY%/maillog-%$YEAR%%$MONTH%%$DAY%%$HOUR%.log" > mail.* -?HourlyMaillog;MaillogTemplate > $template precise,"%timegenerated::fulltime% %HOSTNAME% %syslogfacility-text%/%syslogseverity-text% %syslogtag% %msg%\n" > *.* -/var/log/syslog/everything;precise > mail.* ~ > $template PerAppLogs,"/var/log/syslog/apps/%programname%.log" > *.* -?PerAppLogs > :msg, contains, "ServeRAID" -/var/log/syslog/apps/serveraid.log > :HOSTNAME, !isequal, "loghost1" ~ > *.info;mail.none;authpriv.none;cron.none /var/log/messages > authpriv.* /var/log/secure > mail.* -/var/log/maillog > cron.* /var/log/cron > *.emerg * > uucp,news.crit /var/log/spooler > local7.* /var/log/boot.log > ################################################################################# > > >> options used, > > $ grep -v ^# /etc/sysconfig/rsyslog > SYSLOGD_OPTIONS="-m 0 -r514" > KLOGD_OPTIONS="-x" > SYSLOG_UMASK=077 > >> log entries preceding the crash, ... > > It's a quite busy log server, with about 70 active old style syslog servers > sending logs to it. The second it crashed it wrote 111 log-messages.. (273 > the second before), mostly various postfix daemons, and I'd need to anonymize > them before sharing.. Can't see anything special. > >> If logging forwarded messages, is the remote logger also rsyslog? > > No, all are RHEL3/4/5 with their default syslogd server. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Sep 5 12:25:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 5 Sep 2007 12:25:48 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <46DE7C86.1040308@redhat.com> References: <46D81432.1090302@redhat.com> <46DE7C86.1040308@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278919@grfint2.intern.adiscon.com> I have checked on fulltime, it is an error in sample.conf - there is no need for this option and I think it was actually not present. I'll remove that sample. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of theinric at redhat.com > Sent: Wednesday, September 05, 2007 11:53 AM > To: rsyslog-users > Subject: Re: [rsyslog] v1.19.1 is crashing > > You have an error in your config file, but it's probably harmless: > %timegenerated::fulltime% > > The option fulltime doesn't exist and it looks like it never did. > Additionally, > a colon is missing. I've noticed that this definition is actually > present in > sample.conf, so you've probably picked it up there. (It looks like it > has been > there at least from 0.8.1) > I've did some testing and it probably doesn't have any impact except > for a > warning message in debug mode. > > Jan-Frode Myklebust wrote: > > On 2007-08-31, theinric at redhat.com wrote: > >> could you please provide some more info on your configuration? > >> Configuration file, > > > > > ####################################################################### > ########## > > $ grep -v ^# /etc/rsyslog.conf|grep -v ^$ > > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.lo > g" > > *.* -?DailyPerHostLogs > > $template MaillogTemplate,"%timegenerated::fulltime% %HOSTNAME% > %syslogtag%: %msg%\n" > > $template > HourlyMaillog,"/var/log/syslog/maillog/%$YEAR%/%$MONTH%/%$DAY%/maillog- > %$YEAR%%$MONTH%%$DAY%%$HOUR%.log" > > mail.* -?HourlyMaillog;MaillogTemplate > > $template precise,"%timegenerated::fulltime% %HOSTNAME% > %syslogfacility-text%/%syslogseverity-text% %syslogtag% %msg%\n" > > *.* -/var/log/syslog/everything;precise > > mail.* ~ > > $template PerAppLogs,"/var/log/syslog/apps/%programname%.log" > > *.* -?PerAppLogs > > :msg, contains, "ServeRAID" - > /var/log/syslog/apps/serveraid.log > > :HOSTNAME, !isequal, "loghost1" ~ > > *.info;mail.none;authpriv.none;cron.none > /var/log/messages > > authpriv.* > /var/log/secure > > mail.* - > /var/log/maillog > > cron.* /var/log/cron > > *.emerg * > > uucp,news.crit > /var/log/spooler > > local7.* > /var/log/boot.log > > > ####################################################################### > ########## > > > > > >> options used, > > > > $ grep -v ^# /etc/sysconfig/rsyslog > > SYSLOGD_OPTIONS="-m 0 -r514" > > KLOGD_OPTIONS="-x" > > SYSLOG_UMASK=077 > > > >> log entries preceding the crash, ... > > > > It's a quite busy log server, with about 70 active old style syslog > servers > > sending logs to it. The second it crashed it wrote 111 log-messages.. > (273 > > the second before), mostly various postfix daemons, and I'd need to > anonymize > > them before sharing.. Can't see anything special. > > > >> If logging forwarded messages, is the remote logger also rsyslog? > > > > No, all are RHEL3/4/5 with their default syslogd server. > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Wed Sep 5 12:29:55 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Wed, 05 Sep 2007 12:29:55 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: <1188988195.3362.17.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Dienstag, den 04.09.2007, 18:58 +0200 schrieb Rainer Gerhards: > Hi, > > I noticed that one problem I received patches for (and that are now > included in 1.19.4) is rooted in something that is not fully patched. I > think I've found that root cause now (thanks to the info with that > patches). The bottom line, however, is that 1.19.4 may still have some > stability issues. However, they should surface now only in very obscure > cases (but what is obscure...). Well, at least it was running for almost two hours..... I'm running RHEL5. Here is my config: $AllowedSender UDP, 1.2.3.0/24 $AllowedSender TCP, 1.2.3.0/24 $ModLoad MySQL $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/clamav" $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/exim" $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" *.* /home/local/log/all !rsyslogd :programname, contains, "rsyslogd" /home/local/log/rsyslogd !spamd :msg, contains, "prefork: child states" ~ :msg, contains, "spamd: got connection over /opt/antispam/var/socket/spamd" ~ :msg, contains, "spamd: checking message (unknown) for exim:427" ~ :msg, contains, "spamd: handled cleanup of child pid" ~ mail.* /home/local/log/mail !clamd :msg, contains, "No stats for Database check - forcing reload" ~ :msg, contains, "Reading databases from /var/clamav" ~ :msg, contains, "Database correctly reloaded" ~ :msg, contains, "SelfCheck: Database status OK." ~ local5.* ?clamavFile !exim *.* ?eximFile :msg, contains, "malware detected" ?avFile -- CU, Patrick. From rgerhards at hq.adiscon.com Thu Sep 6 17:58:56 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Sep 2007 17:58:56 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Hi all, We are nearing the point where a decision about the future config file format needs to be made. I have blogged the details: http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html I would deeply appreciate any feedback on the samples and format suggestions. Best regards, Rainer Gerhards From janfrode at tanso.net Fri Sep 7 08:51:13 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 7 Sep 2007 08:51:13 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: On 2007-09-05, Jan-Frode Myklebust wrote: > On 2007-09-04, Rainer Gerhards wrote: >> >> I'd still appreciate if you could apply 1.19.4 and tell me the outcome. >> I am now working on fixing the root cause. That might take a short >> while, as I am thinking about the best *design* to fix the issue. > > OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you > know if it fails again. It failed again yesterday: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xb7209028 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8057e93] rsyslogd[0x804de4a] rsyslogd(llExecFunc+0x3f)[0x805eb0f] rsyslogd[0x804d8aa] rsyslogd[0x804d9d7] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] I have "mon" monitoring that rsyslogd is running, and restart it when it fails. "mon" restarted rsyslogd twice (Thu Sep 6 20:38, and Fri Sep 7 03:27), but I can't find any backtrace from the second crash.. -jf From rgerhards at hq.adiscon.com Fri Sep 7 09:50:32 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 09:50:32 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com><577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278936@grfint2.intern.adiscon.com> Thanks for the feedback. I will probably release a new version today. It has an important fix, which hopefully solves this issue. The bad thing is that I can not reproduce the problem in my lab, so I am basically back to reviewing code and listening to your feedback ;) I have one more area (in the same class) under suspicion. But maybe I do not change that before trying out the current code change. As a side-note, the *actual* root cause was a too-complex internal API, which lead to wrong calling sequences in some parts of the code. I have now re-structured the API and revisited all places where it was called. There is another similar API and this is what I am currently reviewing. I am not sure I like to change that API without real need, because it is used a lot and any such change of course has new bug potential. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, September 07, 2007 8:51 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] v1.19.1 is crashing > > On 2007-09-05, Jan-Frode Myklebust wrote: > > On 2007-09-04, Rainer Gerhards wrote: > >> > >> I'd still appreciate if you could apply 1.19.4 and tell me the > outcome. > >> I am now working on fixing the root cause. That might take a short > >> while, as I am thinking about the best *design* to fix the issue. > > > > OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you > > know if it fails again. > > It failed again yesterday: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xb7209028 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x73)[0x8057e93] > rsyslogd[0x804de4a] > rsyslogd(llExecFunc+0x3f)[0x805eb0f] > rsyslogd[0x804d8aa] > rsyslogd[0x804d9d7] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > I have "mon" monitoring that rsyslogd is running, and restart it when > it fails. "mon" restarted rsyslogd twice (Thu Sep 6 20:38, and Fri > Sep 7 03:27), but I can't find any backtrace from the second crash.. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Sep 7 13:32:13 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 7 Sep 2007 13:32:13 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Message-ID: On 2007-09-06, Rainer Gerhards wrote: > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > I would deeply appreciate any feedback on the samples and format > suggestions. /me thinks you're getting way too little feedback on the blog, or this list. Unfortunately I don't have much more than simple preference to contribute here.. XML-based format: Yikes, you'll need an additional human readable frontend format that's converted to XML for it to be usable. You can't expect us poor sysadmins to be editing XML directly to configure rsyslogd.. syslog-ng like: Fair enough.. It works for my usage. Metalog like: No experience.. Apache like: Not sure I understand this.. Seems like a mix of option/value and xml'ish for some functionality. Programming like..: Of the samples in the wiki, I most prefer the BASIC-like. It resembles python to me, and also "mon"'s config format. Very readable. http://mon.wiki.kernel.org/index.php/Mon_Manual The c-like with functions seems too complex: if1: { if(%severity < "debug" && lower(substr(%msg, 5, 3)) != "err") } action1() { action(type=filewrite, file="/var/log/mail.log") } rule1() { if1() action1() action(type=filewrite, file="/var/log/messages.log") } rule(if1,action1) ruleset(rule1, rule(if1, action(type=filewrite, file="/var/log/messages.log"))) rule(action1(),input="$all") input(type=udp, bind="127.0.0.1") I can't parse this.. Does rule1() break out of if1() is false? Then I guess writes to /var/log/messages.log woun't happen if action1 for some reason failed ? Contrast it to mon's config translated to syslogging: # Define some groups of servers: hostgroup mailservers server1 server2 server3 hostgroup webservers server4 server5 watch mailservers severity > debug SUBMSG = lower(substr(%msg, 5, 3)) SUBMSG != "err" logwrite /var/log/mail.log logwrite /var/log/messages.log SUBMSG == "err" logwrite /var/log/err.log watch webservers programname == httpd severity == crit cmd wall "httpd critical: $msg" logwrite /var/log/crit.log severity < crit logwrite /var/log/httpd.log Each indentation means it's depending on the previous statement being true. You might need to be drinking the python Kool-Aid to see the beauty :-) -jf From skvidal at fedoraproject.org Fri Sep 7 13:40:59 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 07 Sep 2007 07:40:59 -0400 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Message-ID: <1189165259.16157.114.camel@cutter> On Fri, 2007-09-07 at 13:32 +0200, Jan-Frode Myklebust wrote: > On 2007-09-06, Rainer Gerhards wrote: > > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > I would deeply appreciate any feedback on the samples and format > > suggestions. > > /me thinks you're getting way too little feedback on the blog, > or this list. Unfortunately I don't have much more than simple > preference to contribute here.. > > XML-based format: > > Yikes, you'll need an additional human readable frontend > format that's converted to XML for it to be usable. You > can't expect us poor sysadmins to be editing XML > directly to configure rsyslogd.. The nice piece of this is that it is machine parseable easily which enables lots of useful editors. > > syslog-ng like: > > Fair enough.. It works for my usage. The syntax is okay but at that point what distinguishes b/t syslog-ng and rsyslog? > Apache like: > > Not sure I understand this.. Seems like a mix of option/value > and xml'ish for some functionality. This one I'm more interested in. If you think of each log like a vhost and you define the qualities that are added to that inside the definition Destination /path/to/silly.log DestinationMode 0640 DestinationOwner root DestinationGroup log-readers Include mail.info kern.debug cron.emerg etc, etc, etc maybe that doesn't make sense, maybe it does - it is pretty easy to read, though. -sv From rgerhards at hq.adiscon.com Fri Sep 7 14:08:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 14:08:02 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: <1189165259.16157.114.camel@cutter> References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> <1189165259.16157.114.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27893E@grfint2.intern.adiscon.com> I am replying here, but without a real reply. All feedback is deeply appreciate, but I'd like to keep silent for the time being to avoid bringing in my personal bias. Please keep commenting. I'll do a wrap-up later. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of seth vidal > Sent: Friday, September 07, 2007 1:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog config file format - please provide > feedback > > > On Fri, 2007-09-07 at 13:32 +0200, Jan-Frode Myklebust wrote: > > On 2007-09-06, Rainer Gerhards wrote: > > > > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > > > I would deeply appreciate any feedback on the samples and format > > > suggestions. > > > > /me thinks you're getting way too little feedback on the blog, > > or this list. Unfortunately I don't have much more than simple > > preference to contribute here.. > > > > XML-based format: > > > > Yikes, you'll need an additional human readable frontend > > format that's converted to XML for it to be usable. You > > can't expect us poor sysadmins to be editing XML > > directly to configure rsyslogd.. > > The nice piece of this is that it is machine parseable easily which > enables lots of useful editors. > > > > > syslog-ng like: > > > > Fair enough.. It works for my usage. > > The syntax is okay but at that point what distinguishes b/t syslog-ng > and rsyslog? > > > > Apache like: > > > > Not sure I understand this.. Seems like a mix of option/value > > and xml'ish for some functionality. > > This one I'm more interested in. If you think of each log like a vhost > and you define the qualities that are added to that inside the > definition > > > Destination /path/to/silly.log > DestinationMode 0640 > DestinationOwner root > DestinationGroup log-readers > Include mail.info kern.debug cron.emerg > > > etc, etc, etc > > maybe that doesn't make sense, maybe it does - it is pretty easy to > read, though. > > -sv > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 7 18:14:34 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 18:14:34 +0200 Subject: [rsyslog] rsyslog 1.19.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> Hi all, Rsyslog 1.19.5 has been released. It is primarily targeted at fixing rare situations in which a segfault could occur. These have not consistently been able to reproduce in lab. Release 1.19.5 may still have a problem or may hang where previous versions segfaulted. We are actively looking for feedback from the field. Feature-wise, the $ModDir config directive has been added and $ModLoad has been enhanced. We recommend upgrading only for those that experience the problem with earlier versions or those actively interested in helping to solve the bug. If you experience a bug, please report it. Changelog: http://www.rsyslog.com/Article125.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-57.phtml As always, feedback is appreciated. Rainer Gerhards From hagen at rz.uni-karlsruhe.de Fri Sep 7 19:20:12 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Fri, 07 Sep 2007 19:20:12 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> Message-ID: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > Hi all, > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > rare situations in which a segfault could occur. These have not > consistently been able to reproduce in lab. Release 1.19.5 may still The way I see it everyone reporting segfaults so far has been using RHEL5. So perhaps everybody seeing problems could comment on the operating-system? That might ease reproducing the problem in a test-lab. > have a problem or may hang where previous versions segfaulted. We are > actively looking for feedback from the field. Feature-wise, the $ModDir > config directive has been added and $ModLoad has been enhanced. We > recommend upgrading only for those that experience the problem with > earlier versions or those actively interested in helping to solve the > bug. If you experience a bug, please report it. Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be quite "normal", however, I've been lucky and captured some strace-information. Not sure wheter that could be helpful. poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "P*\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7in-a"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "P*\205\200\0\1\0\1\0\6\0\7\00282\003185\00213\003129 \7"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14623]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 165 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\236\222\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\236\222\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14099]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 243 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\377\251\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\377\251\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>spamd[16561]: spamd: identif"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 94 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\202\24\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\202\24\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>spamd[16561]: spamd: result:"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 463 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\33\\\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\33\\\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[15976]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 143 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "I\27\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7in"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "I\27\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \003129"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "v\325\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "v\325\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 futex(0x3627949960, FUTEX_WAKE, 1) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "+\330\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "+\330\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<13>greylistd: Socket error: Bro"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 41 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "+\366\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "+\366\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\233A\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\233A\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14536]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 171 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "M\243\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "M\243\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = ? ERESTARTNOHAND (To be restarted) trace: ptrace(PTRACE_SYSCALL, ...): No such process Process 22923 detached -- CU, Patrick. From infofarmer at FreeBSD.org Fri Sep 7 19:48:14 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Fri, 7 Sep 2007 21:48:14 +0400 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <20070907174813.GE56443@amilo.cenkes.org> On Fri, Sep 07, 2007 at 07:20:12PM +0200, Patrick von der Hagen wrote: > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > > Hi all, > > > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > > rare situations in which a segfault could occur. These have not > > consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. I experience segfaults with 1.19.1 or 1.19.2 on FreeBSD, but 1.19.3 has been up for a week on end now. From mic at npgx.com.au Sat Sep 8 00:20:37 2007 From: mic at npgx.com.au (Michael Mansour) Date: Sat, 8 Sep 2007 08:20:37 +1000 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: <1189165259.16157.114.camel@cutter> References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> <1189165259.16157.114.camel@cutter> Message-ID: <20070907221314.M7794@npgx.com.au> Hi, > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > > > I would deeply appreciate any feedback on the samples and format > > > suggestions. > > > > /me thinks you're getting way too little feedback on the blog, > > or this list. Unfortunately I don't have much more than simple > > preference to contribute here.. > > > > XML-based format: > > > > Yikes, you'll need an additional human readable frontend > > format that's converted to XML for it to be usable. You > > can't expect us poor sysadmins to be editing XML > > directly to configure rsyslogd.. > > The nice piece of this is that it is machine parseable easily which > enables lots of useful editors. I don't agree with the original posters comment there also. As an example, I have been using linuxha.net now for quite some years on many clusters and from day one, linuxha.net has used XML for all it's configuration files. I personally find the "standard" that brings to config files much better than the myriad of conf files I've dealt with the many more years I've been using UNIX and Linux. > > syslog-ng like: > > > > Fair enough.. It works for my usage. > > The syntax is okay but at that point what distinguishes b/t syslog-ng > and rsyslog? I've personally never been a fan of the syntax used in this. Sure I know it now after years for admin work, but I remember the times I needed to learn it thoroughly, it wasn't as easy as other conf files. > > Apache like: > > > > Not sure I understand this.. Seems like a mix of option/value > > and xml'ish for some functionality. > > This one I'm more interested in. If you think of each log like a > vhost and you define the qualities that are added to that inside the > definition > > > Destination /path/to/silly.log > DestinationMode 0640 > DestinationOwner root > DestinationGroup log-readers > Include mail.info kern.debug cron.emerg > > > etc, etc, etc > > maybe that doesn't make sense, maybe it does - it is pretty easy to > read, though. I think every sysadmin has setup a web server and delved into the apache-like configurations with software like apache, proftpd, etc. It's a nice and easy to understand format which has also proved the test of time. I'd be happy with either XML or Apache-like, but my bias is towards XML. Regards, Michael. > -sv > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From theinric at redhat.com Sat Sep 8 22:07:30 2007 From: theinric at redhat.com (Tomas Heinrich) Date: Sat, 08 Sep 2007 22:07:30 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <46E30102.20508@redhat.com> Patrick von der Hagen wrote: > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: >> Hi all, >> >> Rsyslog 1.19.5 has been released. It is primarily targeted at fixing >> rare situations in which a segfault could occur. These have not >> consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. > >> have a problem or may hang where previous versions segfaulted. We are >> actively looking for feedback from the field. Feature-wise, the $ModDir >> config directive has been added and $ModLoad has been enhanced. We >> recommend upgrading only for those that experience the problem with >> earlier versions or those actively interested in helping to solve the >> bug. If you experience a bug, please report it. > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. Thanks for the info. It will be very useful if someone can provide a core dump for 1.19.5. From rgerhards at hq.adiscon.com Mon Sep 10 11:34:17 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 11:34:17 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27896C@grfint2.intern.adiscon.com> Hhmmm... Besides a core-dump (which would definitely be useful), could those of you experiencing this problem try to run rsyslog with debug code enabled? This is NOT debug mode (-d). You enable it via ./configure --enable-debug This will generate additional debug checks in the rsyslog code. The resulting code will probably run 5 to 10 times SLOWER than production code. But it will catch many obscure errors and at least provide a hint to where the problem is (at least I hope so). If you could do that, that would be great. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Friday, September 07, 2007 7:20 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.5 released > > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > > Hi all, > > > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > > rare situations in which a segfault could occur. These have not > > consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. > > > have a problem or may hang where previous versions segfaulted. We are > > actively looking for feedback from the field. Feature-wise, the > $ModDir > > config directive has been added and $ModLoad has been enhanced. We > > recommend upgrading only for those that experience the problem with > > earlier versions or those actively interested in helping to solve the > > bug. If you experience a bug, please report it. > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. > > > > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "P*\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7in-a"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "P*\205\200\0\1\0\1\0\6\0\7\00282\003185\00213\003129 > \7"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14623]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 165 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\236\222\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\236\222\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14099]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 243 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\377\251\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\377\251\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>spamd[16561]: spamd: identif"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 94 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, > "\202\24\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\202\24\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>spamd[16561]: spamd: result:"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 463 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\33\\\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\33\\\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[15976]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 143 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "I\27\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7in"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "I\27\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \003129"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "v\325\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "v\325\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > futex(0x3627949960, FUTEX_WAKE, 1) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "+\330\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "+\330\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<13>greylistd: Socket error: Bro"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 41 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "+\366\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "+\366\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\233A\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\233A\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14536]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 171 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "M\243\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "M\243\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = ? ERESTARTNOHAND (To be > restarted) > trace: ptrace(PTRACE_SYSCALL, ...): No such process > Process 22923 detached > > -- > CU, > Patrick. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Mon Sep 10 14:38:58 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Mon, 10 Sep 2007 14:38:58 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Freitag, den 07.09.2007, 19:20 +0200 schrieb Patrick von der Hagen: [...] > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. I compiled 1.19.5 with "--enable-debug" now and got the following problems: Sep 10 14:24:26 mail11 rsyslogd:could not load module '/usr/local/lib/rsyslog/ommysql.so', dlopen: /usr/local/lib/rsyslog/ommysql.so: cannot open shared object file: No such file or directory Sep 10 14:24:26 mail11 rsyslogd:the last error occured in /etc/rsyslog.conf, line 29 No idea why it tries "/usr/local/lib", the lib has been installed to "/lib" or "/opt/rsyslog/lib/rsyslog/". Hmmm, "/lib" is from 1.18.something, I didn't tidy up properly. Anyway, I currently don't use logging to MySQL, so I uncommented "$ModLoad MySQL". Next try: [root at mail11 log]# /opt/rsyslog/sbin/rsyslogd -r514 -d [...] 1084229952: Lone worker is running... 1084229952: Called fprintlog, logging to builtin-file (/home/local/log/all) 1084229952: programname filter 'rsyslogd' does not match 'spamd' Filter: check for property 'msg' (value ' prefork: child states: BBBBBBBIIBBBBBBB ') contains 'prefork: child states': TRUE 1084229952: Called fprintlog, logging to builtin-discardrsyslogd: omdiscard.c:70: doAction: Assertion `ppString != ((void *)0)' failed. Aborted That's getting a little bit strange now, it has been caused by this rsyslog.conf-lines: !spamd :msg, contains, "prefork: child states" ~ Some other crashes relate to other lines, all of them end with "~". So I uncommented all of them. Those problems are strange, but if those issues were related to my "normal" rsyslog-crashes, rsyslog would not have been able to run up to two hours and would certainly have crashed almost instantly. It's been running for several minutes now... -- CU, Patrick. From rgerhards at hq.adiscon.com Mon Sep 10 14:50:12 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 14:50:12 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com><1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278979@grfint2.intern.adiscon.com> I agree, it looks strange (very strange indeed). I am checking if it could be related to the root cause (or just be a debug-level artifact that slipped through - that may be the case, because discard is the only action which does NOT have any strings at all). Anyhow, it provides me at least another clue where to look at. Thanks Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Monday, September 10, 2007 2:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.5 released > > Am Freitag, den 07.09.2007, 19:20 +0200 schrieb Patrick von der Hagen: > [...] > > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > > quite "normal", however, I've been lucky and captured some > > strace-information. Not sure wheter that could be helpful. > I compiled 1.19.5 with "--enable-debug" now and got the following > problems: > > Sep 10 14:24:26 mail11 rsyslogd:could not load module > '/usr/local/lib/rsyslog/ommysql.so', > dlopen: /usr/local/lib/rsyslog/ommysql.so: cannot open shared object > file: No such file or directory > Sep 10 14:24:26 mail11 rsyslogd:the last error occured > in /etc/rsyslog.conf, line 29 > > No idea why it tries "/usr/local/lib", the lib has been installed to > "/lib" or "/opt/rsyslog/lib/rsyslog/". Hmmm, "/lib" is from > 1.18.something, I didn't tidy up properly. > > Anyway, I currently don't use logging to MySQL, so I uncommented > "$ModLoad MySQL". > > > Next try: > [root at mail11 log]# /opt/rsyslog/sbin/rsyslogd -r514 -d > [...] > 1084229952: Lone worker is running... > 1084229952: Called fprintlog, logging to builtin-file > (/home/local/log/all) > 1084229952: programname filter 'rsyslogd' does not match 'spamd' > Filter: check for property 'msg' (value ' prefork: child states: > BBBBBBBIIBBBBBBB ') contains 'prefork: child states': TRUE > 1084229952: Called fprintlog, logging to builtin-discardrsyslogd: > omdiscard.c:70: doAction: Assertion `ppString != ((void *)0)' failed. > Aborted > > > That's getting a little bit strange now, it has been caused by this > rsyslog.conf-lines: > !spamd > :msg, contains, "prefork: child states" ~ > > > Some other crashes relate to other lines, all of them end with "~". So > I > uncommented all of them. > > Those problems are strange, but if those issues were related to my > "normal" rsyslog-crashes, rsyslog would not have been able to run up to > two hours and would certainly have crashed almost instantly. > > It's been running for several minutes now... > > -- > CU, > Patrick. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Mon Sep 10 15:17:49 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Mon, 10 Sep 2007 15:17:49 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Montag, den 10.09.2007, 14:38 +0200 schrieb Patrick von der Hagen: [...] > It's been running for several minutes now... Now I captured the "real" crash. First, my config: $AllowedSender UDP, 1.2.3.0/24 $AllowedSender TCP, 1.2.3.0/24 $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/clamav" $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/exim" $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" *.* /home/local/log/all !rsyslogd :programname, contains, "rsyslogd" /home/local/log/rsyslogd !spamd mail.* /home/local/log/mail !clamd local5.* ?clamavFile !exim *.* ?eximFile :msg, contains, "malware detected" ?avFile Here the output of "rsyslog -r514 -d" Successful select, descriptor count = 1, Activity on: 1084229952: -1431504256: 8 Called fprintlog, logging to builtin-file1084229952: (eximFile) Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 1IUio2-00054R-EA H=X (Y) [1.2.3.4] Warning: X-Spam-Status: yes, hits=13.9, size=32842') contains 'malware detected': FALSE 1084229952: singleWorker: queue EMPTY, waiting for next message. -1431504256: Message from inetd socket: #8, host: mailin3 -1431504256: Message length: 200, File descriptor: 8. -1431504256: logmsg: mail.info<22>, flags 2, from 'mailin3', msg exim[18550]: 2007-09-10 14:56:03 H=(a.b.c.d) [2.3.4.5] F= temporarily rejected RCPT : greylisted. -1431504256: Message has legacy syslog format. -1431504256: HOSTNAME contains invalid characters, assuming it to be a TAG. -1431504256: EnqueueMsg signaled condition (0) -1431504256: 1084229952: Listening on UDP syslogd socket 7 (IPv6/port 514). -1431504256: Lone worker is running... 1084229952: Called fprintlog, logging to builtin-file (/home/local/log/all) Listening on UDP syslogd socket 8 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 8): 3 7 8 1084229952: programname filter 'rsyslogd' does not match 'exim' 1084229952: programname filter 'spamd' does not match 'exim' 1084229952: programname filter 'clamd' does not match 'exim' 1084229952: Called fprintlog, logging to builtin-file (eximFile) Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 H=(domain) [1.2.3.4] F= temporarily rejected RCPT : greylisted.') contains 'malware detected': FALSE 1084229952: singleWorker: queue EMPTY, waiting for next message. -1431504256: Successful select, descriptor count = 1, Activity on: 8 *** glibc detected *** /opt/rsyslog/sbin/rsyslogd: corrupted double-linked list: 0x00002aaaac001230 *** ======= Backtrace: ========= /lib64/libc.so.6[0x362766cb43] /lib64/libc.so.6[0x362766eea2] /lib64/libc.so.6(__libc_malloc+0x7d)[0x36276706dd] /lib64/libc.so.6[0x362765eb4a] /lib64/libnss_files.so.2[0x2aaaaaad445a] /lib64/libnss_files.so.2(_nss_files_gethostbyaddr_r +0x57)[0x2aaaaaad4b47] /lib64/libc.so.6(gethostbyaddr_r+0xf2)[0x36276e2b42] /lib64/libc.so.6(getnameinfo+0x3ad)[0x36276eb07d] /opt/rsyslog/sbin/rsyslogd(cvthname+0x154)[0x410e64] /opt/rsyslog/sbin/rsyslogd[0x40bec9] /opt/rsyslog/sbin/rsyslogd(main+0x630)[0x40c990] /lib64/libc.so.6(__libc_start_main+0xf4)[0x362761d8a4] /opt/rsyslog/sbin/rsyslogd[0x405899] ======= Memory map: ======== 00400000-00424000 r-xp 00000000 08:03 688189 /opt/rsyslog/sbin/rsyslogd 00624000-00626000 rw-p 00024000 08:03 688189 /opt/rsyslog/sbin/rsyslogd 1c204000-1c249000 rw-p 1c204000 00:00 0 40000000-40001000 ---p 40000000 00:00 0 40001000-40a01000 rw-p 40001000 00:00 0 3627200000-362721a000 r-xp 00000000 08:03 4260124 /lib64/ld-2.5.so 3627419000-362741a000 r--p 00019000 08:03 4260124 /lib64/ld-2.5.so 362741a000-362741b000 rw-p 0001a000 08:03 4260124 /lib64/ld-2.5.so 3627600000-3627744000 r-xp 00000000 08:03 4260125 /lib64/libc-2.5.so 3627744000-3627944000 ---p 00144000 08:03 4260125 /lib64/libc-2.5.so 3627944000-3627948000 r--p 00144000 08:03 4260125 /lib64/libc-2.5.so 3627948000-3627949000 rw-p 00148000 08:03 4260125 /lib64/libc-2.5.so 3627949000-362794e000 rw-p 3627949000 00:00 0 3627e00000-3627e02000 r-xp 00000000 08:03 4260128 /lib64/libdl-2.5.so 3627e02000-3628002000 ---p 00002000 08:03 4260128 /lib64/libdl-2.5.so 3628002000-3628003000 r--p 00002000 08:03 4260128 /lib64/libdl-2.5.so 3628003000-3628004000 rw-p 00003000 08:03 4260128 /lib64/libdl-2.5.so 3628200000-3628215000 r-xp 00000000 08:03 4260022 /lib64/libpthread-2.5.so 3628215000-3628414000 ---p 00015000 08:03 4260022 /lib64/libpthread-2.5.so 3628414000-3628415000 r--p 00014000 08:03 4260022 /lib64/libpthread-2.5.so 3628415000-3628416000 rw-p 00015000 08:03 4260022 /lib64/libpthread-2.5.so 3628416000-362841a000 rw-p 3628416000 00:00 0 3628600000-3628614000 r-xp 00000000 08:03 4547586 /usr/lib64/libz.so.1.2.3 3628614000-3628813000 ---p 00014000 08:03 4547586 /usr/lib64/libz.so.1.2.3 3628813000-3628814000 rw-p 00013000 08:03 4547586 /usr/lib64/libz.so.1.2.3 362d200000-362d207000 r-xp 00000000 08:03 4260133 /lib64/librt-2.5.so 362d207000-362d407000 ---p 00007000 08:03 4260133 /lib64/librt-2.5.so 362d407000-362d408000 r--p 00007000 08:03 4260133 /lib64/librt-2.5.so 362d408000-362d409000 rw-p 00008000 08:03 4260133 /lib64/librt-2.5.so 362ee00000-362ee11000 r-xp 00000000 08:03 4260134 /lib64/libresolv-2.5.so 362ee11000-362f011000 ---p 00011000 08:03 4260134 /lib64/libresolv-2.5.so 362f011000-362f012000 r--p 00011000 08:03 4260134 /lib64/libresolv-2.5.so 362f012000-362f013000 rw-p 00012000 08:03 4260134 /lib64/libresolv-2.5.so 362f013000-362f015000 rw-p 362f013000 00:00 0 3815400000-381540d000 r-xp 00000000 08:03 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 381540d000-381560c000 ---p 0000d000 08:03 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 381560c000-381560d000 rw-p 0000c000 08:03 4259862 /libAborted -- CU, Patrick. From rgerhards at hq.adiscon.com Mon Sep 10 18:26:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 18:26:10 +0200 Subject: [rsyslog] rsyslog segfaults was: rsyslog 1.19.5 released In-Reply-To: <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189441571.2659.56.camel@localhost.localdomain> Patrick, thanks for this report, it is useful. Unfortunately, I had hoped that an assertion failed, which was obviously not the case. I have looked at the code in question, but the root cause seems to be different. It looks like the code that actually aborted was "just" affected by something that happened earlier (the earlier code destroyed the list, which was then detected in cvthname() processing). The assert on discard was a bug in the debug code. I have fixed that in the current CVS (I do not yet want to release a version because of this minor thing - if you like, you can pull it from anon CVS). We have some other things under review. In the mean time, I am asking myself if the problem may be related to a threading issue. Rsyslog can be compiled in single-threading mode. You can do that by: ./configure --disable-pthreads [--enable-debug (if you like)] There is a drawback, though: as message processing and reception is no longer de-coupled, messages in bursts are more likely to be lost. Also, TCP messages sent to an unresponsive server may be lost, as the priority is on reception (there is code that discards messages that block rsyslgogd). However, sysklogd always works in that mode, so I think it is worth giving a try. It would be very helpful to know if the problem persists in single threading mode - or not. Thanks, Rainer On Mon, 2007-09-10 at 15:17 +0200, Patrick von der Hagen wrote: > Am Montag, den 10.09.2007, 14:38 +0200 schrieb Patrick von der Hagen: > [...] > > It's been running for several minutes now... > Now I captured the "real" crash. > > First, my config: > $AllowedSender UDP, 1.2.3.0/24 > $AllowedSender TCP, 1.2.3.0/24 > $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME > %/clamav" > $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME > %/exim" > $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" > *.* /home/local/log/all > !rsyslogd > :programname, contains, "rsyslogd" /home/local/log/rsyslogd > !spamd > mail.* /home/local/log/mail > !clamd > local5.* ?clamavFile > !exim > *.* ?eximFile > :msg, contains, "malware detected" ?avFile > > > Here the output of "rsyslog -r514 -d" > Successful select, descriptor count = 1, Activity on: 1084229952: > -1431504256: 8 > Called fprintlog, logging to builtin-file1084229952: (eximFile) > Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 > 1IUio2-00054R-EA H=X (Y) [1.2.3.4] Warning: X-Spam-Status: yes, > hits=13.9, size=32842') contains 'malware detected': FALSE > 1084229952: singleWorker: queue EMPTY, waiting for next message. > -1431504256: Message from inetd socket: #8, host: mailin3 > -1431504256: Message length: 200, File descriptor: 8. > -1431504256: logmsg: mail.info<22>, flags 2, from 'mailin3', msg > exim[18550]: 2007-09-10 14:56:03 H=(a.b.c.d) [2.3.4.5] F= > temporarily rejected RCPT : greylisted. > -1431504256: Message has legacy syslog format. > -1431504256: HOSTNAME contains invalid characters, assuming it to be a > TAG. > -1431504256: EnqueueMsg signaled condition (0) > -1431504256: 1084229952: Listening on UDP syslogd socket 7 (IPv6/port > 514). > -1431504256: Lone worker is running... > 1084229952: Called fprintlog, logging to builtin-file > (/home/local/log/all) > Listening on UDP syslogd socket 8 (IPv4/port 514). > -1431504256: ---------------------------------------- > -1431504256: Calling select, active file descriptors (max 8): 3 7 8 > 1084229952: programname filter 'rsyslogd' does not match 'exim' > 1084229952: programname filter 'spamd' does not match 'exim' > 1084229952: programname filter 'clamd' does not match 'exim' > 1084229952: Called fprintlog, logging to builtin-file (eximFile) > Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 H=(domain) > [1.2.3.4] F= temporarily rejected RCPT > : greylisted.') contains 'malware detected': FALSE > 1084229952: singleWorker: queue EMPTY, waiting for next message. > -1431504256: > Successful select, descriptor count = 1, Activity on: 8 > *** glibc detected *** /opt/rsyslog/sbin/rsyslogd: corrupted > double-linked list: 0x00002aaaac001230 *** > ======= Backtrace: ========= > /lib64/libc.so.6[0x362766cb43] > /lib64/libc.so.6[0x362766eea2] > /lib64/libc.so.6(__libc_malloc+0x7d)[0x36276706dd] > /lib64/libc.so.6[0x362765eb4a] > /lib64/libnss_files.so.2[0x2aaaaaad445a] > /lib64/libnss_files.so.2(_nss_files_gethostbyaddr_r > +0x57)[0x2aaaaaad4b47] > /lib64/libc.so.6(gethostbyaddr_r+0xf2)[0x36276e2b42] > /lib64/libc.so.6(getnameinfo+0x3ad)[0x36276eb07d] > /opt/rsyslog/sbin/rsyslogd(cvthname+0x154)[0x410e64] > /opt/rsyslog/sbin/rsyslogd[0x40bec9] > /opt/rsyslog/sbin/rsyslogd(main+0x630)[0x40c990] > /lib64/libc.so.6(__libc_start_main+0xf4)[0x362761d8a4] > /opt/rsyslog/sbin/rsyslogd[0x405899] > ======= Memory map: ======== > 00400000-00424000 r-xp 00000000 08:03 > 688189 /opt/rsyslog/sbin/rsyslogd > 00624000-00626000 rw-p 00024000 08:03 > 688189 /opt/rsyslog/sbin/rsyslogd > 1c204000-1c249000 rw-p 1c204000 00:00 0 > 40000000-40001000 ---p 40000000 00:00 0 > 40001000-40a01000 rw-p 40001000 00:00 0 > 3627200000-362721a000 r-xp 00000000 08:03 > 4260124 /lib64/ld-2.5.so > 3627419000-362741a000 r--p 00019000 08:03 > 4260124 /lib64/ld-2.5.so > 362741a000-362741b000 rw-p 0001a000 08:03 > 4260124 /lib64/ld-2.5.so > 3627600000-3627744000 r-xp 00000000 08:03 > 4260125 /lib64/libc-2.5.so > 3627744000-3627944000 ---p 00144000 08:03 > 4260125 /lib64/libc-2.5.so > 3627944000-3627948000 r--p 00144000 08:03 > 4260125 /lib64/libc-2.5.so > 3627948000-3627949000 rw-p 00148000 08:03 > 4260125 /lib64/libc-2.5.so > 3627949000-362794e000 rw-p 3627949000 00:00 0 > 3627e00000-3627e02000 r-xp 00000000 08:03 > 4260128 /lib64/libdl-2.5.so > 3627e02000-3628002000 ---p 00002000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628002000-3628003000 r--p 00002000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628003000-3628004000 rw-p 00003000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628200000-3628215000 r-xp 00000000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628215000-3628414000 ---p 00015000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628414000-3628415000 r--p 00014000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628415000-3628416000 rw-p 00015000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628416000-362841a000 rw-p 3628416000 00:00 0 > 3628600000-3628614000 r-xp 00000000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 3628614000-3628813000 ---p 00014000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 3628813000-3628814000 rw-p 00013000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 362d200000-362d207000 r-xp 00000000 08:03 > 4260133 /lib64/librt-2.5.so > 362d207000-362d407000 ---p 00007000 08:03 > 4260133 /lib64/librt-2.5.so > 362d407000-362d408000 r--p 00007000 08:03 > 4260133 /lib64/librt-2.5.so > 362d408000-362d409000 rw-p 00008000 08:03 > 4260133 /lib64/librt-2.5.so > 362ee00000-362ee11000 r-xp 00000000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362ee11000-362f011000 ---p 00011000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f011000-362f012000 r--p 00011000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f012000-362f013000 rw-p 00012000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f013000-362f015000 rw-p 362f013000 00:00 0 > 3815400000-381540d000 r-xp 00000000 08:03 > 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 > 381540d000-381560c000 ---p 0000d000 08:03 > 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 > 381560c000-381560d000 rw-p 0000c000 08:03 > 4259862 /libAborted > > > From janfrode at tanso.net Sun Sep 9 11:43:58 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Sep 2007 11:43:58 +0200 Subject: [rsyslog] rsyslog 1.19.5 released References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <46E30102.20508@redhat.com> Message-ID: On 2007-09-08, Tomas Heinrich wrote: > > Thanks for the info. It will be very useful if someone can provide a > core dump for 1.19.5. I haven't had the chance to upgrade yet, but any hints for how to get a core dump if it fails ? I've tried this in the init-script: ulimit -c unlimited cd /var/log/syslog echo -n $"Starting system logger (rsyslog): " daemon rsyslogd $SYSLOGD_OPTIONS but haven't gotten any core-dumps in any of the crashes I've had.. -jf From hagen at rz.uni-karlsruhe.de Tue Sep 11 10:25:08 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 11 Sep 2007 10:25:08 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <46E30102.20508@redhat.com> Message-ID: <1189499108.3872.5.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Sonntag, den 09.09.2007, 11:43 +0200 schrieb Jan-Frode Myklebust: > On 2007-09-08, Tomas Heinrich wrote: > > > > Thanks for the info. It will be very useful if someone can provide a > > core dump for 1.19.5. > > I haven't had the chance to upgrade yet, but any hints for how to get > a core dump if it fails ? I've tried this in the init-script: > > ulimit -c unlimited > cd /var/log/syslog > echo -n $"Starting system logger (rsyslog): " > daemon rsyslogd $SYSLOGD_OPTIONS > > but haven't gotten any core-dumps in any of the crashes I've had.. Hmmm. I did "ulimit -c 50000; /opt/rsyslog/sbin/rsyslogd -r514" yesterday evening and found a nice coredump this morning. I sent it to Rainer a minute ago. -- CU, Patrick. From rgerhards at hq.adiscon.com Tue Sep 11 17:09:38 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:09:38 +0200 Subject: [rsyslog] rsyslog 1.19.6 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899A@grfint2.intern.adiscon.com> Release 1.19.6 is a code cleanup and bug fixing release. It provides a number of fixes, cleans up compiler warnings and has some doc improvements. We are still investigating a problem that causes rsyslog to segfault in some constellations and are looking for active feedback. Those that help in this effort are kindly requested to update to this release, as it contains the latest code. The upgrade is recommended only for people who experience problems or would like to help with bugfixing. From rgerhards at hq.adiscon.com Tue Sep 11 17:15:32 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:15:32 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Hi all, I accidently hit the send button, so my release note for 1.19.6 already went on to the list. Anyhow, I guess you can find the relevant links yourself ;) But one important thing: I would appreciate if those of you that experienced the segfaults could try out the new version. I currently think it will *NOT* fix the problem, but I would like to verify that (and I have to admit I am not angry if I am wrong...). If it fails, it would be even greater if you could try it in single-threaded mode: ./configure --disable-pthreads I currently think the problem might be related to a threading and would like to see if that theory has some substance. Also, a test with ./configure --enable-debug Would be useful - but I have not added any asserts so if you already did this, there is no value in re-doing it with 1.9.6. Thanks for all your help. Looking forward to hear back from you. Rainer From rgerhards at hq.adiscon.com Tue Sep 11 17:46:54 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:46:54 +0200 Subject: [rsyslog] 1.19.6 updated Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> Just for your info: I detected one bug that occurred during bug fixing. I fixed it and re-published 1.19.6 with the same version number because I saw now downloads until then. But maybe my counter is wrong. If you have downloaded rsyslog before I sent this message, please re-download it. Sorry for the hassle, but that fix was probably worth it. Thanks, Rainer From infofarmer at FreeBSD.org Tue Sep 11 18:04:40 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Tue, 11 Sep 2007 20:04:40 +0400 Subject: [rsyslog] 1.19.6 updated In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> Message-ID: <20070911160439.GB83726@amilo.cenkes.org> On Tue, Sep 11, 2007 at 05:46:54PM +0200, Rainer Gerhards wrote: > Just for your info: I detected one bug that occurred during bug fixing. > I fixed it and re-published 1.19.6 with the same version number because > I saw now downloads until then. But maybe my counter is wrong. If you > have downloaded rsyslog before I sent this message, please re-download > it. > > Sorry for the hassle, but that fix was probably worth it. I had to run 's/sigaction_t/sigaction/' over rfc3195d.c in order to compile it on FreeBSD. Otherwise it seems to work. Thanks! From rgerhards at hq.adiscon.com Tue Sep 11 18:40:12 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 18:40:12 +0200 Subject: [rsyslog] 1.19.6 updated In-Reply-To: <20070911160439.GB83726@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> <20070911160439.GB83726@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789A1@grfint2.intern.adiscon.com> Ahhh... I have to admit that I currently do not really care about rfc3195d. Currently, RFC 3195 is a failure - nobody uses it and nobody asks for it. I've stopped testing it until I receive at least one real-world implementation note. Thanks for the info, will apply a patch. And, yes, I do not think that 3195 is totally dead. There is a new revision planned, and that may be very interesting. This is why I am not ditching it... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Tuesday, September 11, 2007 6:05 PM > To: rsyslog-users > Subject: Re: [rsyslog] 1.19.6 updated > > On Tue, Sep 11, 2007 at 05:46:54PM +0200, Rainer Gerhards wrote: > > Just for your info: I detected one bug that occurred during bug > fixing. > > I fixed it and re-published 1.19.6 with the same version number > because > > I saw now downloads until then. But maybe my counter is wrong. If you > > have downloaded rsyslog before I sent this message, please re- > download > > it. > > > > Sorry for the hassle, but that fix was probably worth it. > > I had to run 's/sigaction_t/sigaction/' over rfc3195d.c in order > to compile it on FreeBSD. Otherwise it seems to work. Thanks! > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Thu Sep 13 12:31:50 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 12:31:50 +0200 Subject: [rsyslog] some logged lines end with #000 Message-ID: <46E91196.301@ipax.at> hello, today i merged michael biebls debian package source rsyslog_1.19.3-1 with the current upstream rsyslog-1.19.6.tar.gz. the compilation went smooth and i installed the package on a host where there used to by klogd/sysklogd under debian etch. however, i noticed that now a lot of logged lines end with #000. it used to be: > Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. Starting heartbeat 2.1.2 > Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 > Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: 193 > ... > Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 URL:http://localhost:80/server-status [1735/1735] -> "-" [1] it now is: > Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. Starting heartbeat 2.1.2#000 > Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version 2.1.2#000 > Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: 194#000 > ... > Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 URL:http://localhost:80/server-status [1728/1728] -> "-" [1] > Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 URL:http://localhost:80/server-status [1727/1727] -> "-" [1] you will find my configuration file attached. do you have any ideas what might cause this behaviour? cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: etc_default_rsyslog URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: rsyslog.conf URL: From theinric at redhat.com Thu Sep 13 15:01:09 2007 From: theinric at redhat.com (Tomas Heinrich) Date: Thu, 13 Sep 2007 15:01:09 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E91196.301@ipax.at> References: <46E91196.301@ipax.at> Message-ID: <46E93495.8010603@redhat.com> Raoul Bhatia [IPAX] wrote: > hello, > > today i merged michael biebls debian package source rsyslog_1.19.3-1 > with the current upstream rsyslog-1.19.6.tar.gz. > > the compilation went smooth and i installed the package on a host where > there used to by klogd/sysklogd under debian etch. > > however, i noticed that now a lot of logged lines end with #000. > > it used to be: >> Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. >> Starting heartbeat 2.1.2 >> Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 >> Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: 193 >> ... >> Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 >> URL:http://localhost:80/server-status [1735/1735] -> "-" [1] >> Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 >> URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > it now is: >> Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. >> Starting heartbeat 2.1.2#000 >> Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version 2.1.2#000 >> Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: >> 194#000 >> ... >> Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 >> URL:http://localhost:80/server-status [1728/1728] -> "-" [1] >> Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 >> URL:http://localhost:80/server-status [1727/1727] -> "-" [1] > > you will find my configuration file attached. > do you have any ideas what might cause this behaviour? > > cheers, > raoul bhatia > > > ------------------------------------------------------------------------ > > # Options to rsyslogd > # -m 0 disables 'MARK' messages. > # -r enables logging from remote machines > # -x disables DNS lookups on messages recieved with -r > # See rsyslogd(8) for more details > RSYSLOGD_OPTIONS="-m 0" > > # Options to rklogd > # -2 prints all kernel oops messages twice; once for klogd to decode, and > # once for processing with 'ksymoops' > # -x disables all klogd processing of oops messages entirely > # See rklogd(8) for more details > RKLOGD_OPTIONS="-x" > > > > ------------------------------------------------------------------------ > > # /etc/rsyslog.conf Configuration file for rsyslogd. > # > # For more information see > # /usr/share/doc/rsyslog/html/rsyslog_conf.html > > # > # First some standard logfiles. Log by facility. > # > > auth,authpriv.* /var/log/auth.log > *.*;auth,authpriv.none -/var/log/syslog > #cron.* /var/log/cron.log > daemon.* -/var/log/daemon.log > kern.* -/var/log/kern.log > lpr.* -/var/log/lpr.log > mail.* -/var/log/mail.log > user.* -/var/log/user.log > > # > # Logging for the mail system. Split it up so that > # it is easy to write scripts to parse these files. > # > mail.info -/var/log/mail.info > mail.warn -/var/log/mail.warn > mail.err /var/log/mail.err > > # > # Logging for INN news system > # > news.crit /var/log/news/news.crit > news.err /var/log/news/news.err > news.notice -/var/log/news/news.notice > > # > # Some `catch-all' logfiles. > # > *.=debug;\ > auth,authpriv.none;\ > news.none;mail.none -/var/log/debug > *.=info;*.=notice;*.=warn;\ > auth,authpriv.none;\ > cron,daemon.none;\ > mail,news.none -/var/log/messages > > # > # Emergencies are sent to everybody logged in. > # > *.emerg * > > # > # I like to have messages displayed on the console, but only on a virtual > # console I usually leave idle. > # > #daemon,mail.*;\ > # news.=crit;news.=err;news.=notice;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn /dev/tty8 > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, > # you must invoke `xconsole' with the `-file' option: > # > # $ xconsole -file /dev/xconsole [...] > # > # NOTE: adjust the list below, or you'll go crazy if you have a reasonably > # busy site.. > # > daemon.*;mail.*;\ > news.err;\ > *.=debug;*.=info;\ > *.=notice;*.=warn |/dev/xconsole > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog Hi, I've tinkered with this part of code recently, so it could be my fault. But I wasn't able to reproduce it so far and a quick look through the source didn't show anything suspicious. Just a blind guess: $EscapeControlCharactersOnReceive off It will prevent control chars from being escaped but maybe the problem will go away for now. You have a $IncludeConfig /etc/rsyslog.d/ in you conf file; is there any other part of configuration? I will look deeper into this later today. From rgerhards at hq.adiscon.com Thu Sep 13 15:48:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Sep 2007 15:48:02 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E91196.301@ipax.at> References: <46E91196.301@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> I remember the #000. I think in the case that I remember, there was actually a NUL character written by the remote system. It may also be that a too-long message (off by one) is written in the output code. Maybe this is even a clue to the bug we are hunting. You can remove them by turning off control character escaping - but that just a cosmetic fix, it doesn't fix the root issue, which we will need to look at. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Thursday, September 13, 2007 12:32 PM > To: rsyslog-users > Subject: [rsyslog] some logged lines end with #000 > > hello, > > today i merged michael biebls debian package source rsyslog_1.19.3-1 > with the current upstream rsyslog-1.19.6.tar.gz. > > the compilation went smooth and i installed the package on a host where > there used to by klogd/sysklogd under debian etch. > > however, i noticed that now a lot of logged lines end with #000. > > it used to be: > > Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. > Starting heartbeat 2.1.2 > > Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 > > Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: > 193 > > ... > > Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 > URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 > URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > it now is: > > Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. > Starting heartbeat 2.1.2#000 > > Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version > 2.1.2#000 > > Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: > 194#000 > > ... > > Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 > URL:http://localhost:80/server-status [1728/1728] -> "-" [1] > > Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 > URL:http://localhost:80/server-status [1727/1727] -> "-" [1] > > you will find my configuration file attached. > do you have any ideas what might cause this behaviour? > > cheers, > raoul bhatia > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ From r.bhatia at ipax.at Thu Sep 13 16:16:39 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 16:16:39 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> Message-ID: <46E94647.1090709@ipax.at> Rainer Gerhards wrote: > I remember the #000. I think in the case that I remember, there was > actually a NUL character written by the remote system. It may also be > that a too-long message (off by one) is written in the output code. > Maybe this is even a clue to the bug we are hunting. > > You can remove them by turning off control character escaping - but that > just a cosmetic fix, it doesn't fix the root issue, which we will need > to look at. so then please tell me how to help :) cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From r.bhatia at ipax.at Thu Sep 13 16:33:26 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 16:33:26 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E93495.8010603@redhat.com> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> Message-ID: <46E94A36.5050701@ipax.at> Tomas Heinrich wrote: > Hi, > > I've tinkered with this part of code recently, so it could be my fault. > > But I wasn't able to reproduce it so far and a quick look through the > source didn't show anything suspicious. > Just a blind guess: > $EscapeControlCharactersOnReceive off > It will prevent control chars from being escaped but maybe the problem > will go away for now. this does not seem to help. > You have a > $IncludeConfig /etc/rsyslog.d/ > in you conf file; is there any other part of configuration? no, there aren't any configuration files in there. see: > root at xxx /var/log # find /etc/rsyslog.d -type f > root at xxx /var/log # raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Thu Sep 13 17:31:01 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Sep 2007 17:31:01 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E94A36.5050701@ipax.at> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Thursday, September 13, 2007 4:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > Tomas Heinrich wrote: > > Hi, > > > > I've tinkered with this part of code recently, so it could be my > fault. > > > > But I wasn't able to reproduce it so far and a quick look through the > > source didn't show anything suspicious. > > Just a blind guess: > > $EscapeControlCharactersOnReceive off > > It will prevent control chars from being escaped but maybe the > problem > > will go away for now. > > this does not seem to help. Ummm... should have thought about this. NULs are treated special. This is routed in the sysklogd code. Rsyslog uses standard C strings for the most part. Standard C strings are terminated as soon as the first NUL is detected. So we can not allow NULs to be part of the message. Thus, NUL is always escaped, no matter what the control character escape sequences is. Can you provide us with a debug log (rsyslogd -d -n) showing a message with #000 being processed? Rainer > > > You have a > > $IncludeConfig /etc/rsyslog.d/ > > in you conf file; is there any other part of configuration? > > no, there aren't any configuration files in there. see: > > root at xxx /var/log # find /etc/rsyslog.d -type f > > root at xxx /var/log # > > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Fri Sep 14 10:06:33 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 14 Sep 2007 10:06:33 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> Message-ID: <46EA4109.3060009@ipax.at> Rainer Gerhards wrote: > Ummm... should have thought about this. NULs are treated special. This > is routed in the sysklogd code. Rsyslog uses standard C strings for the > most part. Standard C strings are terminated as soon as the first NUL is > detected. So we can not allow NULs to be part of the message. Thus, NUL > is always escaped, no matter what the control character escape sequences > is. > > Can you provide us with a debug log (rsyslogd -d -n) showing a message > with #000 being processed? attached, you will find the output. cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Sep 14 10:17:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Sep 2007 10:17:10 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46EA4109.3060009@ipax.at> References: <46E91196.301@ipax.at><46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at><577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> <46EA4109.3060009@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> The list server is quite picky on attachment types (aka "does not accept most of them" ;)). And I have to admit I introduced that policy and still think it is not a bad one. Can you send me the attachment via private mail please. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Friday, September 14, 2007 10:07 AM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > Rainer Gerhards wrote: > > Ummm... should have thought about this. NULs are treated special. > This > > is routed in the sysklogd code. Rsyslog uses standard C strings for > the > > most part. Standard C strings are terminated as soon as the first NUL > is > > detected. So we can not allow NULs to be part of the message. Thus, > NUL > > is always escaped, no matter what the control character escape > sequences > > is. > > > > Can you provide us with a debug log (rsyslogd -d -n) showing a > message > > with #000 being processed? > > attached, you will find the output. > > cheers, > raoul bhatia > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Sep 14 11:15:35 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Sep 2007 11:15:35 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at><46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at><577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com><46EA4109.3060009@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789E1@grfint2.intern.adiscon.com> I have created a new version which is currently available under the interim name of http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz [***This is NOT 1.19.7 but will become it over time!***] In that version, I have handled NULs that are emitted by faulty senders (and there seem to be lots of them). A quick test both by me and Raoul Bhatia indicates that those NULs disappear. And so do LF NUL (#012#000) as the LF shown is an artifact from the NUL being then the last char of the message). I would now be interested to know if somebody still sees unexpected #000 at the very end of the message. If there are, they now almost for sure identify a rsyslog problem. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, September 14, 2007 10:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > The list server is quite picky on attachment types (aka "does not > accept > most of them" ;)). And I have to admit I introduced that policy and > still think it is not a bad one. > > Can you send me the attachment via private mail please. > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > > Sent: Friday, September 14, 2007 10:07 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] some logged lines end with #000 > > > > Rainer Gerhards wrote: > > > Ummm... should have thought about this. NULs are treated special. > > This > > > is routed in the sysklogd code. Rsyslog uses standard C strings for > > the > > > most part. Standard C strings are terminated as soon as the first > NUL > > is > > > detected. So we can not allow NULs to be part of the message. Thus, > > NUL > > > is always escaped, no matter what the control character escape > > sequences > > > is. > > > > > > Can you provide us with a debug log (rsyslogd -d -n) showing a > > message > > > with #000 being processed? > > > > attached, you will find the output. > > > > cheers, > > raoul bhatia > > -- > > ____________________________________________________________________ > > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > > Technischer Leiter > > > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > > Barawitzkagasse 10/2/2/11 email. office at ipax.at > > 1190 Wien tel. +43 1 3670030 > > FN 277995t HG Wien fax. +43 1 3670030 15 > > ____________________________________________________________________ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 17 14:51:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 14:51:10 +0200 Subject: [rsyslog] rsyslog segfaults? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> Hi all, may I ask if those of you that experienced the segfault problem could re-produce it with 1.19.6? Any feedback would be deeply appreciated. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Sep 17 18:21:09 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 18:21:09 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <1190044055.10861.7.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <1190044055.10861.7.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A19@grfint2.intern.adiscon.com> Patrick, without answering the question directly: could you please try out http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz [**still not "real" 1.19.7 **] I've possibly seen one thing that could be the problem cause. Not sure, though. I'd deeply appreciate feedback if this version (uploaded right NOW) causes the problem to disappear. I am not sure if I have found it, but I'd like to conduct a quick test before going any further. Thus I've no detailed analysis. Thanks, Rainer > -----Original Message----- > From: Patrick von der Hagen [mailto:hagen at rz.uni-karlsruhe.de] > Sent: Monday, September 17, 2007 5:48 PM > To: Rainer Gerhards > Cc: theinric > Subject: Re: [rsyslog] rsyslog segfaults? > > On Mon, 2007-09-17 at 14:51 +0200, Rainer Gerhards wrote: > > Hi all, > > > > may I ask if those of you that experienced the segfault problem could > > re-produce it with 1.19.6? Any feedback would be deeply appreciated. > Still dumps core, haven't tried it single-threaded yet. > > I attach some gdb information I got when examining a core-dump. > > I'm no programmer and don't even understand half of it, but I'm > troubled > by ' ip = "129.13.185.81\000\000 > $a196a97a at akstcacsdatarecoverymnsdgs>,bayes=1.000000,autolearn=spam > \00090$88ab8c55 at 082d6f5a>,bayes=1.000000,autolearn=spam \000n=spam > \000id=<01c7f6df$57aeec90$6a49cd18 at 0aconingham>,bayes=1.000"'. > > It looks like several log-lines merged into one, which might hint at > threading-issues. > > > > [root at mail11 ~]# cd /opt/rsyslog > [root at mail11 rsyslog]# ls > core.14004 lib sbin share > [root at mail11 rsyslog]# gdb -c core.14004 /opt/rsyslog/sbin/rsyslogd > GNU gdb Red Hat Linux (6.5-16.el5rh) > Copyright (C) 2006 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "x86_64-redhat-linux-gnu"...Using host > libthread_db library "/lib64/libthread_db.so.1". > > Reading symbols from /usr/lib64/libz.so.1...done. > Loaded symbols for /usr/lib64/libz.so.1 > Reading symbols from /lib64/libpthread.so.0...done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /lib64/libdl.so.2...done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/librt.so.1...done. > Loaded symbols for /lib64/librt.so.1 > Reading symbols from /lib64/libc.so.6...done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /lib64/ld-linux-x86-64.so.2...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libnss_files.so.2...done. > Loaded symbols for /lib64/libnss_files.so.2 > Reading symbols from /lib64/libnss_dns.so.2...done. > Loaded symbols for /lib64/libnss_dns.so.2 > Reading symbols from /lib64/libresolv.so.2...done. > Loaded symbols for /lib64/libresolv.so.2 > Reading symbols from /lib64/libgcc_s.so.1...done. > Loaded symbols for /lib64/libgcc_s.so.1 > Core was generated by `/opt/rsyslog/sbin/rsyslogd -r514'. > Program terminated with signal 6, Aborted. > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > (gdb) backtrace > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > #1 0x0000003627631980 in abort () from /lib64/libc.so.6 > #2 0x00000036276674db in __libc_message () from /lib64/libc.so.6 > #3 0x000000362766cb43 in malloc_consolidate () from /lib64/libc.so.6 > #4 0x000000362766eea2 in _int_malloc () from /lib64/libc.so.6 > #5 0x00000036276706dd in malloc () from /lib64/libc.so.6 > #6 0x000000362765eb4a in __fopen_internal () from /lib64/libc.so.6 > #7 0x00002aaaaaaf545a in internal_setent () > from /lib64/libnss_files.so.2 > #8 0x00002aaaaaaf5b47 in _nss_files_gethostbyaddr_r () > from /lib64/libnss_files.so.2 > #9 0x00000036276e2b42 in gethostbyaddr_r@@GLIBC_2.2.5 () > from /lib64/libc.so.6 > #10 0x00000036276eb07d in getnameinfo () from /lib64/libc.so.6 > #11 0x00000000004155ee in cvthname (f=0x7ffff040b0d0, > pszHost=0x7ffff040acc0 "mailin3", > pszHostFQDN=0x7ffff040a8b0 "mailin3.rz.uni-karlsruhe.de") at > net.c:137 > #12 0x000000000040e49d in processSelectAfter (maxfds=5, nfds=1, > pReadfds=0x7ffff040ba60, pWritefds=0x7ffff040b9c0) at > syslogd.c:5619 > #13 0x000000000040ee02 in mainloop () at syslogd.c:5869 > #14 0x000000000040fc9c in main (argc=0, argv=0x7ffff040bd08) at > syslogd.c:6315 > (gdb) where full > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > No symbol table info available. > #1 0x0000003627631980 in abort () from /lib64/libc.so.6 > No symbol table info available. > #2 0x00000036276674db in __libc_message () from /lib64/libc.so.6 > No symbol table info available. > #3 0x000000362766cb43 in malloc_consolidate () from /lib64/libc.so.6 > No symbol table info available. > #4 0x000000362766eea2 in _int_malloc () from /lib64/libc.so.6 > No symbol table info available. > #5 0x00000036276706dd in malloc () from /lib64/libc.so.6 > No symbol table info available. > #6 0x000000362765eb4a in __fopen_internal () from /lib64/libc.so.6 > No symbol table info available. > #7 0x00002aaaaaaf545a in internal_setent () > from /lib64/libnss_files.so.2 > No symbol table info available. > #8 0x00002aaaaaaf5b47 in _nss_files_gethostbyaddr_r () > from /lib64/libnss_files.so.2 > No symbol table info available. > #9 0x00000036276e2b42 in gethostbyaddr_r@@GLIBC_2.2.5 () > from /lib64/libc.so.6 > No symbol table info available. > #10 0x00000036276eb07d in getnameinfo () from /lib64/libc.so.6 > No symbol table info available. > ---Type to continue, or q to quit--- > #11 0x00000000004155ee in cvthname (f=0x7ffff040b0d0, > pszHost=0x7ffff040acc0 "mailin3", > pszHostFQDN=0x7ffff040a8b0 "mailin3.rz.uni-karlsruhe.de") at > net.c:137 > p = (uchar *) 0x7ffff040acc7 "" > count = -264195888 > error = 0 > omask = {__val = {0, 4251742, 0, 140737224155248, > 140737224155240, > 14083552, 0, 18446735427676189008, 140737224155264, 4336648, > 140737224155264, 232601472906, 0, 64, 140737224159584, 2047}} > nmask = {__val = {1, 0 }} > ip = "129.13.185.81\000\000 > $a196a97a at akstcacsdatarecoverymnsdgs>,bayes=1.000000,autolearn=spam > \00090$88ab8c55 at 082d6f5a>,bayes=1.000000,autolearn=spam \000n=spam > \000id=<01c7f6df$57aeec90$6a49cd18 at 0aconingham>,bayes=1.000"... > hints = {ai_flags = 4, ai_family = 0, ai_socktype = 2, > ai_protocol = 0, ai_addrlen = 0, ai_addr = 0x0, ai_canonname = 0x0, > ai_next = 0x0} > res = (struct addrinfo *) 0x2e302e3732313d72 > __PRETTY_FUNCTION__ = "cvthname" > #12 0x000000000040e49d in processSelectAfter (maxfds=5, nfds=1, > pReadfds=0x7ffff040ba60, pWritefds=0x7ffff040b9c0) at > syslogd.c:5619 > iRet = RS_RET_OK > iRetLL = RS_RET_OK > i = 1 > ---Type to continue, or q to quit--- > fd = 0 > line = "<22>exim[32680]: 2007-09-14 17:02:19 1IWCgS-0008Tb-B5 > Completed\n", '\0' > writeFDSInfo = {pWritefds = 0x7ffff040b9c0, pMaxfds = > 0x7ffff040a0a8} > f = (selector_t *) 0x0 > frominet = {ss_family = 2, __ss_align = 0, > __ss_padding = "@\023\000??*\000\000@\023\000??*\000\000@\023\000??* > \000\000B\023\000??*\000\000O\023\000??*\000\000@\023\000??*\000\000O > \023\000??*", '\0' , "\017\201B\000\000\000\000\000? > \230\aF\000\000\000"} > socklen = 16 > fromHost = "mailin3\000rz.uni-karlsruhe.de", '\0' times>, "\210_ '6", '\0' , "?? '6", '\0' times>, "??A'6\000\000\000o\b?'6\000\000\000\220{ '6\000\000\000\000 > \000`'6\000\000\000\000 at t'6\000\000\000\220:t'6\000\000\000\220:t'6", > '\0' , "\005\000\000\000\000\000\000\000\000@\224'6", > '\0' , "\001\000\000\000\000\000\000\000\230?\224'6 > \000\000\000\000@\024\000\000\000\000\000\003\000\000\000\000\000\000 > \000\000\000 -6\000\000\000\000p -6\000\000"... > fromHostFQDN = "mailin3.rz.uni-karlsruhe.de", '\0' times>, "\001\000\000\000\000\000\000\000?@??\177\000\000?*q'6", '\0' > , "??e'6", '\0' , "??@??\177\000 > \000/\201B\000\000\000\000\000/\201B", '\0' , > "?4d'6", > '\0' , "\200?@??\177\000\000\000\000\000\000\000\000 > \000\000??@??\177\000\0000?@??\177\000\000*\201B\000\000\000\000\000?@? > ? > \177", '\0' , "????????\000\000\000---Type > to > continue, or q to quit--- > \000\002\000\000\000*"... > iTCPSess = 0 > l = 64 > #13 0x000000000040ee02 in mainloop () at syslogd.c:5869 > readfds = {fds_bits = {32, 0 }} > i = 2 > maxfds = 5 > nfds = 1 > writeFDSInfo = {pWritefds = 0x7ffff040b9c0, pMaxfds = > 0x7ffff040ba5c} > writefds = {fds_bits = {0 }} > f = (selector_t *) 0x0 > iTCPSess = 14003 > #14 0x000000000040fc9c in main (argc=0, argv=0x7ffff040bd08) at > syslogd.c:6315 > i = 1024 > p = 0x63053a "" > num_fds = 1024 > iRet = RS_RET_OK > ppid = 14003 > ch = -1 > hent = (struct hostent *) 0x362794bf80 > pTmp = (uchar *) 0x62fe95 "" > sigAct = {__sigaction_handler = {sa_handler = 0x1, > sa_sigaction = 0x1}, sa_mask = {__val = {0 }}, > ---Type to continue, or q to quit--- > sa_flags = 0, sa_restorer = 0} > (gdb) > > > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- > Patrick von der Hagen > RZ Universit?t Karlsruhe (TH) > Postmaster From rgerhards at hq.adiscon.com Mon Sep 17 18:27:38 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 18:27:38 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <1190046420.10861.9.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <1190044055.10861.7.camel@localhost.localdomain> <577465F99B41C842AAFBE9ED71E70ABA278A19@grfint2.intern.adiscon.com> <1190046420.10861.9.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A1B@grfint2.intern.adiscon.com> Excellent, thx > -----Original Message----- > From: Patrick von der Hagen [mailto:hagen at rz.uni-karlsruhe.de] > Sent: Monday, September 17, 2007 6:27 PM > To: Rainer Gerhards > Cc: rsyslog-users > Subject: RE: [rsyslog] rsyslog segfaults? > > On Mon, 2007-09-17 at 18:21 +0200, Rainer Gerhards wrote: > > Patrick, > > > > without answering the question directly: could you please try out > > > > http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz > > > > [**still not "real" 1.19.7 **] > > > > I've possibly seen one thing that could be the problem cause. Not > sure, though. I'd deeply appreciate feedback if this version (uploaded > right NOW) causes the problem to disappear. I am not sure if I have > found it, but I'd like to conduct a quick test before going any > further. Thus I've no detailed analysis. > I'm currently giving it a try, but don't expect any results today. > -- > Patrick von der Hagen > RZ Universit?t Karlsruhe (TH) > Postmaster From infofarmer at FreeBSD.org Wed Sep 19 10:55:19 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Wed, 19 Sep 2007 12:55:19 +0400 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> Message-ID: <20070919085517.GL98847@amilo.cenkes.org> On Mon, Sep 17, 2007 at 02:51:10PM +0200, Rainer Gerhards wrote: > Hi all, > > may I ask if those of you that experienced the segfault problem could > re-produce it with 1.19.6? Any feedback would be deeply appreciated. The box is in production, so I can't do too much debugging. If it may help, I can recompile rsyslogd with debug flags set and run gdb against it. root at apollo# rsyslogd -v /usr/home/mob/www rsyslogd 1.19.6, compiled with: FEATURE_PTHREADS (dual-threading): Yes FEATURE_REGEXP: Yes FEATURE_DB (MySQL): Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes SYSLOG_INET (Internet/remote support): Yes sat at apollo% gdb =rsyslogd rsyslogd.core <...> #0 0x000000080078189c in pthread_testcancel () from /lib/libpthread.so.2 [New Thread 0x537000 (LWP 100117)] [New Thread 0x52e400 (LWP 100112)] [New Thread 0x52e000 (runnable)] (gdb) bt #0 0x000000080078189c in pthread_testcancel () from /lib/libpthread.so.2 #1 0x000000080076f5c3 in sigaction () from /lib/libpthread.so.2 #2 0x00000008007710e2 in sigaction () from /lib/libpthread.so.2 #3 0x000000080076adb6 in pthread_kill () from /lib/libpthread.so.2 #4 0x000000080076a633 in raise () from /lib/libpthread.so.2 #5 0x000000080095b16d in abort () from /lib/libc.so.6 #6 0x00000008008f4b45 in _UTF8_init () from /lib/libc.so.6 #7 0x00000008008f4b7c in _UTF8_init () from /lib/libc.so.6 #8 0x00000008008f5b1d in _UTF8_init () from /lib/libc.so.6 #9 0x000000000040fa1d in MsgDestruct () #10 0x00000000004066bd in dbgprintf () #11 0x00000000004176b1 in llExecFunc () #12 0x0000000000406f41 in shouldProcessThisMessage () #13 0x0000000000409a9e in logerrorSz () #14 0x0000000800772a99 in pthread_create () from /lib/libpthread.so.2 #15 0x00000008008cfcd4 in makecontext () from /lib/libc.so.6 #16 0x0000000000000000 in ?? () #17 0x0000000000537000 in ?? () #18 0x00000000004099c0 in logerrorSz () #19 0x0000000000000000 in ?? () #20 0x0000000000000000 in ?? () #21 0x0000000000000000 in ?? () #22 0x0000000000000000 in ?? () Cannot access memory at address 0x7fffff5fc000 From rgerhards at hq.adiscon.com Wed Sep 19 13:53:41 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 19 Sep 2007 13:53:41 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <20070919085517.GL98847@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <20070919085517.GL98847@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A3D@grfint2.intern.adiscon.com> Andrew, this is quite helpful. While it does not pinpoint the actual trouble source, it points to a different abort location. All previous segfaults pointed to a different location (Which I have now checked multiple times and not found any notable problems, just cosmetic things). This strengthens the point it may be related to threading - of course, problematic to troubleshoot. I'll see how I proceed from here. Again, if some of you that experience the problem could run rsyslogd compiled for single threading, that would be most helpful. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Wednesday, September 19, 2007 10:55 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog segfaults? > > On Mon, Sep 17, 2007 at 02:51:10PM +0200, Rainer Gerhards wrote: > > Hi all, > > > > may I ask if those of you that experienced the segfault problem could > > re-produce it with 1.19.6? Any feedback would be deeply appreciated. > > The box is in production, so I can't do too much debugging. If it > may help, I can recompile rsyslogd with debug flags set and run > gdb against it. > > root at apollo# rsyslogd -v > /usr/home/mob/www > rsyslogd 1.19.6, compiled with: > FEATURE_PTHREADS (dual-threading): Yes > FEATURE_REGEXP: Yes > FEATURE_DB (MySQL): Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > SYSLOG_INET (Internet/remote support): Yes > > sat at apollo% gdb =rsyslogd rsyslogd.core > <...> > #0 0x000000080078189c in pthread_testcancel () from > /lib/libpthread.so.2 > [New Thread 0x537000 (LWP 100117)] > [New Thread 0x52e400 (LWP 100112)] > [New Thread 0x52e000 (runnable)] > (gdb) bt > #0 0x000000080078189c in pthread_testcancel () from > /lib/libpthread.so.2 > #1 0x000000080076f5c3 in sigaction () from /lib/libpthread.so.2 > #2 0x00000008007710e2 in sigaction () from /lib/libpthread.so.2 > #3 0x000000080076adb6 in pthread_kill () from > /lib/libpthread.so.2 > #4 0x000000080076a633 in raise () from /lib/libpthread.so.2 > #5 0x000000080095b16d in abort () from /lib/libc.so.6 > #6 0x00000008008f4b45 in _UTF8_init () from /lib/libc.so.6 > #7 0x00000008008f4b7c in _UTF8_init () from /lib/libc.so.6 > #8 0x00000008008f5b1d in _UTF8_init () from /lib/libc.so.6 > #9 0x000000000040fa1d in MsgDestruct () > #10 0x00000000004066bd in dbgprintf () > #11 0x00000000004176b1 in llExecFunc () > #12 0x0000000000406f41 in shouldProcessThisMessage () > #13 0x0000000000409a9e in logerrorSz () > #14 0x0000000800772a99 in pthread_create () from > /lib/libpthread.so.2 > #15 0x00000008008cfcd4 in makecontext () from /lib/libc.so.6 > #16 0x0000000000000000 in ?? () > #17 0x0000000000537000 in ?? () > #18 0x00000000004099c0 in logerrorSz () > #19 0x0000000000000000 in ?? () > #20 0x0000000000000000 in ?? () > #21 0x0000000000000000 in ?? () > #22 0x0000000000000000 in ?? () > Cannot access memory at address 0x7fffff5fc000 > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 21 14:07:55 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 21 Sep 2007 14:07:55 +0200 Subject: [rsyslog] rsyslog segfault Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A67@grfint2.intern.adiscon.com> To those of you that experience the problem: I just got note that the problem possibly disappears in single-threading mode. We are looking at this. In the mean time, you may want to use ./configure --disable-pthreads I just thought I pass this around. Rainer From janfrode at tanso.net Sun Sep 23 11:27:53 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 23 Sep 2007 11:27:53 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Message-ID: On 2007-09-11, Rainer Gerhards wrote: > > But one important thing: I would appreciate if those of you that > experienced the segfaults could try out the new version. I currently > think it will *NOT* fix the problem, but I would like to verify that > (and I have to admit I am not angry if I am wrong...). Sorry, you were right: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xb7214c98 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x46)[0x80580b6] rsyslogd[0x804df9a] rsyslogd(llExecFunc+0x3f)[0x805f11f] rsyslogd[0x804d9ea] rsyslogd[0x804db3f] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] With 1.19.6. > If it fails, it would be even greater if you could try it in > single-threaded mode: > > ./configure --disable-pthreads Is this safe to use on a busy system? I wouldn't want to start dropping lots of packets.. -jf From rgerhards at hq.adiscon.com Mon Sep 24 11:16:42 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 11:16:42 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> Thanks for the report. I'd not yet use the single-threading version. The performance hit should not be severe, but I think I'll be able to release another version today. Not sure, of course, if it actually fixes the bug, but I am working on it. I got a report that it is indeed related to a threading issue and I am now checking all runtime functions that get called. At least one I have found to be not thread-safe. I intend to release a version with that fix and a couple of minor things, even if I have not yet completed the full runtime-checks. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Sunday, September 23, 2007 11:28 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > On 2007-09-11, Rainer Gerhards wrote: > > > > But one important thing: I would appreciate if those of you that > > experienced the segfaults could try out the new version. I currently > > think it will *NOT* fix the problem, but I would like to verify that > > (and I have to admit I am not angry if I am wrong...). > > Sorry, you were right: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xb7214c98 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x46)[0x80580b6] > rsyslogd[0x804df9a] > rsyslogd(llExecFunc+0x3f)[0x805f11f] > rsyslogd[0x804d9ea] > rsyslogd[0x804db3f] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > With 1.19.6. > > > If it fails, it would be even greater if you could try it in > > single-threaded mode: > > > > ./configure --disable-pthreads > > Is this safe to use on a busy system? I wouldn't want to start dropping > lots of packets.. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 24 11:25:44 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 11:25:44 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A87@grfint2.intern.adiscon.com> Ummm... Importnat word mising. I'd not use the single-threading version NOW. The reasoning was in the mail. I may ask some time later for it, though. Sorry... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, September 24, 2007 11:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > Thanks for the report. I'd not yet use the single-threading version. > The > performance hit should not be severe, but I think I'll be able to > release another version today. Not sure, of course, if it actually > fixes > the bug, but I am working on it. I got a report that it is indeed > related to a threading issue and I am now checking all runtime > functions > that get called. At least one I have found to be not thread-safe. I > intend to release a version with that fix and a couple of minor things, > even if I have not yet completed the full runtime-checks. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > > Sent: Sunday, September 23, 2007 11:28 AM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > > > On 2007-09-11, Rainer Gerhards wrote: > > > > > > But one important thing: I would appreciate if those of you that > > > experienced the segfaults could try out the new version. I > currently > > > think it will *NOT* fix the problem, but I would like to verify > that > > > (and I have to admit I am not angry if I am wrong...). > > > > Sorry, you were right: > > > > *** glibc detected *** rsyslogd: corrupted double-linked list: > > 0xb7214c98 *** > > ======= Backtrace: ========= > > /lib/libc.so.6[0x4152ce3e] > > /lib/libc.so.6(cfree+0x90)[0x415305d0] > > rsyslogd(MsgDestruct+0x46)[0x80580b6] > > rsyslogd[0x804df9a] > > rsyslogd(llExecFunc+0x3f)[0x805f11f] > > rsyslogd[0x804d9ea] > > rsyslogd[0x804db3f] > > /lib/libpthread.so.0[0x416112db] > > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > > > With 1.19.6. > > > > > If it fails, it would be even greater if you could try it in > > > single-threaded mode: > > > > > > ./configure --disable-pthreads > > > > Is this safe to use on a busy system? I wouldn't want to start > dropping > > lots of packets.. > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 24 16:30:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 16:30:48 +0200 Subject: [rsyslog] new unofficial 1.19.7 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> Hi all, I have posted a new "unofficial" 1.19.7 to the download server: http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz I intend to post it publically tomorrow. But some nits need to be done first, mostly doc issues. I'd appreciate if those with the segfault problem could give it a try (in multi-threading mode) and tell me the outcome. Thanks, Rainer From rgerhards at hq.adiscon.com Tue Sep 25 11:46:14 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 11:46:14 +0200 Subject: [rsyslog] rsyslog 1.19.7 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Hi all, Rsyslog 1.19.7 (the official one) has been released today. It contains some code updated compared to the version from yesterday, so please download it if you already did. Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. Its primary goal is enhanced stability. Feature-wise, it offers support of dropping NUL characters, which are often found at the end of invalidly formatted syslog messages. Upgrading to this release is recommended for all users. Changelog: http://www.rsyslog.com/Article129.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml PLEASE NOTE: the rsyslog site is currently migrated to a new machine. In case you receive a temporary page, your DNS still caches the old machine's address. As always, feedback is appreciated. Feedback is especially appreciated from those that have experienced the segfault problem in the past. Rainer Gerhards From infofarmer at FreeBSD.org Tue Sep 25 12:42:20 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Tue, 25 Sep 2007 14:42:20 +0400 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <20070925104218.GI49755@amilo.cenkes.org> On Tue, Sep 25, 2007 at 11:46:14AM +0200, Rainer Gerhards wrote: > Hi all, > > Rsyslog 1.19.7 (the official one) has been released today. It contains > some code updated compared to the version from yesterday, so please > download it if you already did. > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > Its primary goal is enhanced stability. Feature-wise, it offers support > of dropping NUL characters, which are often found at the end of > invalidly formatted syslog messages. Upgrading to this release is > recommended for all users. You said you don't support rfc3195d at the moment, but anyway: rfc3195d.c:108: error: 'errStr' undeclared (first use in this function) From rgerhards at hq.adiscon.com Tue Sep 25 12:44:33 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 12:44:33 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <20070925104218.GI49755@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <20070925104218.GI49755@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9D@grfint2.intern.adiscon.com> Oops.. looks like I need to check my build procedure, obviously it wasn't build over here... Thanks for the report, it's fixed now. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Tuesday, September 25, 2007 12:42 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > On Tue, Sep 25, 2007 at 11:46:14AM +0200, Rainer Gerhards wrote: > > Hi all, > > > > Rsyslog 1.19.7 (the official one) has been released today. It > contains > > some code updated compared to the version from yesterday, so please > > download it if you already did. > > > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > > Its primary goal is enhanced stability. Feature-wise, it offers > support > > of dropping NUL characters, which are often found at the end of > > invalidly formatted syslog messages. Upgrading to this release is > > recommended for all users. > > You said you don't support rfc3195d at the moment, but anyway: > > rfc3195d.c:108: error: 'errStr' undeclared (first use in this > function) > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Tue Sep 25 13:55:18 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 25 Sep 2007 13:55:18 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <1190721318.3849.13.camel@localhost.localdomain> Am Dienstag, den 25.09.2007, 11:46 +0200 schrieb Rainer Gerhards: [...] > As always, feedback is appreciated. Feedback is especially appreciated > from those that have experienced the segfault problem in the past. Hi all, I'd like to apologize, since it was me who reported to Rainer that the segfault problem seems to have gone with the new release and that everything seems to be better now before he released 1.19.7. Meanwhile I realised that while the rsyslog-Daemon indeed ran without any segfaults, however, it didn't log anything sent to it via UDP. So my segfault problem has been replaced by a new one. ;-) So everybody should be very careful with this update, especially those who previously encountered the segfault problem. -- CU, Patrick. From rgerhards at hq.adiscon.com Tue Sep 25 13:57:46 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 13:57:46 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <1190721318.3849.13.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <1190721318.3849.13.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> Patrick, you mean nothing was logged via UDP while local domain sockets and/or TCP still worked? I have to admit I am a bit puzzled... Do you have a debug log at hand? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Tuesday, September 25, 2007 1:55 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > Am Dienstag, den 25.09.2007, 11:46 +0200 schrieb Rainer Gerhards: > [...] > > As always, feedback is appreciated. Feedback is especially > appreciated > > from those that have experienced the segfault problem in the past. > Hi all, > > I'd like to apologize, since it was me who reported to Rainer that the > segfault problem seems to have gone with the new release and that > everything > seems to be better now before he released 1.19.7. > > Meanwhile I realised that while the rsyslog-Daemon indeed ran without > any > segfaults, however, it didn't log anything sent to it via UDP. So my > segfault > problem has been replaced by a new one. ;-) So everybody should be very > careful with this update, especially those who previously encountered > the > segfault problem. > > -- > CU, > Patrick. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Tue Sep 25 15:39:14 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 25 Sep 2007 15:39:14 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <1190721318.3849.13.camel@localhost.localdomain> <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> Message-ID: <1190727554.3043.14.camel@localhost.localdomain> Am Dienstag, den 25.09.2007, 13:57 +0200 schrieb Rainer Gerhards: > Patrick, > > you mean nothing was logged via UDP while local domain sockets and/or > TCP still worked? I have to admit I am a bit puzzled... I only use rsyslog for UDP-logging, so I can't comment on TCP or local domain sockets. I usually run rsyslogd with "-r514" or "-4 -r514". The only log-entries relate to startup or shutdown of rsyslog itself. > Do you have a debug log at hand? Let's start with compiler warnings. They are identical to 1.19.6, so probably not related to the changed behaviour. syslog.c: In function ?writeSyslogV?: syslog.c:98: warning: function might be possible candidate for ?printf? format attribute syslog.c:98: warning: function might be possible candidate for ?printf? format attribute template.c: In function ?tplPrintList?: template.c:927: warning: cast from pointer to integer of different size modules.c: In function ?modPrintList?: modules.c:309: warning: cast from pointer to integer of different size modules.c:310: warning: cast from pointer to integer of different size modules.c:311: warning: cast from pointer to integer of different size modules.c:312: warning: cast from pointer to integer of different size modules.c:313: warning: cast from pointer to integer of different size cfsysline.c: In function ?dbgPrintCfSysLineHandlers?: cfsysline.c:710: warning: cast from pointer to integer of different size cfsysline.c:711: warning: cast from pointer to integer of different size action.c: In function ?actionDbgPrint?: action.c:174: warning: cast from pointer to integer of different size If I run "rsyslogd -r514 -4 -d" it prints something like this: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: over and over again. I'll sent you a complete output off-list. -- CU, Patrick. From mic at npgx.com.au Tue Sep 25 15:40:36 2007 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 25 Sep 2007 23:40:36 +1000 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <20070925133439.M72922@npgx.com.au> Hi Rainer, > Hi all, > > Rsyslog 1.19.7 (the official one) has been released today. It > contains some code updated compared to the version from yesterday, > so please download it if you already did. I've just downloaded and installed this release, and although I've never experienced the segfault issue (rsyslog continually runs fine for me), when I installed this version and then did a: # service rsyslog restart I got this in my messages file: Sep 25 22:41:46 server kernel: Kernel logging (proc) stopped. Sep 25 22:41:46 server kernel: Kernel log daemon terminating. Sep 25 22:41:47 server rsyslog: rklogd shutdown succeeded Sep 25 22:41:47 server rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="12176"] exiting on signal 15. Sep 25 22:44:08 server rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="10430"][x-configInfo udpRecep tion="No" udpPort="514" tcpReception="No" tcpPort="0"] restart Sep 25 22:44:08 server rsyslog: rsyslogd startup succeeded Sep 25 22:44:08 server kernel: rklogd 1.19.7, log source = /proc/kmsg started. Sep 25 22:44:08 server kernel: rsyslogd[8110]: segfault at 000000000000000a rip 00000038c7568678 rsp 0000007fbfffdc2 0 error 4 Sep 25 22:44:08 server rsyslog: rklogd startup succeeded Sep 25 22:44:08 server rsyslog: rsyslogd shutdown succeeded The above machine is a DL380 G4 with one CPU running hypthreaded. On another machine that's the same as that: Sep 25 22:42:24 server2 kernel: Kernel logging (proc) stopped. Sep 25 22:42:24 server2 kernel: Kernel log daemon terminating. Sep 25 22:42:25 server2 rsyslog: rklogd shutdown succeeded Sep 25 22:42:25 server2 rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="28497"] exiting on signal 15 . Sep 25 22:43:43 server2 rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="16598"][x-configInfo udpRece ption="No" udpPort="514" tcpReception="No" tcpPort="0"] restart Sep 25 22:43:43 server2 rsyslog: rsyslogd startup succeeded Sep 25 22:43:43 server2 kernel: rklogd 1.19.7, log source = /proc/kmsg started. Sep 25 22:43:43 server2 kernel: rsyslogd[15998]: segfault at 000000000000000a rip 00000034d2868678 rsp 0000007fbfffd c20 error 4 Sep 25 22:43:43 server2 rsyslog: rklogd startup succeeded Those two machines above are also running 64bit Linux. Using the same process on other 32 bit linux servers (DL360's), no segfaults occur. Note that I have never (ever) had rsyslog crash on me while it's running, and I currently run it on 6 servers to do the MySQL logging. Regards, Michael. > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > Its primary goal is enhanced stability. Feature-wise, it offers support > of dropping NUL characters, which are often found at the end of > invalidly formatted syslog messages. Upgrading to this release is > recommended for all users. > > Changelog: > > http://www.rsyslog.com/Article129.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml > > PLEASE NOTE: the rsyslog site is currently migrated to a new > machine. In case you receive a temporary page, your DNS still caches > the old machine's address. > > As always, feedback is appreciated. Feedback is especially > appreciated from those that have experienced the segfault problem in > the past. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From halljer at auburn.edu Tue Sep 25 18:09:39 2007 From: halljer at auburn.edu (Dusty Hall) Date: Tue, 25 Sep 2007 11:09:39 -0500 Subject: [rsyslog] new unofficial 1.19.7 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> Message-ID: <46F8EE24.8529.003A.0@auburn.edu> I had some issues with the redhat/rsyslog.init script, default binary locations, etc. I've attached an updated version. Thanks, -Dusty >>> "Rainer Gerhards" 09/24/07 9:30 AM >>> Hi all, I have posted a new "unofficial" 1.19.7 to the download server: http://download.rsyslog.com/rsyslog/rsyslog- 1.19.7.tar.gz I intend to post it publically tomorrow. But some nits need to be done first, mostly doc issues. I'd appreciate if those with the segfault problem could give it a try (in multi- threading mode) and tell me the outcome. Thanks, Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Sep 26 13:53:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 26 Sep 2007 13:53:02 +0200 Subject: [rsyslog] rsyslog 1.19.7 UDP bug Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ACC@grfint2.intern.adiscon.com> Hi all, I have identified and fixed the UDP message loss problem in rsyslog 1.19.7. If you have not yet installed that release, DO NOT INSTALL IT. I will remove it from the download links. A fixed version will be released soon as 1.19.8. If you need it urgently, an interim version is available at http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz That tarball is lacking MySQL functionality (which has now been modularized into a separate package - more news on that later). Sorry for the hassle. Rainer From rgerhards at hq.adiscon.com Wed Sep 26 14:30:24 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 26 Sep 2007 14:30:24 +0200 Subject: [rsyslog] on the udp bug Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ACF@grfint2.intern.adiscon.com> In case you are interested in some more details: http://rgerhards.blogspot.com/2007/09/needed-to-pull-1197-release.html And in CVS: http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/net.c?r1=1.9&r2=1.10 Rainer From rgerhards at hq.adiscon.com Thu Sep 27 10:11:11 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Sep 2007 10:11:11 +0200 Subject: [rsyslog] rsyslog 1.19.8 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Hi all, rsyslog 1.19.8 has been released today. Most importantly, it fixes a bug that could cause severe loss of UDP messages. It also contains improved repeat message processing. The MySQL functionality is now taken out of the core package, but its tarball is still contained in the main tarball (so it is still a single download for everything). This is part of the effort to fully support third-party plugins. Rsyslog 1.19.8 is a recommended update for all users. Change Log: http://www.rsyslog.com/Article132.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-60.phtml As always, feedback is appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Thu Sep 27 10:17:13 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Sep 2007 10:17:13 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <20070925133439.M72922@npgx.com.au> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <20070925133439.M72922@npgx.com.au> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278AE1@grfint2.intern.adiscon.com> Sorry Michael, I just noticed that I had not replied. I looked at the code, but there is too few information to track the source code location down. What I currently assume is a) that we still have a segfault issue (obviously ;)) b) the segfault is a moving target The segfault occurs due to memory corruption. I think (and have often seen) that you do not notice the corruption until it happens to hit some really important piece of memory. So in many cases, it will overwrite something that nobody cares about. Only if some specific structures are hit, a malfunction manifests. This is part of the problem to reproducing it. What is hit, depends on the memory layout. A new version leads to different memory layout. I assume this is what has happened here: you did not experience the bug before, because it hit some memory that wasn't really relevant. With the new versions memory layout, it hits somewhere where it hurts... At least this is my best guess. I'd appreciate if you could try out the 1.19.8 release. I suspect it will also segfault. If it does, it would be great if you could run it in debug mode (-d -n) and send me the output before it segfault (maybe 200 lines or so). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Mansour > Sent: Tuesday, September 25, 2007 3:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > Hi Rainer, > > > Hi all, > > > > Rsyslog 1.19.7 (the official one) has been released today. It > > contains some code updated compared to the version from yesterday, > > so please download it if you already did. > > I've just downloaded and installed this release, and although I've > never > experienced the segfault issue (rsyslog continually runs fine for me), > when I > installed this version and then did a: > > # service rsyslog restart > > I got this in my messages file: > > Sep 25 22:41:46 server kernel: Kernel logging (proc) stopped. > Sep 25 22:41:46 server kernel: Kernel log daemon terminating. > Sep 25 22:41:47 server rsyslog: rklogd shutdown succeeded > Sep 25 22:41:47 server rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="12176"] exiting on signal 15. > Sep 25 22:44:08 server rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="10430"][x-configInfo udpRecep > tion="No" udpPort="514" tcpReception="No" tcpPort="0"] restart > Sep 25 22:44:08 server rsyslog: rsyslogd startup succeeded > Sep 25 22:44:08 server kernel: rklogd 1.19.7, log source = /proc/kmsg > started. > Sep 25 22:44:08 server kernel: rsyslogd[8110]: segfault at > 000000000000000a > rip 00000038c7568678 rsp 0000007fbfffdc2 > 0 error 4 > Sep 25 22:44:08 server rsyslog: rklogd startup succeeded > Sep 25 22:44:08 server rsyslog: rsyslogd shutdown succeeded > > The above machine is a DL380 G4 with one CPU running hypthreaded. On > another > machine that's the same as that: > > Sep 25 22:42:24 server2 kernel: Kernel logging (proc) stopped. > Sep 25 22:42:24 server2 kernel: Kernel log daemon terminating. > Sep 25 22:42:25 server2 rsyslog: rklogd shutdown succeeded > Sep 25 22:42:25 server2 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="28497"] exiting on signal 15 > . > Sep 25 22:43:43 server2 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="16598"][x-configInfo udpRece > ption="No" udpPort="514" tcpReception="No" tcpPort="0"] restart > Sep 25 22:43:43 server2 rsyslog: rsyslogd startup succeeded > Sep 25 22:43:43 server2 kernel: rklogd 1.19.7, log source = /proc/kmsg > started. > Sep 25 22:43:43 server2 kernel: rsyslogd[15998]: segfault at > 000000000000000a > rip 00000034d2868678 rsp 0000007fbfffd > c20 error 4 > Sep 25 22:43:43 server2 rsyslog: rklogd startup succeeded > > Those two machines above are also running 64bit Linux. > > Using the same process on other 32 bit linux servers (DL360's), no > segfaults > occur. > > Note that I have never (ever) had rsyslog crash on me while it's > running, and > I currently run it on 6 servers to do the MySQL logging. > > Regards, > > Michael. > > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > > Its primary goal is enhanced stability. Feature-wise, it offers > support > > of dropping NUL characters, which are often found at the end of > > invalidly formatted syslog messages. Upgrading to this release is > > recommended for all users. > > > > Changelog: > > > > http://www.rsyslog.com/Article129.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml > > > > PLEASE NOTE: the rsyslog site is currently migrated to a new > > machine. In case you receive a temporary page, your DNS still caches > > the old machine's address. > > > > As always, feedback is appreciated. Feedback is especially > > appreciated from those that have experienced the segfault problem in > > the past. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > ------- End of Original Message ------- > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 28 13:46:13 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Sep 2007 13:46:13 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Hi Michael, as I have blogged, I am not yet sure about how to handle the situation. I am also consulting with Autotools experts, any advise is appreciated. Two packages seem useful, especially when more plugins become available (I myself think about email and a couple of other databases). Many folks also do not like the idea of having to have libmysql present on the system just to install rsyslog - with a core package and the plugin, those can only install the core (and that will probably the majority of cases). What I have not yet found - and I have very limited expertise in this area - is how to do that in the best possible way. Oh, and some more background: ones the plugin interace has matured (I expect this in 3 to 6 month), I intend to actually use ommysql with its own version number. Versioning will be handled by the interface (that part is already present, but no code for it yet as there is only a release 1 of it). So, in the medium to long term, ommysql *will* be a separate project - maybe one with a different maintainer, as I am no mysql guy. Does this make sense? As I said, comments are much appreciated... Rainer > -----Original Message----- > From: Michael Biebl [mailto:mbiebl at gmail.com] > Sent: Friday, September 28, 2007 10:23 AM > To: rsyslog-users; Rainer Gerhards > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > 2007/9/27, Rainer Gerhards : > > repeat message processing. The MySQL functionality is now taken out > of > > the core package, but its tarball is still contained in the main > tarball > > (so it is still a single download for everything). This is part of > the > > effort to fully support third-party plugins. Rsyslog 1.19.8 is a > > > To be honest, I don't particularly like this change. It increases the > work for package maintainers (like me). You now have to maintain two > source packages. Having a non-standard tarball inside a tarball > doesn't help there. It even worsens things, as stuff like "make dist" > or "make distcheck" doesn't work anymore for a cvs checkout. > There's also the problem of which version of the plugins will be > compatible with the core version (upgrade scenarios, keeping them in > sync, etc.) > It also adds complexity for the developer (as he has to maintain an > additonal set of build files). > As you were talking about 3rd party plugins (with the emphasis on 3rd > party) I don't understand the benefit of splitting out the mysql > plugin as it is you who develops the mysql plugin, not a third party. > Do you actually intend to create a separate tarball for each plugin in > the future? > > What was wrong with the --enable-mysql configure switch? I don't see > any benefits, only disadvantages. You know, if it ain't broken, why > fix it ;-) > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? From mic at npgx.com.au Fri Sep 28 16:03:59 2007 From: mic at npgx.com.au (Michael Mansour) Date: Sat, 29 Sep 2007 00:03:59 +1000 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: <20070928135248.M3034@npgx.com.au> Hi Rainer, > Hi Michael, > > as I have blogged, I am not yet sure about how to handle the situation. > I am also consulting with Autotools experts, any advise is appreciated. > > Two packages seem useful, especially when more plugins become available > > (I myself think about email and a couple of other databases). Many folks > also do not like the idea of having to have libmysql present on the > system just to install rsyslog - with a core package and the plugin, > those can only install the core (and that will probably the majority > of cases). Yes, I totally agree with what you're doing, as making rsyslog modular is better overall for the product as a whole, as it will allow contributors easier access to have their own plugins in place without needing to release rsyslog each time. > What I have not yet found - and I have very limited expertise in this > area - is how to do that in the best possible way. I've asked both Dag and Dries (from rpmforge) on the best way forward as these guys have years of knowledge behind them for packaging (in the RPM space though). I'm not sure they'll reply though, as sometimes my questions to them are ignored and sometimes they are responded to (some questions I ask go into their "too hard" basket which I think is why they ignore them :P ). I have been querying, requesting, suggesting, etc things off them for more than 7 years now, so they may be sick of me (joking) :). I have been asked to become a packager for them before (and for Axel Thimms who does ATrpms.net - they're all merging into one large repo eventually) but declined as I have little time as is to contribute to the projects I'm currently working with. > Oh, and some more background: ones the plugin interace has matured (I > expect this in 3 to 6 month), I intend to actually use ommysql with its > own version number. Versioning will be handled by the interface (that > part is already present, but no code for it yet as there is only a > release 1 of it). So, in the medium to long term, ommysql *will* be a > separate project - maybe one with a different maintainer, as I am no > mysql guy. > > Does this make sense? As I said, comments are much appreciated... Yes and I do agree with it all. I'm just not personally sure of the best way forward myself, but I would suspect something along the lines of having one spec file for rsyslog, and another for rsyslog-mysql would likely be the right way to go. If neither Dag nor Dries answer me, I might even try Axel and if he also doesn't respond (he usually does) then I'll just start on the packaging myself, doing rsyslog and rsyslog-mysql. In the long term, it would be better though if rpmforge or atrpms.net would take on this packaging system. I realise Red Hat will have a "base" one available in Fedora and then eventually an EL release, but they dont' do updates, just bugfixing, once they release it for an enterprise product. Which is where 3rd party repo's make sense. Regards, Michael. > Rainer > > > -----Original Message----- > > From: Michael Biebl [mailto:mbiebl at gmail.com] > > Sent: Friday, September 28, 2007 10:23 AM > > To: rsyslog-users; Rainer Gerhards > > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > > > 2007/9/27, Rainer Gerhards : > > > repeat message processing. The MySQL functionality is now taken out > > of > > > the core package, but its tarball is still contained in the main > > tarball > > > (so it is still a single download for everything). This is part of > > the > > > effort to fully support third-party plugins. Rsyslog 1.19.8 is a > > > > > > To be honest, I don't particularly like this change. It increases the > > work for package maintainers (like me). You now have to maintain two > > source packages. Having a non-standard tarball inside a tarball > > doesn't help there. It even worsens things, as stuff like "make dist" > > or "make distcheck" doesn't work anymore for a cvs checkout. > > There's also the problem of which version of the plugins will be > > compatible with the core version (upgrade scenarios, keeping them in > > sync, etc.) > > It also adds complexity for the developer (as he has to maintain an > > additonal set of build files). > > As you were talking about 3rd party plugins (with the emphasis on 3rd > > party) I don't understand the benefit of splitting out the mysql > > plugin as it is you who develops the mysql plugin, not a third party. > > Do you actually intend to create a separate tarball for each plugin in > > the future? > > > > What was wrong with the --enable-mysql configure switch? I don't see > > any benefits, only disadvantages. You know, if it ain't broken, why > > fix it ;-) > > > > Cheers, > > Michael > > > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From mbiebl at gmail.com Fri Sep 28 18:06:46 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 18:06:46 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Rainer Gerhards : > Hi Michael, > > as I have blogged, I am not yet sure about how to handle the situation. > I am also consulting with Autotools experts, any advise is appreciated. > > Two packages seem useful, especially when more plugins become available > (I myself think about email and a couple of other databases). Many folks > also do not like the idea of having to have libmysql present on the > system just to install rsyslog - with a core package and the plugin, Well, this is not quite true. If you don't want to have mysql support, you could just use --disable-mysql. This way you don't have to have mysql installed. Such a --enable/disable-feature option could be added for any plugin that will be added in the future. > those can only install the core (and that will probably the majority of > cases). > > What I have not yet found - and I have very limited expertise in this > area - is how to do that in the best possible way. I do have quite some experience with autotools. So if any help is needed, I'll gladly offer it. > release 1 of it). So, in the medium to long term, ommysql *will* be a > separate project - maybe one with a different maintainer, as I am no > mysql guy. This actually is a valid argument for splitting it off. But to reiterate what I posted earlier: For the plugins shipped directly by the rsyslog project, I'd prefer to have them in a single tarball with configure options to turn them on/off and as long as you are the maintainer of the mysql output plugin its more convenient to keep them in one project. Why not make the split when the plugin interface is stable, other output plugins are available (and we have more experience how to handle them) and a external maintainer for ommysql is found? I hope this doesn't sound too negative. I generally like the idea of having a plugin interface, but I only fear that splitting things up prematurely without knowing how it develops, will complicate things (and probably destabilize). Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Fri Sep 28 20:11:20 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Sep 2007 20:11:20 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Michael, I need to prepare for german astronomy day tomorrow, so brief ;) The point was that a single package (RPM or whatever) can be pre-build for the core and the mysql functionality. I think the configure options do not help with that. Or do they? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Friday, September 28, 2007 6:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > 2007/9/28, Rainer Gerhards : > > Hi Michael, > > > > as I have blogged, I am not yet sure about how to handle the > situation. > > I am also consulting with Autotools experts, any advise is > appreciated. > > > > Two packages seem useful, especially when more plugins become > available > > (I myself think about email and a couple of other databases). Many > folks > > also do not like the idea of having to have libmysql present on the > > system just to install rsyslog - with a core package and the plugin, > > Well, this is not quite true. If you don't want to have mysql support, > you could just use --disable-mysql. This way you don't have to have > mysql installed. > Such a --enable/disable-feature option could be added for any plugin > that will be added in the future. > > > those can only install the core (and that will probably the majority > of > > cases). > > > > What I have not yet found - and I have very limited expertise in this > > area - is how to do that in the best possible way. > > I do have quite some experience with autotools. So if any help is > needed, I'll gladly offer it. > > > release 1 of it). So, in the medium to long term, ommysql *will* be a > > separate project - maybe one with a different maintainer, as I am no > > mysql guy. > > This actually is a valid argument for splitting it off. > > But to reiterate what I posted earlier: For the plugins shipped > directly by the rsyslog project, I'd prefer to have them in a single > tarball with configure options to turn them on/off and as long as you > are the maintainer of the mysql output plugin its more convenient to > keep them in one project. Why not make the split when the plugin > interface is stable, other output plugins are available (and we have > more experience how to handle them) and a external maintainer for > ommysql is found? > I hope this doesn't sound too negative. I generally like the idea of > having a plugin interface, but I only fear that splitting things up > prematurely without knowing how it develops, will complicate things > (and probably destabilize). > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Fri Sep 28 21:03:53 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 21:03:53 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Rainer Gerhards : > Michael, > > I need to prepare for german astronomy day tomorrow, so brief ;) > > The point was that a single package (RPM or whatever) can be pre-build > for the core and the mysql functionality. I think the configure options > do not help with that. Or do they? > You can split up the build to create multiple *binary* packages from a single *source* package, the RPM format as well as DEB allows that. Actually the Debian and Fedora packages already do that. The rsyslog [1] binary package contains the core rsyslogd/rklogd binaries, the rsyslog-mysql [2] package the mysql output plugin. It's not necessary to already split up the *source* package for that. Cheers, Michael [1] http://packages.debian.org/sid/rsyslog [2] http://packages.debian.org/sid/rsyslog-mysql -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Fri Sep 28 21:06:05 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 21:06:05 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Michael Biebl : > 2007/9/28, Rainer Gerhards : > > > > The point was that a single package (RPM or whatever) can be pre-build > > for the core and the mysql functionality. I think the configure options > > do not help with that. Or do they? > > > > You can split up the build to create multiple *binary* packages from a > single *source* package, the RPM format as well as DEB allows that. > > Actually the Debian and Fedora packages already do that. The rsyslog > [1] binary package contains the core rsyslogd/rklogd binaries, the > rsyslog-mysql [2] package the mysql output plugin. > It's not necessary to already split up the *source* package for that. > > [1] http://packages.debian.org/sid/rsyslog > [2] http://packages.debian.org/sid/rsyslog-mysql And as you can see from [1] and [2], the core rsyslog package has no libmysql dependencies. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From janfrode at tanso.net Fri Sep 28 22:57:12 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 28 Sep 2007 22:57:12 +0200 Subject: [rsyslog] rsyslog 1.19.8 released References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Message-ID: On 2007-09-27, Rainer Gerhards wrote: > > rsyslog 1.19.8 has been released today. Most importantly, it fixes a bug Still crashes.. twice today, but I didn't find the trace the first time. Here's the trace from the second crash: *** glibc detected *** rsyslogd: corrupted double-linked list: 0x094ca0e8 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152cda9] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8058323] rsyslogd[0x804dfba] rsyslogd(llExecFunc+0x3f)[0x805f43f] rsyslogd[0x804d9ba] rsyslogd[0x804db0f] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] ======= Memory map: ======== 0067d000-0067e000 r-xp 0067d000 00:00 0 [vdso] 08048000-08068000 r-xp 00000000 fd:00 884939 /sbin/rsyslogd 08068000-08069000 rwxp 00020000 fd:00 884939 /sbin/rsyslogd 094c5000-09549000 rwxp 094c5000 00:00 0 414aa000-414c3000 r-xp 00000000 fd:00 1146882 /lib/ld-2.5.so 414c3000-414c4000 r-xp 00018000 fd:00 1146882 /lib/ld-2.5.so 414c4000-414c5000 rwxp 00019000 fd:00 1146882 /lib/ld-2.5.so 414c7000-415fe000 r-xp 00000000 fd:00 1146990 /lib/libc-2.5.so 415fe000-41600000 r-xp 00137000 fd:00 1146990 /lib/libc-2.5.so 41600000-41601000 rwxp 00139000 fd:00 1146990 /lib/libc-2.5.so 41601000-41604000 rwxp 41601000 00:00 0 41606000-41608000 r-xp 00000000 fd:00 1147002 /lib/libdl-2.5.so 41608000-41609000 r-xp 00001000 fd:00 1147002 /lib/libdl-2.5.so 41609000-4160a000 rwxp 00002000 fd:00 1147002 /lib/libdl-2.5.so 4160c000-4161f000 r-xp 00000000 fd:00 1147003 /lib/libpthread-2.5.so 4161f000-41620000 r-xp 00012000 fd:00 1147003 /lib/libpthread-2.5.so 41620000-41621000 rwxp 00013000 fd:00 1147003 /lib/libpthread-2.5.so 41621000-41623000 rwxp 41621000 00:00 0 41667000-41679000 r-xp 00000000 fd:00 1087043 /usr/lib/libz.so.1.2.3 41679000-4167a000 rwxp 00011000 fd:00 1087043 /usr/lib/libz.so.1.2.3 416c4000-416cb000 r-xp 00000000 fd:00 1147009 /lib/librt-2.5.so 416cb000-416cc000 r-xp 00006000 fd:00 1147009 /lib/librt-2.5.so 416cc000-416cd000 rwxp 00007000 fd:00 1147009 /lib/librt-2.5.so 4acb4000-4acbf000 r-xp 00000000 fd:00 1146939 /lib/libgcc_s-4.1.1-20070105.so.1 4acbf000-4acc0000 rwxp 0000a000 fd:00 1146939 /lib/libgcc_s-4.1.1-20070105.so.1 b7200000-b722a000 rw-p b7200000 00:00 0 b722a000-b7300000 ---p b722a000 00:00 0 b7400000-b7430000 rw-p b7400000 00:00 0 b7430000-b7500000 ---p b7430000 00:00 0 b759b000-b759c000 ---p b759b000 00:00 0 b759c000-b7f9e000 rw-p b759c000 00:00 0 b7fa3000-b7fa5000 rw-p b7fa3000 00:00 0 bfc72000-bfc88000 rw-p bfc72000 00:00 0 [stack] -jf From janfrode at tanso.net Mon Sep 3 10:47:26 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 3 Sep 2007 10:47:26 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> Message-ID: Another one of these, this time with v1.19.3: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xae3fa998 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8057393] rsyslogd[0x804de0a] rsyslogd(llExecFunc+0x3f)[0x805ea3f] rsyslogd[0x804d86a] rsyslogd[0x804d997] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] And I didn't get any core-file, maybe because the v1.19.3 overwrote my "ulimit -c unlimited" change to the initscript... Ooops :-) -jf From rgerhards at hq.adiscon.com Tue Sep 4 17:57:55 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 17:57:55 +0200 Subject: [rsyslog] rsyslog 1.19.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27890E@grfint2.intern.adiscon.com> Hi all, rsyslog 1.19.4 is a bug fixing release. It contains no new features, but stability updates. Most importantly, it addresses some bugs that have lead to program aborts. 1.19.4 is a recommended update for all users. Changelog: http://www.rsyslog.com/Article123.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-56.phtml As always, feedback is very appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Tue Sep 4 17:58:39 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 17:58:39 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> Hi, I have just release 1.19.4 and hope that the fixes also address your problem. I'd appreciate if you could try it out. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Monday, September 03, 2007 10:47 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] v1.19.1 is crashing > > > Another one of these, this time with v1.19.3: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xae3fa998 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x73)[0x8057393] > rsyslogd[0x804de0a] > rsyslogd(llExecFunc+0x3f)[0x805ea3f] > rsyslogd[0x804d86a] > rsyslogd[0x804d997] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > And I didn't get any core-file, maybe because the v1.19.3 overwrote my > "ulimit -c unlimited" change to the initscript... Ooops :-) > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Sep 4 18:58:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 18:58:48 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Hi, I noticed that one problem I received patches for (and that are now included in 1.19.4) is rooted in something that is not fully patched. I think I've found that root cause now (thanks to the info with that patches). The bottom line, however, is that 1.19.4 may still have some stability issues. However, they should surface now only in very obscure cases (but what is obscure...). I'd still appreciate if you could apply 1.19.4 and tell me the outcome. I am now working on fixing the root cause. That might take a short while, as I am thinking about the best *design* to fix the issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, September 04, 2007 5:59 PM > To: rsyslog-users > Subject: Re: [rsyslog] v1.19.1 is crashing > > Hi, > > I have just release 1.19.4 and hope that the fixes also address your > problem. I'd appreciate if you could try it out. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > > Sent: Monday, September 03, 2007 10:47 AM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] v1.19.1 is crashing > > > > > > Another one of these, this time with v1.19.3: > > > > *** glibc detected *** rsyslogd: corrupted double-linked list: > > 0xae3fa998 *** > > ======= Backtrace: ========= > > /lib/libc.so.6[0x4152ce3e] > > /lib/libc.so.6(cfree+0x90)[0x415305d0] > > rsyslogd(MsgDestruct+0x73)[0x8057393] > > rsyslogd[0x804de0a] > > rsyslogd(llExecFunc+0x3f)[0x805ea3f] > > rsyslogd[0x804d86a] > > rsyslogd[0x804d997] > > /lib/libpthread.so.0[0x416112db] > > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > > > And I didn't get any core-file, maybe because the v1.19.3 overwrote > my > > "ulimit -c unlimited" change to the initscript... Ooops :-) > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Wed Sep 5 10:15:26 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 5 Sep 2007 10:15:26 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: On 2007-09-04, Rainer Gerhards wrote: > > I'd still appreciate if you could apply 1.19.4 and tell me the outcome. > I am now working on fixing the root cause. That might take a short > while, as I am thinking about the best *design* to fix the issue. OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you know if it fails again. -jf From theinric at redhat.com Wed Sep 5 11:53:10 2007 From: theinric at redhat.com (theinric at redhat.com) Date: Wed, 05 Sep 2007 11:53:10 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com> Message-ID: <46DE7C86.1040308@redhat.com> You have an error in your config file, but it's probably harmless: %timegenerated::fulltime% The option fulltime doesn't exist and it looks like it never did. Additionally, a colon is missing. I've noticed that this definition is actually present in sample.conf, so you've probably picked it up there. (It looks like it has been there at least from 0.8.1) I've did some testing and it probably doesn't have any impact except for a warning message in debug mode. Jan-Frode Myklebust wrote: > On 2007-08-31, theinric at redhat.com wrote: >> could you please provide some more info on your configuration? >> Configuration file, > > ################################################################################# > $ grep -v ^# /etc/rsyslog.conf|grep -v ^$ > $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.log" > *.* -?DailyPerHostLogs > $template MaillogTemplate,"%timegenerated::fulltime% %HOSTNAME% %syslogtag%: %msg%\n" > $template HourlyMaillog,"/var/log/syslog/maillog/%$YEAR%/%$MONTH%/%$DAY%/maillog-%$YEAR%%$MONTH%%$DAY%%$HOUR%.log" > mail.* -?HourlyMaillog;MaillogTemplate > $template precise,"%timegenerated::fulltime% %HOSTNAME% %syslogfacility-text%/%syslogseverity-text% %syslogtag% %msg%\n" > *.* -/var/log/syslog/everything;precise > mail.* ~ > $template PerAppLogs,"/var/log/syslog/apps/%programname%.log" > *.* -?PerAppLogs > :msg, contains, "ServeRAID" -/var/log/syslog/apps/serveraid.log > :HOSTNAME, !isequal, "loghost1" ~ > *.info;mail.none;authpriv.none;cron.none /var/log/messages > authpriv.* /var/log/secure > mail.* -/var/log/maillog > cron.* /var/log/cron > *.emerg * > uucp,news.crit /var/log/spooler > local7.* /var/log/boot.log > ################################################################################# > > >> options used, > > $ grep -v ^# /etc/sysconfig/rsyslog > SYSLOGD_OPTIONS="-m 0 -r514" > KLOGD_OPTIONS="-x" > SYSLOG_UMASK=077 > >> log entries preceding the crash, ... > > It's a quite busy log server, with about 70 active old style syslog servers > sending logs to it. The second it crashed it wrote 111 log-messages.. (273 > the second before), mostly various postfix daemons, and I'd need to anonymize > them before sharing.. Can't see anything special. > >> If logging forwarded messages, is the remote logger also rsyslog? > > No, all are RHEL3/4/5 with their default syslogd server. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Sep 5 12:25:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 5 Sep 2007 12:25:48 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <46DE7C86.1040308@redhat.com> References: <46D81432.1090302@redhat.com> <46DE7C86.1040308@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278919@grfint2.intern.adiscon.com> I have checked on fulltime, it is an error in sample.conf - there is no need for this option and I think it was actually not present. I'll remove that sample. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of theinric at redhat.com > Sent: Wednesday, September 05, 2007 11:53 AM > To: rsyslog-users > Subject: Re: [rsyslog] v1.19.1 is crashing > > You have an error in your config file, but it's probably harmless: > %timegenerated::fulltime% > > The option fulltime doesn't exist and it looks like it never did. > Additionally, > a colon is missing. I've noticed that this definition is actually > present in > sample.conf, so you've probably picked it up there. (It looks like it > has been > there at least from 0.8.1) > I've did some testing and it probably doesn't have any impact except > for a > warning message in debug mode. > > Jan-Frode Myklebust wrote: > > On 2007-08-31, theinric at redhat.com wrote: > >> could you please provide some more info on your configuration? > >> Configuration file, > > > > > ####################################################################### > ########## > > $ grep -v ^# /etc/rsyslog.conf|grep -v ^$ > > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.lo > g" > > *.* -?DailyPerHostLogs > > $template MaillogTemplate,"%timegenerated::fulltime% %HOSTNAME% > %syslogtag%: %msg%\n" > > $template > HourlyMaillog,"/var/log/syslog/maillog/%$YEAR%/%$MONTH%/%$DAY%/maillog- > %$YEAR%%$MONTH%%$DAY%%$HOUR%.log" > > mail.* -?HourlyMaillog;MaillogTemplate > > $template precise,"%timegenerated::fulltime% %HOSTNAME% > %syslogfacility-text%/%syslogseverity-text% %syslogtag% %msg%\n" > > *.* -/var/log/syslog/everything;precise > > mail.* ~ > > $template PerAppLogs,"/var/log/syslog/apps/%programname%.log" > > *.* -?PerAppLogs > > :msg, contains, "ServeRAID" - > /var/log/syslog/apps/serveraid.log > > :HOSTNAME, !isequal, "loghost1" ~ > > *.info;mail.none;authpriv.none;cron.none > /var/log/messages > > authpriv.* > /var/log/secure > > mail.* - > /var/log/maillog > > cron.* /var/log/cron > > *.emerg * > > uucp,news.crit > /var/log/spooler > > local7.* > /var/log/boot.log > > > ####################################################################### > ########## > > > > > >> options used, > > > > $ grep -v ^# /etc/sysconfig/rsyslog > > SYSLOGD_OPTIONS="-m 0 -r514" > > KLOGD_OPTIONS="-x" > > SYSLOG_UMASK=077 > > > >> log entries preceding the crash, ... > > > > It's a quite busy log server, with about 70 active old style syslog > servers > > sending logs to it. The second it crashed it wrote 111 log-messages.. > (273 > > the second before), mostly various postfix daemons, and I'd need to > anonymize > > them before sharing.. Can't see anything special. > > > >> If logging forwarded messages, is the remote logger also rsyslog? > > > > No, all are RHEL3/4/5 with their default syslogd server. > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Wed Sep 5 12:29:55 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Wed, 05 Sep 2007 12:29:55 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: <1188988195.3362.17.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Dienstag, den 04.09.2007, 18:58 +0200 schrieb Rainer Gerhards: > Hi, > > I noticed that one problem I received patches for (and that are now > included in 1.19.4) is rooted in something that is not fully patched. I > think I've found that root cause now (thanks to the info with that > patches). The bottom line, however, is that 1.19.4 may still have some > stability issues. However, they should surface now only in very obscure > cases (but what is obscure...). Well, at least it was running for almost two hours..... I'm running RHEL5. Here is my config: $AllowedSender UDP, 1.2.3.0/24 $AllowedSender TCP, 1.2.3.0/24 $ModLoad MySQL $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/clamav" $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/exim" $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" *.* /home/local/log/all !rsyslogd :programname, contains, "rsyslogd" /home/local/log/rsyslogd !spamd :msg, contains, "prefork: child states" ~ :msg, contains, "spamd: got connection over /opt/antispam/var/socket/spamd" ~ :msg, contains, "spamd: checking message (unknown) for exim:427" ~ :msg, contains, "spamd: handled cleanup of child pid" ~ mail.* /home/local/log/mail !clamd :msg, contains, "No stats for Database check - forcing reload" ~ :msg, contains, "Reading databases from /var/clamav" ~ :msg, contains, "Database correctly reloaded" ~ :msg, contains, "SelfCheck: Database status OK." ~ local5.* ?clamavFile !exim *.* ?eximFile :msg, contains, "malware detected" ?avFile -- CU, Patrick. From rgerhards at hq.adiscon.com Thu Sep 6 17:58:56 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Sep 2007 17:58:56 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Hi all, We are nearing the point where a decision about the future config file format needs to be made. I have blogged the details: http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html I would deeply appreciate any feedback on the samples and format suggestions. Best regards, Rainer Gerhards From janfrode at tanso.net Fri Sep 7 08:51:13 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 7 Sep 2007 08:51:13 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: On 2007-09-05, Jan-Frode Myklebust wrote: > On 2007-09-04, Rainer Gerhards wrote: >> >> I'd still appreciate if you could apply 1.19.4 and tell me the outcome. >> I am now working on fixing the root cause. That might take a short >> while, as I am thinking about the best *design* to fix the issue. > > OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you > know if it fails again. It failed again yesterday: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xb7209028 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8057e93] rsyslogd[0x804de4a] rsyslogd(llExecFunc+0x3f)[0x805eb0f] rsyslogd[0x804d8aa] rsyslogd[0x804d9d7] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] I have "mon" monitoring that rsyslogd is running, and restart it when it fails. "mon" restarted rsyslogd twice (Thu Sep 6 20:38, and Fri Sep 7 03:27), but I can't find any backtrace from the second crash.. -jf From rgerhards at hq.adiscon.com Fri Sep 7 09:50:32 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 09:50:32 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com><577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278936@grfint2.intern.adiscon.com> Thanks for the feedback. I will probably release a new version today. It has an important fix, which hopefully solves this issue. The bad thing is that I can not reproduce the problem in my lab, so I am basically back to reviewing code and listening to your feedback ;) I have one more area (in the same class) under suspicion. But maybe I do not change that before trying out the current code change. As a side-note, the *actual* root cause was a too-complex internal API, which lead to wrong calling sequences in some parts of the code. I have now re-structured the API and revisited all places where it was called. There is another similar API and this is what I am currently reviewing. I am not sure I like to change that API without real need, because it is used a lot and any such change of course has new bug potential. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, September 07, 2007 8:51 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] v1.19.1 is crashing > > On 2007-09-05, Jan-Frode Myklebust wrote: > > On 2007-09-04, Rainer Gerhards wrote: > >> > >> I'd still appreciate if you could apply 1.19.4 and tell me the > outcome. > >> I am now working on fixing the root cause. That might take a short > >> while, as I am thinking about the best *design* to fix the issue. > > > > OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you > > know if it fails again. > > It failed again yesterday: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xb7209028 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x73)[0x8057e93] > rsyslogd[0x804de4a] > rsyslogd(llExecFunc+0x3f)[0x805eb0f] > rsyslogd[0x804d8aa] > rsyslogd[0x804d9d7] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > I have "mon" monitoring that rsyslogd is running, and restart it when > it fails. "mon" restarted rsyslogd twice (Thu Sep 6 20:38, and Fri > Sep 7 03:27), but I can't find any backtrace from the second crash.. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Sep 7 13:32:13 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 7 Sep 2007 13:32:13 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Message-ID: On 2007-09-06, Rainer Gerhards wrote: > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > I would deeply appreciate any feedback on the samples and format > suggestions. /me thinks you're getting way too little feedback on the blog, or this list. Unfortunately I don't have much more than simple preference to contribute here.. XML-based format: Yikes, you'll need an additional human readable frontend format that's converted to XML for it to be usable. You can't expect us poor sysadmins to be editing XML directly to configure rsyslogd.. syslog-ng like: Fair enough.. It works for my usage. Metalog like: No experience.. Apache like: Not sure I understand this.. Seems like a mix of option/value and xml'ish for some functionality. Programming like..: Of the samples in the wiki, I most prefer the BASIC-like. It resembles python to me, and also "mon"'s config format. Very readable. http://mon.wiki.kernel.org/index.php/Mon_Manual The c-like with functions seems too complex: if1: { if(%severity < "debug" && lower(substr(%msg, 5, 3)) != "err") } action1() { action(type=filewrite, file="/var/log/mail.log") } rule1() { if1() action1() action(type=filewrite, file="/var/log/messages.log") } rule(if1,action1) ruleset(rule1, rule(if1, action(type=filewrite, file="/var/log/messages.log"))) rule(action1(),input="$all") input(type=udp, bind="127.0.0.1") I can't parse this.. Does rule1() break out of if1() is false? Then I guess writes to /var/log/messages.log woun't happen if action1 for some reason failed ? Contrast it to mon's config translated to syslogging: # Define some groups of servers: hostgroup mailservers server1 server2 server3 hostgroup webservers server4 server5 watch mailservers severity > debug SUBMSG = lower(substr(%msg, 5, 3)) SUBMSG != "err" logwrite /var/log/mail.log logwrite /var/log/messages.log SUBMSG == "err" logwrite /var/log/err.log watch webservers programname == httpd severity == crit cmd wall "httpd critical: $msg" logwrite /var/log/crit.log severity < crit logwrite /var/log/httpd.log Each indentation means it's depending on the previous statement being true. You might need to be drinking the python Kool-Aid to see the beauty :-) -jf From skvidal at fedoraproject.org Fri Sep 7 13:40:59 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 07 Sep 2007 07:40:59 -0400 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Message-ID: <1189165259.16157.114.camel@cutter> On Fri, 2007-09-07 at 13:32 +0200, Jan-Frode Myklebust wrote: > On 2007-09-06, Rainer Gerhards wrote: > > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > I would deeply appreciate any feedback on the samples and format > > suggestions. > > /me thinks you're getting way too little feedback on the blog, > or this list. Unfortunately I don't have much more than simple > preference to contribute here.. > > XML-based format: > > Yikes, you'll need an additional human readable frontend > format that's converted to XML for it to be usable. You > can't expect us poor sysadmins to be editing XML > directly to configure rsyslogd.. The nice piece of this is that it is machine parseable easily which enables lots of useful editors. > > syslog-ng like: > > Fair enough.. It works for my usage. The syntax is okay but at that point what distinguishes b/t syslog-ng and rsyslog? > Apache like: > > Not sure I understand this.. Seems like a mix of option/value > and xml'ish for some functionality. This one I'm more interested in. If you think of each log like a vhost and you define the qualities that are added to that inside the definition Destination /path/to/silly.log DestinationMode 0640 DestinationOwner root DestinationGroup log-readers Include mail.info kern.debug cron.emerg etc, etc, etc maybe that doesn't make sense, maybe it does - it is pretty easy to read, though. -sv From rgerhards at hq.adiscon.com Fri Sep 7 14:08:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 14:08:02 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: <1189165259.16157.114.camel@cutter> References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> <1189165259.16157.114.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27893E@grfint2.intern.adiscon.com> I am replying here, but without a real reply. All feedback is deeply appreciate, but I'd like to keep silent for the time being to avoid bringing in my personal bias. Please keep commenting. I'll do a wrap-up later. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of seth vidal > Sent: Friday, September 07, 2007 1:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog config file format - please provide > feedback > > > On Fri, 2007-09-07 at 13:32 +0200, Jan-Frode Myklebust wrote: > > On 2007-09-06, Rainer Gerhards wrote: > > > > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > > > I would deeply appreciate any feedback on the samples and format > > > suggestions. > > > > /me thinks you're getting way too little feedback on the blog, > > or this list. Unfortunately I don't have much more than simple > > preference to contribute here.. > > > > XML-based format: > > > > Yikes, you'll need an additional human readable frontend > > format that's converted to XML for it to be usable. You > > can't expect us poor sysadmins to be editing XML > > directly to configure rsyslogd.. > > The nice piece of this is that it is machine parseable easily which > enables lots of useful editors. > > > > > syslog-ng like: > > > > Fair enough.. It works for my usage. > > The syntax is okay but at that point what distinguishes b/t syslog-ng > and rsyslog? > > > > Apache like: > > > > Not sure I understand this.. Seems like a mix of option/value > > and xml'ish for some functionality. > > This one I'm more interested in. If you think of each log like a vhost > and you define the qualities that are added to that inside the > definition > > > Destination /path/to/silly.log > DestinationMode 0640 > DestinationOwner root > DestinationGroup log-readers > Include mail.info kern.debug cron.emerg > > > etc, etc, etc > > maybe that doesn't make sense, maybe it does - it is pretty easy to > read, though. > > -sv > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 7 18:14:34 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 18:14:34 +0200 Subject: [rsyslog] rsyslog 1.19.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> Hi all, Rsyslog 1.19.5 has been released. It is primarily targeted at fixing rare situations in which a segfault could occur. These have not consistently been able to reproduce in lab. Release 1.19.5 may still have a problem or may hang where previous versions segfaulted. We are actively looking for feedback from the field. Feature-wise, the $ModDir config directive has been added and $ModLoad has been enhanced. We recommend upgrading only for those that experience the problem with earlier versions or those actively interested in helping to solve the bug. If you experience a bug, please report it. Changelog: http://www.rsyslog.com/Article125.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-57.phtml As always, feedback is appreciated. Rainer Gerhards From hagen at rz.uni-karlsruhe.de Fri Sep 7 19:20:12 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Fri, 07 Sep 2007 19:20:12 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> Message-ID: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > Hi all, > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > rare situations in which a segfault could occur. These have not > consistently been able to reproduce in lab. Release 1.19.5 may still The way I see it everyone reporting segfaults so far has been using RHEL5. So perhaps everybody seeing problems could comment on the operating-system? That might ease reproducing the problem in a test-lab. > have a problem or may hang where previous versions segfaulted. We are > actively looking for feedback from the field. Feature-wise, the $ModDir > config directive has been added and $ModLoad has been enhanced. We > recommend upgrading only for those that experience the problem with > earlier versions or those actively interested in helping to solve the > bug. If you experience a bug, please report it. Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be quite "normal", however, I've been lucky and captured some strace-information. Not sure wheter that could be helpful. poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "P*\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7in-a"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "P*\205\200\0\1\0\1\0\6\0\7\00282\003185\00213\003129 \7"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14623]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 165 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\236\222\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\236\222\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14099]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 243 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\377\251\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\377\251\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>spamd[16561]: spamd: identif"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 94 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\202\24\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\202\24\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>spamd[16561]: spamd: result:"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 463 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\33\\\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\33\\\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[15976]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 143 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "I\27\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7in"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "I\27\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \003129"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "v\325\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "v\325\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 futex(0x3627949960, FUTEX_WAKE, 1) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "+\330\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "+\330\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<13>greylistd: Socket error: Bro"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 41 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "+\366\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "+\366\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\233A\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\233A\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14536]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 171 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "M\243\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "M\243\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = ? ERESTARTNOHAND (To be restarted) trace: ptrace(PTRACE_SYSCALL, ...): No such process Process 22923 detached -- CU, Patrick. From infofarmer at FreeBSD.org Fri Sep 7 19:48:14 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Fri, 7 Sep 2007 21:48:14 +0400 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <20070907174813.GE56443@amilo.cenkes.org> On Fri, Sep 07, 2007 at 07:20:12PM +0200, Patrick von der Hagen wrote: > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > > Hi all, > > > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > > rare situations in which a segfault could occur. These have not > > consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. I experience segfaults with 1.19.1 or 1.19.2 on FreeBSD, but 1.19.3 has been up for a week on end now. From mic at npgx.com.au Sat Sep 8 00:20:37 2007 From: mic at npgx.com.au (Michael Mansour) Date: Sat, 8 Sep 2007 08:20:37 +1000 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: <1189165259.16157.114.camel@cutter> References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> <1189165259.16157.114.camel@cutter> Message-ID: <20070907221314.M7794@npgx.com.au> Hi, > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > > > I would deeply appreciate any feedback on the samples and format > > > suggestions. > > > > /me thinks you're getting way too little feedback on the blog, > > or this list. Unfortunately I don't have much more than simple > > preference to contribute here.. > > > > XML-based format: > > > > Yikes, you'll need an additional human readable frontend > > format that's converted to XML for it to be usable. You > > can't expect us poor sysadmins to be editing XML > > directly to configure rsyslogd.. > > The nice piece of this is that it is machine parseable easily which > enables lots of useful editors. I don't agree with the original posters comment there also. As an example, I have been using linuxha.net now for quite some years on many clusters and from day one, linuxha.net has used XML for all it's configuration files. I personally find the "standard" that brings to config files much better than the myriad of conf files I've dealt with the many more years I've been using UNIX and Linux. > > syslog-ng like: > > > > Fair enough.. It works for my usage. > > The syntax is okay but at that point what distinguishes b/t syslog-ng > and rsyslog? I've personally never been a fan of the syntax used in this. Sure I know it now after years for admin work, but I remember the times I needed to learn it thoroughly, it wasn't as easy as other conf files. > > Apache like: > > > > Not sure I understand this.. Seems like a mix of option/value > > and xml'ish for some functionality. > > This one I'm more interested in. If you think of each log like a > vhost and you define the qualities that are added to that inside the > definition > > > Destination /path/to/silly.log > DestinationMode 0640 > DestinationOwner root > DestinationGroup log-readers > Include mail.info kern.debug cron.emerg > > > etc, etc, etc > > maybe that doesn't make sense, maybe it does - it is pretty easy to > read, though. I think every sysadmin has setup a web server and delved into the apache-like configurations with software like apache, proftpd, etc. It's a nice and easy to understand format which has also proved the test of time. I'd be happy with either XML or Apache-like, but my bias is towards XML. Regards, Michael. > -sv > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From theinric at redhat.com Sat Sep 8 22:07:30 2007 From: theinric at redhat.com (Tomas Heinrich) Date: Sat, 08 Sep 2007 22:07:30 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <46E30102.20508@redhat.com> Patrick von der Hagen wrote: > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: >> Hi all, >> >> Rsyslog 1.19.5 has been released. It is primarily targeted at fixing >> rare situations in which a segfault could occur. These have not >> consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. > >> have a problem or may hang where previous versions segfaulted. We are >> actively looking for feedback from the field. Feature-wise, the $ModDir >> config directive has been added and $ModLoad has been enhanced. We >> recommend upgrading only for those that experience the problem with >> earlier versions or those actively interested in helping to solve the >> bug. If you experience a bug, please report it. > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. Thanks for the info. It will be very useful if someone can provide a core dump for 1.19.5. From rgerhards at hq.adiscon.com Mon Sep 10 11:34:17 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 11:34:17 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27896C@grfint2.intern.adiscon.com> Hhmmm... Besides a core-dump (which would definitely be useful), could those of you experiencing this problem try to run rsyslog with debug code enabled? This is NOT debug mode (-d). You enable it via ./configure --enable-debug This will generate additional debug checks in the rsyslog code. The resulting code will probably run 5 to 10 times SLOWER than production code. But it will catch many obscure errors and at least provide a hint to where the problem is (at least I hope so). If you could do that, that would be great. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Friday, September 07, 2007 7:20 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.5 released > > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > > Hi all, > > > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > > rare situations in which a segfault could occur. These have not > > consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. > > > have a problem or may hang where previous versions segfaulted. We are > > actively looking for feedback from the field. Feature-wise, the > $ModDir > > config directive has been added and $ModLoad has been enhanced. We > > recommend upgrading only for those that experience the problem with > > earlier versions or those actively interested in helping to solve the > > bug. If you experience a bug, please report it. > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. > > > > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "P*\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7in-a"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "P*\205\200\0\1\0\1\0\6\0\7\00282\003185\00213\003129 > \7"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14623]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 165 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\236\222\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\236\222\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14099]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 243 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\377\251\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\377\251\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>spamd[16561]: spamd: identif"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 94 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, > "\202\24\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\202\24\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>spamd[16561]: spamd: result:"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 463 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\33\\\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\33\\\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[15976]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 143 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "I\27\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7in"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "I\27\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \003129"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "v\325\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "v\325\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > futex(0x3627949960, FUTEX_WAKE, 1) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "+\330\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "+\330\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<13>greylistd: Socket error: Bro"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 41 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "+\366\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "+\366\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\233A\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\233A\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14536]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 171 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "M\243\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "M\243\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = ? ERESTARTNOHAND (To be > restarted) > trace: ptrace(PTRACE_SYSCALL, ...): No such process > Process 22923 detached > > -- > CU, > Patrick. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Mon Sep 10 14:38:58 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Mon, 10 Sep 2007 14:38:58 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Freitag, den 07.09.2007, 19:20 +0200 schrieb Patrick von der Hagen: [...] > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. I compiled 1.19.5 with "--enable-debug" now and got the following problems: Sep 10 14:24:26 mail11 rsyslogd:could not load module '/usr/local/lib/rsyslog/ommysql.so', dlopen: /usr/local/lib/rsyslog/ommysql.so: cannot open shared object file: No such file or directory Sep 10 14:24:26 mail11 rsyslogd:the last error occured in /etc/rsyslog.conf, line 29 No idea why it tries "/usr/local/lib", the lib has been installed to "/lib" or "/opt/rsyslog/lib/rsyslog/". Hmmm, "/lib" is from 1.18.something, I didn't tidy up properly. Anyway, I currently don't use logging to MySQL, so I uncommented "$ModLoad MySQL". Next try: [root at mail11 log]# /opt/rsyslog/sbin/rsyslogd -r514 -d [...] 1084229952: Lone worker is running... 1084229952: Called fprintlog, logging to builtin-file (/home/local/log/all) 1084229952: programname filter 'rsyslogd' does not match 'spamd' Filter: check for property 'msg' (value ' prefork: child states: BBBBBBBIIBBBBBBB ') contains 'prefork: child states': TRUE 1084229952: Called fprintlog, logging to builtin-discardrsyslogd: omdiscard.c:70: doAction: Assertion `ppString != ((void *)0)' failed. Aborted That's getting a little bit strange now, it has been caused by this rsyslog.conf-lines: !spamd :msg, contains, "prefork: child states" ~ Some other crashes relate to other lines, all of them end with "~". So I uncommented all of them. Those problems are strange, but if those issues were related to my "normal" rsyslog-crashes, rsyslog would not have been able to run up to two hours and would certainly have crashed almost instantly. It's been running for several minutes now... -- CU, Patrick. From rgerhards at hq.adiscon.com Mon Sep 10 14:50:12 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 14:50:12 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com><1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278979@grfint2.intern.adiscon.com> I agree, it looks strange (very strange indeed). I am checking if it could be related to the root cause (or just be a debug-level artifact that slipped through - that may be the case, because discard is the only action which does NOT have any strings at all). Anyhow, it provides me at least another clue where to look at. Thanks Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Monday, September 10, 2007 2:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.5 released > > Am Freitag, den 07.09.2007, 19:20 +0200 schrieb Patrick von der Hagen: > [...] > > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > > quite "normal", however, I've been lucky and captured some > > strace-information. Not sure wheter that could be helpful. > I compiled 1.19.5 with "--enable-debug" now and got the following > problems: > > Sep 10 14:24:26 mail11 rsyslogd:could not load module > '/usr/local/lib/rsyslog/ommysql.so', > dlopen: /usr/local/lib/rsyslog/ommysql.so: cannot open shared object > file: No such file or directory > Sep 10 14:24:26 mail11 rsyslogd:the last error occured > in /etc/rsyslog.conf, line 29 > > No idea why it tries "/usr/local/lib", the lib has been installed to > "/lib" or "/opt/rsyslog/lib/rsyslog/". Hmmm, "/lib" is from > 1.18.something, I didn't tidy up properly. > > Anyway, I currently don't use logging to MySQL, so I uncommented > "$ModLoad MySQL". > > > Next try: > [root at mail11 log]# /opt/rsyslog/sbin/rsyslogd -r514 -d > [...] > 1084229952: Lone worker is running... > 1084229952: Called fprintlog, logging to builtin-file > (/home/local/log/all) > 1084229952: programname filter 'rsyslogd' does not match 'spamd' > Filter: check for property 'msg' (value ' prefork: child states: > BBBBBBBIIBBBBBBB ') contains 'prefork: child states': TRUE > 1084229952: Called fprintlog, logging to builtin-discardrsyslogd: > omdiscard.c:70: doAction: Assertion `ppString != ((void *)0)' failed. > Aborted > > > That's getting a little bit strange now, it has been caused by this > rsyslog.conf-lines: > !spamd > :msg, contains, "prefork: child states" ~ > > > Some other crashes relate to other lines, all of them end with "~". So > I > uncommented all of them. > > Those problems are strange, but if those issues were related to my > "normal" rsyslog-crashes, rsyslog would not have been able to run up to > two hours and would certainly have crashed almost instantly. > > It's been running for several minutes now... > > -- > CU, > Patrick. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Mon Sep 10 15:17:49 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Mon, 10 Sep 2007 15:17:49 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Montag, den 10.09.2007, 14:38 +0200 schrieb Patrick von der Hagen: [...] > It's been running for several minutes now... Now I captured the "real" crash. First, my config: $AllowedSender UDP, 1.2.3.0/24 $AllowedSender TCP, 1.2.3.0/24 $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/clamav" $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/exim" $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" *.* /home/local/log/all !rsyslogd :programname, contains, "rsyslogd" /home/local/log/rsyslogd !spamd mail.* /home/local/log/mail !clamd local5.* ?clamavFile !exim *.* ?eximFile :msg, contains, "malware detected" ?avFile Here the output of "rsyslog -r514 -d" Successful select, descriptor count = 1, Activity on: 1084229952: -1431504256: 8 Called fprintlog, logging to builtin-file1084229952: (eximFile) Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 1IUio2-00054R-EA H=X (Y) [1.2.3.4] Warning: X-Spam-Status: yes, hits=13.9, size=32842') contains 'malware detected': FALSE 1084229952: singleWorker: queue EMPTY, waiting for next message. -1431504256: Message from inetd socket: #8, host: mailin3 -1431504256: Message length: 200, File descriptor: 8. -1431504256: logmsg: mail.info<22>, flags 2, from 'mailin3', msg exim[18550]: 2007-09-10 14:56:03 H=(a.b.c.d) [2.3.4.5] F= temporarily rejected RCPT : greylisted. -1431504256: Message has legacy syslog format. -1431504256: HOSTNAME contains invalid characters, assuming it to be a TAG. -1431504256: EnqueueMsg signaled condition (0) -1431504256: 1084229952: Listening on UDP syslogd socket 7 (IPv6/port 514). -1431504256: Lone worker is running... 1084229952: Called fprintlog, logging to builtin-file (/home/local/log/all) Listening on UDP syslogd socket 8 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 8): 3 7 8 1084229952: programname filter 'rsyslogd' does not match 'exim' 1084229952: programname filter 'spamd' does not match 'exim' 1084229952: programname filter 'clamd' does not match 'exim' 1084229952: Called fprintlog, logging to builtin-file (eximFile) Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 H=(domain) [1.2.3.4] F= temporarily rejected RCPT : greylisted.') contains 'malware detected': FALSE 1084229952: singleWorker: queue EMPTY, waiting for next message. -1431504256: Successful select, descriptor count = 1, Activity on: 8 *** glibc detected *** /opt/rsyslog/sbin/rsyslogd: corrupted double-linked list: 0x00002aaaac001230 *** ======= Backtrace: ========= /lib64/libc.so.6[0x362766cb43] /lib64/libc.so.6[0x362766eea2] /lib64/libc.so.6(__libc_malloc+0x7d)[0x36276706dd] /lib64/libc.so.6[0x362765eb4a] /lib64/libnss_files.so.2[0x2aaaaaad445a] /lib64/libnss_files.so.2(_nss_files_gethostbyaddr_r +0x57)[0x2aaaaaad4b47] /lib64/libc.so.6(gethostbyaddr_r+0xf2)[0x36276e2b42] /lib64/libc.so.6(getnameinfo+0x3ad)[0x36276eb07d] /opt/rsyslog/sbin/rsyslogd(cvthname+0x154)[0x410e64] /opt/rsyslog/sbin/rsyslogd[0x40bec9] /opt/rsyslog/sbin/rsyslogd(main+0x630)[0x40c990] /lib64/libc.so.6(__libc_start_main+0xf4)[0x362761d8a4] /opt/rsyslog/sbin/rsyslogd[0x405899] ======= Memory map: ======== 00400000-00424000 r-xp 00000000 08:03 688189 /opt/rsyslog/sbin/rsyslogd 00624000-00626000 rw-p 00024000 08:03 688189 /opt/rsyslog/sbin/rsyslogd 1c204000-1c249000 rw-p 1c204000 00:00 0 40000000-40001000 ---p 40000000 00:00 0 40001000-40a01000 rw-p 40001000 00:00 0 3627200000-362721a000 r-xp 00000000 08:03 4260124 /lib64/ld-2.5.so 3627419000-362741a000 r--p 00019000 08:03 4260124 /lib64/ld-2.5.so 362741a000-362741b000 rw-p 0001a000 08:03 4260124 /lib64/ld-2.5.so 3627600000-3627744000 r-xp 00000000 08:03 4260125 /lib64/libc-2.5.so 3627744000-3627944000 ---p 00144000 08:03 4260125 /lib64/libc-2.5.so 3627944000-3627948000 r--p 00144000 08:03 4260125 /lib64/libc-2.5.so 3627948000-3627949000 rw-p 00148000 08:03 4260125 /lib64/libc-2.5.so 3627949000-362794e000 rw-p 3627949000 00:00 0 3627e00000-3627e02000 r-xp 00000000 08:03 4260128 /lib64/libdl-2.5.so 3627e02000-3628002000 ---p 00002000 08:03 4260128 /lib64/libdl-2.5.so 3628002000-3628003000 r--p 00002000 08:03 4260128 /lib64/libdl-2.5.so 3628003000-3628004000 rw-p 00003000 08:03 4260128 /lib64/libdl-2.5.so 3628200000-3628215000 r-xp 00000000 08:03 4260022 /lib64/libpthread-2.5.so 3628215000-3628414000 ---p 00015000 08:03 4260022 /lib64/libpthread-2.5.so 3628414000-3628415000 r--p 00014000 08:03 4260022 /lib64/libpthread-2.5.so 3628415000-3628416000 rw-p 00015000 08:03 4260022 /lib64/libpthread-2.5.so 3628416000-362841a000 rw-p 3628416000 00:00 0 3628600000-3628614000 r-xp 00000000 08:03 4547586 /usr/lib64/libz.so.1.2.3 3628614000-3628813000 ---p 00014000 08:03 4547586 /usr/lib64/libz.so.1.2.3 3628813000-3628814000 rw-p 00013000 08:03 4547586 /usr/lib64/libz.so.1.2.3 362d200000-362d207000 r-xp 00000000 08:03 4260133 /lib64/librt-2.5.so 362d207000-362d407000 ---p 00007000 08:03 4260133 /lib64/librt-2.5.so 362d407000-362d408000 r--p 00007000 08:03 4260133 /lib64/librt-2.5.so 362d408000-362d409000 rw-p 00008000 08:03 4260133 /lib64/librt-2.5.so 362ee00000-362ee11000 r-xp 00000000 08:03 4260134 /lib64/libresolv-2.5.so 362ee11000-362f011000 ---p 00011000 08:03 4260134 /lib64/libresolv-2.5.so 362f011000-362f012000 r--p 00011000 08:03 4260134 /lib64/libresolv-2.5.so 362f012000-362f013000 rw-p 00012000 08:03 4260134 /lib64/libresolv-2.5.so 362f013000-362f015000 rw-p 362f013000 00:00 0 3815400000-381540d000 r-xp 00000000 08:03 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 381540d000-381560c000 ---p 0000d000 08:03 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 381560c000-381560d000 rw-p 0000c000 08:03 4259862 /libAborted -- CU, Patrick. From rgerhards at hq.adiscon.com Mon Sep 10 18:26:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 18:26:10 +0200 Subject: [rsyslog] rsyslog segfaults was: rsyslog 1.19.5 released In-Reply-To: <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189441571.2659.56.camel@localhost.localdomain> Patrick, thanks for this report, it is useful. Unfortunately, I had hoped that an assertion failed, which was obviously not the case. I have looked at the code in question, but the root cause seems to be different. It looks like the code that actually aborted was "just" affected by something that happened earlier (the earlier code destroyed the list, which was then detected in cvthname() processing). The assert on discard was a bug in the debug code. I have fixed that in the current CVS (I do not yet want to release a version because of this minor thing - if you like, you can pull it from anon CVS). We have some other things under review. In the mean time, I am asking myself if the problem may be related to a threading issue. Rsyslog can be compiled in single-threading mode. You can do that by: ./configure --disable-pthreads [--enable-debug (if you like)] There is a drawback, though: as message processing and reception is no longer de-coupled, messages in bursts are more likely to be lost. Also, TCP messages sent to an unresponsive server may be lost, as the priority is on reception (there is code that discards messages that block rsyslgogd). However, sysklogd always works in that mode, so I think it is worth giving a try. It would be very helpful to know if the problem persists in single threading mode - or not. Thanks, Rainer On Mon, 2007-09-10 at 15:17 +0200, Patrick von der Hagen wrote: > Am Montag, den 10.09.2007, 14:38 +0200 schrieb Patrick von der Hagen: > [...] > > It's been running for several minutes now... > Now I captured the "real" crash. > > First, my config: > $AllowedSender UDP, 1.2.3.0/24 > $AllowedSender TCP, 1.2.3.0/24 > $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME > %/clamav" > $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME > %/exim" > $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" > *.* /home/local/log/all > !rsyslogd > :programname, contains, "rsyslogd" /home/local/log/rsyslogd > !spamd > mail.* /home/local/log/mail > !clamd > local5.* ?clamavFile > !exim > *.* ?eximFile > :msg, contains, "malware detected" ?avFile > > > Here the output of "rsyslog -r514 -d" > Successful select, descriptor count = 1, Activity on: 1084229952: > -1431504256: 8 > Called fprintlog, logging to builtin-file1084229952: (eximFile) > Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 > 1IUio2-00054R-EA H=X (Y) [1.2.3.4] Warning: X-Spam-Status: yes, > hits=13.9, size=32842') contains 'malware detected': FALSE > 1084229952: singleWorker: queue EMPTY, waiting for next message. > -1431504256: Message from inetd socket: #8, host: mailin3 > -1431504256: Message length: 200, File descriptor: 8. > -1431504256: logmsg: mail.info<22>, flags 2, from 'mailin3', msg > exim[18550]: 2007-09-10 14:56:03 H=(a.b.c.d) [2.3.4.5] F= > temporarily rejected RCPT : greylisted. > -1431504256: Message has legacy syslog format. > -1431504256: HOSTNAME contains invalid characters, assuming it to be a > TAG. > -1431504256: EnqueueMsg signaled condition (0) > -1431504256: 1084229952: Listening on UDP syslogd socket 7 (IPv6/port > 514). > -1431504256: Lone worker is running... > 1084229952: Called fprintlog, logging to builtin-file > (/home/local/log/all) > Listening on UDP syslogd socket 8 (IPv4/port 514). > -1431504256: ---------------------------------------- > -1431504256: Calling select, active file descriptors (max 8): 3 7 8 > 1084229952: programname filter 'rsyslogd' does not match 'exim' > 1084229952: programname filter 'spamd' does not match 'exim' > 1084229952: programname filter 'clamd' does not match 'exim' > 1084229952: Called fprintlog, logging to builtin-file (eximFile) > Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 H=(domain) > [1.2.3.4] F= temporarily rejected RCPT > : greylisted.') contains 'malware detected': FALSE > 1084229952: singleWorker: queue EMPTY, waiting for next message. > -1431504256: > Successful select, descriptor count = 1, Activity on: 8 > *** glibc detected *** /opt/rsyslog/sbin/rsyslogd: corrupted > double-linked list: 0x00002aaaac001230 *** > ======= Backtrace: ========= > /lib64/libc.so.6[0x362766cb43] > /lib64/libc.so.6[0x362766eea2] > /lib64/libc.so.6(__libc_malloc+0x7d)[0x36276706dd] > /lib64/libc.so.6[0x362765eb4a] > /lib64/libnss_files.so.2[0x2aaaaaad445a] > /lib64/libnss_files.so.2(_nss_files_gethostbyaddr_r > +0x57)[0x2aaaaaad4b47] > /lib64/libc.so.6(gethostbyaddr_r+0xf2)[0x36276e2b42] > /lib64/libc.so.6(getnameinfo+0x3ad)[0x36276eb07d] > /opt/rsyslog/sbin/rsyslogd(cvthname+0x154)[0x410e64] > /opt/rsyslog/sbin/rsyslogd[0x40bec9] > /opt/rsyslog/sbin/rsyslogd(main+0x630)[0x40c990] > /lib64/libc.so.6(__libc_start_main+0xf4)[0x362761d8a4] > /opt/rsyslog/sbin/rsyslogd[0x405899] > ======= Memory map: ======== > 00400000-00424000 r-xp 00000000 08:03 > 688189 /opt/rsyslog/sbin/rsyslogd > 00624000-00626000 rw-p 00024000 08:03 > 688189 /opt/rsyslog/sbin/rsyslogd > 1c204000-1c249000 rw-p 1c204000 00:00 0 > 40000000-40001000 ---p 40000000 00:00 0 > 40001000-40a01000 rw-p 40001000 00:00 0 > 3627200000-362721a000 r-xp 00000000 08:03 > 4260124 /lib64/ld-2.5.so > 3627419000-362741a000 r--p 00019000 08:03 > 4260124 /lib64/ld-2.5.so > 362741a000-362741b000 rw-p 0001a000 08:03 > 4260124 /lib64/ld-2.5.so > 3627600000-3627744000 r-xp 00000000 08:03 > 4260125 /lib64/libc-2.5.so > 3627744000-3627944000 ---p 00144000 08:03 > 4260125 /lib64/libc-2.5.so > 3627944000-3627948000 r--p 00144000 08:03 > 4260125 /lib64/libc-2.5.so > 3627948000-3627949000 rw-p 00148000 08:03 > 4260125 /lib64/libc-2.5.so > 3627949000-362794e000 rw-p 3627949000 00:00 0 > 3627e00000-3627e02000 r-xp 00000000 08:03 > 4260128 /lib64/libdl-2.5.so > 3627e02000-3628002000 ---p 00002000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628002000-3628003000 r--p 00002000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628003000-3628004000 rw-p 00003000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628200000-3628215000 r-xp 00000000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628215000-3628414000 ---p 00015000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628414000-3628415000 r--p 00014000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628415000-3628416000 rw-p 00015000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628416000-362841a000 rw-p 3628416000 00:00 0 > 3628600000-3628614000 r-xp 00000000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 3628614000-3628813000 ---p 00014000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 3628813000-3628814000 rw-p 00013000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 362d200000-362d207000 r-xp 00000000 08:03 > 4260133 /lib64/librt-2.5.so > 362d207000-362d407000 ---p 00007000 08:03 > 4260133 /lib64/librt-2.5.so > 362d407000-362d408000 r--p 00007000 08:03 > 4260133 /lib64/librt-2.5.so > 362d408000-362d409000 rw-p 00008000 08:03 > 4260133 /lib64/librt-2.5.so > 362ee00000-362ee11000 r-xp 00000000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362ee11000-362f011000 ---p 00011000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f011000-362f012000 r--p 00011000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f012000-362f013000 rw-p 00012000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f013000-362f015000 rw-p 362f013000 00:00 0 > 3815400000-381540d000 r-xp 00000000 08:03 > 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 > 381540d000-381560c000 ---p 0000d000 08:03 > 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 > 381560c000-381560d000 rw-p 0000c000 08:03 > 4259862 /libAborted > > > From janfrode at tanso.net Sun Sep 9 11:43:58 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Sep 2007 11:43:58 +0200 Subject: [rsyslog] rsyslog 1.19.5 released References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <46E30102.20508@redhat.com> Message-ID: On 2007-09-08, Tomas Heinrich wrote: > > Thanks for the info. It will be very useful if someone can provide a > core dump for 1.19.5. I haven't had the chance to upgrade yet, but any hints for how to get a core dump if it fails ? I've tried this in the init-script: ulimit -c unlimited cd /var/log/syslog echo -n $"Starting system logger (rsyslog): " daemon rsyslogd $SYSLOGD_OPTIONS but haven't gotten any core-dumps in any of the crashes I've had.. -jf From hagen at rz.uni-karlsruhe.de Tue Sep 11 10:25:08 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 11 Sep 2007 10:25:08 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <46E30102.20508@redhat.com> Message-ID: <1189499108.3872.5.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Sonntag, den 09.09.2007, 11:43 +0200 schrieb Jan-Frode Myklebust: > On 2007-09-08, Tomas Heinrich wrote: > > > > Thanks for the info. It will be very useful if someone can provide a > > core dump for 1.19.5. > > I haven't had the chance to upgrade yet, but any hints for how to get > a core dump if it fails ? I've tried this in the init-script: > > ulimit -c unlimited > cd /var/log/syslog > echo -n $"Starting system logger (rsyslog): " > daemon rsyslogd $SYSLOGD_OPTIONS > > but haven't gotten any core-dumps in any of the crashes I've had.. Hmmm. I did "ulimit -c 50000; /opt/rsyslog/sbin/rsyslogd -r514" yesterday evening and found a nice coredump this morning. I sent it to Rainer a minute ago. -- CU, Patrick. From rgerhards at hq.adiscon.com Tue Sep 11 17:09:38 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:09:38 +0200 Subject: [rsyslog] rsyslog 1.19.6 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899A@grfint2.intern.adiscon.com> Release 1.19.6 is a code cleanup and bug fixing release. It provides a number of fixes, cleans up compiler warnings and has some doc improvements. We are still investigating a problem that causes rsyslog to segfault in some constellations and are looking for active feedback. Those that help in this effort are kindly requested to update to this release, as it contains the latest code. The upgrade is recommended only for people who experience problems or would like to help with bugfixing. From rgerhards at hq.adiscon.com Tue Sep 11 17:15:32 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:15:32 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Hi all, I accidently hit the send button, so my release note for 1.19.6 already went on to the list. Anyhow, I guess you can find the relevant links yourself ;) But one important thing: I would appreciate if those of you that experienced the segfaults could try out the new version. I currently think it will *NOT* fix the problem, but I would like to verify that (and I have to admit I am not angry if I am wrong...). If it fails, it would be even greater if you could try it in single-threaded mode: ./configure --disable-pthreads I currently think the problem might be related to a threading and would like to see if that theory has some substance. Also, a test with ./configure --enable-debug Would be useful - but I have not added any asserts so if you already did this, there is no value in re-doing it with 1.9.6. Thanks for all your help. Looking forward to hear back from you. Rainer From rgerhards at hq.adiscon.com Tue Sep 11 17:46:54 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:46:54 +0200 Subject: [rsyslog] 1.19.6 updated Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> Just for your info: I detected one bug that occurred during bug fixing. I fixed it and re-published 1.19.6 with the same version number because I saw now downloads until then. But maybe my counter is wrong. If you have downloaded rsyslog before I sent this message, please re-download it. Sorry for the hassle, but that fix was probably worth it. Thanks, Rainer From infofarmer at FreeBSD.org Tue Sep 11 18:04:40 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Tue, 11 Sep 2007 20:04:40 +0400 Subject: [rsyslog] 1.19.6 updated In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> Message-ID: <20070911160439.GB83726@amilo.cenkes.org> On Tue, Sep 11, 2007 at 05:46:54PM +0200, Rainer Gerhards wrote: > Just for your info: I detected one bug that occurred during bug fixing. > I fixed it and re-published 1.19.6 with the same version number because > I saw now downloads until then. But maybe my counter is wrong. If you > have downloaded rsyslog before I sent this message, please re-download > it. > > Sorry for the hassle, but that fix was probably worth it. I had to run 's/sigaction_t/sigaction/' over rfc3195d.c in order to compile it on FreeBSD. Otherwise it seems to work. Thanks! From rgerhards at hq.adiscon.com Tue Sep 11 18:40:12 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 18:40:12 +0200 Subject: [rsyslog] 1.19.6 updated In-Reply-To: <20070911160439.GB83726@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> <20070911160439.GB83726@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789A1@grfint2.intern.adiscon.com> Ahhh... I have to admit that I currently do not really care about rfc3195d. Currently, RFC 3195 is a failure - nobody uses it and nobody asks for it. I've stopped testing it until I receive at least one real-world implementation note. Thanks for the info, will apply a patch. And, yes, I do not think that 3195 is totally dead. There is a new revision planned, and that may be very interesting. This is why I am not ditching it... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Tuesday, September 11, 2007 6:05 PM > To: rsyslog-users > Subject: Re: [rsyslog] 1.19.6 updated > > On Tue, Sep 11, 2007 at 05:46:54PM +0200, Rainer Gerhards wrote: > > Just for your info: I detected one bug that occurred during bug > fixing. > > I fixed it and re-published 1.19.6 with the same version number > because > > I saw now downloads until then. But maybe my counter is wrong. If you > > have downloaded rsyslog before I sent this message, please re- > download > > it. > > > > Sorry for the hassle, but that fix was probably worth it. > > I had to run 's/sigaction_t/sigaction/' over rfc3195d.c in order > to compile it on FreeBSD. Otherwise it seems to work. Thanks! > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Thu Sep 13 12:31:50 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 12:31:50 +0200 Subject: [rsyslog] some logged lines end with #000 Message-ID: <46E91196.301@ipax.at> hello, today i merged michael biebls debian package source rsyslog_1.19.3-1 with the current upstream rsyslog-1.19.6.tar.gz. the compilation went smooth and i installed the package on a host where there used to by klogd/sysklogd under debian etch. however, i noticed that now a lot of logged lines end with #000. it used to be: > Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. Starting heartbeat 2.1.2 > Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 > Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: 193 > ... > Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 URL:http://localhost:80/server-status [1735/1735] -> "-" [1] it now is: > Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. Starting heartbeat 2.1.2#000 > Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version 2.1.2#000 > Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: 194#000 > ... > Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 URL:http://localhost:80/server-status [1728/1728] -> "-" [1] > Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 URL:http://localhost:80/server-status [1727/1727] -> "-" [1] you will find my configuration file attached. do you have any ideas what might cause this behaviour? cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: etc_default_rsyslog URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: rsyslog.conf URL: From theinric at redhat.com Thu Sep 13 15:01:09 2007 From: theinric at redhat.com (Tomas Heinrich) Date: Thu, 13 Sep 2007 15:01:09 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E91196.301@ipax.at> References: <46E91196.301@ipax.at> Message-ID: <46E93495.8010603@redhat.com> Raoul Bhatia [IPAX] wrote: > hello, > > today i merged michael biebls debian package source rsyslog_1.19.3-1 > with the current upstream rsyslog-1.19.6.tar.gz. > > the compilation went smooth and i installed the package on a host where > there used to by klogd/sysklogd under debian etch. > > however, i noticed that now a lot of logged lines end with #000. > > it used to be: >> Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. >> Starting heartbeat 2.1.2 >> Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 >> Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: 193 >> ... >> Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 >> URL:http://localhost:80/server-status [1735/1735] -> "-" [1] >> Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 >> URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > it now is: >> Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. >> Starting heartbeat 2.1.2#000 >> Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version 2.1.2#000 >> Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: >> 194#000 >> ... >> Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 >> URL:http://localhost:80/server-status [1728/1728] -> "-" [1] >> Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 >> URL:http://localhost:80/server-status [1727/1727] -> "-" [1] > > you will find my configuration file attached. > do you have any ideas what might cause this behaviour? > > cheers, > raoul bhatia > > > ------------------------------------------------------------------------ > > # Options to rsyslogd > # -m 0 disables 'MARK' messages. > # -r enables logging from remote machines > # -x disables DNS lookups on messages recieved with -r > # See rsyslogd(8) for more details > RSYSLOGD_OPTIONS="-m 0" > > # Options to rklogd > # -2 prints all kernel oops messages twice; once for klogd to decode, and > # once for processing with 'ksymoops' > # -x disables all klogd processing of oops messages entirely > # See rklogd(8) for more details > RKLOGD_OPTIONS="-x" > > > > ------------------------------------------------------------------------ > > # /etc/rsyslog.conf Configuration file for rsyslogd. > # > # For more information see > # /usr/share/doc/rsyslog/html/rsyslog_conf.html > > # > # First some standard logfiles. Log by facility. > # > > auth,authpriv.* /var/log/auth.log > *.*;auth,authpriv.none -/var/log/syslog > #cron.* /var/log/cron.log > daemon.* -/var/log/daemon.log > kern.* -/var/log/kern.log > lpr.* -/var/log/lpr.log > mail.* -/var/log/mail.log > user.* -/var/log/user.log > > # > # Logging for the mail system. Split it up so that > # it is easy to write scripts to parse these files. > # > mail.info -/var/log/mail.info > mail.warn -/var/log/mail.warn > mail.err /var/log/mail.err > > # > # Logging for INN news system > # > news.crit /var/log/news/news.crit > news.err /var/log/news/news.err > news.notice -/var/log/news/news.notice > > # > # Some `catch-all' logfiles. > # > *.=debug;\ > auth,authpriv.none;\ > news.none;mail.none -/var/log/debug > *.=info;*.=notice;*.=warn;\ > auth,authpriv.none;\ > cron,daemon.none;\ > mail,news.none -/var/log/messages > > # > # Emergencies are sent to everybody logged in. > # > *.emerg * > > # > # I like to have messages displayed on the console, but only on a virtual > # console I usually leave idle. > # > #daemon,mail.*;\ > # news.=crit;news.=err;news.=notice;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn /dev/tty8 > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, > # you must invoke `xconsole' with the `-file' option: > # > # $ xconsole -file /dev/xconsole [...] > # > # NOTE: adjust the list below, or you'll go crazy if you have a reasonably > # busy site.. > # > daemon.*;mail.*;\ > news.err;\ > *.=debug;*.=info;\ > *.=notice;*.=warn |/dev/xconsole > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog Hi, I've tinkered with this part of code recently, so it could be my fault. But I wasn't able to reproduce it so far and a quick look through the source didn't show anything suspicious. Just a blind guess: $EscapeControlCharactersOnReceive off It will prevent control chars from being escaped but maybe the problem will go away for now. You have a $IncludeConfig /etc/rsyslog.d/ in you conf file; is there any other part of configuration? I will look deeper into this later today. From rgerhards at hq.adiscon.com Thu Sep 13 15:48:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Sep 2007 15:48:02 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E91196.301@ipax.at> References: <46E91196.301@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> I remember the #000. I think in the case that I remember, there was actually a NUL character written by the remote system. It may also be that a too-long message (off by one) is written in the output code. Maybe this is even a clue to the bug we are hunting. You can remove them by turning off control character escaping - but that just a cosmetic fix, it doesn't fix the root issue, which we will need to look at. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Thursday, September 13, 2007 12:32 PM > To: rsyslog-users > Subject: [rsyslog] some logged lines end with #000 > > hello, > > today i merged michael biebls debian package source rsyslog_1.19.3-1 > with the current upstream rsyslog-1.19.6.tar.gz. > > the compilation went smooth and i installed the package on a host where > there used to by klogd/sysklogd under debian etch. > > however, i noticed that now a lot of logged lines end with #000. > > it used to be: > > Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. > Starting heartbeat 2.1.2 > > Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 > > Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: > 193 > > ... > > Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 > URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 > URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > it now is: > > Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. > Starting heartbeat 2.1.2#000 > > Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version > 2.1.2#000 > > Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: > 194#000 > > ... > > Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 > URL:http://localhost:80/server-status [1728/1728] -> "-" [1] > > Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 > URL:http://localhost:80/server-status [1727/1727] -> "-" [1] > > you will find my configuration file attached. > do you have any ideas what might cause this behaviour? > > cheers, > raoul bhatia > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ From r.bhatia at ipax.at Thu Sep 13 16:16:39 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 16:16:39 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> Message-ID: <46E94647.1090709@ipax.at> Rainer Gerhards wrote: > I remember the #000. I think in the case that I remember, there was > actually a NUL character written by the remote system. It may also be > that a too-long message (off by one) is written in the output code. > Maybe this is even a clue to the bug we are hunting. > > You can remove them by turning off control character escaping - but that > just a cosmetic fix, it doesn't fix the root issue, which we will need > to look at. so then please tell me how to help :) cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From r.bhatia at ipax.at Thu Sep 13 16:33:26 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 16:33:26 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E93495.8010603@redhat.com> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> Message-ID: <46E94A36.5050701@ipax.at> Tomas Heinrich wrote: > Hi, > > I've tinkered with this part of code recently, so it could be my fault. > > But I wasn't able to reproduce it so far and a quick look through the > source didn't show anything suspicious. > Just a blind guess: > $EscapeControlCharactersOnReceive off > It will prevent control chars from being escaped but maybe the problem > will go away for now. this does not seem to help. > You have a > $IncludeConfig /etc/rsyslog.d/ > in you conf file; is there any other part of configuration? no, there aren't any configuration files in there. see: > root at xxx /var/log # find /etc/rsyslog.d -type f > root at xxx /var/log # raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Thu Sep 13 17:31:01 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Sep 2007 17:31:01 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E94A36.5050701@ipax.at> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Thursday, September 13, 2007 4:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > Tomas Heinrich wrote: > > Hi, > > > > I've tinkered with this part of code recently, so it could be my > fault. > > > > But I wasn't able to reproduce it so far and a quick look through the > > source didn't show anything suspicious. > > Just a blind guess: > > $EscapeControlCharactersOnReceive off > > It will prevent control chars from being escaped but maybe the > problem > > will go away for now. > > this does not seem to help. Ummm... should have thought about this. NULs are treated special. This is routed in the sysklogd code. Rsyslog uses standard C strings for the most part. Standard C strings are terminated as soon as the first NUL is detected. So we can not allow NULs to be part of the message. Thus, NUL is always escaped, no matter what the control character escape sequences is. Can you provide us with a debug log (rsyslogd -d -n) showing a message with #000 being processed? Rainer > > > You have a > > $IncludeConfig /etc/rsyslog.d/ > > in you conf file; is there any other part of configuration? > > no, there aren't any configuration files in there. see: > > root at xxx /var/log # find /etc/rsyslog.d -type f > > root at xxx /var/log # > > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Fri Sep 14 10:06:33 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 14 Sep 2007 10:06:33 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> Message-ID: <46EA4109.3060009@ipax.at> Rainer Gerhards wrote: > Ummm... should have thought about this. NULs are treated special. This > is routed in the sysklogd code. Rsyslog uses standard C strings for the > most part. Standard C strings are terminated as soon as the first NUL is > detected. So we can not allow NULs to be part of the message. Thus, NUL > is always escaped, no matter what the control character escape sequences > is. > > Can you provide us with a debug log (rsyslogd -d -n) showing a message > with #000 being processed? attached, you will find the output. cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Sep 14 10:17:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Sep 2007 10:17:10 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46EA4109.3060009@ipax.at> References: <46E91196.301@ipax.at><46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at><577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> <46EA4109.3060009@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> The list server is quite picky on attachment types (aka "does not accept most of them" ;)). And I have to admit I introduced that policy and still think it is not a bad one. Can you send me the attachment via private mail please. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Friday, September 14, 2007 10:07 AM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > Rainer Gerhards wrote: > > Ummm... should have thought about this. NULs are treated special. > This > > is routed in the sysklogd code. Rsyslog uses standard C strings for > the > > most part. Standard C strings are terminated as soon as the first NUL > is > > detected. So we can not allow NULs to be part of the message. Thus, > NUL > > is always escaped, no matter what the control character escape > sequences > > is. > > > > Can you provide us with a debug log (rsyslogd -d -n) showing a > message > > with #000 being processed? > > attached, you will find the output. > > cheers, > raoul bhatia > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Sep 14 11:15:35 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Sep 2007 11:15:35 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at><46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at><577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com><46EA4109.3060009@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789E1@grfint2.intern.adiscon.com> I have created a new version which is currently available under the interim name of http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz [***This is NOT 1.19.7 but will become it over time!***] In that version, I have handled NULs that are emitted by faulty senders (and there seem to be lots of them). A quick test both by me and Raoul Bhatia indicates that those NULs disappear. And so do LF NUL (#012#000) as the LF shown is an artifact from the NUL being then the last char of the message). I would now be interested to know if somebody still sees unexpected #000 at the very end of the message. If there are, they now almost for sure identify a rsyslog problem. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, September 14, 2007 10:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > The list server is quite picky on attachment types (aka "does not > accept > most of them" ;)). And I have to admit I introduced that policy and > still think it is not a bad one. > > Can you send me the attachment via private mail please. > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > > Sent: Friday, September 14, 2007 10:07 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] some logged lines end with #000 > > > > Rainer Gerhards wrote: > > > Ummm... should have thought about this. NULs are treated special. > > This > > > is routed in the sysklogd code. Rsyslog uses standard C strings for > > the > > > most part. Standard C strings are terminated as soon as the first > NUL > > is > > > detected. So we can not allow NULs to be part of the message. Thus, > > NUL > > > is always escaped, no matter what the control character escape > > sequences > > > is. > > > > > > Can you provide us with a debug log (rsyslogd -d -n) showing a > > message > > > with #000 being processed? > > > > attached, you will find the output. > > > > cheers, > > raoul bhatia > > -- > > ____________________________________________________________________ > > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > > Technischer Leiter > > > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > > Barawitzkagasse 10/2/2/11 email. office at ipax.at > > 1190 Wien tel. +43 1 3670030 > > FN 277995t HG Wien fax. +43 1 3670030 15 > > ____________________________________________________________________ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 17 14:51:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 14:51:10 +0200 Subject: [rsyslog] rsyslog segfaults? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> Hi all, may I ask if those of you that experienced the segfault problem could re-produce it with 1.19.6? Any feedback would be deeply appreciated. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Sep 17 18:21:09 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 18:21:09 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <1190044055.10861.7.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <1190044055.10861.7.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A19@grfint2.intern.adiscon.com> Patrick, without answering the question directly: could you please try out http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz [**still not "real" 1.19.7 **] I've possibly seen one thing that could be the problem cause. Not sure, though. I'd deeply appreciate feedback if this version (uploaded right NOW) causes the problem to disappear. I am not sure if I have found it, but I'd like to conduct a quick test before going any further. Thus I've no detailed analysis. Thanks, Rainer > -----Original Message----- > From: Patrick von der Hagen [mailto:hagen at rz.uni-karlsruhe.de] > Sent: Monday, September 17, 2007 5:48 PM > To: Rainer Gerhards > Cc: theinric > Subject: Re: [rsyslog] rsyslog segfaults? > > On Mon, 2007-09-17 at 14:51 +0200, Rainer Gerhards wrote: > > Hi all, > > > > may I ask if those of you that experienced the segfault problem could > > re-produce it with 1.19.6? Any feedback would be deeply appreciated. > Still dumps core, haven't tried it single-threaded yet. > > I attach some gdb information I got when examining a core-dump. > > I'm no programmer and don't even understand half of it, but I'm > troubled > by ' ip = "129.13.185.81\000\000 > $a196a97a at akstcacsdatarecoverymnsdgs>,bayes=1.000000,autolearn=spam > \00090$88ab8c55 at 082d6f5a>,bayes=1.000000,autolearn=spam \000n=spam > \000id=<01c7f6df$57aeec90$6a49cd18 at 0aconingham>,bayes=1.000"'. > > It looks like several log-lines merged into one, which might hint at > threading-issues. > > > > [root at mail11 ~]# cd /opt/rsyslog > [root at mail11 rsyslog]# ls > core.14004 lib sbin share > [root at mail11 rsyslog]# gdb -c core.14004 /opt/rsyslog/sbin/rsyslogd > GNU gdb Red Hat Linux (6.5-16.el5rh) > Copyright (C) 2006 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "x86_64-redhat-linux-gnu"...Using host > libthread_db library "/lib64/libthread_db.so.1". > > Reading symbols from /usr/lib64/libz.so.1...done. > Loaded symbols for /usr/lib64/libz.so.1 > Reading symbols from /lib64/libpthread.so.0...done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /lib64/libdl.so.2...done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/librt.so.1...done. > Loaded symbols for /lib64/librt.so.1 > Reading symbols from /lib64/libc.so.6...done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /lib64/ld-linux-x86-64.so.2...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libnss_files.so.2...done. > Loaded symbols for /lib64/libnss_files.so.2 > Reading symbols from /lib64/libnss_dns.so.2...done. > Loaded symbols for /lib64/libnss_dns.so.2 > Reading symbols from /lib64/libresolv.so.2...done. > Loaded symbols for /lib64/libresolv.so.2 > Reading symbols from /lib64/libgcc_s.so.1...done. > Loaded symbols for /lib64/libgcc_s.so.1 > Core was generated by `/opt/rsyslog/sbin/rsyslogd -r514'. > Program terminated with signal 6, Aborted. > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > (gdb) backtrace > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > #1 0x0000003627631980 in abort () from /lib64/libc.so.6 > #2 0x00000036276674db in __libc_message () from /lib64/libc.so.6 > #3 0x000000362766cb43 in malloc_consolidate () from /lib64/libc.so.6 > #4 0x000000362766eea2 in _int_malloc () from /lib64/libc.so.6 > #5 0x00000036276706dd in malloc () from /lib64/libc.so.6 > #6 0x000000362765eb4a in __fopen_internal () from /lib64/libc.so.6 > #7 0x00002aaaaaaf545a in internal_setent () > from /lib64/libnss_files.so.2 > #8 0x00002aaaaaaf5b47 in _nss_files_gethostbyaddr_r () > from /lib64/libnss_files.so.2 > #9 0x00000036276e2b42 in gethostbyaddr_r@@GLIBC_2.2.5 () > from /lib64/libc.so.6 > #10 0x00000036276eb07d in getnameinfo () from /lib64/libc.so.6 > #11 0x00000000004155ee in cvthname (f=0x7ffff040b0d0, > pszHost=0x7ffff040acc0 "mailin3", > pszHostFQDN=0x7ffff040a8b0 "mailin3.rz.uni-karlsruhe.de") at > net.c:137 > #12 0x000000000040e49d in processSelectAfter (maxfds=5, nfds=1, > pReadfds=0x7ffff040ba60, pWritefds=0x7ffff040b9c0) at > syslogd.c:5619 > #13 0x000000000040ee02 in mainloop () at syslogd.c:5869 > #14 0x000000000040fc9c in main (argc=0, argv=0x7ffff040bd08) at > syslogd.c:6315 > (gdb) where full > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > No symbol table info available. > #1 0x0000003627631980 in abort () from /lib64/libc.so.6 > No symbol table info available. > #2 0x00000036276674db in __libc_message () from /lib64/libc.so.6 > No symbol table info available. > #3 0x000000362766cb43 in malloc_consolidate () from /lib64/libc.so.6 > No symbol table info available. > #4 0x000000362766eea2 in _int_malloc () from /lib64/libc.so.6 > No symbol table info available. > #5 0x00000036276706dd in malloc () from /lib64/libc.so.6 > No symbol table info available. > #6 0x000000362765eb4a in __fopen_internal () from /lib64/libc.so.6 > No symbol table info available. > #7 0x00002aaaaaaf545a in internal_setent () > from /lib64/libnss_files.so.2 > No symbol table info available. > #8 0x00002aaaaaaf5b47 in _nss_files_gethostbyaddr_r () > from /lib64/libnss_files.so.2 > No symbol table info available. > #9 0x00000036276e2b42 in gethostbyaddr_r@@GLIBC_2.2.5 () > from /lib64/libc.so.6 > No symbol table info available. > #10 0x00000036276eb07d in getnameinfo () from /lib64/libc.so.6 > No symbol table info available. > ---Type to continue, or q to quit--- > #11 0x00000000004155ee in cvthname (f=0x7ffff040b0d0, > pszHost=0x7ffff040acc0 "mailin3", > pszHostFQDN=0x7ffff040a8b0 "mailin3.rz.uni-karlsruhe.de") at > net.c:137 > p = (uchar *) 0x7ffff040acc7 "" > count = -264195888 > error = 0 > omask = {__val = {0, 4251742, 0, 140737224155248, > 140737224155240, > 14083552, 0, 18446735427676189008, 140737224155264, 4336648, > 140737224155264, 232601472906, 0, 64, 140737224159584, 2047}} > nmask = {__val = {1, 0 }} > ip = "129.13.185.81\000\000 > $a196a97a at akstcacsdatarecoverymnsdgs>,bayes=1.000000,autolearn=spam > \00090$88ab8c55 at 082d6f5a>,bayes=1.000000,autolearn=spam \000n=spam > \000id=<01c7f6df$57aeec90$6a49cd18 at 0aconingham>,bayes=1.000"... > hints = {ai_flags = 4, ai_family = 0, ai_socktype = 2, > ai_protocol = 0, ai_addrlen = 0, ai_addr = 0x0, ai_canonname = 0x0, > ai_next = 0x0} > res = (struct addrinfo *) 0x2e302e3732313d72 > __PRETTY_FUNCTION__ = "cvthname" > #12 0x000000000040e49d in processSelectAfter (maxfds=5, nfds=1, > pReadfds=0x7ffff040ba60, pWritefds=0x7ffff040b9c0) at > syslogd.c:5619 > iRet = RS_RET_OK > iRetLL = RS_RET_OK > i = 1 > ---Type to continue, or q to quit--- > fd = 0 > line = "<22>exim[32680]: 2007-09-14 17:02:19 1IWCgS-0008Tb-B5 > Completed\n", '\0' > writeFDSInfo = {pWritefds = 0x7ffff040b9c0, pMaxfds = > 0x7ffff040a0a8} > f = (selector_t *) 0x0 > frominet = {ss_family = 2, __ss_align = 0, > __ss_padding = "@\023\000??*\000\000@\023\000??*\000\000@\023\000??* > \000\000B\023\000??*\000\000O\023\000??*\000\000@\023\000??*\000\000O > \023\000??*", '\0' , "\017\201B\000\000\000\000\000? > \230\aF\000\000\000"} > socklen = 16 > fromHost = "mailin3\000rz.uni-karlsruhe.de", '\0' times>, "\210_ '6", '\0' , "?? '6", '\0' times>, "??A'6\000\000\000o\b?'6\000\000\000\220{ '6\000\000\000\000 > \000`'6\000\000\000\000 at t'6\000\000\000\220:t'6\000\000\000\220:t'6", > '\0' , "\005\000\000\000\000\000\000\000\000@\224'6", > '\0' , "\001\000\000\000\000\000\000\000\230?\224'6 > \000\000\000\000@\024\000\000\000\000\000\003\000\000\000\000\000\000 > \000\000\000 -6\000\000\000\000p -6\000\000"... > fromHostFQDN = "mailin3.rz.uni-karlsruhe.de", '\0' times>, "\001\000\000\000\000\000\000\000?@??\177\000\000?*q'6", '\0' > , "??e'6", '\0' , "??@??\177\000 > \000/\201B\000\000\000\000\000/\201B", '\0' , > "?4d'6", > '\0' , "\200?@??\177\000\000\000\000\000\000\000\000 > \000\000??@??\177\000\0000?@??\177\000\000*\201B\000\000\000\000\000?@? > ? > \177", '\0' , "????????\000\000\000---Type > to > continue, or q to quit--- > \000\002\000\000\000*"... > iTCPSess = 0 > l = 64 > #13 0x000000000040ee02 in mainloop () at syslogd.c:5869 > readfds = {fds_bits = {32, 0 }} > i = 2 > maxfds = 5 > nfds = 1 > writeFDSInfo = {pWritefds = 0x7ffff040b9c0, pMaxfds = > 0x7ffff040ba5c} > writefds = {fds_bits = {0 }} > f = (selector_t *) 0x0 > iTCPSess = 14003 > #14 0x000000000040fc9c in main (argc=0, argv=0x7ffff040bd08) at > syslogd.c:6315 > i = 1024 > p = 0x63053a "" > num_fds = 1024 > iRet = RS_RET_OK > ppid = 14003 > ch = -1 > hent = (struct hostent *) 0x362794bf80 > pTmp = (uchar *) 0x62fe95 "" > sigAct = {__sigaction_handler = {sa_handler = 0x1, > sa_sigaction = 0x1}, sa_mask = {__val = {0 }}, > ---Type to continue, or q to quit--- > sa_flags = 0, sa_restorer = 0} > (gdb) > > > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- > Patrick von der Hagen > RZ Universit?t Karlsruhe (TH) > Postmaster From rgerhards at hq.adiscon.com Mon Sep 17 18:27:38 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 18:27:38 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <1190046420.10861.9.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <1190044055.10861.7.camel@localhost.localdomain> <577465F99B41C842AAFBE9ED71E70ABA278A19@grfint2.intern.adiscon.com> <1190046420.10861.9.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A1B@grfint2.intern.adiscon.com> Excellent, thx > -----Original Message----- > From: Patrick von der Hagen [mailto:hagen at rz.uni-karlsruhe.de] > Sent: Monday, September 17, 2007 6:27 PM > To: Rainer Gerhards > Cc: rsyslog-users > Subject: RE: [rsyslog] rsyslog segfaults? > > On Mon, 2007-09-17 at 18:21 +0200, Rainer Gerhards wrote: > > Patrick, > > > > without answering the question directly: could you please try out > > > > http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz > > > > [**still not "real" 1.19.7 **] > > > > I've possibly seen one thing that could be the problem cause. Not > sure, though. I'd deeply appreciate feedback if this version (uploaded > right NOW) causes the problem to disappear. I am not sure if I have > found it, but I'd like to conduct a quick test before going any > further. Thus I've no detailed analysis. > I'm currently giving it a try, but don't expect any results today. > -- > Patrick von der Hagen > RZ Universit?t Karlsruhe (TH) > Postmaster From infofarmer at FreeBSD.org Wed Sep 19 10:55:19 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Wed, 19 Sep 2007 12:55:19 +0400 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> Message-ID: <20070919085517.GL98847@amilo.cenkes.org> On Mon, Sep 17, 2007 at 02:51:10PM +0200, Rainer Gerhards wrote: > Hi all, > > may I ask if those of you that experienced the segfault problem could > re-produce it with 1.19.6? Any feedback would be deeply appreciated. The box is in production, so I can't do too much debugging. If it may help, I can recompile rsyslogd with debug flags set and run gdb against it. root at apollo# rsyslogd -v /usr/home/mob/www rsyslogd 1.19.6, compiled with: FEATURE_PTHREADS (dual-threading): Yes FEATURE_REGEXP: Yes FEATURE_DB (MySQL): Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes SYSLOG_INET (Internet/remote support): Yes sat at apollo% gdb =rsyslogd rsyslogd.core <...> #0 0x000000080078189c in pthread_testcancel () from /lib/libpthread.so.2 [New Thread 0x537000 (LWP 100117)] [New Thread 0x52e400 (LWP 100112)] [New Thread 0x52e000 (runnable)] (gdb) bt #0 0x000000080078189c in pthread_testcancel () from /lib/libpthread.so.2 #1 0x000000080076f5c3 in sigaction () from /lib/libpthread.so.2 #2 0x00000008007710e2 in sigaction () from /lib/libpthread.so.2 #3 0x000000080076adb6 in pthread_kill () from /lib/libpthread.so.2 #4 0x000000080076a633 in raise () from /lib/libpthread.so.2 #5 0x000000080095b16d in abort () from /lib/libc.so.6 #6 0x00000008008f4b45 in _UTF8_init () from /lib/libc.so.6 #7 0x00000008008f4b7c in _UTF8_init () from /lib/libc.so.6 #8 0x00000008008f5b1d in _UTF8_init () from /lib/libc.so.6 #9 0x000000000040fa1d in MsgDestruct () #10 0x00000000004066bd in dbgprintf () #11 0x00000000004176b1 in llExecFunc () #12 0x0000000000406f41 in shouldProcessThisMessage () #13 0x0000000000409a9e in logerrorSz () #14 0x0000000800772a99 in pthread_create () from /lib/libpthread.so.2 #15 0x00000008008cfcd4 in makecontext () from /lib/libc.so.6 #16 0x0000000000000000 in ?? () #17 0x0000000000537000 in ?? () #18 0x00000000004099c0 in logerrorSz () #19 0x0000000000000000 in ?? () #20 0x0000000000000000 in ?? () #21 0x0000000000000000 in ?? () #22 0x0000000000000000 in ?? () Cannot access memory at address 0x7fffff5fc000 From rgerhards at hq.adiscon.com Wed Sep 19 13:53:41 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 19 Sep 2007 13:53:41 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <20070919085517.GL98847@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <20070919085517.GL98847@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A3D@grfint2.intern.adiscon.com> Andrew, this is quite helpful. While it does not pinpoint the actual trouble source, it points to a different abort location. All previous segfaults pointed to a different location (Which I have now checked multiple times and not found any notable problems, just cosmetic things). This strengthens the point it may be related to threading - of course, problematic to troubleshoot. I'll see how I proceed from here. Again, if some of you that experience the problem could run rsyslogd compiled for single threading, that would be most helpful. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Wednesday, September 19, 2007 10:55 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog segfaults? > > On Mon, Sep 17, 2007 at 02:51:10PM +0200, Rainer Gerhards wrote: > > Hi all, > > > > may I ask if those of you that experienced the segfault problem could > > re-produce it with 1.19.6? Any feedback would be deeply appreciated. > > The box is in production, so I can't do too much debugging. If it > may help, I can recompile rsyslogd with debug flags set and run > gdb against it. > > root at apollo# rsyslogd -v > /usr/home/mob/www > rsyslogd 1.19.6, compiled with: > FEATURE_PTHREADS (dual-threading): Yes > FEATURE_REGEXP: Yes > FEATURE_DB (MySQL): Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > SYSLOG_INET (Internet/remote support): Yes > > sat at apollo% gdb =rsyslogd rsyslogd.core > <...> > #0 0x000000080078189c in pthread_testcancel () from > /lib/libpthread.so.2 > [New Thread 0x537000 (LWP 100117)] > [New Thread 0x52e400 (LWP 100112)] > [New Thread 0x52e000 (runnable)] > (gdb) bt > #0 0x000000080078189c in pthread_testcancel () from > /lib/libpthread.so.2 > #1 0x000000080076f5c3 in sigaction () from /lib/libpthread.so.2 > #2 0x00000008007710e2 in sigaction () from /lib/libpthread.so.2 > #3 0x000000080076adb6 in pthread_kill () from > /lib/libpthread.so.2 > #4 0x000000080076a633 in raise () from /lib/libpthread.so.2 > #5 0x000000080095b16d in abort () from /lib/libc.so.6 > #6 0x00000008008f4b45 in _UTF8_init () from /lib/libc.so.6 > #7 0x00000008008f4b7c in _UTF8_init () from /lib/libc.so.6 > #8 0x00000008008f5b1d in _UTF8_init () from /lib/libc.so.6 > #9 0x000000000040fa1d in MsgDestruct () > #10 0x00000000004066bd in dbgprintf () > #11 0x00000000004176b1 in llExecFunc () > #12 0x0000000000406f41 in shouldProcessThisMessage () > #13 0x0000000000409a9e in logerrorSz () > #14 0x0000000800772a99 in pthread_create () from > /lib/libpthread.so.2 > #15 0x00000008008cfcd4 in makecontext () from /lib/libc.so.6 > #16 0x0000000000000000 in ?? () > #17 0x0000000000537000 in ?? () > #18 0x00000000004099c0 in logerrorSz () > #19 0x0000000000000000 in ?? () > #20 0x0000000000000000 in ?? () > #21 0x0000000000000000 in ?? () > #22 0x0000000000000000 in ?? () > Cannot access memory at address 0x7fffff5fc000 > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 21 14:07:55 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 21 Sep 2007 14:07:55 +0200 Subject: [rsyslog] rsyslog segfault Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A67@grfint2.intern.adiscon.com> To those of you that experience the problem: I just got note that the problem possibly disappears in single-threading mode. We are looking at this. In the mean time, you may want to use ./configure --disable-pthreads I just thought I pass this around. Rainer From janfrode at tanso.net Sun Sep 23 11:27:53 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 23 Sep 2007 11:27:53 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Message-ID: On 2007-09-11, Rainer Gerhards wrote: > > But one important thing: I would appreciate if those of you that > experienced the segfaults could try out the new version. I currently > think it will *NOT* fix the problem, but I would like to verify that > (and I have to admit I am not angry if I am wrong...). Sorry, you were right: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xb7214c98 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x46)[0x80580b6] rsyslogd[0x804df9a] rsyslogd(llExecFunc+0x3f)[0x805f11f] rsyslogd[0x804d9ea] rsyslogd[0x804db3f] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] With 1.19.6. > If it fails, it would be even greater if you could try it in > single-threaded mode: > > ./configure --disable-pthreads Is this safe to use on a busy system? I wouldn't want to start dropping lots of packets.. -jf From rgerhards at hq.adiscon.com Mon Sep 24 11:16:42 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 11:16:42 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> Thanks for the report. I'd not yet use the single-threading version. The performance hit should not be severe, but I think I'll be able to release another version today. Not sure, of course, if it actually fixes the bug, but I am working on it. I got a report that it is indeed related to a threading issue and I am now checking all runtime functions that get called. At least one I have found to be not thread-safe. I intend to release a version with that fix and a couple of minor things, even if I have not yet completed the full runtime-checks. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Sunday, September 23, 2007 11:28 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > On 2007-09-11, Rainer Gerhards wrote: > > > > But one important thing: I would appreciate if those of you that > > experienced the segfaults could try out the new version. I currently > > think it will *NOT* fix the problem, but I would like to verify that > > (and I have to admit I am not angry if I am wrong...). > > Sorry, you were right: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xb7214c98 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x46)[0x80580b6] > rsyslogd[0x804df9a] > rsyslogd(llExecFunc+0x3f)[0x805f11f] > rsyslogd[0x804d9ea] > rsyslogd[0x804db3f] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > With 1.19.6. > > > If it fails, it would be even greater if you could try it in > > single-threaded mode: > > > > ./configure --disable-pthreads > > Is this safe to use on a busy system? I wouldn't want to start dropping > lots of packets.. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 24 11:25:44 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 11:25:44 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A87@grfint2.intern.adiscon.com> Ummm... Importnat word mising. I'd not use the single-threading version NOW. The reasoning was in the mail. I may ask some time later for it, though. Sorry... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, September 24, 2007 11:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > Thanks for the report. I'd not yet use the single-threading version. > The > performance hit should not be severe, but I think I'll be able to > release another version today. Not sure, of course, if it actually > fixes > the bug, but I am working on it. I got a report that it is indeed > related to a threading issue and I am now checking all runtime > functions > that get called. At least one I have found to be not thread-safe. I > intend to release a version with that fix and a couple of minor things, > even if I have not yet completed the full runtime-checks. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > > Sent: Sunday, September 23, 2007 11:28 AM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > > > On 2007-09-11, Rainer Gerhards wrote: > > > > > > But one important thing: I would appreciate if those of you that > > > experienced the segfaults could try out the new version. I > currently > > > think it will *NOT* fix the problem, but I would like to verify > that > > > (and I have to admit I am not angry if I am wrong...). > > > > Sorry, you were right: > > > > *** glibc detected *** rsyslogd: corrupted double-linked list: > > 0xb7214c98 *** > > ======= Backtrace: ========= > > /lib/libc.so.6[0x4152ce3e] > > /lib/libc.so.6(cfree+0x90)[0x415305d0] > > rsyslogd(MsgDestruct+0x46)[0x80580b6] > > rsyslogd[0x804df9a] > > rsyslogd(llExecFunc+0x3f)[0x805f11f] > > rsyslogd[0x804d9ea] > > rsyslogd[0x804db3f] > > /lib/libpthread.so.0[0x416112db] > > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > > > With 1.19.6. > > > > > If it fails, it would be even greater if you could try it in > > > single-threaded mode: > > > > > > ./configure --disable-pthreads > > > > Is this safe to use on a busy system? I wouldn't want to start > dropping > > lots of packets.. > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 24 16:30:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 16:30:48 +0200 Subject: [rsyslog] new unofficial 1.19.7 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> Hi all, I have posted a new "unofficial" 1.19.7 to the download server: http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz I intend to post it publically tomorrow. But some nits need to be done first, mostly doc issues. I'd appreciate if those with the segfault problem could give it a try (in multi-threading mode) and tell me the outcome. Thanks, Rainer From rgerhards at hq.adiscon.com Tue Sep 25 11:46:14 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 11:46:14 +0200 Subject: [rsyslog] rsyslog 1.19.7 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Hi all, Rsyslog 1.19.7 (the official one) has been released today. It contains some code updated compared to the version from yesterday, so please download it if you already did. Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. Its primary goal is enhanced stability. Feature-wise, it offers support of dropping NUL characters, which are often found at the end of invalidly formatted syslog messages. Upgrading to this release is recommended for all users. Changelog: http://www.rsyslog.com/Article129.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml PLEASE NOTE: the rsyslog site is currently migrated to a new machine. In case you receive a temporary page, your DNS still caches the old machine's address. As always, feedback is appreciated. Feedback is especially appreciated from those that have experienced the segfault problem in the past. Rainer Gerhards From infofarmer at FreeBSD.org Tue Sep 25 12:42:20 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Tue, 25 Sep 2007 14:42:20 +0400 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <20070925104218.GI49755@amilo.cenkes.org> On Tue, Sep 25, 2007 at 11:46:14AM +0200, Rainer Gerhards wrote: > Hi all, > > Rsyslog 1.19.7 (the official one) has been released today. It contains > some code updated compared to the version from yesterday, so please > download it if you already did. > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > Its primary goal is enhanced stability. Feature-wise, it offers support > of dropping NUL characters, which are often found at the end of > invalidly formatted syslog messages. Upgrading to this release is > recommended for all users. You said you don't support rfc3195d at the moment, but anyway: rfc3195d.c:108: error: 'errStr' undeclared (first use in this function) From rgerhards at hq.adiscon.com Tue Sep 25 12:44:33 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 12:44:33 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <20070925104218.GI49755@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <20070925104218.GI49755@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9D@grfint2.intern.adiscon.com> Oops.. looks like I need to check my build procedure, obviously it wasn't build over here... Thanks for the report, it's fixed now. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Tuesday, September 25, 2007 12:42 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > On Tue, Sep 25, 2007 at 11:46:14AM +0200, Rainer Gerhards wrote: > > Hi all, > > > > Rsyslog 1.19.7 (the official one) has been released today. It > contains > > some code updated compared to the version from yesterday, so please > > download it if you already did. > > > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > > Its primary goal is enhanced stability. Feature-wise, it offers > support > > of dropping NUL characters, which are often found at the end of > > invalidly formatted syslog messages. Upgrading to this release is > > recommended for all users. > > You said you don't support rfc3195d at the moment, but anyway: > > rfc3195d.c:108: error: 'errStr' undeclared (first use in this > function) > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Tue Sep 25 13:55:18 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 25 Sep 2007 13:55:18 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <1190721318.3849.13.camel@localhost.localdomain> Am Dienstag, den 25.09.2007, 11:46 +0200 schrieb Rainer Gerhards: [...] > As always, feedback is appreciated. Feedback is especially appreciated > from those that have experienced the segfault problem in the past. Hi all, I'd like to apologize, since it was me who reported to Rainer that the segfault problem seems to have gone with the new release and that everything seems to be better now before he released 1.19.7. Meanwhile I realised that while the rsyslog-Daemon indeed ran without any segfaults, however, it didn't log anything sent to it via UDP. So my segfault problem has been replaced by a new one. ;-) So everybody should be very careful with this update, especially those who previously encountered the segfault problem. -- CU, Patrick. From rgerhards at hq.adiscon.com Tue Sep 25 13:57:46 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 13:57:46 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <1190721318.3849.13.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <1190721318.3849.13.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> Patrick, you mean nothing was logged via UDP while local domain sockets and/or TCP still worked? I have to admit I am a bit puzzled... Do you have a debug log at hand? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Tuesday, September 25, 2007 1:55 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > Am Dienstag, den 25.09.2007, 11:46 +0200 schrieb Rainer Gerhards: > [...] > > As always, feedback is appreciated. Feedback is especially > appreciated > > from those that have experienced the segfault problem in the past. > Hi all, > > I'd like to apologize, since it was me who reported to Rainer that the > segfault problem seems to have gone with the new release and that > everything > seems to be better now before he released 1.19.7. > > Meanwhile I realised that while the rsyslog-Daemon indeed ran without > any > segfaults, however, it didn't log anything sent to it via UDP. So my > segfault > problem has been replaced by a new one. ;-) So everybody should be very > careful with this update, especially those who previously encountered > the > segfault problem. > > -- > CU, > Patrick. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Tue Sep 25 15:39:14 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 25 Sep 2007 15:39:14 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <1190721318.3849.13.camel@localhost.localdomain> <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> Message-ID: <1190727554.3043.14.camel@localhost.localdomain> Am Dienstag, den 25.09.2007, 13:57 +0200 schrieb Rainer Gerhards: > Patrick, > > you mean nothing was logged via UDP while local domain sockets and/or > TCP still worked? I have to admit I am a bit puzzled... I only use rsyslog for UDP-logging, so I can't comment on TCP or local domain sockets. I usually run rsyslogd with "-r514" or "-4 -r514". The only log-entries relate to startup or shutdown of rsyslog itself. > Do you have a debug log at hand? Let's start with compiler warnings. They are identical to 1.19.6, so probably not related to the changed behaviour. syslog.c: In function ?writeSyslogV?: syslog.c:98: warning: function might be possible candidate for ?printf? format attribute syslog.c:98: warning: function might be possible candidate for ?printf? format attribute template.c: In function ?tplPrintList?: template.c:927: warning: cast from pointer to integer of different size modules.c: In function ?modPrintList?: modules.c:309: warning: cast from pointer to integer of different size modules.c:310: warning: cast from pointer to integer of different size modules.c:311: warning: cast from pointer to integer of different size modules.c:312: warning: cast from pointer to integer of different size modules.c:313: warning: cast from pointer to integer of different size cfsysline.c: In function ?dbgPrintCfSysLineHandlers?: cfsysline.c:710: warning: cast from pointer to integer of different size cfsysline.c:711: warning: cast from pointer to integer of different size action.c: In function ?actionDbgPrint?: action.c:174: warning: cast from pointer to integer of different size If I run "rsyslogd -r514 -4 -d" it prints something like this: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: over and over again. I'll sent you a complete output off-list. -- CU, Patrick. From mic at npgx.com.au Tue Sep 25 15:40:36 2007 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 25 Sep 2007 23:40:36 +1000 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <20070925133439.M72922@npgx.com.au> Hi Rainer, > Hi all, > > Rsyslog 1.19.7 (the official one) has been released today. It > contains some code updated compared to the version from yesterday, > so please download it if you already did. I've just downloaded and installed this release, and although I've never experienced the segfault issue (rsyslog continually runs fine for me), when I installed this version and then did a: # service rsyslog restart I got this in my messages file: Sep 25 22:41:46 server kernel: Kernel logging (proc) stopped. Sep 25 22:41:46 server kernel: Kernel log daemon terminating. Sep 25 22:41:47 server rsyslog: rklogd shutdown succeeded Sep 25 22:41:47 server rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="12176"] exiting on signal 15. Sep 25 22:44:08 server rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="10430"][x-configInfo udpRecep tion="No" udpPort="514" tcpReception="No" tcpPort="0"] restart Sep 25 22:44:08 server rsyslog: rsyslogd startup succeeded Sep 25 22:44:08 server kernel: rklogd 1.19.7, log source = /proc/kmsg started. Sep 25 22:44:08 server kernel: rsyslogd[8110]: segfault at 000000000000000a rip 00000038c7568678 rsp 0000007fbfffdc2 0 error 4 Sep 25 22:44:08 server rsyslog: rklogd startup succeeded Sep 25 22:44:08 server rsyslog: rsyslogd shutdown succeeded The above machine is a DL380 G4 with one CPU running hypthreaded. On another machine that's the same as that: Sep 25 22:42:24 server2 kernel: Kernel logging (proc) stopped. Sep 25 22:42:24 server2 kernel: Kernel log daemon terminating. Sep 25 22:42:25 server2 rsyslog: rklogd shutdown succeeded Sep 25 22:42:25 server2 rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="28497"] exiting on signal 15 . Sep 25 22:43:43 server2 rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="16598"][x-configInfo udpRece ption="No" udpPort="514" tcpReception="No" tcpPort="0"] restart Sep 25 22:43:43 server2 rsyslog: rsyslogd startup succeeded Sep 25 22:43:43 server2 kernel: rklogd 1.19.7, log source = /proc/kmsg started. Sep 25 22:43:43 server2 kernel: rsyslogd[15998]: segfault at 000000000000000a rip 00000034d2868678 rsp 0000007fbfffd c20 error 4 Sep 25 22:43:43 server2 rsyslog: rklogd startup succeeded Those two machines above are also running 64bit Linux. Using the same process on other 32 bit linux servers (DL360's), no segfaults occur. Note that I have never (ever) had rsyslog crash on me while it's running, and I currently run it on 6 servers to do the MySQL logging. Regards, Michael. > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > Its primary goal is enhanced stability. Feature-wise, it offers support > of dropping NUL characters, which are often found at the end of > invalidly formatted syslog messages. Upgrading to this release is > recommended for all users. > > Changelog: > > http://www.rsyslog.com/Article129.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml > > PLEASE NOTE: the rsyslog site is currently migrated to a new > machine. In case you receive a temporary page, your DNS still caches > the old machine's address. > > As always, feedback is appreciated. Feedback is especially > appreciated from those that have experienced the segfault problem in > the past. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From halljer at auburn.edu Tue Sep 25 18:09:39 2007 From: halljer at auburn.edu (Dusty Hall) Date: Tue, 25 Sep 2007 11:09:39 -0500 Subject: [rsyslog] new unofficial 1.19.7 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> Message-ID: <46F8EE24.8529.003A.0@auburn.edu> I had some issues with the redhat/rsyslog.init script, default binary locations, etc. I've attached an updated version. Thanks, -Dusty >>> "Rainer Gerhards" 09/24/07 9:30 AM >>> Hi all, I have posted a new "unofficial" 1.19.7 to the download server: http://download.rsyslog.com/rsyslog/rsyslog- 1.19.7.tar.gz I intend to post it publically tomorrow. But some nits need to be done first, mostly doc issues. I'd appreciate if those with the segfault problem could give it a try (in multi- threading mode) and tell me the outcome. Thanks, Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Sep 26 13:53:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 26 Sep 2007 13:53:02 +0200 Subject: [rsyslog] rsyslog 1.19.7 UDP bug Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ACC@grfint2.intern.adiscon.com> Hi all, I have identified and fixed the UDP message loss problem in rsyslog 1.19.7. If you have not yet installed that release, DO NOT INSTALL IT. I will remove it from the download links. A fixed version will be released soon as 1.19.8. If you need it urgently, an interim version is available at http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz That tarball is lacking MySQL functionality (which has now been modularized into a separate package - more news on that later). Sorry for the hassle. Rainer From rgerhards at hq.adiscon.com Wed Sep 26 14:30:24 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 26 Sep 2007 14:30:24 +0200 Subject: [rsyslog] on the udp bug Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ACF@grfint2.intern.adiscon.com> In case you are interested in some more details: http://rgerhards.blogspot.com/2007/09/needed-to-pull-1197-release.html And in CVS: http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/net.c?r1=1.9&r2=1.10 Rainer From rgerhards at hq.adiscon.com Thu Sep 27 10:11:11 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Sep 2007 10:11:11 +0200 Subject: [rsyslog] rsyslog 1.19.8 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Hi all, rsyslog 1.19.8 has been released today. Most importantly, it fixes a bug that could cause severe loss of UDP messages. It also contains improved repeat message processing. The MySQL functionality is now taken out of the core package, but its tarball is still contained in the main tarball (so it is still a single download for everything). This is part of the effort to fully support third-party plugins. Rsyslog 1.19.8 is a recommended update for all users. Change Log: http://www.rsyslog.com/Article132.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-60.phtml As always, feedback is appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Thu Sep 27 10:17:13 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Sep 2007 10:17:13 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <20070925133439.M72922@npgx.com.au> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <20070925133439.M72922@npgx.com.au> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278AE1@grfint2.intern.adiscon.com> Sorry Michael, I just noticed that I had not replied. I looked at the code, but there is too few information to track the source code location down. What I currently assume is a) that we still have a segfault issue (obviously ;)) b) the segfault is a moving target The segfault occurs due to memory corruption. I think (and have often seen) that you do not notice the corruption until it happens to hit some really important piece of memory. So in many cases, it will overwrite something that nobody cares about. Only if some specific structures are hit, a malfunction manifests. This is part of the problem to reproducing it. What is hit, depends on the memory layout. A new version leads to different memory layout. I assume this is what has happened here: you did not experience the bug before, because it hit some memory that wasn't really relevant. With the new versions memory layout, it hits somewhere where it hurts... At least this is my best guess. I'd appreciate if you could try out the 1.19.8 release. I suspect it will also segfault. If it does, it would be great if you could run it in debug mode (-d -n) and send me the output before it segfault (maybe 200 lines or so). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Mansour > Sent: Tuesday, September 25, 2007 3:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > Hi Rainer, > > > Hi all, > > > > Rsyslog 1.19.7 (the official one) has been released today. It > > contains some code updated compared to the version from yesterday, > > so please download it if you already did. > > I've just downloaded and installed this release, and although I've > never > experienced the segfault issue (rsyslog continually runs fine for me), > when I > installed this version and then did a: > > # service rsyslog restart > > I got this in my messages file: > > Sep 25 22:41:46 server kernel: Kernel logging (proc) stopped. > Sep 25 22:41:46 server kernel: Kernel log daemon terminating. > Sep 25 22:41:47 server rsyslog: rklogd shutdown succeeded > Sep 25 22:41:47 server rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="12176"] exiting on signal 15. > Sep 25 22:44:08 server rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="10430"][x-configInfo udpRecep > tion="No" udpPort="514" tcpReception="No" tcpPort="0"] restart > Sep 25 22:44:08 server rsyslog: rsyslogd startup succeeded > Sep 25 22:44:08 server kernel: rklogd 1.19.7, log source = /proc/kmsg > started. > Sep 25 22:44:08 server kernel: rsyslogd[8110]: segfault at > 000000000000000a > rip 00000038c7568678 rsp 0000007fbfffdc2 > 0 error 4 > Sep 25 22:44:08 server rsyslog: rklogd startup succeeded > Sep 25 22:44:08 server rsyslog: rsyslogd shutdown succeeded > > The above machine is a DL380 G4 with one CPU running hypthreaded. On > another > machine that's the same as that: > > Sep 25 22:42:24 server2 kernel: Kernel logging (proc) stopped. > Sep 25 22:42:24 server2 kernel: Kernel log daemon terminating. > Sep 25 22:42:25 server2 rsyslog: rklogd shutdown succeeded > Sep 25 22:42:25 server2 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="28497"] exiting on signal 15 > . > Sep 25 22:43:43 server2 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="16598"][x-configInfo udpRece > ption="No" udpPort="514" tcpReception="No" tcpPort="0"] restart > Sep 25 22:43:43 server2 rsyslog: rsyslogd startup succeeded > Sep 25 22:43:43 server2 kernel: rklogd 1.19.7, log source = /proc/kmsg > started. > Sep 25 22:43:43 server2 kernel: rsyslogd[15998]: segfault at > 000000000000000a > rip 00000034d2868678 rsp 0000007fbfffd > c20 error 4 > Sep 25 22:43:43 server2 rsyslog: rklogd startup succeeded > > Those two machines above are also running 64bit Linux. > > Using the same process on other 32 bit linux servers (DL360's), no > segfaults > occur. > > Note that I have never (ever) had rsyslog crash on me while it's > running, and > I currently run it on 6 servers to do the MySQL logging. > > Regards, > > Michael. > > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > > Its primary goal is enhanced stability. Feature-wise, it offers > support > > of dropping NUL characters, which are often found at the end of > > invalidly formatted syslog messages. Upgrading to this release is > > recommended for all users. > > > > Changelog: > > > > http://www.rsyslog.com/Article129.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml > > > > PLEASE NOTE: the rsyslog site is currently migrated to a new > > machine. In case you receive a temporary page, your DNS still caches > > the old machine's address. > > > > As always, feedback is appreciated. Feedback is especially > > appreciated from those that have experienced the segfault problem in > > the past. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > ------- End of Original Message ------- > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 28 13:46:13 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Sep 2007 13:46:13 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Hi Michael, as I have blogged, I am not yet sure about how to handle the situation. I am also consulting with Autotools experts, any advise is appreciated. Two packages seem useful, especially when more plugins become available (I myself think about email and a couple of other databases). Many folks also do not like the idea of having to have libmysql present on the system just to install rsyslog - with a core package and the plugin, those can only install the core (and that will probably the majority of cases). What I have not yet found - and I have very limited expertise in this area - is how to do that in the best possible way. Oh, and some more background: ones the plugin interace has matured (I expect this in 3 to 6 month), I intend to actually use ommysql with its own version number. Versioning will be handled by the interface (that part is already present, but no code for it yet as there is only a release 1 of it). So, in the medium to long term, ommysql *will* be a separate project - maybe one with a different maintainer, as I am no mysql guy. Does this make sense? As I said, comments are much appreciated... Rainer > -----Original Message----- > From: Michael Biebl [mailto:mbiebl at gmail.com] > Sent: Friday, September 28, 2007 10:23 AM > To: rsyslog-users; Rainer Gerhards > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > 2007/9/27, Rainer Gerhards : > > repeat message processing. The MySQL functionality is now taken out > of > > the core package, but its tarball is still contained in the main > tarball > > (so it is still a single download for everything). This is part of > the > > effort to fully support third-party plugins. Rsyslog 1.19.8 is a > > > To be honest, I don't particularly like this change. It increases the > work for package maintainers (like me). You now have to maintain two > source packages. Having a non-standard tarball inside a tarball > doesn't help there. It even worsens things, as stuff like "make dist" > or "make distcheck" doesn't work anymore for a cvs checkout. > There's also the problem of which version of the plugins will be > compatible with the core version (upgrade scenarios, keeping them in > sync, etc.) > It also adds complexity for the developer (as he has to maintain an > additonal set of build files). > As you were talking about 3rd party plugins (with the emphasis on 3rd > party) I don't understand the benefit of splitting out the mysql > plugin as it is you who develops the mysql plugin, not a third party. > Do you actually intend to create a separate tarball for each plugin in > the future? > > What was wrong with the --enable-mysql configure switch? I don't see > any benefits, only disadvantages. You know, if it ain't broken, why > fix it ;-) > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? From mic at npgx.com.au Fri Sep 28 16:03:59 2007 From: mic at npgx.com.au (Michael Mansour) Date: Sat, 29 Sep 2007 00:03:59 +1000 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: <20070928135248.M3034@npgx.com.au> Hi Rainer, > Hi Michael, > > as I have blogged, I am not yet sure about how to handle the situation. > I am also consulting with Autotools experts, any advise is appreciated. > > Two packages seem useful, especially when more plugins become available > > (I myself think about email and a couple of other databases). Many folks > also do not like the idea of having to have libmysql present on the > system just to install rsyslog - with a core package and the plugin, > those can only install the core (and that will probably the majority > of cases). Yes, I totally agree with what you're doing, as making rsyslog modular is better overall for the product as a whole, as it will allow contributors easier access to have their own plugins in place without needing to release rsyslog each time. > What I have not yet found - and I have very limited expertise in this > area - is how to do that in the best possible way. I've asked both Dag and Dries (from rpmforge) on the best way forward as these guys have years of knowledge behind them for packaging (in the RPM space though). I'm not sure they'll reply though, as sometimes my questions to them are ignored and sometimes they are responded to (some questions I ask go into their "too hard" basket which I think is why they ignore them :P ). I have been querying, requesting, suggesting, etc things off them for more than 7 years now, so they may be sick of me (joking) :). I have been asked to become a packager for them before (and for Axel Thimms who does ATrpms.net - they're all merging into one large repo eventually) but declined as I have little time as is to contribute to the projects I'm currently working with. > Oh, and some more background: ones the plugin interace has matured (I > expect this in 3 to 6 month), I intend to actually use ommysql with its > own version number. Versioning will be handled by the interface (that > part is already present, but no code for it yet as there is only a > release 1 of it). So, in the medium to long term, ommysql *will* be a > separate project - maybe one with a different maintainer, as I am no > mysql guy. > > Does this make sense? As I said, comments are much appreciated... Yes and I do agree with it all. I'm just not personally sure of the best way forward myself, but I would suspect something along the lines of having one spec file for rsyslog, and another for rsyslog-mysql would likely be the right way to go. If neither Dag nor Dries answer me, I might even try Axel and if he also doesn't respond (he usually does) then I'll just start on the packaging myself, doing rsyslog and rsyslog-mysql. In the long term, it would be better though if rpmforge or atrpms.net would take on this packaging system. I realise Red Hat will have a "base" one available in Fedora and then eventually an EL release, but they dont' do updates, just bugfixing, once they release it for an enterprise product. Which is where 3rd party repo's make sense. Regards, Michael. > Rainer > > > -----Original Message----- > > From: Michael Biebl [mailto:mbiebl at gmail.com] > > Sent: Friday, September 28, 2007 10:23 AM > > To: rsyslog-users; Rainer Gerhards > > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > > > 2007/9/27, Rainer Gerhards : > > > repeat message processing. The MySQL functionality is now taken out > > of > > > the core package, but its tarball is still contained in the main > > tarball > > > (so it is still a single download for everything). This is part of > > the > > > effort to fully support third-party plugins. Rsyslog 1.19.8 is a > > > > > > To be honest, I don't particularly like this change. It increases the > > work for package maintainers (like me). You now have to maintain two > > source packages. Having a non-standard tarball inside a tarball > > doesn't help there. It even worsens things, as stuff like "make dist" > > or "make distcheck" doesn't work anymore for a cvs checkout. > > There's also the problem of which version of the plugins will be > > compatible with the core version (upgrade scenarios, keeping them in > > sync, etc.) > > It also adds complexity for the developer (as he has to maintain an > > additonal set of build files). > > As you were talking about 3rd party plugins (with the emphasis on 3rd > > party) I don't understand the benefit of splitting out the mysql > > plugin as it is you who develops the mysql plugin, not a third party. > > Do you actually intend to create a separate tarball for each plugin in > > the future? > > > > What was wrong with the --enable-mysql configure switch? I don't see > > any benefits, only disadvantages. You know, if it ain't broken, why > > fix it ;-) > > > > Cheers, > > Michael > > > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From mbiebl at gmail.com Fri Sep 28 18:06:46 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 18:06:46 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Rainer Gerhards : > Hi Michael, > > as I have blogged, I am not yet sure about how to handle the situation. > I am also consulting with Autotools experts, any advise is appreciated. > > Two packages seem useful, especially when more plugins become available > (I myself think about email and a couple of other databases). Many folks > also do not like the idea of having to have libmysql present on the > system just to install rsyslog - with a core package and the plugin, Well, this is not quite true. If you don't want to have mysql support, you could just use --disable-mysql. This way you don't have to have mysql installed. Such a --enable/disable-feature option could be added for any plugin that will be added in the future. > those can only install the core (and that will probably the majority of > cases). > > What I have not yet found - and I have very limited expertise in this > area - is how to do that in the best possible way. I do have quite some experience with autotools. So if any help is needed, I'll gladly offer it. > release 1 of it). So, in the medium to long term, ommysql *will* be a > separate project - maybe one with a different maintainer, as I am no > mysql guy. This actually is a valid argument for splitting it off. But to reiterate what I posted earlier: For the plugins shipped directly by the rsyslog project, I'd prefer to have them in a single tarball with configure options to turn them on/off and as long as you are the maintainer of the mysql output plugin its more convenient to keep them in one project. Why not make the split when the plugin interface is stable, other output plugins are available (and we have more experience how to handle them) and a external maintainer for ommysql is found? I hope this doesn't sound too negative. I generally like the idea of having a plugin interface, but I only fear that splitting things up prematurely without knowing how it develops, will complicate things (and probably destabilize). Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Fri Sep 28 20:11:20 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Sep 2007 20:11:20 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Michael, I need to prepare for german astronomy day tomorrow, so brief ;) The point was that a single package (RPM or whatever) can be pre-build for the core and the mysql functionality. I think the configure options do not help with that. Or do they? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Friday, September 28, 2007 6:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > 2007/9/28, Rainer Gerhards : > > Hi Michael, > > > > as I have blogged, I am not yet sure about how to handle the > situation. > > I am also consulting with Autotools experts, any advise is > appreciated. > > > > Two packages seem useful, especially when more plugins become > available > > (I myself think about email and a couple of other databases). Many > folks > > also do not like the idea of having to have libmysql present on the > > system just to install rsyslog - with a core package and the plugin, > > Well, this is not quite true. If you don't want to have mysql support, > you could just use --disable-mysql. This way you don't have to have > mysql installed. > Such a --enable/disable-feature option could be added for any plugin > that will be added in the future. > > > those can only install the core (and that will probably the majority > of > > cases). > > > > What I have not yet found - and I have very limited expertise in this > > area - is how to do that in the best possible way. > > I do have quite some experience with autotools. So if any help is > needed, I'll gladly offer it. > > > release 1 of it). So, in the medium to long term, ommysql *will* be a > > separate project - maybe one with a different maintainer, as I am no > > mysql guy. > > This actually is a valid argument for splitting it off. > > But to reiterate what I posted earlier: For the plugins shipped > directly by the rsyslog project, I'd prefer to have them in a single > tarball with configure options to turn them on/off and as long as you > are the maintainer of the mysql output plugin its more convenient to > keep them in one project. Why not make the split when the plugin > interface is stable, other output plugins are available (and we have > more experience how to handle them) and a external maintainer for > ommysql is found? > I hope this doesn't sound too negative. I generally like the idea of > having a plugin interface, but I only fear that splitting things up > prematurely without knowing how it develops, will complicate things > (and probably destabilize). > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Fri Sep 28 21:03:53 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 21:03:53 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Rainer Gerhards : > Michael, > > I need to prepare for german astronomy day tomorrow, so brief ;) > > The point was that a single package (RPM or whatever) can be pre-build > for the core and the mysql functionality. I think the configure options > do not help with that. Or do they? > You can split up the build to create multiple *binary* packages from a single *source* package, the RPM format as well as DEB allows that. Actually the Debian and Fedora packages already do that. The rsyslog [1] binary package contains the core rsyslogd/rklogd binaries, the rsyslog-mysql [2] package the mysql output plugin. It's not necessary to already split up the *source* package for that. Cheers, Michael [1] http://packages.debian.org/sid/rsyslog [2] http://packages.debian.org/sid/rsyslog-mysql -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Fri Sep 28 21:06:05 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 21:06:05 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Michael Biebl : > 2007/9/28, Rainer Gerhards : > > > > The point was that a single package (RPM or whatever) can be pre-build > > for the core and the mysql functionality. I think the configure options > > do not help with that. Or do they? > > > > You can split up the build to create multiple *binary* packages from a > single *source* package, the RPM format as well as DEB allows that. > > Actually the Debian and Fedora packages already do that. The rsyslog > [1] binary package contains the core rsyslogd/rklogd binaries, the > rsyslog-mysql [2] package the mysql output plugin. > It's not necessary to already split up the *source* package for that. > > [1] http://packages.debian.org/sid/rsyslog > [2] http://packages.debian.org/sid/rsyslog-mysql And as you can see from [1] and [2], the core rsyslog package has no libmysql dependencies. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From janfrode at tanso.net Fri Sep 28 22:57:12 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 28 Sep 2007 22:57:12 +0200 Subject: [rsyslog] rsyslog 1.19.8 released References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Message-ID: On 2007-09-27, Rainer Gerhards wrote: > > rsyslog 1.19.8 has been released today. Most importantly, it fixes a bug Still crashes.. twice today, but I didn't find the trace the first time. Here's the trace from the second crash: *** glibc detected *** rsyslogd: corrupted double-linked list: 0x094ca0e8 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152cda9] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8058323] rsyslogd[0x804dfba] rsyslogd(llExecFunc+0x3f)[0x805f43f] rsyslogd[0x804d9ba] rsyslogd[0x804db0f] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] ======= Memory map: ======== 0067d000-0067e000 r-xp 0067d000 00:00 0 [vdso] 08048000-08068000 r-xp 00000000 fd:00 884939 /sbin/rsyslogd 08068000-08069000 rwxp 00020000 fd:00 884939 /sbin/rsyslogd 094c5000-09549000 rwxp 094c5000 00:00 0 414aa000-414c3000 r-xp 00000000 fd:00 1146882 /lib/ld-2.5.so 414c3000-414c4000 r-xp 00018000 fd:00 1146882 /lib/ld-2.5.so 414c4000-414c5000 rwxp 00019000 fd:00 1146882 /lib/ld-2.5.so 414c7000-415fe000 r-xp 00000000 fd:00 1146990 /lib/libc-2.5.so 415fe000-41600000 r-xp 00137000 fd:00 1146990 /lib/libc-2.5.so 41600000-41601000 rwxp 00139000 fd:00 1146990 /lib/libc-2.5.so 41601000-41604000 rwxp 41601000 00:00 0 41606000-41608000 r-xp 00000000 fd:00 1147002 /lib/libdl-2.5.so 41608000-41609000 r-xp 00001000 fd:00 1147002 /lib/libdl-2.5.so 41609000-4160a000 rwxp 00002000 fd:00 1147002 /lib/libdl-2.5.so 4160c000-4161f000 r-xp 00000000 fd:00 1147003 /lib/libpthread-2.5.so 4161f000-41620000 r-xp 00012000 fd:00 1147003 /lib/libpthread-2.5.so 41620000-41621000 rwxp 00013000 fd:00 1147003 /lib/libpthread-2.5.so 41621000-41623000 rwxp 41621000 00:00 0 41667000-41679000 r-xp 00000000 fd:00 1087043 /usr/lib/libz.so.1.2.3 41679000-4167a000 rwxp 00011000 fd:00 1087043 /usr/lib/libz.so.1.2.3 416c4000-416cb000 r-xp 00000000 fd:00 1147009 /lib/librt-2.5.so 416cb000-416cc000 r-xp 00006000 fd:00 1147009 /lib/librt-2.5.so 416cc000-416cd000 rwxp 00007000 fd:00 1147009 /lib/librt-2.5.so 4acb4000-4acbf000 r-xp 00000000 fd:00 1146939 /lib/libgcc_s-4.1.1-20070105.so.1 4acbf000-4acc0000 rwxp 0000a000 fd:00 1146939 /lib/libgcc_s-4.1.1-20070105.so.1 b7200000-b722a000 rw-p b7200000 00:00 0 b722a000-b7300000 ---p b722a000 00:00 0 b7400000-b7430000 rw-p b7400000 00:00 0 b7430000-b7500000 ---p b7430000 00:00 0 b759b000-b759c000 ---p b759b000 00:00 0 b759c000-b7f9e000 rw-p b759c000 00:00 0 b7fa3000-b7fa5000 rw-p b7fa3000 00:00 0 bfc72000-bfc88000 rw-p bfc72000 00:00 0 [stack] -jf From janfrode at tanso.net Mon Sep 3 10:47:26 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 3 Sep 2007 10:47:26 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> Message-ID: Another one of these, this time with v1.19.3: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xae3fa998 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8057393] rsyslogd[0x804de0a] rsyslogd(llExecFunc+0x3f)[0x805ea3f] rsyslogd[0x804d86a] rsyslogd[0x804d997] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] And I didn't get any core-file, maybe because the v1.19.3 overwrote my "ulimit -c unlimited" change to the initscript... Ooops :-) -jf From rgerhards at hq.adiscon.com Tue Sep 4 17:57:55 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 17:57:55 +0200 Subject: [rsyslog] rsyslog 1.19.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27890E@grfint2.intern.adiscon.com> Hi all, rsyslog 1.19.4 is a bug fixing release. It contains no new features, but stability updates. Most importantly, it addresses some bugs that have lead to program aborts. 1.19.4 is a recommended update for all users. Changelog: http://www.rsyslog.com/Article123.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-56.phtml As always, feedback is very appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Tue Sep 4 17:58:39 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 17:58:39 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> Hi, I have just release 1.19.4 and hope that the fixes also address your problem. I'd appreciate if you could try it out. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Monday, September 03, 2007 10:47 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] v1.19.1 is crashing > > > Another one of these, this time with v1.19.3: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xae3fa998 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x73)[0x8057393] > rsyslogd[0x804de0a] > rsyslogd(llExecFunc+0x3f)[0x805ea3f] > rsyslogd[0x804d86a] > rsyslogd[0x804d997] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > And I didn't get any core-file, maybe because the v1.19.3 overwrote my > "ulimit -c unlimited" change to the initscript... Ooops :-) > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Sep 4 18:58:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Sep 2007 18:58:48 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Hi, I noticed that one problem I received patches for (and that are now included in 1.19.4) is rooted in something that is not fully patched. I think I've found that root cause now (thanks to the info with that patches). The bottom line, however, is that 1.19.4 may still have some stability issues. However, they should surface now only in very obscure cases (but what is obscure...). I'd still appreciate if you could apply 1.19.4 and tell me the outcome. I am now working on fixing the root cause. That might take a short while, as I am thinking about the best *design* to fix the issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, September 04, 2007 5:59 PM > To: rsyslog-users > Subject: Re: [rsyslog] v1.19.1 is crashing > > Hi, > > I have just release 1.19.4 and hope that the fixes also address your > problem. I'd appreciate if you could try it out. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > > Sent: Monday, September 03, 2007 10:47 AM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] v1.19.1 is crashing > > > > > > Another one of these, this time with v1.19.3: > > > > *** glibc detected *** rsyslogd: corrupted double-linked list: > > 0xae3fa998 *** > > ======= Backtrace: ========= > > /lib/libc.so.6[0x4152ce3e] > > /lib/libc.so.6(cfree+0x90)[0x415305d0] > > rsyslogd(MsgDestruct+0x73)[0x8057393] > > rsyslogd[0x804de0a] > > rsyslogd(llExecFunc+0x3f)[0x805ea3f] > > rsyslogd[0x804d86a] > > rsyslogd[0x804d997] > > /lib/libpthread.so.0[0x416112db] > > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > > > And I didn't get any core-file, maybe because the v1.19.3 overwrote > my > > "ulimit -c unlimited" change to the initscript... Ooops :-) > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Wed Sep 5 10:15:26 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 5 Sep 2007 10:15:26 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: On 2007-09-04, Rainer Gerhards wrote: > > I'd still appreciate if you could apply 1.19.4 and tell me the outcome. > I am now working on fixing the root cause. That might take a short > while, as I am thinking about the best *design* to fix the issue. OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you know if it fails again. -jf From theinric at redhat.com Wed Sep 5 11:53:10 2007 From: theinric at redhat.com (theinric at redhat.com) Date: Wed, 05 Sep 2007 11:53:10 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com> Message-ID: <46DE7C86.1040308@redhat.com> You have an error in your config file, but it's probably harmless: %timegenerated::fulltime% The option fulltime doesn't exist and it looks like it never did. Additionally, a colon is missing. I've noticed that this definition is actually present in sample.conf, so you've probably picked it up there. (It looks like it has been there at least from 0.8.1) I've did some testing and it probably doesn't have any impact except for a warning message in debug mode. Jan-Frode Myklebust wrote: > On 2007-08-31, theinric at redhat.com wrote: >> could you please provide some more info on your configuration? >> Configuration file, > > ################################################################################# > $ grep -v ^# /etc/rsyslog.conf|grep -v ^$ > $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.log" > *.* -?DailyPerHostLogs > $template MaillogTemplate,"%timegenerated::fulltime% %HOSTNAME% %syslogtag%: %msg%\n" > $template HourlyMaillog,"/var/log/syslog/maillog/%$YEAR%/%$MONTH%/%$DAY%/maillog-%$YEAR%%$MONTH%%$DAY%%$HOUR%.log" > mail.* -?HourlyMaillog;MaillogTemplate > $template precise,"%timegenerated::fulltime% %HOSTNAME% %syslogfacility-text%/%syslogseverity-text% %syslogtag% %msg%\n" > *.* -/var/log/syslog/everything;precise > mail.* ~ > $template PerAppLogs,"/var/log/syslog/apps/%programname%.log" > *.* -?PerAppLogs > :msg, contains, "ServeRAID" -/var/log/syslog/apps/serveraid.log > :HOSTNAME, !isequal, "loghost1" ~ > *.info;mail.none;authpriv.none;cron.none /var/log/messages > authpriv.* /var/log/secure > mail.* -/var/log/maillog > cron.* /var/log/cron > *.emerg * > uucp,news.crit /var/log/spooler > local7.* /var/log/boot.log > ################################################################################# > > >> options used, > > $ grep -v ^# /etc/sysconfig/rsyslog > SYSLOGD_OPTIONS="-m 0 -r514" > KLOGD_OPTIONS="-x" > SYSLOG_UMASK=077 > >> log entries preceding the crash, ... > > It's a quite busy log server, with about 70 active old style syslog servers > sending logs to it. The second it crashed it wrote 111 log-messages.. (273 > the second before), mostly various postfix daemons, and I'd need to anonymize > them before sharing.. Can't see anything special. > >> If logging forwarded messages, is the remote logger also rsyslog? > > No, all are RHEL3/4/5 with their default syslogd server. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Sep 5 12:25:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 5 Sep 2007 12:25:48 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <46DE7C86.1040308@redhat.com> References: <46D81432.1090302@redhat.com> <46DE7C86.1040308@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278919@grfint2.intern.adiscon.com> I have checked on fulltime, it is an error in sample.conf - there is no need for this option and I think it was actually not present. I'll remove that sample. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of theinric at redhat.com > Sent: Wednesday, September 05, 2007 11:53 AM > To: rsyslog-users > Subject: Re: [rsyslog] v1.19.1 is crashing > > You have an error in your config file, but it's probably harmless: > %timegenerated::fulltime% > > The option fulltime doesn't exist and it looks like it never did. > Additionally, > a colon is missing. I've noticed that this definition is actually > present in > sample.conf, so you've probably picked it up there. (It looks like it > has been > there at least from 0.8.1) > I've did some testing and it probably doesn't have any impact except > for a > warning message in debug mode. > > Jan-Frode Myklebust wrote: > > On 2007-08-31, theinric at redhat.com wrote: > >> could you please provide some more info on your configuration? > >> Configuration file, > > > > > ####################################################################### > ########## > > $ grep -v ^# /etc/rsyslog.conf|grep -v ^$ > > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.lo > g" > > *.* -?DailyPerHostLogs > > $template MaillogTemplate,"%timegenerated::fulltime% %HOSTNAME% > %syslogtag%: %msg%\n" > > $template > HourlyMaillog,"/var/log/syslog/maillog/%$YEAR%/%$MONTH%/%$DAY%/maillog- > %$YEAR%%$MONTH%%$DAY%%$HOUR%.log" > > mail.* -?HourlyMaillog;MaillogTemplate > > $template precise,"%timegenerated::fulltime% %HOSTNAME% > %syslogfacility-text%/%syslogseverity-text% %syslogtag% %msg%\n" > > *.* -/var/log/syslog/everything;precise > > mail.* ~ > > $template PerAppLogs,"/var/log/syslog/apps/%programname%.log" > > *.* -?PerAppLogs > > :msg, contains, "ServeRAID" - > /var/log/syslog/apps/serveraid.log > > :HOSTNAME, !isequal, "loghost1" ~ > > *.info;mail.none;authpriv.none;cron.none > /var/log/messages > > authpriv.* > /var/log/secure > > mail.* - > /var/log/maillog > > cron.* /var/log/cron > > *.emerg * > > uucp,news.crit > /var/log/spooler > > local7.* > /var/log/boot.log > > > ####################################################################### > ########## > > > > > >> options used, > > > > $ grep -v ^# /etc/sysconfig/rsyslog > > SYSLOGD_OPTIONS="-m 0 -r514" > > KLOGD_OPTIONS="-x" > > SYSLOG_UMASK=077 > > > >> log entries preceding the crash, ... > > > > It's a quite busy log server, with about 70 active old style syslog > servers > > sending logs to it. The second it crashed it wrote 111 log-messages.. > (273 > > the second before), mostly various postfix daemons, and I'd need to > anonymize > > them before sharing.. Can't see anything special. > > > >> If logging forwarded messages, is the remote logger also rsyslog? > > > > No, all are RHEL3/4/5 with their default syslogd server. > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Wed Sep 5 12:29:55 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Wed, 05 Sep 2007 12:29:55 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: <1188988195.3362.17.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Dienstag, den 04.09.2007, 18:58 +0200 schrieb Rainer Gerhards: > Hi, > > I noticed that one problem I received patches for (and that are now > included in 1.19.4) is rooted in something that is not fully patched. I > think I've found that root cause now (thanks to the info with that > patches). The bottom line, however, is that 1.19.4 may still have some > stability issues. However, they should surface now only in very obscure > cases (but what is obscure...). Well, at least it was running for almost two hours..... I'm running RHEL5. Here is my config: $AllowedSender UDP, 1.2.3.0/24 $AllowedSender TCP, 1.2.3.0/24 $ModLoad MySQL $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/clamav" $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/exim" $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" *.* /home/local/log/all !rsyslogd :programname, contains, "rsyslogd" /home/local/log/rsyslogd !spamd :msg, contains, "prefork: child states" ~ :msg, contains, "spamd: got connection over /opt/antispam/var/socket/spamd" ~ :msg, contains, "spamd: checking message (unknown) for exim:427" ~ :msg, contains, "spamd: handled cleanup of child pid" ~ mail.* /home/local/log/mail !clamd :msg, contains, "No stats for Database check - forcing reload" ~ :msg, contains, "Reading databases from /var/clamav" ~ :msg, contains, "Database correctly reloaded" ~ :msg, contains, "SelfCheck: Database status OK." ~ local5.* ?clamavFile !exim *.* ?eximFile :msg, contains, "malware detected" ?avFile -- CU, Patrick. From rgerhards at hq.adiscon.com Thu Sep 6 17:58:56 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Sep 2007 17:58:56 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Hi all, We are nearing the point where a decision about the future config file format needs to be made. I have blogged the details: http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html I would deeply appreciate any feedback on the samples and format suggestions. Best regards, Rainer Gerhards From janfrode at tanso.net Fri Sep 7 08:51:13 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 7 Sep 2007 08:51:13 +0200 Subject: [rsyslog] v1.19.1 is crashing References: <46D81432.1090302@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: On 2007-09-05, Jan-Frode Myklebust wrote: > On 2007-09-04, Rainer Gerhards wrote: >> >> I'd still appreciate if you could apply 1.19.4 and tell me the outcome. >> I am now working on fixing the root cause. That might take a short >> while, as I am thinking about the best *design* to fix the issue. > > OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you > know if it fails again. It failed again yesterday: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xb7209028 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8057e93] rsyslogd[0x804de4a] rsyslogd(llExecFunc+0x3f)[0x805eb0f] rsyslogd[0x804d8aa] rsyslogd[0x804d9d7] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] I have "mon" monitoring that rsyslogd is running, and restart it when it fails. "mon" restarted rsyslogd twice (Thu Sep 6 20:38, and Fri Sep 7 03:27), but I can't find any backtrace from the second crash.. -jf From rgerhards at hq.adiscon.com Fri Sep 7 09:50:32 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 09:50:32 +0200 Subject: [rsyslog] v1.19.1 is crashing In-Reply-To: References: <46D81432.1090302@redhat.com><577465F99B41C842AAFBE9ED71E70ABA27890F@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA278912@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278936@grfint2.intern.adiscon.com> Thanks for the feedback. I will probably release a new version today. It has an important fix, which hopefully solves this issue. The bad thing is that I can not reproduce the problem in my lab, so I am basically back to reviewing code and listening to your feedback ;) I have one more area (in the same class) under suspicion. But maybe I do not change that before trying out the current code change. As a side-note, the *actual* root cause was a too-complex internal API, which lead to wrong calling sequences in some parts of the code. I have now re-structured the API and revisited all places where it was called. There is another similar API and this is what I am currently reviewing. I am not sure I like to change that API without real need, because it is used a lot and any such change of course has new bug potential. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, September 07, 2007 8:51 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] v1.19.1 is crashing > > On 2007-09-05, Jan-Frode Myklebust wrote: > > On 2007-09-04, Rainer Gerhards wrote: > >> > >> I'd still appreciate if you could apply 1.19.4 and tell me the > outcome. > >> I am now working on fixing the root cause. That might take a short > >> while, as I am thinking about the best *design* to fix the issue. > > > > OK, thanks. I've upgraded my loghost to 1.19.4 now, will let you > > know if it fails again. > > It failed again yesterday: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xb7209028 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x73)[0x8057e93] > rsyslogd[0x804de4a] > rsyslogd(llExecFunc+0x3f)[0x805eb0f] > rsyslogd[0x804d8aa] > rsyslogd[0x804d9d7] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > I have "mon" monitoring that rsyslogd is running, and restart it when > it fails. "mon" restarted rsyslogd twice (Thu Sep 6 20:38, and Fri > Sep 7 03:27), but I can't find any backtrace from the second crash.. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Sep 7 13:32:13 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 7 Sep 2007 13:32:13 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Message-ID: On 2007-09-06, Rainer Gerhards wrote: > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > I would deeply appreciate any feedback on the samples and format > suggestions. /me thinks you're getting way too little feedback on the blog, or this list. Unfortunately I don't have much more than simple preference to contribute here.. XML-based format: Yikes, you'll need an additional human readable frontend format that's converted to XML for it to be usable. You can't expect us poor sysadmins to be editing XML directly to configure rsyslogd.. syslog-ng like: Fair enough.. It works for my usage. Metalog like: No experience.. Apache like: Not sure I understand this.. Seems like a mix of option/value and xml'ish for some functionality. Programming like..: Of the samples in the wiki, I most prefer the BASIC-like. It resembles python to me, and also "mon"'s config format. Very readable. http://mon.wiki.kernel.org/index.php/Mon_Manual The c-like with functions seems too complex: if1: { if(%severity < "debug" && lower(substr(%msg, 5, 3)) != "err") } action1() { action(type=filewrite, file="/var/log/mail.log") } rule1() { if1() action1() action(type=filewrite, file="/var/log/messages.log") } rule(if1,action1) ruleset(rule1, rule(if1, action(type=filewrite, file="/var/log/messages.log"))) rule(action1(),input="$all") input(type=udp, bind="127.0.0.1") I can't parse this.. Does rule1() break out of if1() is false? Then I guess writes to /var/log/messages.log woun't happen if action1 for some reason failed ? Contrast it to mon's config translated to syslogging: # Define some groups of servers: hostgroup mailservers server1 server2 server3 hostgroup webservers server4 server5 watch mailservers severity > debug SUBMSG = lower(substr(%msg, 5, 3)) SUBMSG != "err" logwrite /var/log/mail.log logwrite /var/log/messages.log SUBMSG == "err" logwrite /var/log/err.log watch webservers programname == httpd severity == crit cmd wall "httpd critical: $msg" logwrite /var/log/crit.log severity < crit logwrite /var/log/httpd.log Each indentation means it's depending on the previous statement being true. You might need to be drinking the python Kool-Aid to see the beauty :-) -jf From skvidal at fedoraproject.org Fri Sep 7 13:40:59 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 07 Sep 2007 07:40:59 -0400 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> Message-ID: <1189165259.16157.114.camel@cutter> On Fri, 2007-09-07 at 13:32 +0200, Jan-Frode Myklebust wrote: > On 2007-09-06, Rainer Gerhards wrote: > > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > I would deeply appreciate any feedback on the samples and format > > suggestions. > > /me thinks you're getting way too little feedback on the blog, > or this list. Unfortunately I don't have much more than simple > preference to contribute here.. > > XML-based format: > > Yikes, you'll need an additional human readable frontend > format that's converted to XML for it to be usable. You > can't expect us poor sysadmins to be editing XML > directly to configure rsyslogd.. The nice piece of this is that it is machine parseable easily which enables lots of useful editors. > > syslog-ng like: > > Fair enough.. It works for my usage. The syntax is okay but at that point what distinguishes b/t syslog-ng and rsyslog? > Apache like: > > Not sure I understand this.. Seems like a mix of option/value > and xml'ish for some functionality. This one I'm more interested in. If you think of each log like a vhost and you define the qualities that are added to that inside the definition Destination /path/to/silly.log DestinationMode 0640 DestinationOwner root DestinationGroup log-readers Include mail.info kern.debug cron.emerg etc, etc, etc maybe that doesn't make sense, maybe it does - it is pretty easy to read, though. -sv From rgerhards at hq.adiscon.com Fri Sep 7 14:08:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 14:08:02 +0200 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: <1189165259.16157.114.camel@cutter> References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> <1189165259.16157.114.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27893E@grfint2.intern.adiscon.com> I am replying here, but without a real reply. All feedback is deeply appreciate, but I'd like to keep silent for the time being to avoid bringing in my personal bias. Please keep commenting. I'll do a wrap-up later. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of seth vidal > Sent: Friday, September 07, 2007 1:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog config file format - please provide > feedback > > > On Fri, 2007-09-07 at 13:32 +0200, Jan-Frode Myklebust wrote: > > On 2007-09-06, Rainer Gerhards wrote: > > > > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > > > I would deeply appreciate any feedback on the samples and format > > > suggestions. > > > > /me thinks you're getting way too little feedback on the blog, > > or this list. Unfortunately I don't have much more than simple > > preference to contribute here.. > > > > XML-based format: > > > > Yikes, you'll need an additional human readable frontend > > format that's converted to XML for it to be usable. You > > can't expect us poor sysadmins to be editing XML > > directly to configure rsyslogd.. > > The nice piece of this is that it is machine parseable easily which > enables lots of useful editors. > > > > > syslog-ng like: > > > > Fair enough.. It works for my usage. > > The syntax is okay but at that point what distinguishes b/t syslog-ng > and rsyslog? > > > > Apache like: > > > > Not sure I understand this.. Seems like a mix of option/value > > and xml'ish for some functionality. > > This one I'm more interested in. If you think of each log like a vhost > and you define the qualities that are added to that inside the > definition > > > Destination /path/to/silly.log > DestinationMode 0640 > DestinationOwner root > DestinationGroup log-readers > Include mail.info kern.debug cron.emerg > > > etc, etc, etc > > maybe that doesn't make sense, maybe it does - it is pretty easy to > read, though. > > -sv > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 7 18:14:34 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 7 Sep 2007 18:14:34 +0200 Subject: [rsyslog] rsyslog 1.19.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> Hi all, Rsyslog 1.19.5 has been released. It is primarily targeted at fixing rare situations in which a segfault could occur. These have not consistently been able to reproduce in lab. Release 1.19.5 may still have a problem or may hang where previous versions segfaulted. We are actively looking for feedback from the field. Feature-wise, the $ModDir config directive has been added and $ModLoad has been enhanced. We recommend upgrading only for those that experience the problem with earlier versions or those actively interested in helping to solve the bug. If you experience a bug, please report it. Changelog: http://www.rsyslog.com/Article125.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-57.phtml As always, feedback is appreciated. Rainer Gerhards From hagen at rz.uni-karlsruhe.de Fri Sep 7 19:20:12 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Fri, 07 Sep 2007 19:20:12 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> Message-ID: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > Hi all, > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > rare situations in which a segfault could occur. These have not > consistently been able to reproduce in lab. Release 1.19.5 may still The way I see it everyone reporting segfaults so far has been using RHEL5. So perhaps everybody seeing problems could comment on the operating-system? That might ease reproducing the problem in a test-lab. > have a problem or may hang where previous versions segfaulted. We are > actively looking for feedback from the field. Feature-wise, the $ModDir > config directive has been added and $ModLoad has been enhanced. We > recommend upgrading only for those that experience the problem with > earlier versions or those actively interested in helping to solve the > bug. If you experience a bug, please report it. Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be quite "normal", however, I've been lucky and captured some strace-information. Not sure wheter that could be helpful. poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "P*\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7in-a"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "P*\205\200\0\1\0\1\0\6\0\7\00282\003185\00213\003129 \7"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14623]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 165 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\236\222\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\236\222\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14099]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 243 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\377\251\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\377\251\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>spamd[16561]: spamd: identif"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 94 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\202\24\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\202\24\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>spamd[16561]: spamd: result:"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 463 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\33\\\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\33\\\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[15976]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 143 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "I\27\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7in"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "I\27\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \003129"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "v\325\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "v\325\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 futex(0x3627949960, FUTEX_WAKE, 1) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "+\330\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "+\330\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<13>greylistd: Socket error: Bro"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 41 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "+\366\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "+\366\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "\233A\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "\233A\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) recvfrom(5, "<22>exim[14536]: 2007-09-07 18:3"..., 2047, 0, {sa_family=AF_INET, sin_port=htons(514), sin_addr=inet_addr("129.13.185.82")}, [16]) = 171 rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 open("/etc/hosts", O_RDONLY) = 17 fcntl(17, F_GETFD) = 0 fcntl(17, F_SETFD, FD_CLOEXEC) = 0 fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaab4000000 read(17, "# Do not remove the following li"..., 4096) = 234 read(17, "", 4096) = 0 close(17) = 0 munmap(0x2aaab4000000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 connect(17, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, 28) = 0 fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 sendto(17, "M\243\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7i"..., 44, MSG_NOSIGNAL, NULL, 0) = 44 poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 ioctl(17, FIONREAD, [344]) = 0 recvfrom(17, "M\243\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 close(17) = 0 socket(PF_NETLINK, SOCK_RAW, 0) = 17 bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, [4294967308]) = 0 sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(17) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 select(6, [0 4 5], [], NULL, NULL) = ? ERESTARTNOHAND (To be restarted) trace: ptrace(PTRACE_SYSCALL, ...): No such process Process 22923 detached -- CU, Patrick. From infofarmer at FreeBSD.org Fri Sep 7 19:48:14 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Fri, 7 Sep 2007 21:48:14 +0400 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <20070907174813.GE56443@amilo.cenkes.org> On Fri, Sep 07, 2007 at 07:20:12PM +0200, Patrick von der Hagen wrote: > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > > Hi all, > > > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > > rare situations in which a segfault could occur. These have not > > consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. I experience segfaults with 1.19.1 or 1.19.2 on FreeBSD, but 1.19.3 has been up for a week on end now. From mic at npgx.com.au Sat Sep 8 00:20:37 2007 From: mic at npgx.com.au (Michael Mansour) Date: Sat, 8 Sep 2007 08:20:37 +1000 Subject: [rsyslog] rsyslog config file format - please provide feedback In-Reply-To: <1189165259.16157.114.camel@cutter> References: <577465F99B41C842AAFBE9ED71E70ABA278931@grfint2.intern.adiscon.com> <1189165259.16157.114.camel@cutter> Message-ID: <20070907221314.M7794@npgx.com.au> Hi, > > > http://rgerhards.blogspot.com/2007/09/rsyslog-config-again.html > > > > > > I would deeply appreciate any feedback on the samples and format > > > suggestions. > > > > /me thinks you're getting way too little feedback on the blog, > > or this list. Unfortunately I don't have much more than simple > > preference to contribute here.. > > > > XML-based format: > > > > Yikes, you'll need an additional human readable frontend > > format that's converted to XML for it to be usable. You > > can't expect us poor sysadmins to be editing XML > > directly to configure rsyslogd.. > > The nice piece of this is that it is machine parseable easily which > enables lots of useful editors. I don't agree with the original posters comment there also. As an example, I have been using linuxha.net now for quite some years on many clusters and from day one, linuxha.net has used XML for all it's configuration files. I personally find the "standard" that brings to config files much better than the myriad of conf files I've dealt with the many more years I've been using UNIX and Linux. > > syslog-ng like: > > > > Fair enough.. It works for my usage. > > The syntax is okay but at that point what distinguishes b/t syslog-ng > and rsyslog? I've personally never been a fan of the syntax used in this. Sure I know it now after years for admin work, but I remember the times I needed to learn it thoroughly, it wasn't as easy as other conf files. > > Apache like: > > > > Not sure I understand this.. Seems like a mix of option/value > > and xml'ish for some functionality. > > This one I'm more interested in. If you think of each log like a > vhost and you define the qualities that are added to that inside the > definition > > > Destination /path/to/silly.log > DestinationMode 0640 > DestinationOwner root > DestinationGroup log-readers > Include mail.info kern.debug cron.emerg > > > etc, etc, etc > > maybe that doesn't make sense, maybe it does - it is pretty easy to > read, though. I think every sysadmin has setup a web server and delved into the apache-like configurations with software like apache, proftpd, etc. It's a nice and easy to understand format which has also proved the test of time. I'd be happy with either XML or Apache-like, but my bias is towards XML. Regards, Michael. > -sv > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From theinric at redhat.com Sat Sep 8 22:07:30 2007 From: theinric at redhat.com (Tomas Heinrich) Date: Sat, 08 Sep 2007 22:07:30 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <46E30102.20508@redhat.com> Patrick von der Hagen wrote: > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: >> Hi all, >> >> Rsyslog 1.19.5 has been released. It is primarily targeted at fixing >> rare situations in which a segfault could occur. These have not >> consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. > >> have a problem or may hang where previous versions segfaulted. We are >> actively looking for feedback from the field. Feature-wise, the $ModDir >> config directive has been added and $ModLoad has been enhanced. We >> recommend upgrading only for those that experience the problem with >> earlier versions or those actively interested in helping to solve the >> bug. If you experience a bug, please report it. > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. Thanks for the info. It will be very useful if someone can provide a core dump for 1.19.5. From rgerhards at hq.adiscon.com Mon Sep 10 11:34:17 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 11:34:17 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27896C@grfint2.intern.adiscon.com> Hhmmm... Besides a core-dump (which would definitely be useful), could those of you experiencing this problem try to run rsyslog with debug code enabled? This is NOT debug mode (-d). You enable it via ./configure --enable-debug This will generate additional debug checks in the rsyslog code. The resulting code will probably run 5 to 10 times SLOWER than production code. But it will catch many obscure errors and at least provide a hint to where the problem is (at least I hope so). If you could do that, that would be great. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Friday, September 07, 2007 7:20 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.5 released > > Am Freitag, den 07.09.2007, 18:14 +0200 schrieb Rainer Gerhards: > > Hi all, > > > > Rsyslog 1.19.5 has been released. It is primarily targeted at fixing > > rare situations in which a segfault could occur. These have not > > consistently been able to reproduce in lab. Release 1.19.5 may still > The way I see it everyone reporting segfaults so far has been using > RHEL5. So perhaps everybody seeing problems could comment on the > operating-system? That might ease reproducing the problem in a > test-lab. > > > have a problem or may hang where previous versions segfaulted. We are > > actively looking for feedback from the field. Feature-wise, the > $ModDir > > config directive has been added and $ModLoad has been enhanced. We > > recommend upgrading only for those that experience the problem with > > earlier versions or those actively interested in helping to solve the > > bug. If you experience a bug, please report it. > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. > > > > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "P*\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7in-a"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "P*\205\200\0\1\0\1\0\6\0\7\00282\003185\00213\003129 > \7"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14623]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 165 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\236\222\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\236\222\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14099]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 243 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\377\251\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\377\251\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>spamd[16561]: spamd: identif"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 94 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, > "\202\24\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\202\24\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \003"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>spamd[16561]: spamd: result:"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 463 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\33\\\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\33\\\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[15976]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 143 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "I\27\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7in"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "I\27\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \003129"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "v\325\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "v\325\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > futex(0x3627949960, FUTEX_WAKE, 1) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "+\330\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "+\330\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<13>greylistd: Socket error: Bro"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 41 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "+\366\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "+\366\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<21>exim[15583]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.81")}, [16]) = 318 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "\233A\1\0\0\1\0\0\0\0\0\0\00281\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "\233A\205\200\0\1\0\1\0\6\0\7\00281\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = 1 (in [5]) > recvfrom(5, "<22>exim[14536]: 2007-09-07 18:3"..., 2047, 0, > {sa_family=AF_INET, sin_port=htons(514), > sin_addr=inet_addr("129.13.185.82")}, [16]) = 171 > rt_sigprocmask(SIG_BLOCK, [HUP], [], 8) = 0 > open("/etc/hosts", O_RDONLY) = 17 > fcntl(17, F_GETFD) = 0 > fcntl(17, F_SETFD, FD_CLOEXEC) = 0 > fstat(17, {st_mode=S_IFREG|0644, st_size=234, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) > = 0x2aaab4000000 > read(17, "# Do not remove the following li"..., 4096) = 234 > read(17, "", 4096) = 0 > close(17) = 0 > munmap(0x2aaab4000000, 4096) = 0 > socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 17 > connect(17, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, 28) = 0 > fcntl(17, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(17, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > poll([{fd=17, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1 > sendto(17, "M\243\1\0\0\1\0\0\0\0\0\0\00282\003185\00213\003129\7i"..., > 44, MSG_NOSIGNAL, NULL, 0) = 44 > poll([{fd=17, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 > ioctl(17, FIONREAD, [344]) = 0 > recvfrom(17, "M\243\205\200\0\1\0\1\0\6\0\7\00282\003185\00213 > \00312"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), > sin_addr=inet_addr("129.13.96.2")}, [16]) = 344 > close(17) = 0 > socket(PF_NETLINK, SOCK_RAW, 0) = 17 > bind(17, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 > getsockname(17, {sa_family=AF_NETLINK, pid=22923, groups=00000000}, > [4294967308]) = 0 > sendto(17, "\24\0\0\0\26\0\1\3\213~\341F\0\0\0\0\0\0\0\0", 20, 0, > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"<\0\0\0\24\0\2\0\213~\341F\213Y\0\0\2\10 > \200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\213~\341F\213Y\0\0\n > \200\200\376\1\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = > 128 > recvmsg(17, {msg_name(12)={sa_family=AF_NETLINK, pid=0, > groups=00000000}, > msg_iov(1)=[{"\24\0\0\0\3\0\2\0\213~\341F\213Y\0\0\0\0 > \0\0\1\0\0\0\24"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 > close(17) = 0 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > futex(0x192fd054, 0x5 /* FUTEX_??? */, 1) = 1 > select(6, [0 4 5], [], NULL, NULL) = ? ERESTARTNOHAND (To be > restarted) > trace: ptrace(PTRACE_SYSCALL, ...): No such process > Process 22923 detached > > -- > CU, > Patrick. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Mon Sep 10 14:38:58 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Mon, 10 Sep 2007 14:38:58 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Freitag, den 07.09.2007, 19:20 +0200 schrieb Patrick von der Hagen: [...] > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > quite "normal", however, I've been lucky and captured some > strace-information. Not sure wheter that could be helpful. I compiled 1.19.5 with "--enable-debug" now and got the following problems: Sep 10 14:24:26 mail11 rsyslogd:could not load module '/usr/local/lib/rsyslog/ommysql.so', dlopen: /usr/local/lib/rsyslog/ommysql.so: cannot open shared object file: No such file or directory Sep 10 14:24:26 mail11 rsyslogd:the last error occured in /etc/rsyslog.conf, line 29 No idea why it tries "/usr/local/lib", the lib has been installed to "/lib" or "/opt/rsyslog/lib/rsyslog/". Hmmm, "/lib" is from 1.18.something, I didn't tidy up properly. Anyway, I currently don't use logging to MySQL, so I uncommented "$ModLoad MySQL". Next try: [root at mail11 log]# /opt/rsyslog/sbin/rsyslogd -r514 -d [...] 1084229952: Lone worker is running... 1084229952: Called fprintlog, logging to builtin-file (/home/local/log/all) 1084229952: programname filter 'rsyslogd' does not match 'spamd' Filter: check for property 'msg' (value ' prefork: child states: BBBBBBBIIBBBBBBB ') contains 'prefork: child states': TRUE 1084229952: Called fprintlog, logging to builtin-discardrsyslogd: omdiscard.c:70: doAction: Assertion `ppString != ((void *)0)' failed. Aborted That's getting a little bit strange now, it has been caused by this rsyslog.conf-lines: !spamd :msg, contains, "prefork: child states" ~ Some other crashes relate to other lines, all of them end with "~". So I uncommented all of them. Those problems are strange, but if those issues were related to my "normal" rsyslog-crashes, rsyslog would not have been able to run up to two hours and would certainly have crashed almost instantly. It's been running for several minutes now... -- CU, Patrick. From rgerhards at hq.adiscon.com Mon Sep 10 14:50:12 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 14:50:12 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com><1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278979@grfint2.intern.adiscon.com> I agree, it looks strange (very strange indeed). I am checking if it could be related to the root cause (or just be a debug-level artifact that slipped through - that may be the case, because discard is the only action which does NOT have any strings at all). Anyhow, it provides me at least another clue where to look at. Thanks Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Monday, September 10, 2007 2:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.5 released > > Am Freitag, den 07.09.2007, 19:20 +0200 schrieb Patrick von der Hagen: > [...] > > Crash of 1.19.5 after less than 30 minutes. Stack-trace seems to be > > quite "normal", however, I've been lucky and captured some > > strace-information. Not sure wheter that could be helpful. > I compiled 1.19.5 with "--enable-debug" now and got the following > problems: > > Sep 10 14:24:26 mail11 rsyslogd:could not load module > '/usr/local/lib/rsyslog/ommysql.so', > dlopen: /usr/local/lib/rsyslog/ommysql.so: cannot open shared object > file: No such file or directory > Sep 10 14:24:26 mail11 rsyslogd:the last error occured > in /etc/rsyslog.conf, line 29 > > No idea why it tries "/usr/local/lib", the lib has been installed to > "/lib" or "/opt/rsyslog/lib/rsyslog/". Hmmm, "/lib" is from > 1.18.something, I didn't tidy up properly. > > Anyway, I currently don't use logging to MySQL, so I uncommented > "$ModLoad MySQL". > > > Next try: > [root at mail11 log]# /opt/rsyslog/sbin/rsyslogd -r514 -d > [...] > 1084229952: Lone worker is running... > 1084229952: Called fprintlog, logging to builtin-file > (/home/local/log/all) > 1084229952: programname filter 'rsyslogd' does not match 'spamd' > Filter: check for property 'msg' (value ' prefork: child states: > BBBBBBBIIBBBBBBB ') contains 'prefork: child states': TRUE > 1084229952: Called fprintlog, logging to builtin-discardrsyslogd: > omdiscard.c:70: doAction: Assertion `ppString != ((void *)0)' failed. > Aborted > > > That's getting a little bit strange now, it has been caused by this > rsyslog.conf-lines: > !spamd > :msg, contains, "prefork: child states" ~ > > > Some other crashes relate to other lines, all of them end with "~". So > I > uncommented all of them. > > Those problems are strange, but if those issues were related to my > "normal" rsyslog-crashes, rsyslog would not have been able to run up to > two hours and would certainly have crashed almost instantly. > > It's been running for several minutes now... > > -- > CU, > Patrick. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Mon Sep 10 15:17:49 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Mon, 10 Sep 2007 15:17:49 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Montag, den 10.09.2007, 14:38 +0200 schrieb Patrick von der Hagen: [...] > It's been running for several minutes now... Now I captured the "real" crash. First, my config: $AllowedSender UDP, 1.2.3.0/24 $AllowedSender TCP, 1.2.3.0/24 $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/clamav" $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME %/exim" $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" *.* /home/local/log/all !rsyslogd :programname, contains, "rsyslogd" /home/local/log/rsyslogd !spamd mail.* /home/local/log/mail !clamd local5.* ?clamavFile !exim *.* ?eximFile :msg, contains, "malware detected" ?avFile Here the output of "rsyslog -r514 -d" Successful select, descriptor count = 1, Activity on: 1084229952: -1431504256: 8 Called fprintlog, logging to builtin-file1084229952: (eximFile) Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 1IUio2-00054R-EA H=X (Y) [1.2.3.4] Warning: X-Spam-Status: yes, hits=13.9, size=32842') contains 'malware detected': FALSE 1084229952: singleWorker: queue EMPTY, waiting for next message. -1431504256: Message from inetd socket: #8, host: mailin3 -1431504256: Message length: 200, File descriptor: 8. -1431504256: logmsg: mail.info<22>, flags 2, from 'mailin3', msg exim[18550]: 2007-09-10 14:56:03 H=(a.b.c.d) [2.3.4.5] F= temporarily rejected RCPT : greylisted. -1431504256: Message has legacy syslog format. -1431504256: HOSTNAME contains invalid characters, assuming it to be a TAG. -1431504256: EnqueueMsg signaled condition (0) -1431504256: 1084229952: Listening on UDP syslogd socket 7 (IPv6/port 514). -1431504256: Lone worker is running... 1084229952: Called fprintlog, logging to builtin-file (/home/local/log/all) Listening on UDP syslogd socket 8 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 8): 3 7 8 1084229952: programname filter 'rsyslogd' does not match 'exim' 1084229952: programname filter 'spamd' does not match 'exim' 1084229952: programname filter 'clamd' does not match 'exim' 1084229952: Called fprintlog, logging to builtin-file (eximFile) Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 H=(domain) [1.2.3.4] F= temporarily rejected RCPT : greylisted.') contains 'malware detected': FALSE 1084229952: singleWorker: queue EMPTY, waiting for next message. -1431504256: Successful select, descriptor count = 1, Activity on: 8 *** glibc detected *** /opt/rsyslog/sbin/rsyslogd: corrupted double-linked list: 0x00002aaaac001230 *** ======= Backtrace: ========= /lib64/libc.so.6[0x362766cb43] /lib64/libc.so.6[0x362766eea2] /lib64/libc.so.6(__libc_malloc+0x7d)[0x36276706dd] /lib64/libc.so.6[0x362765eb4a] /lib64/libnss_files.so.2[0x2aaaaaad445a] /lib64/libnss_files.so.2(_nss_files_gethostbyaddr_r +0x57)[0x2aaaaaad4b47] /lib64/libc.so.6(gethostbyaddr_r+0xf2)[0x36276e2b42] /lib64/libc.so.6(getnameinfo+0x3ad)[0x36276eb07d] /opt/rsyslog/sbin/rsyslogd(cvthname+0x154)[0x410e64] /opt/rsyslog/sbin/rsyslogd[0x40bec9] /opt/rsyslog/sbin/rsyslogd(main+0x630)[0x40c990] /lib64/libc.so.6(__libc_start_main+0xf4)[0x362761d8a4] /opt/rsyslog/sbin/rsyslogd[0x405899] ======= Memory map: ======== 00400000-00424000 r-xp 00000000 08:03 688189 /opt/rsyslog/sbin/rsyslogd 00624000-00626000 rw-p 00024000 08:03 688189 /opt/rsyslog/sbin/rsyslogd 1c204000-1c249000 rw-p 1c204000 00:00 0 40000000-40001000 ---p 40000000 00:00 0 40001000-40a01000 rw-p 40001000 00:00 0 3627200000-362721a000 r-xp 00000000 08:03 4260124 /lib64/ld-2.5.so 3627419000-362741a000 r--p 00019000 08:03 4260124 /lib64/ld-2.5.so 362741a000-362741b000 rw-p 0001a000 08:03 4260124 /lib64/ld-2.5.so 3627600000-3627744000 r-xp 00000000 08:03 4260125 /lib64/libc-2.5.so 3627744000-3627944000 ---p 00144000 08:03 4260125 /lib64/libc-2.5.so 3627944000-3627948000 r--p 00144000 08:03 4260125 /lib64/libc-2.5.so 3627948000-3627949000 rw-p 00148000 08:03 4260125 /lib64/libc-2.5.so 3627949000-362794e000 rw-p 3627949000 00:00 0 3627e00000-3627e02000 r-xp 00000000 08:03 4260128 /lib64/libdl-2.5.so 3627e02000-3628002000 ---p 00002000 08:03 4260128 /lib64/libdl-2.5.so 3628002000-3628003000 r--p 00002000 08:03 4260128 /lib64/libdl-2.5.so 3628003000-3628004000 rw-p 00003000 08:03 4260128 /lib64/libdl-2.5.so 3628200000-3628215000 r-xp 00000000 08:03 4260022 /lib64/libpthread-2.5.so 3628215000-3628414000 ---p 00015000 08:03 4260022 /lib64/libpthread-2.5.so 3628414000-3628415000 r--p 00014000 08:03 4260022 /lib64/libpthread-2.5.so 3628415000-3628416000 rw-p 00015000 08:03 4260022 /lib64/libpthread-2.5.so 3628416000-362841a000 rw-p 3628416000 00:00 0 3628600000-3628614000 r-xp 00000000 08:03 4547586 /usr/lib64/libz.so.1.2.3 3628614000-3628813000 ---p 00014000 08:03 4547586 /usr/lib64/libz.so.1.2.3 3628813000-3628814000 rw-p 00013000 08:03 4547586 /usr/lib64/libz.so.1.2.3 362d200000-362d207000 r-xp 00000000 08:03 4260133 /lib64/librt-2.5.so 362d207000-362d407000 ---p 00007000 08:03 4260133 /lib64/librt-2.5.so 362d407000-362d408000 r--p 00007000 08:03 4260133 /lib64/librt-2.5.so 362d408000-362d409000 rw-p 00008000 08:03 4260133 /lib64/librt-2.5.so 362ee00000-362ee11000 r-xp 00000000 08:03 4260134 /lib64/libresolv-2.5.so 362ee11000-362f011000 ---p 00011000 08:03 4260134 /lib64/libresolv-2.5.so 362f011000-362f012000 r--p 00011000 08:03 4260134 /lib64/libresolv-2.5.so 362f012000-362f013000 rw-p 00012000 08:03 4260134 /lib64/libresolv-2.5.so 362f013000-362f015000 rw-p 362f013000 00:00 0 3815400000-381540d000 r-xp 00000000 08:03 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 381540d000-381560c000 ---p 0000d000 08:03 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 381560c000-381560d000 rw-p 0000c000 08:03 4259862 /libAborted -- CU, Patrick. From rgerhards at hq.adiscon.com Mon Sep 10 18:26:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Sep 2007 18:26:10 +0200 Subject: [rsyslog] rsyslog segfaults was: rsyslog 1.19.5 released In-Reply-To: <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189427938.3136.29.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <1189430269.9930.4.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Message-ID: <1189441571.2659.56.camel@localhost.localdomain> Patrick, thanks for this report, it is useful. Unfortunately, I had hoped that an assertion failed, which was obviously not the case. I have looked at the code in question, but the root cause seems to be different. It looks like the code that actually aborted was "just" affected by something that happened earlier (the earlier code destroyed the list, which was then detected in cvthname() processing). The assert on discard was a bug in the debug code. I have fixed that in the current CVS (I do not yet want to release a version because of this minor thing - if you like, you can pull it from anon CVS). We have some other things under review. In the mean time, I am asking myself if the problem may be related to a threading issue. Rsyslog can be compiled in single-threading mode. You can do that by: ./configure --disable-pthreads [--enable-debug (if you like)] There is a drawback, though: as message processing and reception is no longer de-coupled, messages in bursts are more likely to be lost. Also, TCP messages sent to an unresponsive server may be lost, as the priority is on reception (there is code that discards messages that block rsyslgogd). However, sysklogd always works in that mode, so I think it is worth giving a try. It would be very helpful to know if the problem persists in single threading mode - or not. Thanks, Rainer On Mon, 2007-09-10 at 15:17 +0200, Patrick von der Hagen wrote: > Am Montag, den 10.09.2007, 14:38 +0200 schrieb Patrick von der Hagen: > [...] > > It's been running for several minutes now... > Now I captured the "real" crash. > > First, my config: > $AllowedSender UDP, 1.2.3.0/24 > $AllowedSender TCP, 1.2.3.0/24 > $template clamavFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME > %/clamav" > $template eximFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME > %/exim" > $template avFile,"/home/local/log/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/av" > *.* /home/local/log/all > !rsyslogd > :programname, contains, "rsyslogd" /home/local/log/rsyslogd > !spamd > mail.* /home/local/log/mail > !clamd > local5.* ?clamavFile > !exim > *.* ?eximFile > :msg, contains, "malware detected" ?avFile > > > Here the output of "rsyslog -r514 -d" > Successful select, descriptor count = 1, Activity on: 1084229952: > -1431504256: 8 > Called fprintlog, logging to builtin-file1084229952: (eximFile) > Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 > 1IUio2-00054R-EA H=X (Y) [1.2.3.4] Warning: X-Spam-Status: yes, > hits=13.9, size=32842') contains 'malware detected': FALSE > 1084229952: singleWorker: queue EMPTY, waiting for next message. > -1431504256: Message from inetd socket: #8, host: mailin3 > -1431504256: Message length: 200, File descriptor: 8. > -1431504256: logmsg: mail.info<22>, flags 2, from 'mailin3', msg > exim[18550]: 2007-09-10 14:56:03 H=(a.b.c.d) [2.3.4.5] F= > temporarily rejected RCPT : greylisted. > -1431504256: Message has legacy syslog format. > -1431504256: HOSTNAME contains invalid characters, assuming it to be a > TAG. > -1431504256: EnqueueMsg signaled condition (0) > -1431504256: 1084229952: Listening on UDP syslogd socket 7 (IPv6/port > 514). > -1431504256: Lone worker is running... > 1084229952: Called fprintlog, logging to builtin-file > (/home/local/log/all) > Listening on UDP syslogd socket 8 (IPv4/port 514). > -1431504256: ---------------------------------------- > -1431504256: Calling select, active file descriptors (max 8): 3 7 8 > 1084229952: programname filter 'rsyslogd' does not match 'exim' > 1084229952: programname filter 'spamd' does not match 'exim' > 1084229952: programname filter 'clamd' does not match 'exim' > 1084229952: Called fprintlog, logging to builtin-file (eximFile) > Filter: check for property 'msg' (value ' 2007-09-10 14:56:03 H=(domain) > [1.2.3.4] F= temporarily rejected RCPT > : greylisted.') contains 'malware detected': FALSE > 1084229952: singleWorker: queue EMPTY, waiting for next message. > -1431504256: > Successful select, descriptor count = 1, Activity on: 8 > *** glibc detected *** /opt/rsyslog/sbin/rsyslogd: corrupted > double-linked list: 0x00002aaaac001230 *** > ======= Backtrace: ========= > /lib64/libc.so.6[0x362766cb43] > /lib64/libc.so.6[0x362766eea2] > /lib64/libc.so.6(__libc_malloc+0x7d)[0x36276706dd] > /lib64/libc.so.6[0x362765eb4a] > /lib64/libnss_files.so.2[0x2aaaaaad445a] > /lib64/libnss_files.so.2(_nss_files_gethostbyaddr_r > +0x57)[0x2aaaaaad4b47] > /lib64/libc.so.6(gethostbyaddr_r+0xf2)[0x36276e2b42] > /lib64/libc.so.6(getnameinfo+0x3ad)[0x36276eb07d] > /opt/rsyslog/sbin/rsyslogd(cvthname+0x154)[0x410e64] > /opt/rsyslog/sbin/rsyslogd[0x40bec9] > /opt/rsyslog/sbin/rsyslogd(main+0x630)[0x40c990] > /lib64/libc.so.6(__libc_start_main+0xf4)[0x362761d8a4] > /opt/rsyslog/sbin/rsyslogd[0x405899] > ======= Memory map: ======== > 00400000-00424000 r-xp 00000000 08:03 > 688189 /opt/rsyslog/sbin/rsyslogd > 00624000-00626000 rw-p 00024000 08:03 > 688189 /opt/rsyslog/sbin/rsyslogd > 1c204000-1c249000 rw-p 1c204000 00:00 0 > 40000000-40001000 ---p 40000000 00:00 0 > 40001000-40a01000 rw-p 40001000 00:00 0 > 3627200000-362721a000 r-xp 00000000 08:03 > 4260124 /lib64/ld-2.5.so > 3627419000-362741a000 r--p 00019000 08:03 > 4260124 /lib64/ld-2.5.so > 362741a000-362741b000 rw-p 0001a000 08:03 > 4260124 /lib64/ld-2.5.so > 3627600000-3627744000 r-xp 00000000 08:03 > 4260125 /lib64/libc-2.5.so > 3627744000-3627944000 ---p 00144000 08:03 > 4260125 /lib64/libc-2.5.so > 3627944000-3627948000 r--p 00144000 08:03 > 4260125 /lib64/libc-2.5.so > 3627948000-3627949000 rw-p 00148000 08:03 > 4260125 /lib64/libc-2.5.so > 3627949000-362794e000 rw-p 3627949000 00:00 0 > 3627e00000-3627e02000 r-xp 00000000 08:03 > 4260128 /lib64/libdl-2.5.so > 3627e02000-3628002000 ---p 00002000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628002000-3628003000 r--p 00002000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628003000-3628004000 rw-p 00003000 08:03 > 4260128 /lib64/libdl-2.5.so > 3628200000-3628215000 r-xp 00000000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628215000-3628414000 ---p 00015000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628414000-3628415000 r--p 00014000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628415000-3628416000 rw-p 00015000 08:03 > 4260022 /lib64/libpthread-2.5.so > 3628416000-362841a000 rw-p 3628416000 00:00 0 > 3628600000-3628614000 r-xp 00000000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 3628614000-3628813000 ---p 00014000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 3628813000-3628814000 rw-p 00013000 08:03 > 4547586 /usr/lib64/libz.so.1.2.3 > 362d200000-362d207000 r-xp 00000000 08:03 > 4260133 /lib64/librt-2.5.so > 362d207000-362d407000 ---p 00007000 08:03 > 4260133 /lib64/librt-2.5.so > 362d407000-362d408000 r--p 00007000 08:03 > 4260133 /lib64/librt-2.5.so > 362d408000-362d409000 rw-p 00008000 08:03 > 4260133 /lib64/librt-2.5.so > 362ee00000-362ee11000 r-xp 00000000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362ee11000-362f011000 ---p 00011000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f011000-362f012000 r--p 00011000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f012000-362f013000 rw-p 00012000 08:03 > 4260134 /lib64/libresolv-2.5.so > 362f013000-362f015000 rw-p 362f013000 00:00 0 > 3815400000-381540d000 r-xp 00000000 08:03 > 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 > 381540d000-381560c000 ---p 0000d000 08:03 > 4259862 /lib64/libgcc_s-4.1.1-20070105.so.1 > 381560c000-381560d000 rw-p 0000c000 08:03 > 4259862 /libAborted > > > From janfrode at tanso.net Sun Sep 9 11:43:58 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Sep 2007 11:43:58 +0200 Subject: [rsyslog] rsyslog 1.19.5 released References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <46E30102.20508@redhat.com> Message-ID: On 2007-09-08, Tomas Heinrich wrote: > > Thanks for the info. It will be very useful if someone can provide a > core dump for 1.19.5. I haven't had the chance to upgrade yet, but any hints for how to get a core dump if it fails ? I've tried this in the init-script: ulimit -c unlimited cd /var/log/syslog echo -n $"Starting system logger (rsyslog): " daemon rsyslogd $SYSLOGD_OPTIONS but haven't gotten any core-dumps in any of the crashes I've had.. -jf From hagen at rz.uni-karlsruhe.de Tue Sep 11 10:25:08 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 11 Sep 2007 10:25:08 +0200 Subject: [rsyslog] rsyslog 1.19.5 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278954@grfint2.intern.adiscon.com> <1189185612.15694.6.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> <46E30102.20508@redhat.com> Message-ID: <1189499108.3872.5.camel@rzm-hagen-lt.rz.uni-karlsruhe.de> Am Sonntag, den 09.09.2007, 11:43 +0200 schrieb Jan-Frode Myklebust: > On 2007-09-08, Tomas Heinrich wrote: > > > > Thanks for the info. It will be very useful if someone can provide a > > core dump for 1.19.5. > > I haven't had the chance to upgrade yet, but any hints for how to get > a core dump if it fails ? I've tried this in the init-script: > > ulimit -c unlimited > cd /var/log/syslog > echo -n $"Starting system logger (rsyslog): " > daemon rsyslogd $SYSLOGD_OPTIONS > > but haven't gotten any core-dumps in any of the crashes I've had.. Hmmm. I did "ulimit -c 50000; /opt/rsyslog/sbin/rsyslogd -r514" yesterday evening and found a nice coredump this morning. I sent it to Rainer a minute ago. -- CU, Patrick. From rgerhards at hq.adiscon.com Tue Sep 11 17:09:38 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:09:38 +0200 Subject: [rsyslog] rsyslog 1.19.6 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899A@grfint2.intern.adiscon.com> Release 1.19.6 is a code cleanup and bug fixing release. It provides a number of fixes, cleans up compiler warnings and has some doc improvements. We are still investigating a problem that causes rsyslog to segfault in some constellations and are looking for active feedback. Those that help in this effort are kindly requested to update to this release, as it contains the latest code. The upgrade is recommended only for people who experience problems or would like to help with bugfixing. From rgerhards at hq.adiscon.com Tue Sep 11 17:15:32 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:15:32 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Hi all, I accidently hit the send button, so my release note for 1.19.6 already went on to the list. Anyhow, I guess you can find the relevant links yourself ;) But one important thing: I would appreciate if those of you that experienced the segfaults could try out the new version. I currently think it will *NOT* fix the problem, but I would like to verify that (and I have to admit I am not angry if I am wrong...). If it fails, it would be even greater if you could try it in single-threaded mode: ./configure --disable-pthreads I currently think the problem might be related to a threading and would like to see if that theory has some substance. Also, a test with ./configure --enable-debug Would be useful - but I have not added any asserts so if you already did this, there is no value in re-doing it with 1.9.6. Thanks for all your help. Looking forward to hear back from you. Rainer From rgerhards at hq.adiscon.com Tue Sep 11 17:46:54 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 17:46:54 +0200 Subject: [rsyslog] 1.19.6 updated Message-ID: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> Just for your info: I detected one bug that occurred during bug fixing. I fixed it and re-published 1.19.6 with the same version number because I saw now downloads until then. But maybe my counter is wrong. If you have downloaded rsyslog before I sent this message, please re-download it. Sorry for the hassle, but that fix was probably worth it. Thanks, Rainer From infofarmer at FreeBSD.org Tue Sep 11 18:04:40 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Tue, 11 Sep 2007 20:04:40 +0400 Subject: [rsyslog] 1.19.6 updated In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> Message-ID: <20070911160439.GB83726@amilo.cenkes.org> On Tue, Sep 11, 2007 at 05:46:54PM +0200, Rainer Gerhards wrote: > Just for your info: I detected one bug that occurred during bug fixing. > I fixed it and re-published 1.19.6 with the same version number because > I saw now downloads until then. But maybe my counter is wrong. If you > have downloaded rsyslog before I sent this message, please re-download > it. > > Sorry for the hassle, but that fix was probably worth it. I had to run 's/sigaction_t/sigaction/' over rfc3195d.c in order to compile it on FreeBSD. Otherwise it seems to work. Thanks! From rgerhards at hq.adiscon.com Tue Sep 11 18:40:12 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Sep 2007 18:40:12 +0200 Subject: [rsyslog] 1.19.6 updated In-Reply-To: <20070911160439.GB83726@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA27899D@grfint2.intern.adiscon.com> <20070911160439.GB83726@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789A1@grfint2.intern.adiscon.com> Ahhh... I have to admit that I currently do not really care about rfc3195d. Currently, RFC 3195 is a failure - nobody uses it and nobody asks for it. I've stopped testing it until I receive at least one real-world implementation note. Thanks for the info, will apply a patch. And, yes, I do not think that 3195 is totally dead. There is a new revision planned, and that may be very interesting. This is why I am not ditching it... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Tuesday, September 11, 2007 6:05 PM > To: rsyslog-users > Subject: Re: [rsyslog] 1.19.6 updated > > On Tue, Sep 11, 2007 at 05:46:54PM +0200, Rainer Gerhards wrote: > > Just for your info: I detected one bug that occurred during bug > fixing. > > I fixed it and re-published 1.19.6 with the same version number > because > > I saw now downloads until then. But maybe my counter is wrong. If you > > have downloaded rsyslog before I sent this message, please re- > download > > it. > > > > Sorry for the hassle, but that fix was probably worth it. > > I had to run 's/sigaction_t/sigaction/' over rfc3195d.c in order > to compile it on FreeBSD. Otherwise it seems to work. Thanks! > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Thu Sep 13 12:31:50 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 12:31:50 +0200 Subject: [rsyslog] some logged lines end with #000 Message-ID: <46E91196.301@ipax.at> hello, today i merged michael biebls debian package source rsyslog_1.19.3-1 with the current upstream rsyslog-1.19.6.tar.gz. the compilation went smooth and i installed the package on a host where there used to by klogd/sysklogd under debian etch. however, i noticed that now a lot of logged lines end with #000. it used to be: > Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. Starting heartbeat 2.1.2 > Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 > Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: 193 > ... > Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 URL:http://localhost:80/server-status [1735/1735] -> "-" [1] it now is: > Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. Starting heartbeat 2.1.2#000 > Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version 2.1.2#000 > Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: 194#000 > ... > Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 URL:http://localhost:80/server-status [1728/1728] -> "-" [1] > Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 URL:http://localhost:80/server-status [1727/1727] -> "-" [1] you will find my configuration file attached. do you have any ideas what might cause this behaviour? cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: etc_default_rsyslog URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: rsyslog.conf URL: From theinric at redhat.com Thu Sep 13 15:01:09 2007 From: theinric at redhat.com (Tomas Heinrich) Date: Thu, 13 Sep 2007 15:01:09 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E91196.301@ipax.at> References: <46E91196.301@ipax.at> Message-ID: <46E93495.8010603@redhat.com> Raoul Bhatia [IPAX] wrote: > hello, > > today i merged michael biebls debian package source rsyslog_1.19.3-1 > with the current upstream rsyslog-1.19.6.tar.gz. > > the compilation went smooth and i installed the package on a host where > there used to by klogd/sysklogd under debian etch. > > however, i noticed that now a lot of logged lines end with #000. > > it used to be: >> Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. >> Starting heartbeat 2.1.2 >> Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 >> Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: 193 >> ... >> Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 >> URL:http://localhost:80/server-status [1735/1735] -> "-" [1] >> Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 >> URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > it now is: >> Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. >> Starting heartbeat 2.1.2#000 >> Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version 2.1.2#000 >> Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: >> 194#000 >> ... >> Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 >> URL:http://localhost:80/server-status [1728/1728] -> "-" [1] >> Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 >> URL:http://localhost:80/server-status [1727/1727] -> "-" [1] > > you will find my configuration file attached. > do you have any ideas what might cause this behaviour? > > cheers, > raoul bhatia > > > ------------------------------------------------------------------------ > > # Options to rsyslogd > # -m 0 disables 'MARK' messages. > # -r enables logging from remote machines > # -x disables DNS lookups on messages recieved with -r > # See rsyslogd(8) for more details > RSYSLOGD_OPTIONS="-m 0" > > # Options to rklogd > # -2 prints all kernel oops messages twice; once for klogd to decode, and > # once for processing with 'ksymoops' > # -x disables all klogd processing of oops messages entirely > # See rklogd(8) for more details > RKLOGD_OPTIONS="-x" > > > > ------------------------------------------------------------------------ > > # /etc/rsyslog.conf Configuration file for rsyslogd. > # > # For more information see > # /usr/share/doc/rsyslog/html/rsyslog_conf.html > > # > # First some standard logfiles. Log by facility. > # > > auth,authpriv.* /var/log/auth.log > *.*;auth,authpriv.none -/var/log/syslog > #cron.* /var/log/cron.log > daemon.* -/var/log/daemon.log > kern.* -/var/log/kern.log > lpr.* -/var/log/lpr.log > mail.* -/var/log/mail.log > user.* -/var/log/user.log > > # > # Logging for the mail system. Split it up so that > # it is easy to write scripts to parse these files. > # > mail.info -/var/log/mail.info > mail.warn -/var/log/mail.warn > mail.err /var/log/mail.err > > # > # Logging for INN news system > # > news.crit /var/log/news/news.crit > news.err /var/log/news/news.err > news.notice -/var/log/news/news.notice > > # > # Some `catch-all' logfiles. > # > *.=debug;\ > auth,authpriv.none;\ > news.none;mail.none -/var/log/debug > *.=info;*.=notice;*.=warn;\ > auth,authpriv.none;\ > cron,daemon.none;\ > mail,news.none -/var/log/messages > > # > # Emergencies are sent to everybody logged in. > # > *.emerg * > > # > # I like to have messages displayed on the console, but only on a virtual > # console I usually leave idle. > # > #daemon,mail.*;\ > # news.=crit;news.=err;news.=notice;\ > # *.=debug;*.=info;\ > # *.=notice;*.=warn /dev/tty8 > > # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, > # you must invoke `xconsole' with the `-file' option: > # > # $ xconsole -file /dev/xconsole [...] > # > # NOTE: adjust the list below, or you'll go crazy if you have a reasonably > # busy site.. > # > daemon.*;mail.*;\ > news.err;\ > *.=debug;*.=info;\ > *.=notice;*.=warn |/dev/xconsole > # > # Include all config files in /etc/rsyslog.d/ > # > $IncludeConfig /etc/rsyslog.d/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog Hi, I've tinkered with this part of code recently, so it could be my fault. But I wasn't able to reproduce it so far and a quick look through the source didn't show anything suspicious. Just a blind guess: $EscapeControlCharactersOnReceive off It will prevent control chars from being escaped but maybe the problem will go away for now. You have a $IncludeConfig /etc/rsyslog.d/ in you conf file; is there any other part of configuration? I will look deeper into this later today. From rgerhards at hq.adiscon.com Thu Sep 13 15:48:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Sep 2007 15:48:02 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E91196.301@ipax.at> References: <46E91196.301@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> I remember the #000. I think in the case that I remember, there was actually a NUL character written by the remote system. It may also be that a too-long message (off by one) is written in the output code. Maybe this is even a clue to the bug we are hunting. You can remove them by turning off control character escaping - but that just a cosmetic fix, it doesn't fix the root issue, which we will need to look at. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Thursday, September 13, 2007 12:32 PM > To: rsyslog-users > Subject: [rsyslog] some logged lines end with #000 > > hello, > > today i merged michael biebls debian package source rsyslog_1.19.3-1 > with the current upstream rsyslog-1.19.6.tar.gz. > > the compilation went smooth and i installed the package on a host where > there used to by klogd/sysklogd under debian etch. > > however, i noticed that now a lot of logged lines end with #000. > > it used to be: > > Sep 13 09:27:02 xxx heartbeat: [2454]: info: Configuration validated. > Starting heartbeat 2.1.2 > > Sep 13 09:27:02 xxx heartbeat: [2455]: info: heartbeat: version 2.1.2 > > Sep 13 09:27:02 xxx heartbeat: [2455]: info: Heartbeat generation: > 193 > > ... > > Sep 13 11:08:52 xxx apache[24790]: INFO: 11:08:52 > URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > Sep 13 11:09:02 xxx apache[24872]: INFO: 11:09:02 > URL:http://localhost:80/server-status [1735/1735] -> "-" [1] > > it now is: > > Sep 13 12:12:30 xxx heartbeat: [5096]: info: Configuration validated. > Starting heartbeat 2.1.2#000 > > Sep 13 12:12:30 xxx heartbeat: [5097]: info: heartbeat: version > 2.1.2#000 > > Sep 13 12:12:30 xxx heartbeat: [5097]: info: Heartbeat generation: > 194#000 > > ... > > Sep 13 12:28:35 xxx apache[18033]: INFO: 12:28:35 > URL:http://localhost:80/server-status [1728/1728] -> "-" [1] > > Sep 13 12:28:46 xxx apache[18243]: INFO: 12:28:46 > URL:http://localhost:80/server-status [1727/1727] -> "-" [1] > > you will find my configuration file attached. > do you have any ideas what might cause this behaviour? > > cheers, > raoul bhatia > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ From r.bhatia at ipax.at Thu Sep 13 16:16:39 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 16:16:39 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789BB@grfint2.intern.adiscon.com> Message-ID: <46E94647.1090709@ipax.at> Rainer Gerhards wrote: > I remember the #000. I think in the case that I remember, there was > actually a NUL character written by the remote system. It may also be > that a too-long message (off by one) is written in the output code. > Maybe this is even a clue to the bug we are hunting. > > You can remove them by turning off control character escaping - but that > just a cosmetic fix, it doesn't fix the root issue, which we will need > to look at. so then please tell me how to help :) cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From r.bhatia at ipax.at Thu Sep 13 16:33:26 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Thu, 13 Sep 2007 16:33:26 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E93495.8010603@redhat.com> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> Message-ID: <46E94A36.5050701@ipax.at> Tomas Heinrich wrote: > Hi, > > I've tinkered with this part of code recently, so it could be my fault. > > But I wasn't able to reproduce it so far and a quick look through the > source didn't show anything suspicious. > Just a blind guess: > $EscapeControlCharactersOnReceive off > It will prevent control chars from being escaped but maybe the problem > will go away for now. this does not seem to help. > You have a > $IncludeConfig /etc/rsyslog.d/ > in you conf file; is there any other part of configuration? no, there aren't any configuration files in there. see: > root at xxx /var/log # find /etc/rsyslog.d -type f > root at xxx /var/log # raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Thu Sep 13 17:31:01 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Sep 2007 17:31:01 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46E94A36.5050701@ipax.at> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Thursday, September 13, 2007 4:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > Tomas Heinrich wrote: > > Hi, > > > > I've tinkered with this part of code recently, so it could be my > fault. > > > > But I wasn't able to reproduce it so far and a quick look through the > > source didn't show anything suspicious. > > Just a blind guess: > > $EscapeControlCharactersOnReceive off > > It will prevent control chars from being escaped but maybe the > problem > > will go away for now. > > this does not seem to help. Ummm... should have thought about this. NULs are treated special. This is routed in the sysklogd code. Rsyslog uses standard C strings for the most part. Standard C strings are terminated as soon as the first NUL is detected. So we can not allow NULs to be part of the message. Thus, NUL is always escaped, no matter what the control character escape sequences is. Can you provide us with a debug log (rsyslogd -d -n) showing a message with #000 being processed? Rainer > > > You have a > > $IncludeConfig /etc/rsyslog.d/ > > in you conf file; is there any other part of configuration? > > no, there aren't any configuration files in there. see: > > root at xxx /var/log # find /etc/rsyslog.d -type f > > root at xxx /var/log # > > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Fri Sep 14 10:06:33 2007 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 14 Sep 2007 10:06:33 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at> <46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> Message-ID: <46EA4109.3060009@ipax.at> Rainer Gerhards wrote: > Ummm... should have thought about this. NULs are treated special. This > is routed in the sysklogd code. Rsyslog uses standard C strings for the > most part. Standard C strings are terminated as soon as the first NUL is > detected. So we can not allow NULs to be part of the message. Thus, NUL > is always escaped, no matter what the control character escape sequences > is. > > Can you provide us with a debug log (rsyslogd -d -n) showing a message > with #000 being processed? attached, you will find the output. cheers, raoul bhatia -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Sep 14 10:17:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Sep 2007 10:17:10 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <46EA4109.3060009@ipax.at> References: <46E91196.301@ipax.at><46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at><577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com> <46EA4109.3060009@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> The list server is quite picky on attachment types (aka "does not accept most of them" ;)). And I have to admit I introduced that policy and still think it is not a bad one. Can you send me the attachment via private mail please. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Friday, September 14, 2007 10:07 AM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > Rainer Gerhards wrote: > > Ummm... should have thought about this. NULs are treated special. > This > > is routed in the sysklogd code. Rsyslog uses standard C strings for > the > > most part. Standard C strings are terminated as soon as the first NUL > is > > detected. So we can not allow NULs to be part of the message. Thus, > NUL > > is always escaped, no matter what the control character escape > sequences > > is. > > > > Can you provide us with a debug log (rsyslogd -d -n) showing a > message > > with #000 being processed? > > attached, you will find the output. > > cheers, > raoul bhatia > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Sep 14 11:15:35 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Sep 2007 11:15:35 +0200 Subject: [rsyslog] some logged lines end with #000 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> References: <46E91196.301@ipax.at><46E93495.8010603@redhat.com> <46E94A36.5050701@ipax.at><577465F99B41C842AAFBE9ED71E70ABA2789BE@grfint2.intern.adiscon.com><46EA4109.3060009@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA2789DC@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA2789E1@grfint2.intern.adiscon.com> I have created a new version which is currently available under the interim name of http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz [***This is NOT 1.19.7 but will become it over time!***] In that version, I have handled NULs that are emitted by faulty senders (and there seem to be lots of them). A quick test both by me and Raoul Bhatia indicates that those NULs disappear. And so do LF NUL (#012#000) as the LF shown is an artifact from the NUL being then the last char of the message). I would now be interested to know if somebody still sees unexpected #000 at the very end of the message. If there are, they now almost for sure identify a rsyslog problem. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, September 14, 2007 10:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] some logged lines end with #000 > > The list server is quite picky on attachment types (aka "does not > accept > most of them" ;)). And I have to admit I introduced that policy and > still think it is not a bad one. > > Can you send me the attachment via private mail please. > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > > Sent: Friday, September 14, 2007 10:07 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] some logged lines end with #000 > > > > Rainer Gerhards wrote: > > > Ummm... should have thought about this. NULs are treated special. > > This > > > is routed in the sysklogd code. Rsyslog uses standard C strings for > > the > > > most part. Standard C strings are terminated as soon as the first > NUL > > is > > > detected. So we can not allow NULs to be part of the message. Thus, > > NUL > > > is always escaped, no matter what the control character escape > > sequences > > > is. > > > > > > Can you provide us with a debug log (rsyslogd -d -n) showing a > > message > > > with #000 being processed? > > > > attached, you will find the output. > > > > cheers, > > raoul bhatia > > -- > > ____________________________________________________________________ > > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > > Technischer Leiter > > > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > > Barawitzkagasse 10/2/2/11 email. office at ipax.at > > 1190 Wien tel. +43 1 3670030 > > FN 277995t HG Wien fax. +43 1 3670030 15 > > ____________________________________________________________________ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 17 14:51:10 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 14:51:10 +0200 Subject: [rsyslog] rsyslog segfaults? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> Hi all, may I ask if those of you that experienced the segfault problem could re-produce it with 1.19.6? Any feedback would be deeply appreciated. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Sep 17 18:21:09 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 18:21:09 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <1190044055.10861.7.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <1190044055.10861.7.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A19@grfint2.intern.adiscon.com> Patrick, without answering the question directly: could you please try out http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz [**still not "real" 1.19.7 **] I've possibly seen one thing that could be the problem cause. Not sure, though. I'd deeply appreciate feedback if this version (uploaded right NOW) causes the problem to disappear. I am not sure if I have found it, but I'd like to conduct a quick test before going any further. Thus I've no detailed analysis. Thanks, Rainer > -----Original Message----- > From: Patrick von der Hagen [mailto:hagen at rz.uni-karlsruhe.de] > Sent: Monday, September 17, 2007 5:48 PM > To: Rainer Gerhards > Cc: theinric > Subject: Re: [rsyslog] rsyslog segfaults? > > On Mon, 2007-09-17 at 14:51 +0200, Rainer Gerhards wrote: > > Hi all, > > > > may I ask if those of you that experienced the segfault problem could > > re-produce it with 1.19.6? Any feedback would be deeply appreciated. > Still dumps core, haven't tried it single-threaded yet. > > I attach some gdb information I got when examining a core-dump. > > I'm no programmer and don't even understand half of it, but I'm > troubled > by ' ip = "129.13.185.81\000\000 > $a196a97a at akstcacsdatarecoverymnsdgs>,bayes=1.000000,autolearn=spam > \00090$88ab8c55 at 082d6f5a>,bayes=1.000000,autolearn=spam \000n=spam > \000id=<01c7f6df$57aeec90$6a49cd18 at 0aconingham>,bayes=1.000"'. > > It looks like several log-lines merged into one, which might hint at > threading-issues. > > > > [root at mail11 ~]# cd /opt/rsyslog > [root at mail11 rsyslog]# ls > core.14004 lib sbin share > [root at mail11 rsyslog]# gdb -c core.14004 /opt/rsyslog/sbin/rsyslogd > GNU gdb Red Hat Linux (6.5-16.el5rh) > Copyright (C) 2006 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "x86_64-redhat-linux-gnu"...Using host > libthread_db library "/lib64/libthread_db.so.1". > > Reading symbols from /usr/lib64/libz.so.1...done. > Loaded symbols for /usr/lib64/libz.so.1 > Reading symbols from /lib64/libpthread.so.0...done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /lib64/libdl.so.2...done. > Loaded symbols for /lib64/libdl.so.2 > Reading symbols from /lib64/librt.so.1...done. > Loaded symbols for /lib64/librt.so.1 > Reading symbols from /lib64/libc.so.6...done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /lib64/ld-linux-x86-64.so.2...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libnss_files.so.2...done. > Loaded symbols for /lib64/libnss_files.so.2 > Reading symbols from /lib64/libnss_dns.so.2...done. > Loaded symbols for /lib64/libnss_dns.so.2 > Reading symbols from /lib64/libresolv.so.2...done. > Loaded symbols for /lib64/libresolv.so.2 > Reading symbols from /lib64/libgcc_s.so.1...done. > Loaded symbols for /lib64/libgcc_s.so.1 > Core was generated by `/opt/rsyslog/sbin/rsyslogd -r514'. > Program terminated with signal 6, Aborted. > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > (gdb) backtrace > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > #1 0x0000003627631980 in abort () from /lib64/libc.so.6 > #2 0x00000036276674db in __libc_message () from /lib64/libc.so.6 > #3 0x000000362766cb43 in malloc_consolidate () from /lib64/libc.so.6 > #4 0x000000362766eea2 in _int_malloc () from /lib64/libc.so.6 > #5 0x00000036276706dd in malloc () from /lib64/libc.so.6 > #6 0x000000362765eb4a in __fopen_internal () from /lib64/libc.so.6 > #7 0x00002aaaaaaf545a in internal_setent () > from /lib64/libnss_files.so.2 > #8 0x00002aaaaaaf5b47 in _nss_files_gethostbyaddr_r () > from /lib64/libnss_files.so.2 > #9 0x00000036276e2b42 in gethostbyaddr_r@@GLIBC_2.2.5 () > from /lib64/libc.so.6 > #10 0x00000036276eb07d in getnameinfo () from /lib64/libc.so.6 > #11 0x00000000004155ee in cvthname (f=0x7ffff040b0d0, > pszHost=0x7ffff040acc0 "mailin3", > pszHostFQDN=0x7ffff040a8b0 "mailin3.rz.uni-karlsruhe.de") at > net.c:137 > #12 0x000000000040e49d in processSelectAfter (maxfds=5, nfds=1, > pReadfds=0x7ffff040ba60, pWritefds=0x7ffff040b9c0) at > syslogd.c:5619 > #13 0x000000000040ee02 in mainloop () at syslogd.c:5869 > #14 0x000000000040fc9c in main (argc=0, argv=0x7ffff040bd08) at > syslogd.c:6315 > (gdb) where full > #0 0x0000003627630015 in raise () from /lib64/libc.so.6 > No symbol table info available. > #1 0x0000003627631980 in abort () from /lib64/libc.so.6 > No symbol table info available. > #2 0x00000036276674db in __libc_message () from /lib64/libc.so.6 > No symbol table info available. > #3 0x000000362766cb43 in malloc_consolidate () from /lib64/libc.so.6 > No symbol table info available. > #4 0x000000362766eea2 in _int_malloc () from /lib64/libc.so.6 > No symbol table info available. > #5 0x00000036276706dd in malloc () from /lib64/libc.so.6 > No symbol table info available. > #6 0x000000362765eb4a in __fopen_internal () from /lib64/libc.so.6 > No symbol table info available. > #7 0x00002aaaaaaf545a in internal_setent () > from /lib64/libnss_files.so.2 > No symbol table info available. > #8 0x00002aaaaaaf5b47 in _nss_files_gethostbyaddr_r () > from /lib64/libnss_files.so.2 > No symbol table info available. > #9 0x00000036276e2b42 in gethostbyaddr_r@@GLIBC_2.2.5 () > from /lib64/libc.so.6 > No symbol table info available. > #10 0x00000036276eb07d in getnameinfo () from /lib64/libc.so.6 > No symbol table info available. > ---Type to continue, or q to quit--- > #11 0x00000000004155ee in cvthname (f=0x7ffff040b0d0, > pszHost=0x7ffff040acc0 "mailin3", > pszHostFQDN=0x7ffff040a8b0 "mailin3.rz.uni-karlsruhe.de") at > net.c:137 > p = (uchar *) 0x7ffff040acc7 "" > count = -264195888 > error = 0 > omask = {__val = {0, 4251742, 0, 140737224155248, > 140737224155240, > 14083552, 0, 18446735427676189008, 140737224155264, 4336648, > 140737224155264, 232601472906, 0, 64, 140737224159584, 2047}} > nmask = {__val = {1, 0 }} > ip = "129.13.185.81\000\000 > $a196a97a at akstcacsdatarecoverymnsdgs>,bayes=1.000000,autolearn=spam > \00090$88ab8c55 at 082d6f5a>,bayes=1.000000,autolearn=spam \000n=spam > \000id=<01c7f6df$57aeec90$6a49cd18 at 0aconingham>,bayes=1.000"... > hints = {ai_flags = 4, ai_family = 0, ai_socktype = 2, > ai_protocol = 0, ai_addrlen = 0, ai_addr = 0x0, ai_canonname = 0x0, > ai_next = 0x0} > res = (struct addrinfo *) 0x2e302e3732313d72 > __PRETTY_FUNCTION__ = "cvthname" > #12 0x000000000040e49d in processSelectAfter (maxfds=5, nfds=1, > pReadfds=0x7ffff040ba60, pWritefds=0x7ffff040b9c0) at > syslogd.c:5619 > iRet = RS_RET_OK > iRetLL = RS_RET_OK > i = 1 > ---Type to continue, or q to quit--- > fd = 0 > line = "<22>exim[32680]: 2007-09-14 17:02:19 1IWCgS-0008Tb-B5 > Completed\n", '\0' > writeFDSInfo = {pWritefds = 0x7ffff040b9c0, pMaxfds = > 0x7ffff040a0a8} > f = (selector_t *) 0x0 > frominet = {ss_family = 2, __ss_align = 0, > __ss_padding = "@\023\000??*\000\000@\023\000??*\000\000@\023\000??* > \000\000B\023\000??*\000\000O\023\000??*\000\000@\023\000??*\000\000O > \023\000??*", '\0' , "\017\201B\000\000\000\000\000? > \230\aF\000\000\000"} > socklen = 16 > fromHost = "mailin3\000rz.uni-karlsruhe.de", '\0' times>, "\210_ '6", '\0' , "?? '6", '\0' times>, "??A'6\000\000\000o\b?'6\000\000\000\220{ '6\000\000\000\000 > \000`'6\000\000\000\000 at t'6\000\000\000\220:t'6\000\000\000\220:t'6", > '\0' , "\005\000\000\000\000\000\000\000\000@\224'6", > '\0' , "\001\000\000\000\000\000\000\000\230?\224'6 > \000\000\000\000@\024\000\000\000\000\000\003\000\000\000\000\000\000 > \000\000\000 -6\000\000\000\000p -6\000\000"... > fromHostFQDN = "mailin3.rz.uni-karlsruhe.de", '\0' times>, "\001\000\000\000\000\000\000\000?@??\177\000\000?*q'6", '\0' > , "??e'6", '\0' , "??@??\177\000 > \000/\201B\000\000\000\000\000/\201B", '\0' , > "?4d'6", > '\0' , "\200?@??\177\000\000\000\000\000\000\000\000 > \000\000??@??\177\000\0000?@??\177\000\000*\201B\000\000\000\000\000?@? > ? > \177", '\0' , "????????\000\000\000---Type > to > continue, or q to quit--- > \000\002\000\000\000*"... > iTCPSess = 0 > l = 64 > #13 0x000000000040ee02 in mainloop () at syslogd.c:5869 > readfds = {fds_bits = {32, 0 }} > i = 2 > maxfds = 5 > nfds = 1 > writeFDSInfo = {pWritefds = 0x7ffff040b9c0, pMaxfds = > 0x7ffff040ba5c} > writefds = {fds_bits = {0 }} > f = (selector_t *) 0x0 > iTCPSess = 14003 > #14 0x000000000040fc9c in main (argc=0, argv=0x7ffff040bd08) at > syslogd.c:6315 > i = 1024 > p = 0x63053a "" > num_fds = 1024 > iRet = RS_RET_OK > ppid = 14003 > ch = -1 > hent = (struct hostent *) 0x362794bf80 > pTmp = (uchar *) 0x62fe95 "" > sigAct = {__sigaction_handler = {sa_handler = 0x1, > sa_sigaction = 0x1}, sa_mask = {__val = {0 }}, > ---Type to continue, or q to quit--- > sa_flags = 0, sa_restorer = 0} > (gdb) > > > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- > Patrick von der Hagen > RZ Universit?t Karlsruhe (TH) > Postmaster From rgerhards at hq.adiscon.com Mon Sep 17 18:27:38 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 17 Sep 2007 18:27:38 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <1190046420.10861.9.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <1190044055.10861.7.camel@localhost.localdomain> <577465F99B41C842AAFBE9ED71E70ABA278A19@grfint2.intern.adiscon.com> <1190046420.10861.9.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A1B@grfint2.intern.adiscon.com> Excellent, thx > -----Original Message----- > From: Patrick von der Hagen [mailto:hagen at rz.uni-karlsruhe.de] > Sent: Monday, September 17, 2007 6:27 PM > To: Rainer Gerhards > Cc: rsyslog-users > Subject: RE: [rsyslog] rsyslog segfaults? > > On Mon, 2007-09-17 at 18:21 +0200, Rainer Gerhards wrote: > > Patrick, > > > > without answering the question directly: could you please try out > > > > http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz > > > > [**still not "real" 1.19.7 **] > > > > I've possibly seen one thing that could be the problem cause. Not > sure, though. I'd deeply appreciate feedback if this version (uploaded > right NOW) causes the problem to disappear. I am not sure if I have > found it, but I'd like to conduct a quick test before going any > further. Thus I've no detailed analysis. > I'm currently giving it a try, but don't expect any results today. > -- > Patrick von der Hagen > RZ Universit?t Karlsruhe (TH) > Postmaster From infofarmer at FreeBSD.org Wed Sep 19 10:55:19 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Wed, 19 Sep 2007 12:55:19 +0400 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> Message-ID: <20070919085517.GL98847@amilo.cenkes.org> On Mon, Sep 17, 2007 at 02:51:10PM +0200, Rainer Gerhards wrote: > Hi all, > > may I ask if those of you that experienced the segfault problem could > re-produce it with 1.19.6? Any feedback would be deeply appreciated. The box is in production, so I can't do too much debugging. If it may help, I can recompile rsyslogd with debug flags set and run gdb against it. root at apollo# rsyslogd -v /usr/home/mob/www rsyslogd 1.19.6, compiled with: FEATURE_PTHREADS (dual-threading): Yes FEATURE_REGEXP: Yes FEATURE_DB (MySQL): Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes SYSLOG_INET (Internet/remote support): Yes sat at apollo% gdb =rsyslogd rsyslogd.core <...> #0 0x000000080078189c in pthread_testcancel () from /lib/libpthread.so.2 [New Thread 0x537000 (LWP 100117)] [New Thread 0x52e400 (LWP 100112)] [New Thread 0x52e000 (runnable)] (gdb) bt #0 0x000000080078189c in pthread_testcancel () from /lib/libpthread.so.2 #1 0x000000080076f5c3 in sigaction () from /lib/libpthread.so.2 #2 0x00000008007710e2 in sigaction () from /lib/libpthread.so.2 #3 0x000000080076adb6 in pthread_kill () from /lib/libpthread.so.2 #4 0x000000080076a633 in raise () from /lib/libpthread.so.2 #5 0x000000080095b16d in abort () from /lib/libc.so.6 #6 0x00000008008f4b45 in _UTF8_init () from /lib/libc.so.6 #7 0x00000008008f4b7c in _UTF8_init () from /lib/libc.so.6 #8 0x00000008008f5b1d in _UTF8_init () from /lib/libc.so.6 #9 0x000000000040fa1d in MsgDestruct () #10 0x00000000004066bd in dbgprintf () #11 0x00000000004176b1 in llExecFunc () #12 0x0000000000406f41 in shouldProcessThisMessage () #13 0x0000000000409a9e in logerrorSz () #14 0x0000000800772a99 in pthread_create () from /lib/libpthread.so.2 #15 0x00000008008cfcd4 in makecontext () from /lib/libc.so.6 #16 0x0000000000000000 in ?? () #17 0x0000000000537000 in ?? () #18 0x00000000004099c0 in logerrorSz () #19 0x0000000000000000 in ?? () #20 0x0000000000000000 in ?? () #21 0x0000000000000000 in ?? () #22 0x0000000000000000 in ?? () Cannot access memory at address 0x7fffff5fc000 From rgerhards at hq.adiscon.com Wed Sep 19 13:53:41 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 19 Sep 2007 13:53:41 +0200 Subject: [rsyslog] rsyslog segfaults? In-Reply-To: <20070919085517.GL98847@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA278A0E@grfint2.intern.adiscon.com> <20070919085517.GL98847@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A3D@grfint2.intern.adiscon.com> Andrew, this is quite helpful. While it does not pinpoint the actual trouble source, it points to a different abort location. All previous segfaults pointed to a different location (Which I have now checked multiple times and not found any notable problems, just cosmetic things). This strengthens the point it may be related to threading - of course, problematic to troubleshoot. I'll see how I proceed from here. Again, if some of you that experience the problem could run rsyslogd compiled for single threading, that would be most helpful. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Wednesday, September 19, 2007 10:55 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog segfaults? > > On Mon, Sep 17, 2007 at 02:51:10PM +0200, Rainer Gerhards wrote: > > Hi all, > > > > may I ask if those of you that experienced the segfault problem could > > re-produce it with 1.19.6? Any feedback would be deeply appreciated. > > The box is in production, so I can't do too much debugging. If it > may help, I can recompile rsyslogd with debug flags set and run > gdb against it. > > root at apollo# rsyslogd -v > /usr/home/mob/www > rsyslogd 1.19.6, compiled with: > FEATURE_PTHREADS (dual-threading): Yes > FEATURE_REGEXP: Yes > FEATURE_DB (MySQL): Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > SYSLOG_INET (Internet/remote support): Yes > > sat at apollo% gdb =rsyslogd rsyslogd.core > <...> > #0 0x000000080078189c in pthread_testcancel () from > /lib/libpthread.so.2 > [New Thread 0x537000 (LWP 100117)] > [New Thread 0x52e400 (LWP 100112)] > [New Thread 0x52e000 (runnable)] > (gdb) bt > #0 0x000000080078189c in pthread_testcancel () from > /lib/libpthread.so.2 > #1 0x000000080076f5c3 in sigaction () from /lib/libpthread.so.2 > #2 0x00000008007710e2 in sigaction () from /lib/libpthread.so.2 > #3 0x000000080076adb6 in pthread_kill () from > /lib/libpthread.so.2 > #4 0x000000080076a633 in raise () from /lib/libpthread.so.2 > #5 0x000000080095b16d in abort () from /lib/libc.so.6 > #6 0x00000008008f4b45 in _UTF8_init () from /lib/libc.so.6 > #7 0x00000008008f4b7c in _UTF8_init () from /lib/libc.so.6 > #8 0x00000008008f5b1d in _UTF8_init () from /lib/libc.so.6 > #9 0x000000000040fa1d in MsgDestruct () > #10 0x00000000004066bd in dbgprintf () > #11 0x00000000004176b1 in llExecFunc () > #12 0x0000000000406f41 in shouldProcessThisMessage () > #13 0x0000000000409a9e in logerrorSz () > #14 0x0000000800772a99 in pthread_create () from > /lib/libpthread.so.2 > #15 0x00000008008cfcd4 in makecontext () from /lib/libc.so.6 > #16 0x0000000000000000 in ?? () > #17 0x0000000000537000 in ?? () > #18 0x00000000004099c0 in logerrorSz () > #19 0x0000000000000000 in ?? () > #20 0x0000000000000000 in ?? () > #21 0x0000000000000000 in ?? () > #22 0x0000000000000000 in ?? () > Cannot access memory at address 0x7fffff5fc000 > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 21 14:07:55 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 21 Sep 2007 14:07:55 +0200 Subject: [rsyslog] rsyslog segfault Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A67@grfint2.intern.adiscon.com> To those of you that experience the problem: I just got note that the problem possibly disappears in single-threading mode. We are looking at this. In the mean time, you may want to use ./configure --disable-pthreads I just thought I pass this around. Rainer From janfrode at tanso.net Sun Sep 23 11:27:53 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 23 Sep 2007 11:27:53 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Message-ID: On 2007-09-11, Rainer Gerhards wrote: > > But one important thing: I would appreciate if those of you that > experienced the segfaults could try out the new version. I currently > think it will *NOT* fix the problem, but I would like to verify that > (and I have to admit I am not angry if I am wrong...). Sorry, you were right: *** glibc detected *** rsyslogd: corrupted double-linked list: 0xb7214c98 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152ce3e] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x46)[0x80580b6] rsyslogd[0x804df9a] rsyslogd(llExecFunc+0x3f)[0x805f11f] rsyslogd[0x804d9ea] rsyslogd[0x804db3f] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] With 1.19.6. > If it fails, it would be even greater if you could try it in > single-threaded mode: > > ./configure --disable-pthreads Is this safe to use on a busy system? I wouldn't want to start dropping lots of packets.. -jf From rgerhards at hq.adiscon.com Mon Sep 24 11:16:42 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 11:16:42 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> Thanks for the report. I'd not yet use the single-threading version. The performance hit should not be severe, but I think I'll be able to release another version today. Not sure, of course, if it actually fixes the bug, but I am working on it. I got a report that it is indeed related to a threading issue and I am now checking all runtime functions that get called. At least one I have found to be not thread-safe. I intend to release a version with that fix and a couple of minor things, even if I have not yet completed the full runtime-checks. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Sunday, September 23, 2007 11:28 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > On 2007-09-11, Rainer Gerhards wrote: > > > > But one important thing: I would appreciate if those of you that > > experienced the segfaults could try out the new version. I currently > > think it will *NOT* fix the problem, but I would like to verify that > > (and I have to admit I am not angry if I am wrong...). > > Sorry, you were right: > > *** glibc detected *** rsyslogd: corrupted double-linked list: > 0xb7214c98 *** > ======= Backtrace: ========= > /lib/libc.so.6[0x4152ce3e] > /lib/libc.so.6(cfree+0x90)[0x415305d0] > rsyslogd(MsgDestruct+0x46)[0x80580b6] > rsyslogd[0x804df9a] > rsyslogd(llExecFunc+0x3f)[0x805f11f] > rsyslogd[0x804d9ea] > rsyslogd[0x804db3f] > /lib/libpthread.so.0[0x416112db] > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > With 1.19.6. > > > If it fails, it would be even greater if you could try it in > > single-threaded mode: > > > > ./configure --disable-pthreads > > Is this safe to use on a busy system? I wouldn't want to start dropping > lots of packets.. > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 24 11:25:44 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 11:25:44 +0200 Subject: [rsyslog] rsyslog 1.19.6 and segfaults In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA27899B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278A86@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A87@grfint2.intern.adiscon.com> Ummm... Importnat word mising. I'd not use the single-threading version NOW. The reasoning was in the mail. I may ask some time later for it, though. Sorry... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, September 24, 2007 11:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > Thanks for the report. I'd not yet use the single-threading version. > The > performance hit should not be severe, but I think I'll be able to > release another version today. Not sure, of course, if it actually > fixes > the bug, but I am working on it. I got a report that it is indeed > related to a threading issue and I am now checking all runtime > functions > that get called. At least one I have found to be not thread-safe. I > intend to release a version with that fix and a couple of minor things, > even if I have not yet completed the full runtime-checks. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > > Sent: Sunday, September 23, 2007 11:28 AM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] rsyslog 1.19.6 and segfaults > > > > On 2007-09-11, Rainer Gerhards wrote: > > > > > > But one important thing: I would appreciate if those of you that > > > experienced the segfaults could try out the new version. I > currently > > > think it will *NOT* fix the problem, but I would like to verify > that > > > (and I have to admit I am not angry if I am wrong...). > > > > Sorry, you were right: > > > > *** glibc detected *** rsyslogd: corrupted double-linked list: > > 0xb7214c98 *** > > ======= Backtrace: ========= > > /lib/libc.so.6[0x4152ce3e] > > /lib/libc.so.6(cfree+0x90)[0x415305d0] > > rsyslogd(MsgDestruct+0x46)[0x80580b6] > > rsyslogd[0x804df9a] > > rsyslogd(llExecFunc+0x3f)[0x805f11f] > > rsyslogd[0x804d9ea] > > rsyslogd[0x804db3f] > > /lib/libpthread.so.0[0x416112db] > > /lib/libc.so.6(clone+0x5e)[0x4159414e] > > > > With 1.19.6. > > > > > If it fails, it would be even greater if you could try it in > > > single-threaded mode: > > > > > > ./configure --disable-pthreads > > > > Is this safe to use on a busy system? I wouldn't want to start > dropping > > lots of packets.. > > > > > > -jf > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Sep 24 16:30:48 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Sep 2007 16:30:48 +0200 Subject: [rsyslog] new unofficial 1.19.7 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> Hi all, I have posted a new "unofficial" 1.19.7 to the download server: http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz I intend to post it publically tomorrow. But some nits need to be done first, mostly doc issues. I'd appreciate if those with the segfault problem could give it a try (in multi-threading mode) and tell me the outcome. Thanks, Rainer From rgerhards at hq.adiscon.com Tue Sep 25 11:46:14 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 11:46:14 +0200 Subject: [rsyslog] rsyslog 1.19.7 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Hi all, Rsyslog 1.19.7 (the official one) has been released today. It contains some code updated compared to the version from yesterday, so please download it if you already did. Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. Its primary goal is enhanced stability. Feature-wise, it offers support of dropping NUL characters, which are often found at the end of invalidly formatted syslog messages. Upgrading to this release is recommended for all users. Changelog: http://www.rsyslog.com/Article129.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml PLEASE NOTE: the rsyslog site is currently migrated to a new machine. In case you receive a temporary page, your DNS still caches the old machine's address. As always, feedback is appreciated. Feedback is especially appreciated from those that have experienced the segfault problem in the past. Rainer Gerhards From infofarmer at FreeBSD.org Tue Sep 25 12:42:20 2007 From: infofarmer at FreeBSD.org (Andrew Pantyukhin) Date: Tue, 25 Sep 2007 14:42:20 +0400 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <20070925104218.GI49755@amilo.cenkes.org> On Tue, Sep 25, 2007 at 11:46:14AM +0200, Rainer Gerhards wrote: > Hi all, > > Rsyslog 1.19.7 (the official one) has been released today. It contains > some code updated compared to the version from yesterday, so please > download it if you already did. > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > Its primary goal is enhanced stability. Feature-wise, it offers support > of dropping NUL characters, which are often found at the end of > invalidly formatted syslog messages. Upgrading to this release is > recommended for all users. You said you don't support rfc3195d at the moment, but anyway: rfc3195d.c:108: error: 'errStr' undeclared (first use in this function) From rgerhards at hq.adiscon.com Tue Sep 25 12:44:33 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 12:44:33 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <20070925104218.GI49755@amilo.cenkes.org> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <20070925104218.GI49755@amilo.cenkes.org> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9D@grfint2.intern.adiscon.com> Oops.. looks like I need to check my build procedure, obviously it wasn't build over here... Thanks for the report, it's fixed now. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Andrew Pantyukhin > Sent: Tuesday, September 25, 2007 12:42 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > On Tue, Sep 25, 2007 at 11:46:14AM +0200, Rainer Gerhards wrote: > > Hi all, > > > > Rsyslog 1.19.7 (the official one) has been released today. It > contains > > some code updated compared to the version from yesterday, so please > > download it if you already did. > > > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > > Its primary goal is enhanced stability. Feature-wise, it offers > support > > of dropping NUL characters, which are often found at the end of > > invalidly formatted syslog messages. Upgrading to this release is > > recommended for all users. > > You said you don't support rfc3195d at the moment, but anyway: > > rfc3195d.c:108: error: 'errStr' undeclared (first use in this > function) > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Tue Sep 25 13:55:18 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 25 Sep 2007 13:55:18 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <1190721318.3849.13.camel@localhost.localdomain> Am Dienstag, den 25.09.2007, 11:46 +0200 schrieb Rainer Gerhards: [...] > As always, feedback is appreciated. Feedback is especially appreciated > from those that have experienced the segfault problem in the past. Hi all, I'd like to apologize, since it was me who reported to Rainer that the segfault problem seems to have gone with the new release and that everything seems to be better now before he released 1.19.7. Meanwhile I realised that while the rsyslog-Daemon indeed ran without any segfaults, however, it didn't log anything sent to it via UDP. So my segfault problem has been replaced by a new one. ;-) So everybody should be very careful with this update, especially those who previously encountered the segfault problem. -- CU, Patrick. From rgerhards at hq.adiscon.com Tue Sep 25 13:57:46 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Sep 2007 13:57:46 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <1190721318.3849.13.camel@localhost.localdomain> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <1190721318.3849.13.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> Patrick, you mean nothing was logged via UDP while local domain sockets and/or TCP still worked? I have to admit I am a bit puzzled... Do you have a debug log at hand? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick von der Hagen > Sent: Tuesday, September 25, 2007 1:55 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > Am Dienstag, den 25.09.2007, 11:46 +0200 schrieb Rainer Gerhards: > [...] > > As always, feedback is appreciated. Feedback is especially > appreciated > > from those that have experienced the segfault problem in the past. > Hi all, > > I'd like to apologize, since it was me who reported to Rainer that the > segfault problem seems to have gone with the new release and that > everything > seems to be better now before he released 1.19.7. > > Meanwhile I realised that while the rsyslog-Daemon indeed ran without > any > segfaults, however, it didn't log anything sent to it via UDP. So my > segfault > problem has been replaced by a new one. ;-) So everybody should be very > careful with this update, especially those who previously encountered > the > segfault problem. > > -- > CU, > Patrick. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From hagen at rz.uni-karlsruhe.de Tue Sep 25 15:39:14 2007 From: hagen at rz.uni-karlsruhe.de (Patrick von der Hagen) Date: Tue, 25 Sep 2007 15:39:14 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <1190721318.3849.13.camel@localhost.localdomain> <577465F99B41C842AAFBE9ED71E70ABA278A9F@grfint2.intern.adiscon.com> Message-ID: <1190727554.3043.14.camel@localhost.localdomain> Am Dienstag, den 25.09.2007, 13:57 +0200 schrieb Rainer Gerhards: > Patrick, > > you mean nothing was logged via UDP while local domain sockets and/or > TCP still worked? I have to admit I am a bit puzzled... I only use rsyslog for UDP-logging, so I can't comment on TCP or local domain sockets. I usually run rsyslogd with "-r514" or "-4 -r514". The only log-entries relate to startup or shutdown of rsyslog itself. > Do you have a debug log at hand? Let's start with compiler warnings. They are identical to 1.19.6, so probably not related to the changed behaviour. syslog.c: In function ?writeSyslogV?: syslog.c:98: warning: function might be possible candidate for ?printf? format attribute syslog.c:98: warning: function might be possible candidate for ?printf? format attribute template.c: In function ?tplPrintList?: template.c:927: warning: cast from pointer to integer of different size modules.c: In function ?modPrintList?: modules.c:309: warning: cast from pointer to integer of different size modules.c:310: warning: cast from pointer to integer of different size modules.c:311: warning: cast from pointer to integer of different size modules.c:312: warning: cast from pointer to integer of different size modules.c:313: warning: cast from pointer to integer of different size cfsysline.c: In function ?dbgPrintCfSysLineHandlers?: cfsysline.c:710: warning: cast from pointer to integer of different size cfsysline.c:711: warning: cast from pointer to integer of different size action.c: In function ?actionDbgPrint?: action.c:174: warning: cast from pointer to integer of different size If I run "rsyslogd -r514 -4 -d" it prints something like this: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: Successful select, descriptor count = 1, Activity on: 6 -1431504256: Listening on UDP syslogd socket 6 (IPv4/port 514). -1431504256: ---------------------------------------- -1431504256: Calling select, active file descriptors (max 6): 3 6 -1431504256: over and over again. I'll sent you a complete output off-list. -- CU, Patrick. From mic at npgx.com.au Tue Sep 25 15:40:36 2007 From: mic at npgx.com.au (Michael Mansour) Date: Tue, 25 Sep 2007 23:40:36 +1000 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> Message-ID: <20070925133439.M72922@npgx.com.au> Hi Rainer, > Hi all, > > Rsyslog 1.19.7 (the official one) has been released today. It > contains some code updated compared to the version from yesterday, > so please download it if you already did. I've just downloaded and installed this release, and although I've never experienced the segfault issue (rsyslog continually runs fine for me), when I installed this version and then did a: # service rsyslog restart I got this in my messages file: Sep 25 22:41:46 server kernel: Kernel logging (proc) stopped. Sep 25 22:41:46 server kernel: Kernel log daemon terminating. Sep 25 22:41:47 server rsyslog: rklogd shutdown succeeded Sep 25 22:41:47 server rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="12176"] exiting on signal 15. Sep 25 22:44:08 server rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="10430"][x-configInfo udpRecep tion="No" udpPort="514" tcpReception="No" tcpPort="0"] restart Sep 25 22:44:08 server rsyslog: rsyslogd startup succeeded Sep 25 22:44:08 server kernel: rklogd 1.19.7, log source = /proc/kmsg started. Sep 25 22:44:08 server kernel: rsyslogd[8110]: segfault at 000000000000000a rip 00000038c7568678 rsp 0000007fbfffdc2 0 error 4 Sep 25 22:44:08 server rsyslog: rklogd startup succeeded Sep 25 22:44:08 server rsyslog: rsyslogd shutdown succeeded The above machine is a DL380 G4 with one CPU running hypthreaded. On another machine that's the same as that: Sep 25 22:42:24 server2 kernel: Kernel logging (proc) stopped. Sep 25 22:42:24 server2 kernel: Kernel log daemon terminating. Sep 25 22:42:25 server2 rsyslog: rklogd shutdown succeeded Sep 25 22:42:25 server2 rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="28497"] exiting on signal 15 . Sep 25 22:43:43 server2 rsyslogd: [origin software="rsyslogd" swVersion="1.19.7" x-pid="16598"][x-configInfo udpRece ption="No" udpPort="514" tcpReception="No" tcpPort="0"] restart Sep 25 22:43:43 server2 rsyslog: rsyslogd startup succeeded Sep 25 22:43:43 server2 kernel: rklogd 1.19.7, log source = /proc/kmsg started. Sep 25 22:43:43 server2 kernel: rsyslogd[15998]: segfault at 000000000000000a rip 00000034d2868678 rsp 0000007fbfffd c20 error 4 Sep 25 22:43:43 server2 rsyslog: rklogd startup succeeded Those two machines above are also running 64bit Linux. Using the same process on other 32 bit linux servers (DL360's), no segfaults occur. Note that I have never (ever) had rsyslog crash on me while it's running, and I currently run it on 6 servers to do the MySQL logging. Regards, Michael. > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > Its primary goal is enhanced stability. Feature-wise, it offers support > of dropping NUL characters, which are often found at the end of > invalidly formatted syslog messages. Upgrading to this release is > recommended for all users. > > Changelog: > > http://www.rsyslog.com/Article129.phtml > > Download: > > http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml > > PLEASE NOTE: the rsyslog site is currently migrated to a new > machine. In case you receive a temporary page, your DNS still caches > the old machine's address. > > As always, feedback is appreciated. Feedback is especially > appreciated from those that have experienced the segfault problem in > the past. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From halljer at auburn.edu Tue Sep 25 18:09:39 2007 From: halljer at auburn.edu (Dusty Hall) Date: Tue, 25 Sep 2007 11:09:39 -0500 Subject: [rsyslog] new unofficial 1.19.7 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278A8E@grfint2.intern.adiscon.com> Message-ID: <46F8EE24.8529.003A.0@auburn.edu> I had some issues with the redhat/rsyslog.init script, default binary locations, etc. I've attached an updated version. Thanks, -Dusty >>> "Rainer Gerhards" 09/24/07 9:30 AM >>> Hi all, I have posted a new "unofficial" 1.19.7 to the download server: http://download.rsyslog.com/rsyslog/rsyslog- 1.19.7.tar.gz I intend to post it publically tomorrow. But some nits need to be done first, mostly doc issues. I'd appreciate if those with the segfault problem could give it a try (in multi- threading mode) and tell me the outcome. Thanks, Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Sep 26 13:53:02 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 26 Sep 2007 13:53:02 +0200 Subject: [rsyslog] rsyslog 1.19.7 UDP bug Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ACC@grfint2.intern.adiscon.com> Hi all, I have identified and fixed the UDP message loss problem in rsyslog 1.19.7. If you have not yet installed that release, DO NOT INSTALL IT. I will remove it from the download links. A fixed version will be released soon as 1.19.8. If you need it urgently, an interim version is available at http://download.rsyslog.com/rsyslog/rsyslog-1.19.7.tar.gz That tarball is lacking MySQL functionality (which has now been modularized into a separate package - more news on that later). Sorry for the hassle. Rainer From rgerhards at hq.adiscon.com Wed Sep 26 14:30:24 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 26 Sep 2007 14:30:24 +0200 Subject: [rsyslog] on the udp bug Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ACF@grfint2.intern.adiscon.com> In case you are interested in some more details: http://rgerhards.blogspot.com/2007/09/needed-to-pull-1197-release.html And in CVS: http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/net.c?r1=1.9&r2=1.10 Rainer From rgerhards at hq.adiscon.com Thu Sep 27 10:11:11 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Sep 2007 10:11:11 +0200 Subject: [rsyslog] rsyslog 1.19.8 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Hi all, rsyslog 1.19.8 has been released today. Most importantly, it fixes a bug that could cause severe loss of UDP messages. It also contains improved repeat message processing. The MySQL functionality is now taken out of the core package, but its tarball is still contained in the main tarball (so it is still a single download for everything). This is part of the effort to fully support third-party plugins. Rsyslog 1.19.8 is a recommended update for all users. Change Log: http://www.rsyslog.com/Article132.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-60.phtml As always, feedback is appreciated. Rainer Gerhards From rgerhards at hq.adiscon.com Thu Sep 27 10:17:13 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Sep 2007 10:17:13 +0200 Subject: [rsyslog] rsyslog 1.19.7 released In-Reply-To: <20070925133439.M72922@npgx.com.au> References: <577465F99B41C842AAFBE9ED71E70ABA278A9C@grfint2.intern.adiscon.com> <20070925133439.M72922@npgx.com.au> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278AE1@grfint2.intern.adiscon.com> Sorry Michael, I just noticed that I had not replied. I looked at the code, but there is too few information to track the source code location down. What I currently assume is a) that we still have a segfault issue (obviously ;)) b) the segfault is a moving target The segfault occurs due to memory corruption. I think (and have often seen) that you do not notice the corruption until it happens to hit some really important piece of memory. So in many cases, it will overwrite something that nobody cares about. Only if some specific structures are hit, a malfunction manifests. This is part of the problem to reproducing it. What is hit, depends on the memory layout. A new version leads to different memory layout. I assume this is what has happened here: you did not experience the bug before, because it hit some memory that wasn't really relevant. With the new versions memory layout, it hits somewhere where it hurts... At least this is my best guess. I'd appreciate if you could try out the 1.19.8 release. I suspect it will also segfault. If it does, it would be great if you could run it in debug mode (-d -n) and send me the output before it segfault (maybe 200 lines or so). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Mansour > Sent: Tuesday, September 25, 2007 3:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.7 released > > Hi Rainer, > > > Hi all, > > > > Rsyslog 1.19.7 (the official one) has been released today. It > > contains some code updated compared to the version from yesterday, > > so please download it if you already did. > > I've just downloaded and installed this release, and although I've > never > experienced the segfault issue (rsyslog continually runs fine for me), > when I > installed this version and then did a: > > # service rsyslog restart > > I got this in my messages file: > > Sep 25 22:41:46 server kernel: Kernel logging (proc) stopped. > Sep 25 22:41:46 server kernel: Kernel log daemon terminating. > Sep 25 22:41:47 server rsyslog: rklogd shutdown succeeded > Sep 25 22:41:47 server rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="12176"] exiting on signal 15. > Sep 25 22:44:08 server rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="10430"][x-configInfo udpRecep > tion="No" udpPort="514" tcpReception="No" tcpPort="0"] restart > Sep 25 22:44:08 server rsyslog: rsyslogd startup succeeded > Sep 25 22:44:08 server kernel: rklogd 1.19.7, log source = /proc/kmsg > started. > Sep 25 22:44:08 server kernel: rsyslogd[8110]: segfault at > 000000000000000a > rip 00000038c7568678 rsp 0000007fbfffdc2 > 0 error 4 > Sep 25 22:44:08 server rsyslog: rklogd startup succeeded > Sep 25 22:44:08 server rsyslog: rsyslogd shutdown succeeded > > The above machine is a DL380 G4 with one CPU running hypthreaded. On > another > machine that's the same as that: > > Sep 25 22:42:24 server2 kernel: Kernel logging (proc) stopped. > Sep 25 22:42:24 server2 kernel: Kernel log daemon terminating. > Sep 25 22:42:25 server2 rsyslog: rklogd shutdown succeeded > Sep 25 22:42:25 server2 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="28497"] exiting on signal 15 > . > Sep 25 22:43:43 server2 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="16598"][x-configInfo udpRece > ption="No" udpPort="514" tcpReception="No" tcpPort="0"] restart > Sep 25 22:43:43 server2 rsyslog: rsyslogd startup succeeded > Sep 25 22:43:43 server2 kernel: rklogd 1.19.7, log source = /proc/kmsg > started. > Sep 25 22:43:43 server2 kernel: rsyslogd[15998]: segfault at > 000000000000000a > rip 00000034d2868678 rsp 0000007fbfffd > c20 error 4 > Sep 25 22:43:43 server2 rsyslog: rklogd startup succeeded > > Those two machines above are also running 64bit Linux. > > Using the same process on other 32 bit linux servers (DL360's), no > segfaults > occur. > > Note that I have never (ever) had rsyslog crash on me while it's > running, and > I currently run it on 6 servers to do the MySQL logging. > > Regards, > > Michael. > > > Rsyslog 1.19.7 is a code cleanup, bug fixing and maintenance release. > > Its primary goal is enhanced stability. Feature-wise, it offers > support > > of dropping NUL characters, which are often found at the end of > > invalidly formatted syslog messages. Upgrading to this release is > > recommended for all users. > > > > Changelog: > > > > http://www.rsyslog.com/Article129.phtml > > > > Download: > > > > http://www.rsyslog.com/Downloads-req-getit-lid-59.phtml > > > > PLEASE NOTE: the rsyslog site is currently migrated to a new > > machine. In case you receive a temporary page, your DNS still caches > > the old machine's address. > > > > As always, feedback is appreciated. Feedback is especially > > appreciated from those that have experienced the segfault problem in > > the past. > > > > Rainer Gerhards > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > ------- End of Original Message ------- > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Sep 28 13:46:13 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Sep 2007 13:46:13 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Hi Michael, as I have blogged, I am not yet sure about how to handle the situation. I am also consulting with Autotools experts, any advise is appreciated. Two packages seem useful, especially when more plugins become available (I myself think about email and a couple of other databases). Many folks also do not like the idea of having to have libmysql present on the system just to install rsyslog - with a core package and the plugin, those can only install the core (and that will probably the majority of cases). What I have not yet found - and I have very limited expertise in this area - is how to do that in the best possible way. Oh, and some more background: ones the plugin interace has matured (I expect this in 3 to 6 month), I intend to actually use ommysql with its own version number. Versioning will be handled by the interface (that part is already present, but no code for it yet as there is only a release 1 of it). So, in the medium to long term, ommysql *will* be a separate project - maybe one with a different maintainer, as I am no mysql guy. Does this make sense? As I said, comments are much appreciated... Rainer > -----Original Message----- > From: Michael Biebl [mailto:mbiebl at gmail.com] > Sent: Friday, September 28, 2007 10:23 AM > To: rsyslog-users; Rainer Gerhards > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > 2007/9/27, Rainer Gerhards : > > repeat message processing. The MySQL functionality is now taken out > of > > the core package, but its tarball is still contained in the main > tarball > > (so it is still a single download for everything). This is part of > the > > effort to fully support third-party plugins. Rsyslog 1.19.8 is a > > > To be honest, I don't particularly like this change. It increases the > work for package maintainers (like me). You now have to maintain two > source packages. Having a non-standard tarball inside a tarball > doesn't help there. It even worsens things, as stuff like "make dist" > or "make distcheck" doesn't work anymore for a cvs checkout. > There's also the problem of which version of the plugins will be > compatible with the core version (upgrade scenarios, keeping them in > sync, etc.) > It also adds complexity for the developer (as he has to maintain an > additonal set of build files). > As you were talking about 3rd party plugins (with the emphasis on 3rd > party) I don't understand the benefit of splitting out the mysql > plugin as it is you who develops the mysql plugin, not a third party. > Do you actually intend to create a separate tarball for each plugin in > the future? > > What was wrong with the --enable-mysql configure switch? I don't see > any benefits, only disadvantages. You know, if it ain't broken, why > fix it ;-) > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? From mic at npgx.com.au Fri Sep 28 16:03:59 2007 From: mic at npgx.com.au (Michael Mansour) Date: Sat, 29 Sep 2007 00:03:59 +1000 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: <20070928135248.M3034@npgx.com.au> Hi Rainer, > Hi Michael, > > as I have blogged, I am not yet sure about how to handle the situation. > I am also consulting with Autotools experts, any advise is appreciated. > > Two packages seem useful, especially when more plugins become available > > (I myself think about email and a couple of other databases). Many folks > also do not like the idea of having to have libmysql present on the > system just to install rsyslog - with a core package and the plugin, > those can only install the core (and that will probably the majority > of cases). Yes, I totally agree with what you're doing, as making rsyslog modular is better overall for the product as a whole, as it will allow contributors easier access to have their own plugins in place without needing to release rsyslog each time. > What I have not yet found - and I have very limited expertise in this > area - is how to do that in the best possible way. I've asked both Dag and Dries (from rpmforge) on the best way forward as these guys have years of knowledge behind them for packaging (in the RPM space though). I'm not sure they'll reply though, as sometimes my questions to them are ignored and sometimes they are responded to (some questions I ask go into their "too hard" basket which I think is why they ignore them :P ). I have been querying, requesting, suggesting, etc things off them for more than 7 years now, so they may be sick of me (joking) :). I have been asked to become a packager for them before (and for Axel Thimms who does ATrpms.net - they're all merging into one large repo eventually) but declined as I have little time as is to contribute to the projects I'm currently working with. > Oh, and some more background: ones the plugin interace has matured (I > expect this in 3 to 6 month), I intend to actually use ommysql with its > own version number. Versioning will be handled by the interface (that > part is already present, but no code for it yet as there is only a > release 1 of it). So, in the medium to long term, ommysql *will* be a > separate project - maybe one with a different maintainer, as I am no > mysql guy. > > Does this make sense? As I said, comments are much appreciated... Yes and I do agree with it all. I'm just not personally sure of the best way forward myself, but I would suspect something along the lines of having one spec file for rsyslog, and another for rsyslog-mysql would likely be the right way to go. If neither Dag nor Dries answer me, I might even try Axel and if he also doesn't respond (he usually does) then I'll just start on the packaging myself, doing rsyslog and rsyslog-mysql. In the long term, it would be better though if rpmforge or atrpms.net would take on this packaging system. I realise Red Hat will have a "base" one available in Fedora and then eventually an EL release, but they dont' do updates, just bugfixing, once they release it for an enterprise product. Which is where 3rd party repo's make sense. Regards, Michael. > Rainer > > > -----Original Message----- > > From: Michael Biebl [mailto:mbiebl at gmail.com] > > Sent: Friday, September 28, 2007 10:23 AM > > To: rsyslog-users; Rainer Gerhards > > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > > > 2007/9/27, Rainer Gerhards : > > > repeat message processing. The MySQL functionality is now taken out > > of > > > the core package, but its tarball is still contained in the main > > tarball > > > (so it is still a single download for everything). This is part of > > the > > > effort to fully support third-party plugins. Rsyslog 1.19.8 is a > > > > > > To be honest, I don't particularly like this change. It increases the > > work for package maintainers (like me). You now have to maintain two > > source packages. Having a non-standard tarball inside a tarball > > doesn't help there. It even worsens things, as stuff like "make dist" > > or "make distcheck" doesn't work anymore for a cvs checkout. > > There's also the problem of which version of the plugins will be > > compatible with the core version (upgrade scenarios, keeping them in > > sync, etc.) > > It also adds complexity for the developer (as he has to maintain an > > additonal set of build files). > > As you were talking about 3rd party plugins (with the emphasis on 3rd > > party) I don't understand the benefit of splitting out the mysql > > plugin as it is you who develops the mysql plugin, not a third party. > > Do you actually intend to create a separate tarball for each plugin in > > the future? > > > > What was wrong with the --enable-mysql configure switch? I don't see > > any benefits, only disadvantages. You know, if it ain't broken, why > > fix it ;-) > > > > Cheers, > > Michael > > > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog ------- End of Original Message ------- From mbiebl at gmail.com Fri Sep 28 18:06:46 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 18:06:46 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Rainer Gerhards : > Hi Michael, > > as I have blogged, I am not yet sure about how to handle the situation. > I am also consulting with Autotools experts, any advise is appreciated. > > Two packages seem useful, especially when more plugins become available > (I myself think about email and a couple of other databases). Many folks > also do not like the idea of having to have libmysql present on the > system just to install rsyslog - with a core package and the plugin, Well, this is not quite true. If you don't want to have mysql support, you could just use --disable-mysql. This way you don't have to have mysql installed. Such a --enable/disable-feature option could be added for any plugin that will be added in the future. > those can only install the core (and that will probably the majority of > cases). > > What I have not yet found - and I have very limited expertise in this > area - is how to do that in the best possible way. I do have quite some experience with autotools. So if any help is needed, I'll gladly offer it. > release 1 of it). So, in the medium to long term, ommysql *will* be a > separate project - maybe one with a different maintainer, as I am no > mysql guy. This actually is a valid argument for splitting it off. But to reiterate what I posted earlier: For the plugins shipped directly by the rsyslog project, I'd prefer to have them in a single tarball with configure options to turn them on/off and as long as you are the maintainer of the mysql output plugin its more convenient to keep them in one project. Why not make the split when the plugin interface is stable, other output plugins are available (and we have more experience how to handle them) and a external maintainer for ommysql is found? I hope this doesn't sound too negative. I generally like the idea of having a plugin interface, but I only fear that splitting things up prematurely without knowing how it develops, will complicate things (and probably destabilize). Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Fri Sep 28 20:11:20 2007 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Sep 2007 20:11:20 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Michael, I need to prepare for german astronomy day tomorrow, so brief ;) The point was that a single package (RPM or whatever) can be pre-build for the core and the mysql functionality. I think the configure options do not help with that. Or do they? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Friday, September 28, 2007 6:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 1.19.8 released > > 2007/9/28, Rainer Gerhards : > > Hi Michael, > > > > as I have blogged, I am not yet sure about how to handle the > situation. > > I am also consulting with Autotools experts, any advise is > appreciated. > > > > Two packages seem useful, especially when more plugins become > available > > (I myself think about email and a couple of other databases). Many > folks > > also do not like the idea of having to have libmysql present on the > > system just to install rsyslog - with a core package and the plugin, > > Well, this is not quite true. If you don't want to have mysql support, > you could just use --disable-mysql. This way you don't have to have > mysql installed. > Such a --enable/disable-feature option could be added for any plugin > that will be added in the future. > > > those can only install the core (and that will probably the majority > of > > cases). > > > > What I have not yet found - and I have very limited expertise in this > > area - is how to do that in the best possible way. > > I do have quite some experience with autotools. So if any help is > needed, I'll gladly offer it. > > > release 1 of it). So, in the medium to long term, ommysql *will* be a > > separate project - maybe one with a different maintainer, as I am no > > mysql guy. > > This actually is a valid argument for splitting it off. > > But to reiterate what I posted earlier: For the plugins shipped > directly by the rsyslog project, I'd prefer to have them in a single > tarball with configure options to turn them on/off and as long as you > are the maintainer of the mysql output plugin its more convenient to > keep them in one project. Why not make the split when the plugin > interface is stable, other output plugins are available (and we have > more experience how to handle them) and a external maintainer for > ommysql is found? > I hope this doesn't sound too negative. I generally like the idea of > having a plugin interface, but I only fear that splitting things up > prematurely without knowing how it develops, will complicate things > (and probably destabilize). > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Fri Sep 28 21:03:53 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 21:03:53 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Rainer Gerhards : > Michael, > > I need to prepare for german astronomy day tomorrow, so brief ;) > > The point was that a single package (RPM or whatever) can be pre-build > for the core and the mysql functionality. I think the configure options > do not help with that. Or do they? > You can split up the build to create multiple *binary* packages from a single *source* package, the RPM format as well as DEB allows that. Actually the Debian and Fedora packages already do that. The rsyslog [1] binary package contains the core rsyslogd/rklogd binaries, the rsyslog-mysql [2] package the mysql output plugin. It's not necessary to already split up the *source* package for that. Cheers, Michael [1] http://packages.debian.org/sid/rsyslog [2] http://packages.debian.org/sid/rsyslog-mysql -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Fri Sep 28 21:06:05 2007 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 28 Sep 2007 21:06:05 +0200 Subject: [rsyslog] rsyslog 1.19.8 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B13@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA278B2B@grfint2.intern.adiscon.com> Message-ID: 2007/9/28, Michael Biebl : > 2007/9/28, Rainer Gerhards : > > > > The point was that a single package (RPM or whatever) can be pre-build > > for the core and the mysql functionality. I think the configure options > > do not help with that. Or do they? > > > > You can split up the build to create multiple *binary* packages from a > single *source* package, the RPM format as well as DEB allows that. > > Actually the Debian and Fedora packages already do that. The rsyslog > [1] binary package contains the core rsyslogd/rklogd binaries, the > rsyslog-mysql [2] package the mysql output plugin. > It's not necessary to already split up the *source* package for that. > > [1] http://packages.debian.org/sid/rsyslog > [2] http://packages.debian.org/sid/rsyslog-mysql And as you can see from [1] and [2], the core rsyslog package has no libmysql dependencies. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From janfrode at tanso.net Fri Sep 28 22:57:12 2007 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 28 Sep 2007 22:57:12 +0200 Subject: [rsyslog] rsyslog 1.19.8 released References: <577465F99B41C842AAFBE9ED71E70ABA278ADF@grfint2.intern.adiscon.com> Message-ID: On 2007-09-27, Rainer Gerhards wrote: > > rsyslog 1.19.8 has been released today. Most importantly, it fixes a bug Still crashes.. twice today, but I didn't find the trace the first time. Here's the trace from the second crash: *** glibc detected *** rsyslogd: corrupted double-linked list: 0x094ca0e8 *** ======= Backtrace: ========= /lib/libc.so.6[0x4152cda9] /lib/libc.so.6(cfree+0x90)[0x415305d0] rsyslogd(MsgDestruct+0x73)[0x8058323] rsyslogd[0x804dfba] rsyslogd(llExecFunc+0x3f)[0x805f43f] rsyslogd[0x804d9ba] rsyslogd[0x804db0f] /lib/libpthread.so.0[0x416112db] /lib/libc.so.6(clone+0x5e)[0x4159414e] ======= Memory map: ======== 0067d000-0067e000 r-xp 0067d000 00:00 0 [vdso] 08048000-08068000 r-xp 00000000 fd:00 884939 /sbin/rsyslogd 08068000-08069000 rwxp 00020000 fd:00 884939 /sbin/rsyslogd 094c5000-09549000 rwxp 094c5000 00:00 0 414aa000-414c3000 r-xp 00000000 fd:00 1146882 /lib/ld-2.5.so 414c3000-414c4000 r-xp 00018000 fd:00 1146882 /lib/ld-2.5.so 414c4000-414c5000 rwxp 00019000 fd:00 1146882 /lib/ld-2.5.so 414c7000-415fe000 r-xp 00000000 fd:00 1146990 /lib/libc-2.5.so 415fe000-41600000 r-xp 00137000 fd:00 1146990 /lib/libc-2.5.so 41600000-41601000 rwxp 00139000 fd:00 1146990 /lib/libc-2.5.so 41601000-41604000 rwxp 41601000 00:00 0 41606000-41608000 r-xp 00000000 fd:00 1147002 /lib/libdl-2.5.so 41608000-41609000 r-xp 00001000 fd:00 1147002 /lib/libdl-2.5.so 41609000-4160a000 rwxp 00002000 fd:00 1147002 /lib/libdl-2.5.so 4160c000-4161f000 r-xp 00000000 fd:00 1147003 /lib/libpthread-2.5.so 4161f000-41620000 r-xp 00012000 fd:00 1147003 /lib/libpthread-2.5.so 41620000-41621000 rwxp 00013000 fd:00 1147003 /lib/libpthread-2.5.so 41621000-41623000 rwxp 41621000 00:00 0 41667000-41679000 r-xp 00000000 fd:00 1087043 /usr/lib/libz.so.1.2.3 41679000-4167a000 rwxp 00011000 fd:00 1087043 /usr/lib/libz.so.1.2.3 416c4000-416cb000 r-xp 00000000 fd:00 1147009 /lib/librt-2.5.so 416cb000-416cc000 r-xp 00006000 fd:00 1147009 /lib/librt-2.5.so 416cc000-416cd000 rwxp 00007000 fd:00 1147009 /lib/librt-2.5.so 4acb4000-4acbf000 r-xp 00000000 fd:00 1146939 /lib/libgcc_s-4.1.1-20070105.so.1 4acbf000-4acc0000 rwxp 0000a000 fd:00 1146939 /lib/libgcc_s-4.1.1-20070105.so.1 b7200000-b722a000 rw-p b7200000 00:00 0 b722a000-b7300000 ---p b722a000 00:00 0 b7400000-b7430000 rw-p b7400000 00:00 0 b7430000-b7500000 ---p b7430000 00:00 0 b759b000-b759c000 ---p b759b000 00:00 0 b759c000-b7f9e000 rw-p b759c000 00:00 0 b7fa3000-b7fa5000 rw-p b7fa3000 00:00 0 bfc72000-bfc88000 rw-p bfc72000 00:00 0 [stack] -jf