From rgerhards at hq.adiscon.com Mon Dec 1 10:02:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 10:02:01 +0100 Subject: [rsyslog] rsyslog doc licenses Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Hi all, Michael Biebl made me aware that the doc files have inconsistent license information (thanks!). Some state GPL, some FDL, some even nothing. I would like to unify them to use GPLv3, which is consistent with the rest of this project. Does anybody object this move? If so what are the reasons and alternatives? Feedback is appreciated. Many thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 1 10:55:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 10:55:12 +0100 Subject: [rsyslog] security issue in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> Hi folks, thanks to a bug report, I found out that the $AllowedSender directive does not work in all releases. The bug in question is: http://bugzilla.adiscon.com/show_bug.cgi?id=111 Im am currently working on the bug. Obviously, this can lead to messages being received from systems that are not permitted so. As a work-around, proper firewalling should be set up on the vulnerable hosts. Until further note, I would assume that all versions of rsyslog are affected (I will provide more detail during my analysis). Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 1 11:26:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 11:26:56 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> Version v2-stable is NOT vulnerable. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 10:55 AM > To: rsyslog-users > Subject: [rsyslog] security issue in rsyslog > > Hi folks, > > thanks to a bug report, I found out that the $AllowedSender directive > does not work in all releases. The bug in question is: > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > Im am currently working on the bug. Obviously, this can lead to > messages > being received from systems that are not permitted so. As a work- > around, > proper firewalling should be set up on the vulnerable hosts. Until > further note, I would assume that all versions of rsyslog are affected > (I will provide more detail during my analysis). > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 15:31:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 15:31:36 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> Hi all, this is patch for v3-stable: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae6 d9bbf6b07e2f06c4dd676 I have not tried yet, but I think it will work on almost all other versions, too. I keep you posted on the progress. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 11:27 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Version v2-stable is NOT vulnerable. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 10:55 AM > > To: rsyslog-users > > Subject: [rsyslog] security issue in rsyslog > > > > Hi folks, > > > > thanks to a bug report, I found out that the $AllowedSender directive > > does not work in all releases. The bug in question is: > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > Im am currently working on the bug. Obviously, this can lead to > > messages > > being received from systems that are not permitted so. As a work- > > around, > > proper firewalling should be set up on the vulnerable hosts. Until > > further note, I would assume that all versions of rsyslog are > affected > > (I will provide more detail during my analysis). > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 16:30:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 16:30:36 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> I now clarified the affected versions. Affected are 3.12.2 and above. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 3:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Hi all, > > this is patch for v3-stable: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > 6 > d9bbf6b07e2f06c4dd676 > > I have not tried yet, but I think it will work on almost all other > versions, too. I keep you posted on the progress. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 11:27 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Version v2-stable is NOT vulnerable. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 10:55 AM > > > To: rsyslog-users > > > Subject: [rsyslog] security issue in rsyslog > > > > > > Hi folks, > > > > > > thanks to a bug report, I found out that the $AllowedSender > directive > > > does not work in all releases. The bug in question is: > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > messages > > > being received from systems that are not permitted so. As a work- > > > around, > > > proper firewalling should be set up on the vulnerable hosts. Until > > > further note, I would assume that all versions of rsyslog are > > affected > > > (I will provide more detail during my analysis). > > > > > > Thanks, > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 16:37:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 16:37:23 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> ... and the patch will not work on all of these version, due to the introduction of the netstream driver functionality. Please note that anything older than current v3-stable is outdated, so the proper way to replace the faulty code is to upgrade to the current v3-stable and apply the patch. I will also release a new v3-stable soon, hopefully today (but I'd like to conduct some more tests). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 4:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > I now clarified the affected versions. Affected are 3.12.2 and above. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 3:32 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Hi all, > > > > this is patch for v3-stable: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > 6 > > d9bbf6b07e2f06c4dd676 > > > > I have not tried yet, but I think it will work on almost all other > > versions, too. I keep you posted on the progress. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 11:27 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Version v2-stable is NOT vulnerable. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > To: rsyslog-users > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > Hi folks, > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > directive > > > > does not work in all releases. The bug in question is: > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > > messages > > > > being received from systems that are not permitted so. As a work- > > > > around, > > > > proper firewalling should be set up on the vulnerable hosts. > Until > > > > further note, I would assume that all versions of rsyslog are > > > affected > > > > (I will provide more detail during my analysis). > > > > > > > > Thanks, > > > > Rainer > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 17:08:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 17:08:05 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> The issue also exists in TCP mode, but analysis shows this is not a trial fix. The design overlooked the situation. In theory, a whole new access control feature would be needed. I am checking out if it is possible to "just" enhance the interface. With the current netstreams defined that should be possible. I am tempted to release the UDP-fixed version and release the next version with the TCP fix. Feedback from packagers is appreciated. The TCP fix may take a day or two, depending on how smart a way I find. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 4:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > ... and the patch will not work on all of these version, due to the > introduction of the netstream driver functionality. Please note that > anything older than current v3-stable is outdated, so the proper way to > replace the faulty code is to upgrade to the current v3-stable and > apply > the patch. I will also release a new v3-stable soon, hopefully today > (but I'd like to conduct some more tests). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 4:31 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > I now clarified the affected versions. Affected are 3.12.2 and above. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 3:32 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Hi all, > > > > > > this is patch for v3-stable: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > 6 > > > d9bbf6b07e2f06c4dd676 > > > > > > I have not tried yet, but I think it will work on almost all other > > > versions, too. I keep you posted on the progress. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > To: rsyslog-users > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > Hi folks, > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > directive > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > > > messages > > > > > being received from systems that are not permitted so. As a > work- > > > > > around, > > > > > proper firewalling should be set up on the vulnerable hosts. > > Until > > > > > further note, I would assume that all versions of rsyslog are > > > > affected > > > > > (I will provide more detail during my analysis). > > > > > > > > > > Thanks, > > > > > Rainer > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 17:51:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 17:51:38 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> Ok, looks like I found a work-around. Not that elegant, but seems to work quite well. Patch for TCP is here: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9e 18747b55d701e360d5aac Please note that this effectively disables GSS functionality. I'll updated the GSS drivers in the next step. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 5:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The issue also exists in TCP mode, but analysis shows this is not a > trial fix. The design overlooked the situation. In theory, a whole new > access control feature would be needed. I am checking out if it is > possible to "just" enhance the interface. With the current netstreams > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 4:37 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > ... and the patch will not work on all of these version, due to the > > introduction of the netstream driver functionality. Please note that > > anything older than current v3-stable is outdated, so the proper way > to > > replace the faulty code is to upgrade to the current v3-stable and > > apply > > the patch. I will also release a new v3-stable soon, hopefully today > > (but I'd like to conduct some more tests). > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 4:31 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > above. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Hi all, > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > 6 > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > I have not tried yet, but I think it will work on almost all > other > > > > versions, too. I keep you posted on the progress. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > To: rsyslog-users > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > > directive > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can lead > to > > > > > > messages > > > > > > being received from systems that are not permitted so. As a > > work- > > > > > > around, > > > > > > proper firewalling should be set up on the vulnerable hosts. > > > Until > > > > > > further note, I would assume that all versions of rsyslog are > > > > > affected > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > Thanks, > > > > > > Rainer > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Mon Dec 1 18:23:47 2008 From: david at lang.hm (david at lang.hm) Date: Mon, 1 Dec 2008 09:23:47 -0800 (PST) Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: On Mon, 1 Dec 2008, Rainer Gerhards wrote: > The issue also exists in TCP mode, but analysis shows this is not a > trial fix. The design overlooked the situation. In theory, a whole new > access control feature would be needed. I am checking out if it is > possible to "just" enhance the interface. With the current netstreams > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. for UDP it's trivial to forge the source IP address anyway, so the 'security' gained by this feature in that mode is questionable to start with. that being said, I'm very pleased to see how you are handling this. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >> Sent: Monday, December 01, 2008 4:37 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] security issue in rsyslog >> >> ... and the patch will not work on all of these version, due to the >> introduction of the netstream driver functionality. Please note that >> anything older than current v3-stable is outdated, so the proper way > to >> replace the faulty code is to upgrade to the current v3-stable and >> apply >> the patch. I will also release a new v3-stable soon, hopefully today >> (but I'd like to conduct some more tests). >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>> Sent: Monday, December 01, 2008 4:31 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] security issue in rsyslog >>> >>> I now clarified the affected versions. Affected are 3.12.2 and > above. >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>> Sent: Monday, December 01, 2008 3:32 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] security issue in rsyslog >>>> >>>> Hi all, >>>> >>>> this is patch for v3-stable: >>>> >>>> >>> >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae >>>> 6 >>>> d9bbf6b07e2f06c4dd676 >>>> >>>> I have not tried yet, but I think it will work on almost all other >>>> versions, too. I keep you posted on the progress. >>>> >>>> Rainer >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>>> Sent: Monday, December 01, 2008 11:27 AM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] security issue in rsyslog >>>>> >>>>> Version v2-stable is NOT vulnerable. >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>>>> Sent: Monday, December 01, 2008 10:55 AM >>>>>> To: rsyslog-users >>>>>> Subject: [rsyslog] security issue in rsyslog >>>>>> >>>>>> Hi folks, >>>>>> >>>>>> thanks to a bug report, I found out that the $AllowedSender >>>> directive >>>>>> does not work in all releases. The bug in question is: >>>>>> >>>>>> http://bugzilla.adiscon.com/show_bug.cgi?id=111 >>>>>> >>>>>> Im am currently working on the bug. Obviously, this can lead > to >>>>>> messages >>>>>> being received from systems that are not permitted so. As a >> work- >>>>>> around, >>>>>> proper firewalling should be set up on the vulnerable hosts. >>> Until >>>>>> further note, I would assume that all versions of rsyslog are >>>>> affected >>>>>> (I will provide more detail during my analysis). >>>>>> >>>>>> Thanks, >>>>>> Rainer >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From mbiebl at gmail.com Mon Dec 1 18:17:31 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 18:17:31 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: 2008/12/1 Rainer Gerhards : > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. I'd say, take the time for proper fixing and testing, even if it takes a day or two and release afterwards. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Dec 1 18:47:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 18:47:00 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> And now there is an *untested* fix for the TLS driver: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec063 83fc5969178887d00abfc Testing takes a bit more of time, I need to set up the test environment for TLS again (looks like it would really pay to have a fixed test suite for all those cases - also the issue here would have never occurred...). Please note that I mistook GSSAPI with TLS in my previous mail. The TLS part should not be really affected by the problem: there are so much better access control features in TLS... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 5:52 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Ok, looks like I found a work-around. Not that elegant, but seems to > work quite well. Patch for TCP is here: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > e > 18747b55d701e360d5aac > > Please note that this effectively disables GSS functionality. I'll > updated the GSS drivers in the next step. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 5:08 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > The issue also exists in TCP mode, but analysis shows this is not a > > trial fix. The design overlooked the situation. In theory, a whole > new > > access control feature would be needed. I am checking out if it is > > possible to "just" enhance the interface. With the current netstreams > > defined that should be possible. I am tempted to release the UDP- > fixed > > version and release the next version with the TCP fix. Feedback from > > packagers is appreciated. The TCP fix may take a day or two, > depending > > on how smart a way I find. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 4:37 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > ... and the patch will not work on all of these version, due to the > > > introduction of the netstream driver functionality. Please note > that > > > anything older than current v3-stable is outdated, so the proper > way > > to > > > replace the faulty code is to upgrade to the current v3-stable and > > > apply > > > the patch. I will also release a new v3-stable soon, hopefully > today > > > (but I'd like to conduct some more tests). > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > > above. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Hi all, > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > 6 > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > I have not tried yet, but I think it will work on almost all > > other > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > To: rsyslog-users > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > > > directive > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > lead > > to > > > > > > > messages > > > > > > > being received from systems that are not permitted so. As a > > > work- > > > > > > > around, > > > > > > > proper firewalling should be set up on the vulnerable > hosts. > > > > Until > > > > > > > further note, I would assume that all versions of rsyslog > are > > > > > > affected > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > Thanks, > > > > > > > Rainer > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Mon Dec 1 19:30:57 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 19:30:57 +0100 Subject: [rsyslog] rsyslog doc licenses In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Message-ID: 2008/12/1 Rainer Gerhards : > Hi all, > > Michael Biebl made me aware that the doc files have inconsistent license > information (thanks!). Some state GPL, some FDL, some even nothing. > > I would like to unify them to use GPLv3, which is consistent with the > rest of this project. Does anybody object this move? My contributions to the documentation where minuscule. But anyways, you have my ok for the relicensing. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From aoz.syn at gmail.com Mon Dec 1 20:00:07 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 1 Dec 2008 12:00:07 -0700 Subject: [rsyslog] rsyslog doc licenses In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Message-ID: <4255c2570812011100l7183e18g4999de9bdddf5955@mail.gmail.com> On Mon, Dec 1, 2008 at 02:02, Rainer Gerhards wrote: > I would like to unify them to use GPLv3, which is consistent with the > rest of this project. Does anybody object this move? If so what are the > reasons and alternatives? One of the more documentation-focused licenses (e.g. GFDL or one of the Creative Commons family) may be better-suited to protect the IP concerns particular to documentation. I'm not sure I perfectly understand why separate documentation & source licenses exist, but the fact that legal minds greater than mine (doesn't take much) decided the split was necessary causes me to at least pause and consider them. A particularly interesting problem some have thrown at GFDL is that it is incompatible with the GPL - you cannot convert in either direction without re-licensing from the originator. Feh. Licensing is a sticky subject. From peter.matulis at canonical.com Mon Dec 1 20:48:41 2008 From: peter.matulis at canonical.com (Peter Matulis) Date: Mon, 01 Dec 2008 14:48:41 -0500 Subject: [rsyslog] TLS certificates Message-ID: <49343F99.4030607@canonical.com> Why does the documentation state that a client does not need to include its private key and certificate in its configuration? It says to only use the CA public certificate (ca.pem). Peter From mbiebl at gmail.com Mon Dec 1 20:53:19 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 20:53:19 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <49343F99.4030607@canonical.com> References: <49343F99.4030607@canonical.com> Message-ID: 2008/12/1 Peter Matulis : > Why does the documentation state that a client does not need to include > its private key and certificate in its configuration? It says to only > use the CA public certificate (ca.pem). That depends on what you want to do. If you only want to verify, that a client is sending it's messages to the right server, then it only needs the CA to verify that. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From peter.matulis at canonical.com Mon Dec 1 21:04:29 2008 From: peter.matulis at canonical.com (Peter Matulis) Date: Mon, 01 Dec 2008 15:04:29 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: References: <49343F99.4030607@canonical.com> Message-ID: <4934434D.4070703@canonical.com> Michael Biebl wrote: > 2008/12/1 Peter Matulis : >> Why does the documentation state that a client does not need to include >> its private key and certificate in its configuration? It says to only >> use the CA public certificate (ca.pem). > > That depends on what you want to do. > > If you only want to verify, that a client is sending it's messages to > the right server, then it only needs the CA to verify that. > > Michael > Right. I want to encrypt. Peter From aoz.syn at gmail.com Mon Dec 1 21:08:29 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 1 Dec 2008 13:08:29 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: <4934434D.4070703@canonical.com> References: <49343F99.4030607@canonical.com> <4934434D.4070703@canonical.com> Message-ID: <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> On Mon, Dec 1, 2008 at 13:04, Peter Matulis wrote: > Right. I want to encrypt. Encryption is still possible without a client certificate. Generally speaking, in SSL all a client cert adds is the ability for the server to verify the client's identity at the transport layer. RB From rgerhards at hq.adiscon.com Mon Dec 1 21:10:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 21:10:24 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> References: <49343F99.4030607@canonical.com><4934434D.4070703@canonical.com> <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F77A@grfint2.intern.adiscon.com> Jupp. I'd also like to mention the more elaborate guide. It is part of the doc set, but also available online: http://www.rsyslog.com/doc-rsyslog_secure_tls.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of RB > Sent: Monday, December 01, 2008 9:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > On Mon, Dec 1, 2008 at 13:04, Peter Matulis > wrote: > > Right. I want to encrypt. > > Encryption is still possible without a client certificate. Generally > speaking, in SSL all a client cert adds is the ability for the server > to verify the client's identity at the transport layer. > > > RB > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jmiscaro at gmail.com Tue Dec 2 14:55:56 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 08:55:56 -0500 Subject: [rsyslog] TLS certificates Message-ID: I has reading the docs [1] and this confusd me to. It says "neither the client nor the server are authenticated. So while the message transfer is encrypted, you can not be sure which peer you are talking to" Also, how can client encrypt without having any keys specified in its config? Example for the client shows: $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem $ActionSendStreamDriverAuthMode anon # server is NOT authenticated 2nd question: Why is the server not authenticated? /juan [1] http://www.rsyslog.com/doc-rsyslog_tls.html From aoz.syn at gmail.com Tue Dec 2 16:56:37 2008 From: aoz.syn at gmail.com (RB) Date: Tue, 2 Dec 2008 08:56:37 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: References: Message-ID: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: > "neither the client nor the server are authenticated. So while the > message transfer is encrypted, you can not be sure which peer you are > talking to" I'm hoping Rainer will jump in and clarify precisely how much handshake validation he's implemented. The fact that the client must have a copy of the CA's public material seems to indicate he is at least verifying that the server's certificate was issued by the CA. It's possible to not do so, but the result is rather susceptible to MITM. > Also, how can client encrypt without having any keys specified in its config? This isn't the forum to discuss the particulars of the SSL handshake, but suffice it to say that SSL incorporates a challenge/response mechanism (using the server's presented certificate) followed by negotiation of an ephemeral session key. See also: public-key cryptography. > $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > > 2nd question: Why is the server not authenticated? Without looking at the code, I presume the 'anon' AuthMode is the switch used to tell the SSL library whether or not to check the server certificate against the CA. If so, it should make specifying the CA public key redundant - the client just accepts whatever certificate the server (or MITM) presents and starts encrypting to it. From rgerhards at hq.adiscon.com Tue Dec 2 17:00:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:00:43 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: Tuesday, December 02, 2008 4:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: > > "neither the client nor the server are authenticated. So while the > > message transfer is encrypted, you can not be sure which peer you are > > talking to" > > I'm hoping Rainer will jump in and clarify precisely how much > handshake validation he's implemented. The fact that the client must > have a copy of the CA's public material seems to indicate he is at > least verifying that the server's certificate was issued by the CA. > It's possible to not do so, but the result is rather susceptible to > MITM. Just a quick note, I am quite busy at the moment (guess what ;)). If the auth is set to "anon" nothing at all is validated and MITM *is* absolutely possible. That's why the doc does not recommend to use that mode. I posted a link to the long TLS setup guide, which creates a fairly safe scenario (but your milage may vary... ;)). > > > Also, how can client encrypt without having any keys specified in its > config? > > This isn't the forum to discuss the particulars of the SSL handshake, > but suffice it to say that SSL incorporates a challenge/response > mechanism (using the server's presented certificate) followed by > negotiation of an ephemeral session key. See also: public-key > cryptography. > > > $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > > $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > > > > 2nd question: Why is the server not authenticated? > > Without looking at the code, I presume the 'anon' AuthMode is the > switch used to tell the SSL library whether or not to check the server > certificate against the CA. If so, it should make specifying the CA > public key redundant - the client just accepts whatever certificate > the server (or MITM) presents and starts encrypting to it. The modes are mostly rooted in the upcoming RFC5225 (or 5226, don't remember correctly). Anon is an insecure extension. While being insecure, it is a mode that allows low end devices deployed in no-knowledge environmebnt (hopefully read: home users) to have at least the benefit of encryption (obfuscation would be more precise) but nothing (nothing!) above that. Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Tue Dec 2 17:18:30 2008 From: aoz.syn at gmail.com (RB) Date: Tue, 2 Dec 2008 09:18:30 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> Message-ID: <4255c2570812020818w50a7a8b7k1c506ab8455b511e@mail.gmail.com> On Tue, Dec 2, 2008 at 09:00, Rainer Gerhards wrote: > Just a quick note, I am quite busy at the moment (guess what ;)). If the > auth is set to "anon" nothing at all is validated and MITM *is* > absolutely possible. That's why the doc does not recommend to use that > mode. I posted a link to the long TLS setup guide, which creates a > fairly safe scenario (but your milage may vary... ;)). Understood. For everyone else's edification, here is the comment in the related code, outlining what modes are used: /* Set the authentication mode. For us, the following is supported: * anon - no certificate checks whatsoever (discouraged, but supported) * x509/certvalid - (just) check certificate validity * x509/fingerprint - certificate fingerprint * x509/name - cerfificate name check * mode == NULL is valid and defaults to x509/name * rgerhards, 2008-05-16 */ From jmiscaro at gmail.com Tue Dec 2 17:30:42 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 11:30:42 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: 2008/12/2 RB : > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: >> "neither the client nor the server are authenticated. So while the >> message transfer is encrypted, you can not be sure which peer you are >> talking to" > > I'm hoping Rainer will jump in and clarify precisely how much > handshake validation he's implemented. The fact that the client must > have a copy of the CA's public material seems to indicate he is at > least verifying that the server's certificate was issued by the CA. > It's possible to not do so, but the result is rather susceptible to > MITM. > >> Also, how can client encrypt without having any keys specified in its config? > > This isn't the forum to discuss the particulars of the SSL handshake, > but suffice it to say that SSL incorporates a challenge/response > mechanism (using the server's presented certificate) followed by > negotiation of an ephemeral session key. See also: public-key > cryptography. > >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated >> >> 2nd question: Why is the server not authenticated? > > Without looking at the code, I presume the 'anon' AuthMode is the > switch used to tell the SSL library whether or not to check the server > certificate against the CA. If so, it should make specifying the CA > public key redundant - the client just accepts whatever certificate > the server (or MITM) presents and starts encrypting to it. Thank you. I change my config and logging is hapenning on the server end. However, I get such lines in the logs on the server end when I restart the client system: invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 42 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 45 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 46 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 47 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 49 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 51 This happens for each TLS line in my client config (comments removed): $DefaultNetstreamDriver gtls $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client-cert.pem $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client-key.pem $ActionSendStreamDriverAuthMode x509/name $ActionSendStreamDriverMode 1 *.* @@192.168.4.102:10514 /juan From rgerhards at hq.adiscon.com Tue Dec 2 17:31:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:31:13 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Too old version? > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > Sent: Tuesday, December 02, 2008 5:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > 2008/12/2 RB : > > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro > wrote: > >> "neither the client nor the server are authenticated. So while the > >> message transfer is encrypted, you can not be sure which peer you > are > >> talking to" > > > > I'm hoping Rainer will jump in and clarify precisely how much > > handshake validation he's implemented. The fact that the client must > > have a copy of the CA's public material seems to indicate he is at > > least verifying that the server's certificate was issued by the CA. > > It's possible to not do so, but the result is rather susceptible to > > MITM. > > > >> Also, how can client encrypt without having any keys specified in > its config? > > > > This isn't the forum to discuss the particulars of the SSL handshake, > > but suffice it to say that SSL incorporates a challenge/response > > mechanism (using the server's presented certificate) followed by > > negotiation of an ephemeral session key. See also: public-key > > cryptography. > > > >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > >> > >> 2nd question: Why is the server not authenticated? > > > > Without looking at the code, I presume the 'anon' AuthMode is the > > switch used to tell the SSL library whether or not to check the > server > > certificate against the CA. If so, it should make specifying the CA > > public key redundant - the client just accepts whatever certificate > > the server (or MITM) presents and starts encrypting to it. > > Thank you. I change my config and logging is hapenning on the server > end. However, I get such lines in the logs on the server end when I > restart the client system: > > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 42 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 45 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 46 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 47 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 49 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 51 > > > This happens for each TLS line in my client config (comments removed): > > $DefaultNetstreamDriver gtls > $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem > $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- > cert.pem > $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- > key.pem > $ActionSendStreamDriverAuthMode x509/name > $ActionSendStreamDriverMode 1 > *.* @@192.168.4.102:10514 > > > /juan > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jmiscaro at gmail.com Tue Dec 2 17:39:49 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 11:39:49 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Message-ID: My boxes are running 3.18.1 /juan 2008/12/2 Rainer Gerhards : > Too old version? > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Juan Miscaro >> Sent: Tuesday, December 02, 2008 5:31 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] TLS certificates >> >> 2008/12/2 RB : >> > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro >> wrote: >> >> "neither the client nor the server are authenticated. So while the >> >> message transfer is encrypted, you can not be sure which peer you >> are >> >> talking to" >> > >> > I'm hoping Rainer will jump in and clarify precisely how much >> > handshake validation he's implemented. The fact that the client > must >> > have a copy of the CA's public material seems to indicate he is at >> > least verifying that the server's certificate was issued by the CA. >> > It's possible to not do so, but the result is rather susceptible to >> > MITM. >> > >> >> Also, how can client encrypt without having any keys specified in >> its config? >> > >> > This isn't the forum to discuss the particulars of the SSL > handshake, >> > but suffice it to say that SSL incorporates a challenge/response >> > mechanism (using the server's presented certificate) followed by >> > negotiation of an ephemeral session key. See also: public-key >> > cryptography. >> > >> >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem >> >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated >> >> >> >> 2nd question: Why is the server not authenticated? >> > >> > Without looking at the code, I presume the 'anon' AuthMode is the >> > switch used to tell the SSL library whether or not to check the >> server >> > certificate against the CA. If so, it should make specifying the CA >> > public key redundant - the client just accepts whatever certificate >> > the server (or MITM) presents and starts encrypting to it. >> >> Thank you. I change my config and logging is hapenning on the server >> end. However, I get such lines in the logs on the server end when I >> restart the client system: >> >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 42 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 45 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 46 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 47 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 49 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 51 >> >> >> This happens for each TLS line in my client config (comments removed): >> >> $DefaultNetstreamDriver gtls >> $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem >> $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- >> cert.pem >> $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- >> key.pem >> $ActionSendStreamDriverAuthMode x509/name >> $ActionSendStreamDriverMode 1 >> *.* @@192.168.4.102:10514 >> >> >> /juan >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Tue Dec 2 17:52:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:52:25 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7B5@grfint2.intern.adiscon.com> Jup, that's the problem. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > Sent: Tuesday, December 02, 2008 5:40 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > My boxes are running 3.18.1 > > /juan > > 2008/12/2 Rainer Gerhards : > > Too old version? > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > >> Sent: Tuesday, December 02, 2008 5:31 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] TLS certificates > >> > >> 2008/12/2 RB : > >> > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro > >> wrote: > >> >> "neither the client nor the server are authenticated. So while > the > >> >> message transfer is encrypted, you can not be sure which peer you > >> are > >> >> talking to" > >> > > >> > I'm hoping Rainer will jump in and clarify precisely how much > >> > handshake validation he's implemented. The fact that the client > > must > >> > have a copy of the CA's public material seems to indicate he is at > >> > least verifying that the server's certificate was issued by the > CA. > >> > It's possible to not do so, but the result is rather susceptible > to > >> > MITM. > >> > > >> >> Also, how can client encrypt without having any keys specified in > >> its config? > >> > > >> > This isn't the forum to discuss the particulars of the SSL > > handshake, > >> > but suffice it to say that SSL incorporates a challenge/response > >> > mechanism (using the server's presented certificate) followed by > >> > negotiation of an ephemeral session key. See also: public-key > >> > cryptography. > >> > > >> >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > >> >> $ActionSendStreamDriverAuthMode anon # server is NOT > authenticated > >> >> > >> >> 2nd question: Why is the server not authenticated? > >> > > >> > Without looking at the code, I presume the 'anon' AuthMode is the > >> > switch used to tell the SSL library whether or not to check the > >> server > >> > certificate against the CA. If so, it should make specifying the > CA > >> > public key redundant - the client just accepts whatever > certificate > >> > the server (or MITM) presents and starts encrypting to it. > >> > >> Thank you. I change my config and logging is hapenning on the > server > >> end. However, I get such lines in the logs on the server end when I > >> restart the client system: > >> > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 42 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 45 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 46 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 47 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 49 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 51 > >> > >> > >> This happens for each TLS line in my client config (comments > removed): > >> > >> $DefaultNetstreamDriver gtls > >> $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem > >> $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- > >> cert.pem > >> $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- > >> key.pem > >> $ActionSendStreamDriverAuthMode x509/name > >> $ActionSendStreamDriverMode 1 > >> *.* @@192.168.4.102:10514 > >> > >> > >> /juan > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 10:54:20 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 10:54:20 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> Ok, I ran this fix through a couple of tests yesterday. It looks well for TLS, too. Note that there is an implication that $AllowedSender TCP,... applies to TLS to (because it is TCP). I'd consider this to be a side-effect, but I do not think it is worth fixing. With TLS, there is much finer and better control. An issue may only exists if someone decides to run non-tls tcp and tls tcp together AND use $AllowedSender. Workaround in that case is to use the firewall, so I don't consider it is worth fixing now. Please note that my testing revealed a potential memory leak as side-effect of the fixes. This could be abused to a remote DoS, so I will investigate that before releasing. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 6:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > And now there is an *untested* fix for the TLS driver: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > 3 > 83fc5969178887d00abfc > > Testing takes a bit more of time, I need to set up the test environment > for TLS again (looks like it would really pay to have a fixed test > suite > for all those cases - also the issue here would have never > occurred...). > > Please note that I mistook GSSAPI with TLS in my previous mail. The TLS > part should not be really affected by the problem: there are so much > better access control features in TLS... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 5:52 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, looks like I found a work-around. Not that elegant, but seems to > > work quite well. Patch for TCP is here: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > e > > 18747b55d701e360d5aac > > > > Please note that this effectively disables GSS functionality. I'll > > updated the GSS drivers in the next step. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 5:08 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > The issue also exists in TCP mode, but analysis shows this is not a > > > trial fix. The design overlooked the situation. In theory, a whole > > new > > > access control feature would be needed. I am checking out if it is > > > possible to "just" enhance the interface. With the current > netstreams > > > defined that should be possible. I am tempted to release the UDP- > > fixed > > > version and release the next version with the TCP fix. Feedback > from > > > packagers is appreciated. The TCP fix may take a day or two, > > depending > > > on how smart a way I find. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > ... and the patch will not work on all of these version, due to > the > > > > introduction of the netstream driver functionality. Please note > > that > > > > anything older than current v3-stable is outdated, so the proper > > way > > > to > > > > replace the faulty code is to upgrade to the current v3-stable > and > > > > apply > > > > the patch. I will also release a new v3-stable soon, hopefully > > today > > > > (but I'd like to conduct some more tests). > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > > > above. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Hi all, > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > 6 > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > I have not tried yet, but I think it will work on almost all > > > other > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > $AllowedSender > > > > > > directive > > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > > lead > > > to > > > > > > > > messages > > > > > > > > being received from systems that are not permitted so. As > a > > > > work- > > > > > > > > around, > > > > > > > > proper firewalling should be set up on the vulnerable > > hosts. > > > > > Until > > > > > > > > further note, I would assume that all versions of rsyslog > > are > > > > > > > affected > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Rainer > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 11:30:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 11:30:29 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> The memory leak is now also fixed, I just quickly re-run some TLS tests to make sure nothing is broken and it works there, too. Patch (on top of the others): http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54ddd cb8703560c750f04a6370 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 10:54 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Ok, I ran this fix through a couple of tests yesterday. It looks well > for TLS, too. Note that there is an implication that $AllowedSender > TCP,... applies to TLS to (because it is TCP). I'd consider this to be > a > side-effect, but I do not think it is worth fixing. With TLS, there is > much finer and better control. An issue may only exists if someone > decides to run non-tls tcp and tls tcp together AND use $AllowedSender. > Workaround in that case is to use the firewall, so I don't consider it > is worth fixing now. > > Please note that my testing revealed a potential memory leak as > side-effect of the fixes. This could be abused to a remote DoS, so I > will investigate that before releasing. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 6:47 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > And now there is an *untested* fix for the TLS driver: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > 3 > > 83fc5969178887d00abfc > > > > Testing takes a bit more of time, I need to set up the test > environment > > for TLS again (looks like it would really pay to have a fixed test > > suite > > for all those cases - also the issue here would have never > > occurred...). > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > TLS > > part should not be really affected by the problem: there are so much > > better access control features in TLS... > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 5:52 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > to > > > work quite well. Patch for TCP is here: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > e > > > 18747b55d701e360d5aac > > > > > > Please note that this effectively disables GSS functionality. I'll > > > updated the GSS drivers in the next step. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > The issue also exists in TCP mode, but analysis shows this is not > a > > > > trial fix. The design overlooked the situation. In theory, a > whole > > > new > > > > access control feature would be needed. I am checking out if it > is > > > > possible to "just" enhance the interface. With the current > > netstreams > > > > defined that should be possible. I am tempted to release the UDP- > > > fixed > > > > version and release the next version with the TCP fix. Feedback > > from > > > > packagers is appreciated. The TCP fix may take a day or two, > > > depending > > > > on how smart a way I find. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > ... and the patch will not work on all of these version, due to > > the > > > > > introduction of the netstream driver functionality. Please note > > > that > > > > > anything older than current v3-stable is outdated, so the > proper > > > way > > > > to > > > > > replace the faulty code is to upgrade to the current v3-stable > > and > > > > > apply > > > > > the patch. I will also release a new v3-stable soon, hopefully > > > today > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > and > > > > above. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > 6 > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > all > > > > other > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > $AllowedSender > > > > > > > directive > > > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > > > lead > > > > to > > > > > > > > > messages > > > > > > > > > being received from systems that are not permitted so. > As > > a > > > > > work- > > > > > > > > > around, > > > > > > > > > proper firewalling should be set up on the vulnerable > > > hosts. > > > > > > Until > > > > > > > > > further note, I would assume that all versions of > rsyslog > > > are > > > > > > > > affected > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Rainer > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 16:47:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 16:47:45 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7D5@grfint2.intern.adiscon.com> ...TLS testing always takes an awful lot of time... But I was also able to identify another memory leak, which is nice. I think I have now finished the release (less some doc update, maybe). I'd appreciate if some of you could give it a try. I would then do the actual release tomorrow. Download is: http://download.rsyslog.com/rsyslog/rsyslog-3.20.1.tar.gz md5sum: 2786d0d8de85fc9e6e83ff4ed9f468a7 If you try it out, please let me know the results. As I said, I don't expect anything bad, so it should be suitable for production use. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 11:30 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The memory leak is now also fixed, I just quickly re-run some TLS tests > to make sure nothing is broken and it works there, too. > > Patch (on top of the others): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > d > cb8703560c750f04a6370 > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 10:54 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, I ran this fix through a couple of tests yesterday. It looks well > > for TLS, too. Note that there is an implication that $AllowedSender > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > be > > a > > side-effect, but I do not think it is worth fixing. With TLS, there > is > > much finer and better control. An issue may only exists if someone > > decides to run non-tls tcp and tls tcp together AND use > $AllowedSender. > > Workaround in that case is to use the firewall, so I don't consider > it > > is worth fixing now. > > > > Please note that my testing revealed a potential memory leak as > > side-effect of the fixes. This could be abused to a remote DoS, so I > > will investigate that before releasing. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 6:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > 3 > > > 83fc5969178887d00abfc > > > > > > Testing takes a bit more of time, I need to set up the test > > environment > > > for TLS again (looks like it would really pay to have a fixed test > > > suite > > > for all those cases - also the issue here would have never > > > occurred...). > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > > TLS > > > part should not be really affected by the problem: there are so > much > > > better access control features in TLS... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > > to > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > e > > > > 18747b55d701e360d5aac > > > > > > > > Please note that this effectively disables GSS functionality. > I'll > > > > updated the GSS drivers in the next step. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > not > > a > > > > > trial fix. The design overlooked the situation. In theory, a > > whole > > > > new > > > > > access control feature would be needed. I am checking out if it > > is > > > > > possible to "just" enhance the interface. With the current > > > netstreams > > > > > defined that should be possible. I am tempted to release the > UDP- > > > > fixed > > > > > version and release the next version with the TCP fix. Feedback > > > from > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > depending > > > > > on how smart a way I find. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > ... and the patch will not work on all of these version, due > to > > > the > > > > > > introduction of the netstream driver functionality. Please > note > > > > that > > > > > > anything older than current v3-stable is outdated, so the > > proper > > > > way > > > > > to > > > > > > replace the faulty code is to upgrade to the current v3- > stable > > > and > > > > > > apply > > > > > > the patch. I will also release a new v3-stable soon, > hopefully > > > > today > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > > and > > > > > above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > 6 > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > > all > > > > > other > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > $AllowedSender > > > > > > > > directive > > > > > > > > > > does not work in all releases. The bug in question > is: > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > can > > > > lead > > > > > to > > > > > > > > > > messages > > > > > > > > > > being received from systems that are not permitted > so. > > As > > > a > > > > > > work- > > > > > > > > > > around, > > > > > > > > > > proper firewalling should be set up on the vulnerable > > > > hosts. > > > > > > > Until > > > > > > > > > > further note, I would assume that all versions of > > rsyslog > > > > are > > > > > > > > > affected > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Rainer > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Dec 4 12:30:58 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Dec 2008 12:30:58 +0100 Subject: [rsyslog] rsyslog 3.20.1 (v3-stable) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E3@grfint2.intern.adiscon.com> Hello, We have just released rsyslog 3.20.1, a member of the v3-stable branch. Most importantly, this release addresses a security vulnerability that renders the $AllowedSender directive useless. This issue has already been discussed here on the list. In addition to this, the release also contains some bug fixes, including a memory leak that could cause real trouble in environments that use TLS, as well as some very small enhancements (that were considered small enough to be a applied to the stable branch) Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-141.phtml Change Log: http://www.rsyslog.com/Article323.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. Releases for the beta and devel branches will hopefully be posted later today. The git archive has all relevant patches if someone has an urgent need. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Thu Dec 4 13:34:08 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Dec 2008 13:34:08 +0100 Subject: [rsyslog] rsyslog 3.21.8 (v3-beta) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 3.21.8, a member of the v3-beta branch. Most importantly, this release addresses a security vulnerability the renders the $AllowedSender directive useless. This issue has already been discussed here on the list. In addition to this, the release also contains all the bug fixes and enhancements from the stable release 3.20.1. Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-142.phtml Change Log: http://www.rsyslog.com/Article326.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. Releases for the devel branch will hopefully be posted later today. The git archive has all relevant patches if someone has an urgent need. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Thu Dec 4 13:38:17 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 13:38:17 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> Grrr... One more issue. I noticed that while I resolved some conflicts on the devel branch integration. There is an option that a log message is emitted by rsyslog itself, when a remote machine's message is discarded due to no permission. This was requested so that people know when something goes wrong. This is only in the UDP code. HOWEVER, this is not rate-limited so if someone carries out a heavy attack, he can still flood the local disk by these messages. I'll change it so that the message is emited only once every minute and will then re-release what already has been released... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 11:30 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The memory leak is now also fixed, I just quickly re-run some TLS tests > to make sure nothing is broken and it works there, too. > > Patch (on top of the others): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > d > cb8703560c750f04a6370 > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 10:54 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, I ran this fix through a couple of tests yesterday. It looks well > > for TLS, too. Note that there is an implication that $AllowedSender > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > be > > a > > side-effect, but I do not think it is worth fixing. With TLS, there > is > > much finer and better control. An issue may only exists if someone > > decides to run non-tls tcp and tls tcp together AND use > $AllowedSender. > > Workaround in that case is to use the firewall, so I don't consider > it > > is worth fixing now. > > > > Please note that my testing revealed a potential memory leak as > > side-effect of the fixes. This could be abused to a remote DoS, so I > > will investigate that before releasing. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 6:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > 3 > > > 83fc5969178887d00abfc > > > > > > Testing takes a bit more of time, I need to set up the test > > environment > > > for TLS again (looks like it would really pay to have a fixed test > > > suite > > > for all those cases - also the issue here would have never > > > occurred...). > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > > TLS > > > part should not be really affected by the problem: there are so > much > > > better access control features in TLS... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > > to > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > e > > > > 18747b55d701e360d5aac > > > > > > > > Please note that this effectively disables GSS functionality. > I'll > > > > updated the GSS drivers in the next step. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > not > > a > > > > > trial fix. The design overlooked the situation. In theory, a > > whole > > > > new > > > > > access control feature would be needed. I am checking out if it > > is > > > > > possible to "just" enhance the interface. With the current > > > netstreams > > > > > defined that should be possible. I am tempted to release the > UDP- > > > > fixed > > > > > version and release the next version with the TCP fix. Feedback > > > from > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > depending > > > > > on how smart a way I find. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > ... and the patch will not work on all of these version, due > to > > > the > > > > > > introduction of the netstream driver functionality. Please > note > > > > that > > > > > > anything older than current v3-stable is outdated, so the > > proper > > > > way > > > > > to > > > > > > replace the faulty code is to upgrade to the current v3- > stable > > > and > > > > > > apply > > > > > > the patch. I will also release a new v3-stable soon, > hopefully > > > > today > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > > and > > > > > above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > 6 > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > > all > > > > > other > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > $AllowedSender > > > > > > > > directive > > > > > > > > > > does not work in all releases. The bug in question > is: > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > can > > > > lead > > > > > to > > > > > > > > > > messages > > > > > > > > > > being received from systems that are not permitted > so. > > As > > > a > > > > > > work- > > > > > > > > > > around, > > > > > > > > > > proper firewalling should be set up on the vulnerable > > > > hosts. > > > > > > > Until > > > > > > > > > > further note, I would assume that all versions of > > rsyslog > > > > are > > > > > > > > > affected > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Rainer > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 4 14:48:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 14:48:03 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> OK, 3.20.1 is now re-released as 3.20.2 (there were a few downloads...). The download link is still correct, it is updated (including the md5sum ;)). 3.21.8 is pulled and I'll restore it next. Sorry for the hassle. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, December 04, 2008 1:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Grrr... One more issue. I noticed that while I resolved some conflicts > on the devel branch integration. There is an option that a log message > is emitted by rsyslog itself, when a remote machine's message is > discarded due to no permission. This was requested so that people know > when something goes wrong. This is only in the UDP code. > > HOWEVER, this is not rate-limited so if someone carries out a heavy > attack, he can still flood the local disk by these messages. I'll > change > it so that the message is emited only once every minute and will then > re-release what already has been released... > > Rainer > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 11:30 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > The memory leak is now also fixed, I just quickly re-run some TLS > tests > > to make sure nothing is broken and it works there, too. > > > > Patch (on top of the others): > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > > d > > cb8703560c750f04a6370 > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Wednesday, December 03, 2008 10:54 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Ok, I ran this fix through a couple of tests yesterday. It looks > well > > > for TLS, too. Note that there is an implication that $AllowedSender > > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > > be > > > a > > > side-effect, but I do not think it is worth fixing. With TLS, there > > is > > > much finer and better control. An issue may only exists if someone > > > decides to run non-tls tcp and tls tcp together AND use > > $AllowedSender. > > > Workaround in that case is to use the firewall, so I don't consider > > it > > > is worth fixing now. > > > > > > Please note that my testing revealed a potential memory leak as > > > side-effect of the fixes. This could be abused to a remote DoS, so > I > > > will investigate that before releasing. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 6:47 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > > 3 > > > > 83fc5969178887d00abfc > > > > > > > > Testing takes a bit more of time, I need to set up the test > > > environment > > > > for TLS again (looks like it would really pay to have a fixed > test > > > > suite > > > > for all those cases - also the issue here would have never > > > > occurred...). > > > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. > The > > > TLS > > > > part should not be really affected by the problem: there are so > > much > > > > better access control features in TLS... > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but > seems > > > to > > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > > e > > > > > 18747b55d701e360d5aac > > > > > > > > > > Please note that this effectively disables GSS functionality. > > I'll > > > > > updated the GSS drivers in the next step. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > > not > > > a > > > > > > trial fix. The design overlooked the situation. In theory, a > > > whole > > > > > new > > > > > > access control feature would be needed. I am checking out if > it > > > is > > > > > > possible to "just" enhance the interface. With the current > > > > netstreams > > > > > > defined that should be possible. I am tempted to release the > > UDP- > > > > > fixed > > > > > > version and release the next version with the TCP fix. > Feedback > > > > from > > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > > depending > > > > > > on how smart a way I find. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > ... and the patch will not work on all of these version, > due > > to > > > > the > > > > > > > introduction of the netstream driver functionality. Please > > note > > > > > that > > > > > > > anything older than current v3-stable is outdated, so the > > > proper > > > > > way > > > > > > to > > > > > > > replace the faulty code is to upgrade to the current v3- > > stable > > > > and > > > > > > > apply > > > > > > > the patch. I will also release a new v3-stable soon, > > hopefully > > > > > today > > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > I now clarified the affected versions. Affected are > 3.12.2 > > > and > > > > > > above. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > > 6 > > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on > almost > > > all > > > > > > other > > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog- > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > Gerhards > > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > > $AllowedSender > > > > > > > > > directive > > > > > > > > > > > does not work in all releases. The bug in question > > is: > > > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > > can > > > > > lead > > > > > > to > > > > > > > > > > > messages > > > > > > > > > > > being received from systems that are not permitted > > so. > > > As > > > > a > > > > > > > work- > > > > > > > > > > > around, > > > > > > > > > > > proper firewalling should be set up on the > vulnerable > > > > > hosts. > > > > > > > > Until > > > > > > > > > > > further note, I would assume that all versions of > > > rsyslog > > > > > are > > > > > > > > > > affected > > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Rainer > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Thu Dec 4 16:03:30 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 4 Dec 2008 08:03:30 -0700 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> Message-ID: <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> > Sorry for the hassle. It's not a hassle when you're being open and honest about issues. Too few projects call an apple an apple, so it's pleasing to be able to understand precisely what issues are. From Gerhardus.Geldenhuis at gta-travel.com Thu Dec 4 16:09:07 2008 From: Gerhardus.Geldenhuis at gta-travel.com (Gerhardus.Geldenhuis at gta-travel.com) Date: Thu, 4 Dec 2008 15:09:07 -0000 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> Message-ID: <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> I must concur, I am not very active on this mailing list in either form, but rsyslog does represent to me what open source was always about. The openness, speed and intelligence with which bugs/issues are addressed are exemplary. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: 04 December 2008 15:04 > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > > Sorry for the hassle. > > It's not a hassle when you're being open and honest about issues. Too > few projects call an apple an apple, so it's pleasing to be able to > understand precisely what issues are. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ From rgerhards at hq.adiscon.com Thu Dec 4 16:39:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 16:39:45 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com><4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7F1@grfint2.intern.adiscon.com> Thanks folks for the nice statements. Obviously much appreciated ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gerhardus.Geldenhuis at gta- > travel.com > Sent: Thursday, December 04, 2008 4:09 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] security issue in rsyslog > > I must concur, I am not very active on this mailing list in either > form, > but rsyslog does represent to me what open source was always about. The > openness, speed and intelligence with which bugs/issues are addressed > are exemplary. > > Regards > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of RB > > Sent: 04 December 2008 15:04 > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > Sorry for the hassle. > > > > It's not a hassle when you're being open and honest about issues. > Too > > few projects call an apple an apple, so it's pleasing to be able to > > understand precisely what issues are. > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 4 17:40:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 17:40:43 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7FF@grfint2.intern.adiscon.com> 3.21.8 has now also been replaced by 3.21.9. As with 3.20.2, links remain intact. 3.21.8 has probably never been downloaded, but I thought it is saver to use a new version number, especially as it is a security issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, December 04, 2008 2:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > OK, 3.20.1 is now re-released as 3.20.2 (there were a few > downloads...). > The download link is still correct, it is updated (including the md5sum > ;)). 3.21.8 is pulled and I'll restore it next. > > Sorry for the hassle. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Thursday, December 04, 2008 1:38 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Grrr... One more issue. I noticed that while I resolved some > conflicts > > on the devel branch integration. There is an option that a log > message > > is emitted by rsyslog itself, when a remote machine's message is > > discarded due to no permission. This was requested so that people > know > > when something goes wrong. This is only in the UDP code. > > > > HOWEVER, this is not rate-limited so if someone carries out a heavy > > attack, he can still flood the local disk by these messages. I'll > > change > > it so that the message is emited only once every minute and will then > > re-release what already has been released... > > > > Rainer > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Wednesday, December 03, 2008 11:30 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > The memory leak is now also fixed, I just quickly re-run some TLS > > tests > > > to make sure nothing is broken and it works there, too. > > > > > > Patch (on top of the others): > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > > > d > > > cb8703560c750f04a6370 > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Wednesday, December 03, 2008 10:54 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, I ran this fix through a couple of tests yesterday. It looks > > well > > > > for TLS, too. Note that there is an implication that > $AllowedSender > > > > TCP,... applies to TLS to (because it is TCP). I'd consider this > to > > > be > > > > a > > > > side-effect, but I do not think it is worth fixing. With TLS, > there > > > is > > > > much finer and better control. An issue may only exists if > someone > > > > decides to run non-tls tcp and tls tcp together AND use > > > $AllowedSender. > > > > Workaround in that case is to use the firewall, so I don't > consider > > > it > > > > is worth fixing now. > > > > > > > > Please note that my testing revealed a potential memory leak as > > > > side-effect of the fixes. This could be abused to a remote DoS, > so > > I > > > > will investigate that before releasing. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 6:47 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > > > 3 > > > > > 83fc5969178887d00abfc > > > > > > > > > > Testing takes a bit more of time, I need to set up the test > > > > environment > > > > > for TLS again (looks like it would really pay to have a fixed > > test > > > > > suite > > > > > for all those cases - also the issue here would have never > > > > > occurred...). > > > > > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. > > The > > > > TLS > > > > > part should not be really affected by the problem: there are so > > > much > > > > > better access control features in TLS... > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but > > seems > > > > to > > > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > > > e > > > > > > 18747b55d701e360d5aac > > > > > > > > > > > > Please note that this effectively disables GSS functionality. > > > I'll > > > > > > updated the GSS drivers in the next step. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this > is > > > not > > > > a > > > > > > > trial fix. The design overlooked the situation. In theory, > a > > > > whole > > > > > > new > > > > > > > access control feature would be needed. I am checking out > if > > it > > > > is > > > > > > > possible to "just" enhance the interface. With the current > > > > > netstreams > > > > > > > defined that should be possible. I am tempted to release > the > > > UDP- > > > > > > fixed > > > > > > > version and release the next version with the TCP fix. > > Feedback > > > > > from > > > > > > > packagers is appreciated. The TCP fix may take a day or > two, > > > > > > depending > > > > > > > on how smart a way I find. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > ... and the patch will not work on all of these version, > > due > > > to > > > > > the > > > > > > > > introduction of the netstream driver functionality. > Please > > > note > > > > > > that > > > > > > > > anything older than current v3-stable is outdated, so the > > > > proper > > > > > > way > > > > > > > to > > > > > > > > replace the faulty code is to upgrade to the current v3- > > > stable > > > > > and > > > > > > > > apply > > > > > > > > the patch. I will also release a new v3-stable soon, > > > hopefully > > > > > > today > > > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > I now clarified the affected versions. Affected are > > 3.12.2 > > > > and > > > > > > > above. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > > > 6 > > > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on > > almost > > > > all > > > > > > > other > > > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog- > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > Gerhards > > > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > > [mailto:rsyslog- > > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > > Gerhards > > > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > > > $AllowedSender > > > > > > > > > > directive > > > > > > > > > > > > does not work in all releases. The bug in > question > > > is: > > > > > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, > this > > > can > > > > > > lead > > > > > > > to > > > > > > > > > > > > messages > > > > > > > > > > > > being received from systems that are not > permitted > > > so. > > > > As > > > > > a > > > > > > > > work- > > > > > > > > > > > > around, > > > > > > > > > > > > proper firewalling should be set up on the > > vulnerable > > > > > > hosts. > > > > > > > > > Until > > > > > > > > > > > > further note, I would assume that all versions of > > > > rsyslog > > > > > > are > > > > > > > > > > > affected > > > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Rainer > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Fri Dec 5 15:59:29 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 5 Dec 2008 15:59:29 +0100 Subject: [rsyslog] rsyslog 4.1.2 (v4-devel) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F81C@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 4.1.2, a member of the v4-development branch. Most importantly, this release addresses a security vulnerability that renders the $AllowedSender directive useless. This release has another security fix, which addresses a imudp which emitted a message each time a non-permitted sender tried to send a message. This could have filled the disk. Now it only emits a message once per minute. Further, all fixes and changes from 3.21.9 (beta) and 3.20.2 (stable) have been included in this release. Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-143.phtml Change Log: http://www.rsyslog.com/Article331.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From samuel at dragonboricua.net Mon Dec 8 06:25:30 2008 From: samuel at dragonboricua.net (Elisamuel Resto) Date: Mon, 8 Dec 2008 01:25:30 -0400 Subject: [rsyslog] rsyslog 3.21.8 (v3-beta) released - IMPORTANT SECURITY RELEASE In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> Message-ID: <20081208012530.20571aa8@zeus.dragonboricua.com> On Thu, 4 Dec 2008 13:34:08 +0100, Florian Riedl wrote: > Hi all, > > We have just released rsyslog 3.21.8, a member of the v3-beta branch. > Most importantly, this release addresses a security vulnerability the > renders the $AllowedSender directive useless. This issue has already > been discussed here on the list. In addition to this, the release also > contains all the bug fixes and enhancements from the stable release > 3.20.1. I believe the following is worthy of note in regards to this. On Thu, 4 Dec 2008 17:40:43 +0100, Rainer Gerhards wrote: > 3.21.8 has now also been replaced by 3.21.9. As with 3.20.2, links > remain intact. 3.21.8 has probably never been downloaded, but I thought > it is saver to use a new version number, especially as it is a security > issue. > > Rainer -- Elisamuel Resto Source Mage Tome Lead / http://sourcemage.org GPG ID: 18615F19/1024D / http://simplysam.us From nyerup at one.com Mon Dec 8 09:06:22 2008 From: nyerup at one.com (Jesper Dahl Nyerup) Date: Mon, 08 Dec 2008 09:06:22 +0100 Subject: [rsyslog] Cleanup after ugly shutdown Message-ID: <1228723582.3628.26.camel@iapetus.one.com> Hi all. I recently encountered an incident where rsyslogd died in an "ugly" manner. This was due to no fault of its own, but simply a matter of running out of disk space, because an instance, running with a disk-assisted memory queue, failed to reconnect to the syslog server, and therefore didn't have anywhere to put the log data, except on its own hard drive. When rsyslog dies like this (or in any other way that prevents a decent shutdown), it obviously do not write a .qi-file, indicating where it left off, and thus all the data written to disk is not handled when rsyslogd is started again. This behaviour is correct, as I see it, as at least some of the data written to disk, almost certainly will be inconsistent. But still I would like the opportunity to manually process these data, so that they eventually will be sent to the syslog server. What is the smartest way to properly do this? Has it been considered to let rsyslogd take a command line argument invoking such behaviour, or has someone written a seperate tool to do something like this? Any ideas are welcome. Kind regards, Jesper Nyerup. From rgerhards at hq.adiscon.com Mon Dec 8 10:42:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 10:42:46 +0100 Subject: [rsyslog] analysis of security issue in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F82E@grfint2.intern.adiscon.com> Hi all, I have done a bit of analysis of the issue and blogged about it: http://blog.gerhards.net/2008/12/root-cause-of-security-issue-in-rsyslog .html I hope this is useful information. As I wrote, I will try to focus on getting at basic test suite using DejaGNU (or anything else quickly suggested ;)) up and running. Any help would be appreciated. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 8 10:47:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 10:47:57 +0100 Subject: [rsyslog] make distcheck - multiple configure options possible? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F82F@grfint2.intern.adiscon.com> Hi all, I thought I ask this list before I search further. One thing that is annoying is that from time to time I have some minor nits because not all code pathes are even compiled due to the many configure options that exists (obviously even less are executed). I regularly run make distcheck, but as far as I know, I can only supply one set of configure options to it. Does anybody know a way to have multiple "make distcheck"s running, all with different set of configure options? All feedback is appreciated. Thanks. Rainer From rgerhards at hq.adiscon.com Mon Dec 8 12:33:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 12:33:30 +0100 Subject: [rsyslog] Security Fix for 3.18.5 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F83C@grfint2.intern.adiscon.com> Hi all, Michael Biebl made me aware that Debian Lenny uses rsyslog 3.18.5 and is unable to move to 3.20.x due to release freeze. I have now created a debian_lenny branch in git and backported the fixes to that branch. A quick test succeeded. So if anyone is interested in that version, he or she may pull it from git. If anyone finds a problem with that backport, please let me know. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 8 12:37:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 12:37:42 +0100 Subject: [rsyslog] Cleanup after ugly shutdown In-Reply-To: <1228723582.3628.26.camel@iapetus.one.com> References: <1228723582.3628.26.camel@iapetus.one.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F83D@grfint2.intern.adiscon.com> Hi Jesper, one solution is to use the checkpoint interval. This ensures a queue file is written every n-th time a message is enqueued. This is inside the queue doc. On the "restart queue" option, I agree that it would be useful, but I unfortunately currently neither have time nor funding to implement it. You may want to add this as a feature request to the bugzilla, so that it is not forgotten if time arises... Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jesper Dahl Nyerup > Sent: Monday, December 08, 2008 9:06 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Cleanup after ugly shutdown > > Hi all. > > I recently encountered an incident where rsyslogd died in an "ugly" > manner. This was due to no fault of its own, but simply a matter of > running out of disk space, because an instance, running with a > disk-assisted memory queue, failed to reconnect to the syslog server, > and therefore didn't have anywhere to put the log data, except on its > own hard drive. > > When rsyslog dies like this (or in any other way that prevents a decent > shutdown), it obviously do not write a .qi-file, indicating where it > left off, and thus all the data written to disk is not handled when > rsyslogd is started again. This behaviour is correct, as I see it, as > at > least some of the data written to disk, almost certainly will be > inconsistent. > > But still I would like the opportunity to manually process these data, > so that they eventually will be sent to the syslog server. What is the > smartest way to properly do this? Has it been considered to let > rsyslogd > take a command line argument invoking such behaviour, or has someone > written a seperate tool to do something like this? > > Any ideas are welcome. > > > Kind regards, > > Jesper Nyerup. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Fri Dec 12 07:48:48 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Fri, 12 Dec 2008 07:48:48 +0100 Subject: [rsyslog] Disk error when using rsyslog Message-ID: <49420950.2030904@net-m.de> Hi Rainer, We're using rsyslog(3.20.0) as our central logging server and clients. We experienced an DISK ERROR on server last month. At that time, we were using TCP to transport logs from client to server. And we also setup the configuration just like [1]. But unfortunately our central logging server got DISK error for one hour. So we lost logfiles of that period of time. I've a look at doc [1] carefully, I guess "RELIABLE" only means when server got offline or rsyslogd on it isn't running, then clients will save logs in buffer or write to a file on disk. If server is still online and rsyslogd is running, but with IO/Error or Disk Full, then client will still transfer logs to server even with RELP, coz I guess RELP only protects logs could be transferred via network successfully, it doesn't care the logs are written successfully to file on server. Am I right? So I guess if we need to prevent this, we need do some work on server? Do we have some "directives" options that we could transfer logs to a failover server if local disk fails or buffer in memory before disk got corrected? [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html Thanks, -- Patrick Shen Operations Engineer From hks.private at gmail.com Fri Dec 12 16:09:42 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 12 Dec 2008 10:09:42 -0500 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <49420950.2030904@net-m.de> References: <49420950.2030904@net-m.de> Message-ID: On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen wrote: > Hi Rainer, > > We're using rsyslog(3.20.0) as our central logging server and clients. > > We experienced an DISK ERROR on server last month. At that time, we were > using TCP to transport logs from client to server. And we also setup the > configuration just like [1]. But unfortunately our central logging > server got DISK error for one hour. So we lost logfiles of that period > of time. > > I've a look at doc [1] carefully, I guess "RELIABLE" only means when > server got offline or rsyslogd on it isn't running, then clients will > save logs in buffer or write to a file on disk. If server is still > online and rsyslogd is running, but with IO/Error or Disk Full, then > client will still transfer logs to server even with RELP, coz I guess > RELP only protects logs could be transferred via network successfully, > it doesn't care the logs are written successfully to file on server. Am > I right? Yes. RELP Is a protocol for the reliable exchange of event logs over a network. What the destination daemon does once it has the logs is no concern of the client's. > So I guess if we need to prevent this, we need do some work on server? > Do we have some "directives" options that we could transfer logs to a > failover server if local disk fails or buffer in memory before disk got > corrected? Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer -HKS > [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html > > Thanks, > > -- > Patrick Shen > Operations Engineer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Dec 12 16:16:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Dec 2008 16:16:21 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: References: <49420950.2030904@net-m.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> I seem to have overlooked the initial message (spam filter maybe...). HKS is right (thx), but I think this looks like a bug in that the output write does not care about the write failure (what it should). The output writer is pretty old legacy code, so that's quite possible. I'll look into it ASAP, but I got a new machine (hopefully fast enough to disply some troubles) today and currently I am happy that at least mail does work again (so far it's a mess). So... some time next week ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Friday, December 12, 2008 4:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Disk error when using rsyslog > > On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen > wrote: > > Hi Rainer, > > > > We're using rsyslog(3.20.0) as our central logging server and > clients. > > > > We experienced an DISK ERROR on server last month. At that time, we > were > > using TCP to transport logs from client to server. And we also setup > the > > configuration just like [1]. But unfortunately our central logging > > server got DISK error for one hour. So we lost logfiles of that > period > > of time. > > > > I've a look at doc [1] carefully, I guess "RELIABLE" only means when > > server got offline or rsyslogd on it isn't running, then clients will > > save logs in buffer or write to a file on disk. If server is still > > online and rsyslogd is running, but with IO/Error or Disk Full, then > > client will still transfer logs to server even with RELP, coz I guess > > RELP only protects logs could be transferred via network > successfully, > > it doesn't care the logs are written successfully to file on server. > Am > > I right? > > Yes. RELP Is a protocol for the reliable exchange of event logs over a > network. What the destination daemon does once it has the logs is no > concern of the client's. > > > > So I guess if we need to prevent this, we need do some work on > server? > > Do we have some "directives" options that we could transfer logs to a > > failover server if local disk fails or buffer in memory before disk > got > > corrected? > > Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > -HKS > > > [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html > > > > Thanks, > > > > -- > > Patrick Shen > > Operations Engineer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Sun Dec 14 04:46:38 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Sun, 14 Dec 2008 04:46:38 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: References: <49420950.2030904@net-m.de> Message-ID: <4944819E.90101@net-m.de> Hi HKS, Thanks for your good advice. It's helpful. Patrick (private) HKS wrote: > On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen wrote: >> Hi Rainer, >> >> We're using rsyslog(3.20.0) as our central logging server and clients. >> >> We experienced an DISK ERROR on server last month. At that time, we were >> using TCP to transport logs from client to server. And we also setup the >> configuration just like [1]. But unfortunately our central logging >> server got DISK error for one hour. So we lost logfiles of that period >> of time. >> >> I've a look at doc [1] carefully, I guess "RELIABLE" only means when >> server got offline or rsyslogd on it isn't running, then clients will >> save logs in buffer or write to a file on disk. If server is still >> online and rsyslogd is running, but with IO/Error or Disk Full, then >> client will still transfer logs to server even with RELP, coz I guess >> RELP only protects logs could be transferred via network successfully, >> it doesn't care the logs are written successfully to file on server. Am >> I right? > > Yes. RELP Is a protocol for the reliable exchange of event logs over a > network. What the destination daemon does once it has the logs is no > concern of the client's. > > >> So I guess if we need to prevent this, we need do some work on server? >> Do we have some "directives" options that we could transfer logs to a >> failover server if local disk fails or buffer in memory before disk got >> corrected? > > Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > -HKS > >> [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html >> >> Thanks, >> >> -- >> Patrick Shen >> Operations Engineer >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Sun Dec 14 04:56:16 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Sun, 14 Dec 2008 04:56:16 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> Message-ID: <494483E0.3030702@net-m.de> Hi Rainer, Rainer Gerhards wrote: > I seem to have overlooked the initial message (spam filter maybe...). Bad luck for my mail account :-( > HKS is right (thx), but I think this looks like a bug in that the output > write does not care about the write failure (what it should). The output > writer is pretty old legacy code, so that's quite possible. I'll look > into it ASAP, but I got a new machine (hopefully fast enough to disply > some troubles) today and currently I am happy that at least mail does > work again (so far it's a mess). So... some time next week ;) Quite appreciate if you have a look at write failure. I think it's quite Enterprise demand feature :-) Many thanks, Patrick From rgerhards at hq.adiscon.com Mon Dec 15 12:03:53 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 15 Dec 2008 12:03:53 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <494483E0.3030702@net-m.de> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> <494483E0.3030702@net-m.de> Message-ID: <1229339033.2482.3.camel@localhost.localdomain> On Sun, 2008-12-14 at 04:56 +0100, Patrick Shen wrote: > Hi Rainer, > > Rainer Gerhards wrote: > > I seem to have overlooked the initial message (spam filter maybe...). > > Bad luck for my mail account :-( > > > HKS is right (thx), but I think this looks like a bug in that the output > > write does not care about the write failure (what it should). The output > > writer is pretty old legacy code, so that's quite possible. I'll look > > into it ASAP, but I got a new machine (hopefully fast enough to disply > > some troubles) today and currently I am happy that at least mail does > > work again (so far it's a mess). So... some time next week ;) > > Quite appreciate if you have a look at write failure. I think it's quite > Enterprise demand feature :-) I have now verified that the code (by intension) ignores write errors. That, of course, is legacy from a long gone era. However, I need to think a bit about how to handle this most intelligently. The problem is partial writes. Maybe I just try to write a LF after a failure and, if that succeeds, simply continue. This results in a partial record begin written and then the same record being "duplicated" (actually, the partial part being duplicated). Does anyone have a suggestion on how to best handle such a case? Or I could try to write what could not yet be written. Maybe this is better, but it wont' be able to survive a daemon restart... Feedback appreciated. Rainer From david at lang.hm Tue Dec 16 10:23:37 2008 From: david at lang.hm (david at lang.hm) Date: Tue, 16 Dec 2008 01:23:37 -0800 (PST) Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <1229339033.2482.3.camel@localhost.localdomain> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> <494483E0.3030702@net-m.de> <1229339033.2482.3.camel@localhost.localdomain> Message-ID: On Mon, 15 Dec 2008, Rainer Gerhards wrote: > On Sun, 2008-12-14 at 04:56 +0100, Patrick Shen wrote: >> Hi Rainer, >> >> Rainer Gerhards wrote: >>> I seem to have overlooked the initial message (spam filter maybe...). >> >> Bad luck for my mail account :-( >> >>> HKS is right (thx), but I think this looks like a bug in that the output >>> write does not care about the write failure (what it should). The output >>> writer is pretty old legacy code, so that's quite possible. I'll look >>> into it ASAP, but I got a new machine (hopefully fast enough to disply >>> some troubles) today and currently I am happy that at least mail does >>> work again (so far it's a mess). So... some time next week ;) >> >> Quite appreciate if you have a look at write failure. I think it's quite >> Enterprise demand feature :-) > > I have now verified that the code (by intension) ignores write errors. > That, of course, is legacy from a long gone era. However, I need to > think a bit about how to handle this most intelligently. The problem is > partial writes. Maybe I just try to write a LF after a failure and, if > that succeeds, simply continue. This results in a partial record begin > written and then the same record being "duplicated" (actually, the > partial part being duplicated). > > Does anyone have a suggestion on how to best handle such a case? Or I > could try to write what could not yet be written. Maybe this is better, > but it wont' be able to survive a daemon restart... > > Feedback appreciated. this sounds like a smaller portion of the problem we were discussing when we were talking about de-queuing multiple messages. this approach would work, but if you know you have a file you could also truncate the file to the end of the previous record (the beginning of the record you are trying to write). David Lang From william at blocket.se Tue Dec 16 17:39:53 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Tue, 16 Dec 2008 17:39:53 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs Message-ID: Hi, I don't know if this is an intended feature or not, but when CreateDirs is set to off, rsyslog can no longer create files (time stamp file names). Lets say you have a template for writing files: / log//.log, this will let anybody to create directories in / log when CreateDirs is on. This may cause a lot of directories and can be abused by regular users. I've patched 2.0.2 to separate file and directory creation here: https://bugzilla.redhat.com/attachment.cgi?id=327100 And steps to reproduce this can be found here: https://bugzilla.redhat.com/show_bug.cgi?id=473419 Comments anyone? William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From rgerhards at hq.adiscon.com Thu Dec 18 12:01:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 12:01:25 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: References: Message-ID: <1229598085.7471.2.camel@localhost.localdomain> Hi William, thanks for the bug report and the patch. I have now looked into the issue and could reproduce the situation. The patch obviously resolves it. There is one thing, though: I do not fully understand why you have patched syslogd.c (RS_RET_ERR). I would appreciate if you could provide a bit more insight into the problem you intend to solve. So far, I can not see why this part of the patch is actually needed. I'll integrate the omfile part now. I will also see that I apply this to the other official branches. Thanks, Rainer On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > Hi, > > I don't know if this is an intended feature or not, but when > CreateDirs is set to off, rsyslog can no longer create files (time > stamp file names). Lets say you have a template for writing files: / > log//.log, this will let anybody to create directories in / > log when CreateDirs is on. This may cause a lot of directories and can > be abused by regular users. > > I've patched 2.0.2 to separate file and directory creation here: > https://bugzilla.redhat.com/attachment.cgi?id=327100 > > And steps to reproduce this can be found here: > https://bugzilla.redhat.com/show_bug.cgi?id=473419 > > Comments anyone? > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Dec 18 13:03:48 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 18 Dec 2008 13:03:48 +0100 Subject: [rsyslog] rsyslog 4.1.3 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F8F9@grfint2.intern.adiscon.com> Hi all, rsyslog 4.1.3, a member of the development branch, has been released today. Most importantly, it offers a bugfix for some situations where the imudp input module could run in a tight loop. The release also offers enhanced functionality, among others a work-around to handle the invalid syslog/tcp framing used by NetScreen. There are two more configuration-options which can be used for fine-tuning functionality (see changelog for details). We would like to express my gratitude to BlinkMind, Inc. (http://www.blinkmind.com), who sponsored the development of the $PreserveFQDN configuration option. Download http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-145.phtml Changelog http://www.rsyslog.com/Article335.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From william at blocket.se Thu Dec 18 14:35:00 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 14:35:00 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229598085.7471.2.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> Message-ID: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Hi, Lets see if I can get this right: My modification in prepareFile() will return with (pData->fd = -1) if the log file can't be created. In prepareDynFile() we run prepareFile() and return with -1 if pData- >fd is set to -1. In writeFile() we run prepareDynFile() and return RS_RET_ERR if prepareDynFile() is not returned with 0. writeFile() is wrapped in doAction(). doAction() is exectued in fprintlog() where RS_RET_ERR never will be catched. I discard the log message and sets the error flag to tell the "msg repeated"-check to not log this message ("msg repeated" is executed before we try to open the file if the message content match the previous message). I tried without this catch in the first attempt, but I could see the message stuck in the loop, every action to rsyslog tried to open the file. This and some traffic volume caused rsyslog to hang (and use a lot of i/o). William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > Hi William, > > thanks for the bug report and the patch. I have now looked into the > issue and could reproduce the situation. The patch obviously resolves > it. > > There is one thing, though: I do not fully understand why you have > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > provide > a bit more insight into the problem you intend to solve. So far, I can > not see why this part of the patch is actually needed. > > I'll integrate the omfile part now. I will also see that I apply > this to > the other official branches. > > Thanks, > Rainer > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >> Hi, >> >> I don't know if this is an intended feature or not, but when >> CreateDirs is set to off, rsyslog can no longer create files (time >> stamp file names). Lets say you have a template for writing files: / >> log//.log, this will let anybody to create directories >> in / >> log when CreateDirs is on. This may cause a lot of directories and >> can >> be abused by regular users. >> >> I've patched 2.0.2 to separate file and directory creation here: >> https://bugzilla.redhat.com/attachment.cgi?id=327100 >> >> And steps to reproduce this can be found here: >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >> >> Comments anyone? >> >> >> William Tis?ter >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From william at blocket.se Thu Dec 18 15:22:53 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 15:22:53 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Message-ID: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> I've debugged some more, this only happens when you trigger the repeat- loop to the missing directory. E.g: # logger -t non_existent_dir -p local0.crit test # logger -t non_existent_dir -p local0.crit test # logger -t existing_dir -p local0.crit test Debug output from last command: Filter: check for property 'programname' (value 'existing_dir') NOT regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE 1092925760: Called fprintlog, logging to builtin-file (CritByDay) 1092925760: Called fprintlog, logging to builtin-file (CritByDay) Filter: check for property 'msg' (value ' test') contains '(CRIT)': FALSE 1092925760: Called fprintlog, logging to builtin-file (DirByTagFileByDay) 1092925760: Called logerr, msg: Could not open dynamic file '/log/ non_existent_dir/2008-12-18.log' - discarding message 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding message: No such file or directory 1092925760: Message has legacy syslog format. 1092925760: EnqueueMsg signaled condition (0) 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. 1092925760: Lone worker is running... 1092925760: msg repeated 1 times, 10 sec of 30. Filter: check for property 'syslogfacility-text' (value 'syslog') NOT isequal 'local0': TRUE 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: queue EMPTY, waiting for next message. William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > Hi, > > Lets see if I can get this right: > > My modification in prepareFile() will return with (pData->fd = -1) > if the log file can't be created. > In prepareDynFile() we run prepareFile() and return with -1 if pData- > >fd is set to -1. > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > prepareDynFile() is not returned with 0. > > writeFile() is wrapped in doAction(). > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > catched. I discard the log message and sets the error flag to tell > the "msg repeated"-check to not log this message ("msg repeated" is > executed before we try to open the file if the message content match > the previous message). > > I tried without this catch in the first attempt, but I could see the > message stuck in the loop, every action to rsyslog tried to open the > file. This and some traffic volume caused rsyslog to hang (and use a > lot of i/o). > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > >> Hi William, >> >> thanks for the bug report and the patch. I have now looked into the >> issue and could reproduce the situation. The patch obviously resolves >> it. >> >> There is one thing, though: I do not fully understand why you have >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >> provide >> a bit more insight into the problem you intend to solve. So far, I >> can >> not see why this part of the patch is actually needed. >> >> I'll integrate the omfile part now. I will also see that I apply >> this to >> the other official branches. >> >> Thanks, >> Rainer >> >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>> Hi, >>> >>> I don't know if this is an intended feature or not, but when >>> CreateDirs is set to off, rsyslog can no longer create files (time >>> stamp file names). Lets say you have a template for writing files: / >>> log//.log, this will let anybody to create directories >>> in / >>> log when CreateDirs is on. This may cause a lot of directories and >>> can >>> be abused by regular users. >>> >>> I've patched 2.0.2 to separate file and directory creation here: >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>> >>> And steps to reproduce this can be found here: >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>> >>> Comments anyone? >>> >>> >>> William Tis?ter >>> >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>> http://www.blocket.se/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 18 15:58:50 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 15:58:50 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Message-ID: <1229612330.12594.8.camel@localhost.localdomain> Hi William, I had another look at the code. I am not picky, I just want to make sure I understand the failure scenario. The v3 versions have a different approach at this, and I need to be sure I understand the issue so that I can check the newer releases for it. more inline below... On Thu, 2008-12-18 at 14:35 +0100, William Tis?ter wrote: > Hi, > > Lets see if I can get this right: > > My modification in prepareFile() will return with (pData->fd = -1) if > the log file can't be created. > In prepareDynFile() we run prepareFile() and return with -1 if pData- > >fd is set to -1. > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > prepareDynFile() is not returned with 0. > > writeFile() is wrapped in doAction(). > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > catched. Well.. kind of. I do not check for a specific error state, but I check if all went well (if iRet == RS_RE_OK). Only then the previous count is reset, otherwise it is left as is. An output module may return a large number of errors, so it is not sufficient to check for a specific error case. That's also why I am trying to understand the issue you still see. I am sure it can occur under other circumstances, too (as I said, there are various error codes). > I discard the log message and sets the error flag to tell the > "msg repeated"-check to not log this message ("msg repeated" is > executed before we try to open the file if the message content match > the previous message). ... but this is done via fprintlog(), so the logic to try open the file is still included. > > I tried without this catch in the first attempt, but I could see the > message stuck in the loop, As of my understanding, it should not be the very same message, because that is discarded. It can be the next message, which of course will trigger the "last message repeated n times" text. > every action to rsyslog tried to open the > file. This and some traffic volume caused rsyslog to hang (and use a > lot of i/o). Do you have a procedure to reproduce that? Would be most interesting. As a side-Note, I think there is a memory leak in the patch, you will probably want to correct it. The return in line 3392 does not do the necessary cleanup (the return further UP is a different case, see its comments). Use "ABORT_FINALIZE(RS_RET_DISCARDMSG);" instead, it will invoke the finalizer, which will free some string space. > Rainer > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > Hi William, > > > > thanks for the bug report and the patch. I have now looked into the > > issue and could reproduce the situation. The patch obviously > resolves > > it. > > > > There is one thing, though: I do not fully understand why you have > > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > > provide > > a bit more insight into the problem you intend to solve. So far, I > can > > not see why this part of the patch is actually needed. > > > > I'll integrate the omfile part now. I will also see that I apply > > this to > > the other official branches. > > > > Thanks, > > Rainer > > > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >> Hi, > >> > >> I don't know if this is an intended feature or not, but when > >> CreateDirs is set to off, rsyslog can no longer create files (time > >> stamp file names). Lets say you have a template for writing > files: / > >> log//.log, this will let anybody to create directories > >> in / > >> log when CreateDirs is on. This may cause a lot of directories and > >> can > >> be abused by regular users. > >> > >> I've patched 2.0.2 to separate file and directory creation here: > >> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >> > >> And steps to reproduce this can be found here: > >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >> > >> Comments anyone? > >> > >> > >> William Tis?ter > >> > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >> http://www.blocket.se/ > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:03:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:03:24 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229612330.12594.8.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <1229612330.12594.8.camel@localhost.localdomain> Message-ID: <1229612604.12594.11.camel@localhost.localdomain> oops, one thing that occured to me just now. RS_RET_DISCARDMSG is probably not what you want. This totally *discards* the message from the queue. This state was introduced for the discard action. For example, if your config ist *.* ?dynafile *.* file1 *.* file2 and you get in issue during ?dynafile processing, the message will never be written to file1 and file2. So it is better to leave it at RS_RET_ERR. This, however, brings us close to the code as it currently is (except for the additional flag, which I am trying to understand ;)) Rainer On Thu, 2008-12-18 at 15:58 +0100, Rainer Gerhards wrote: > Hi William, > > I had another look at the code. I am not picky, I just want to make > sure > I understand the failure scenario. The v3 versions have a different > approach at this, and I need to be sure I understand the issue so that > I > can check the newer releases for it. > > more inline below... > > On Thu, 2008-12-18 at 14:35 +0100, William Tis?ter wrote: > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > if > > the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > > catched. > > Well.. kind of. I do not check for a specific error state, but I check > if all went well (if iRet == RS_RE_OK). Only then the previous count > is > reset, otherwise it is left as is. An output module may return a large > number of errors, so it is not sufficient to check for a specific > error > case. That's also why I am trying to understand the issue you still > see. > I am sure it can occur under other circumstances, too (as I said, > there > are various error codes). > > > I discard the log message and sets the error flag to tell the > > "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content match > > the previous message). > > ... but this is done via fprintlog(), so the logic to try open the > file > is still included. > > > > I tried without this catch in the first attempt, but I could see the > > message stuck in the loop, > > As of my understanding, it should not be the very same message, > because > that is discarded. It can be the next message, which of course will > trigger the "last message repeated n times" text. > > > every action to rsyslog tried to open the > > file. This and some traffic volume caused rsyslog to hang (and use a > > lot of i/o). > > Do you have a procedure to reproduce that? Would be most interesting. > > As a side-Note, I think there is a memory leak in the patch, you will > probably want to correct it. The return in line 3392 does not do the > necessary cleanup (the return further UP is a different case, see its > comments). Use "ABORT_FINALIZE(RS_RET_DISCARDMSG);" instead, it will > invoke the finalizer, which will free some string space. > > > > Rainer > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > > > Hi William, > > > > > > thanks for the bug report and the patch. I have now looked into > the > > > issue and could reproduce the situation. The patch obviously > > resolves > > > it. > > > > > > There is one thing, though: I do not fully understand why you have > > > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > > > provide > > > a bit more insight into the problem you intend to solve. So far, I > > can > > > not see why this part of the patch is actually needed. > > > > > > I'll integrate the omfile part now. I will also see that I apply > > > this to > > > the other official branches. > > > > > > Thanks, > > > Rainer > > > > > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > > >> Hi, > > >> > > >> I don't know if this is an intended feature or not, but when > > >> CreateDirs is set to off, rsyslog can no longer create files > (time > > >> stamp file names). Lets say you have a template for writing > > files: / > > >> log//.log, this will let anybody to create directories > > >> in / > > >> log when CreateDirs is on. This may cause a lot of directories > and > > >> can > > >> be abused by regular users. > > >> > > >> I've patched 2.0.2 to separate file and directory creation here: > > >> https://bugzilla.redhat.com/attachment.cgi?id=327100 > > >> > > >> And steps to reproduce this can be found here: > > >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > > >> > > >> Comments anyone? > > >> > > >> > > >> William Tis?ter > > >> > > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > >> http://www.blocket.se/ > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:04:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:04:19 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> Message-ID: <1229612659.12594.12.camel@localhost.localdomain> The spam filter caught this message - sorry. I'll test this scenario. Rainer On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > I've debugged some more, this only happens when you trigger the > repeat- > loop to the missing directory. > > E.g: > > # logger -t non_existent_dir -p local0.crit test > # logger -t non_existent_dir -p local0.crit test > # logger -t existing_dir -p local0.crit test > > Debug output from last command: > > Filter: check for property 'programname' (value 'existing_dir') NOT > regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > Filter: check for property 'msg' (value ' test') contains '(CRIT)': > FALSE > 1092925760: Called fprintlog, logging to builtin-file > (DirByTagFileByDay) > 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > non_existent_dir/2008-12-18.log' - discarding message > 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding > message: No such file or directory > 1092925760: Message has legacy syslog format. > 1092925760: EnqueueMsg signaled condition (0) > 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. > 1092925760: Lone worker is running... > 1092925760: msg repeated 1 times, 10 sec of 30. > Filter: check for property 'syslogfacility-text' (value 'syslog') NOT > isequal 'local0': TRUE > 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: > queue EMPTY, waiting for next message. > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > > if the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will > be > > catched. I discard the log message and sets the error flag to tell > > the "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content > match > > the previous message). > > > > I tried without this catch in the first attempt, but I could see > the > > message stuck in the loop, every action to rsyslog tried to open > the > > file. This and some traffic volume caused rsyslog to hang (and use > a > > lot of i/o). > > > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > >> Hi William, > >> > >> thanks for the bug report and the patch. I have now looked into the > >> issue and could reproduce the situation. The patch obviously > resolves > >> it. > >> > >> There is one thing, though: I do not fully understand why you have > >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >> provide > >> a bit more insight into the problem you intend to solve. So far, I > >> can > >> not see why this part of the patch is actually needed. > >> > >> I'll integrate the omfile part now. I will also see that I apply > >> this to > >> the other official branches. > >> > >> Thanks, > >> Rainer > >> > >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>> Hi, > >>> > >>> I don't know if this is an intended feature or not, but when > >>> CreateDirs is set to off, rsyslog can no longer create files (time > >>> stamp file names). Lets say you have a template for writing > files: / > >>> log//.log, this will let anybody to create directories > >>> in / > >>> log when CreateDirs is on. This may cause a lot of directories > and > >>> can > >>> be abused by regular users. > >>> > >>> I've patched 2.0.2 to separate file and directory creation here: > >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>> > >>> And steps to reproduce this can be found here: > >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>> > >>> Comments anyone? > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:12:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:12:34 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> Message-ID: <1229613154.12594.13.camel@localhost.localdomain> mmhhhh... I can not reproduce the issue with my state of the code. Can you verify that it still occurs? A git snapshot is available here: http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz Thanks, Rainer On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > I've debugged some more, this only happens when you trigger the > repeat- > loop to the missing directory. > > E.g: > > # logger -t non_existent_dir -p local0.crit test > # logger -t non_existent_dir -p local0.crit test > # logger -t existing_dir -p local0.crit test > > Debug output from last command: > > Filter: check for property 'programname' (value 'existing_dir') NOT > regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > Filter: check for property 'msg' (value ' test') contains '(CRIT)': > FALSE > 1092925760: Called fprintlog, logging to builtin-file > (DirByTagFileByDay) > 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > non_existent_dir/2008-12-18.log' - discarding message > 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding > message: No such file or directory > 1092925760: Message has legacy syslog format. > 1092925760: EnqueueMsg signaled condition (0) > 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. > 1092925760: Lone worker is running... > 1092925760: msg repeated 1 times, 10 sec of 30. > Filter: check for property 'syslogfacility-text' (value 'syslog') NOT > isequal 'local0': TRUE > 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: > queue EMPTY, waiting for next message. > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > > if the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will > be > > catched. I discard the log message and sets the error flag to tell > > the "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content > match > > the previous message). > > > > I tried without this catch in the first attempt, but I could see > the > > message stuck in the loop, every action to rsyslog tried to open > the > > file. This and some traffic volume caused rsyslog to hang (and use > a > > lot of i/o). > > > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > >> Hi William, > >> > >> thanks for the bug report and the patch. I have now looked into the > >> issue and could reproduce the situation. The patch obviously > resolves > >> it. > >> > >> There is one thing, though: I do not fully understand why you have > >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >> provide > >> a bit more insight into the problem you intend to solve. So far, I > >> can > >> not see why this part of the patch is actually needed. > >> > >> I'll integrate the omfile part now. I will also see that I apply > >> this to > >> the other official branches. > >> > >> Thanks, > >> Rainer > >> > >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>> Hi, > >>> > >>> I don't know if this is an intended feature or not, but when > >>> CreateDirs is set to off, rsyslog can no longer create files (time > >>> stamp file names). Lets say you have a template for writing > files: / > >>> log//.log, this will let anybody to create directories > >>> in / > >>> log when CreateDirs is on. This may cause a lot of directories > and > >>> can > >>> be abused by regular users. > >>> > >>> I've patched 2.0.2 to separate file and directory creation here: > >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>> > >>> And steps to reproduce this can be found here: > >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>> > >>> Comments anyone? > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From william at blocket.se Thu Dec 18 16:35:08 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 16:35:08 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229613154.12594.13.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> Message-ID: <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> Yes, I could reproduce my error, the same way as described earlier. My /etc/rsyslog.conf: $template DirByTagFileByDay,"/log/%programname%/%timegenerated: 1:10:date-rfc3339%.log" $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" # Discard all but local0 :syslogfacility-text, !isequal, "local0" ~ $umask 0000 $FileCreateMode 0644 $DirCreateMode 0755 $CreateDirs off # Discard strange tags :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ badtags.log :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ # Collect all crits in one log local0.crit ?CritByDay :msg, contains, "(CRIT)" ?CritByDay # All local0's buffered to their own dirs local0.* -?DirByTagFileByDay / William On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: > mmhhhh... I can not reproduce the issue with my state of the code. Can > you verify that it still occurs? A git snapshot is available here: > > http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz > > Thanks, > Rainer > > On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: >> I've debugged some more, this only happens when you trigger the >> repeat- >> loop to the missing directory. >> >> E.g: >> >> # logger -t non_existent_dir -p local0.crit test >> # logger -t non_existent_dir -p local0.crit test >> # logger -t existing_dir -p local0.crit test >> >> Debug output from last command: >> >> Filter: check for property 'programname' (value 'existing_dir') NOT >> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >> Filter: check for property 'msg' (value ' test') contains '(CRIT)': >> FALSE >> 1092925760: Called fprintlog, logging to builtin-file >> (DirByTagFileByDay) >> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ >> non_existent_dir/2008-12-18.log' - discarding message >> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not >> open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding >> message: No such file or directory >> 1092925760: Message has legacy syslog format. >> 1092925760: EnqueueMsg signaled condition (0) >> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. >> 1092925760: Lone worker is running... >> 1092925760: msg repeated 1 times, 10 sec of 30. >> Filter: check for property 'syslogfacility-text' (value 'syslog') NOT >> isequal 'local0': TRUE >> 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: >> queue EMPTY, waiting for next message. >> >> >> William Tis?ter >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> >> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: >> >>> Hi, >>> >>> Lets see if I can get this right: >>> >>> My modification in prepareFile() will return with (pData->fd = -1) >>> if the log file can't be created. >>> In prepareDynFile() we run prepareFile() and return with -1 if >> pData- >>>> fd is set to -1. >>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if >>> prepareDynFile() is not returned with 0. >>> >>> writeFile() is wrapped in doAction(). >>> >>> doAction() is exectued in fprintlog() where RS_RET_ERR never will >> be >>> catched. I discard the log message and sets the error flag to tell >>> the "msg repeated"-check to not log this message ("msg repeated" is >>> executed before we try to open the file if the message content >> match >>> the previous message). >>> >>> I tried without this catch in the first attempt, but I could see >> the >>> message stuck in the loop, every action to rsyslog tried to open >> the >>> file. This and some traffic volume caused rsyslog to hang (and use >> a >>> lot of i/o). >>> >>> >>> William Tis?ter >>> >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>> http://www.blocket.se/ >>> >>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: >>> >>>> Hi William, >>>> >>>> thanks for the bug report and the patch. I have now looked into the >>>> issue and could reproduce the situation. The patch obviously >> resolves >>>> it. >>>> >>>> There is one thing, though: I do not fully understand why you have >>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >>>> provide >>>> a bit more insight into the problem you intend to solve. So far, I >>>> can >>>> not see why this part of the patch is actually needed. >>>> >>>> I'll integrate the omfile part now. I will also see that I apply >>>> this to >>>> the other official branches. >>>> >>>> Thanks, >>>> Rainer >>>> >>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>>>> Hi, >>>>> >>>>> I don't know if this is an intended feature or not, but when >>>>> CreateDirs is set to off, rsyslog can no longer create files (time >>>>> stamp file names). Lets say you have a template for writing >> files: / >>>>> log//.log, this will let anybody to create directories >>>>> in / >>>>> log when CreateDirs is on. This may cause a lot of directories >> and >>>>> can >>>>> be abused by regular users. >>>>> >>>>> I've patched 2.0.2 to separate file and directory creation here: >>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>>>> >>>>> And steps to reproduce this can be found here: >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>>>> >>>>> Comments anyone? >>>>> >>>>> >>>>> William Tis?ter >>>>> >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>> http://www.blocket.se/ >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >> >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com William Tis?ter Cell: +46 (0)76 3377182 Mail: william at blocket.se Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From rgerhards at hq.adiscon.com Thu Dec 18 16:44:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:44:04 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> Message-ID: <1229615044.12594.14.camel@localhost.localdomain> would it be possible that you mail me (privately) a full debug log from such a run. Not sure if there is too much confidential data in it... Rainer On Thu, 2008-12-18 at 16:35 +0100, William Tis?ter wrote: > Yes, I could reproduce my error, the same way as described earlier. > > My /etc/rsyslog.conf: > > $template DirByTagFileByDay,"/log/%programname%/%timegenerated: > 1:10:date-rfc3339%.log" > $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" > > # Discard all but local0 > :syslogfacility-text, !isequal, "local0" ~ > > $umask 0000 > $FileCreateMode 0644 > $DirCreateMode 0755 > $CreateDirs off > > # Discard strange tags > :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ > badtags.log > :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ > > # Collect all crits in one log > local0.crit ?CritByDay > :msg, contains, "(CRIT)" ?CritByDay > > # All local0's buffered to their own dirs > local0.* -?DirByTagFileByDay > > > / William > > On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: > > > mmhhhh... I can not reproduce the issue with my state of the code. > Can > > you verify that it still occurs? A git snapshot is available here: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz > > > > Thanks, > > Rainer > > > > On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > >> I've debugged some more, this only happens when you trigger the > >> repeat- > >> loop to the missing directory. > >> > >> E.g: > >> > >> # logger -t non_existent_dir -p local0.crit test > >> # logger -t non_existent_dir -p local0.crit test > >> # logger -t existing_dir -p local0.crit test > >> > >> Debug output from last command: > >> > >> Filter: check for property 'programname' (value 'existing_dir') NOT > >> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > >> Filter: check for property 'msg' (value ' test') contains '(CRIT)': > >> FALSE > >> 1092925760: Called fprintlog, logging to builtin-file > >> (DirByTagFileByDay) > >> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > >> non_existent_dir/2008-12-18.log' - discarding message > >> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > >> open dynamic file '/log/non_existent_dir/2008-12-18.log' - > discarding > >> message: No such file or directory > >> 1092925760: Message has legacy syslog format. > >> 1092925760: EnqueueMsg signaled condition (0) > >> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from > dynaCache. > >> 1092925760: Lone worker is running... > >> 1092925760: msg repeated 1 times, 10 sec of 30. > >> Filter: check for property 'syslogfacility-text' (value 'syslog') > NOT > >> isequal 'local0': TRUE > >> 1092925760: Called fprintlog, logging to > builtin-discardsingleWorker: > >> queue EMPTY, waiting for next message. > >> > >> > >> William Tis?ter > >> > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >> http://www.blocket.se/ > >> > >> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > >> > >>> Hi, > >>> > >>> Lets see if I can get this right: > >>> > >>> My modification in prepareFile() will return with (pData->fd = -1) > >>> if the log file can't be created. > >>> In prepareDynFile() we run prepareFile() and return with -1 if > >> pData- > >>>> fd is set to -1. > >>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if > >>> prepareDynFile() is not returned with 0. > >>> > >>> writeFile() is wrapped in doAction(). > >>> > >>> doAction() is exectued in fprintlog() where RS_RET_ERR never will > >> be > >>> catched. I discard the log message and sets the error flag to tell > >>> the "msg repeated"-check to not log this message ("msg repeated" > is > >>> executed before we try to open the file if the message content > >> match > >>> the previous message). > >>> > >>> I tried without this catch in the first attempt, but I could see > >> the > >>> message stuck in the loop, every action to rsyslog tried to open > >> the > >>> file. This and some traffic volume caused rsyslog to hang (and use > >> a > >>> lot of i/o). > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> > >>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > >>> > >>>> Hi William, > >>>> > >>>> thanks for the bug report and the patch. I have now looked into > the > >>>> issue and could reproduce the situation. The patch obviously > >> resolves > >>>> it. > >>>> > >>>> There is one thing, though: I do not fully understand why you > have > >>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >>>> provide > >>>> a bit more insight into the problem you intend to solve. So far, > I > >>>> can > >>>> not see why this part of the patch is actually needed. > >>>> > >>>> I'll integrate the omfile part now. I will also see that I apply > >>>> this to > >>>> the other official branches. > >>>> > >>>> Thanks, > >>>> Rainer > >>>> > >>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>>>> Hi, > >>>>> > >>>>> I don't know if this is an intended feature or not, but when > >>>>> CreateDirs is set to off, rsyslog can no longer create files > (time > >>>>> stamp file names). Lets say you have a template for writing > >> files: / > >>>>> log//.log, this will let anybody to create > directories > >>>>> in / > >>>>> log when CreateDirs is on. This may cause a lot of directories > >> and > >>>>> can > >>>>> be abused by regular users. > >>>>> > >>>>> I've patched 2.0.2 to separate file and directory creation here: > >>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>>>> > >>>>> And steps to reproduce this can be found here: > >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>>>> > >>>>> Comments anyone? > >>>>> > >>>>> > >>>>> William Tis?ter > >>>>> > >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>>>> http://www.blocket.se/ > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >> > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > William Tis?ter > > Cell: +46 (0)76 3377182 > Mail: william at blocket.se > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From william at blocket.se Thu Dec 18 16:47:28 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 16:47:28 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229615044.12594.14.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> <1229615044.12594.14.camel@localhost.localdomain> Message-ID: * Sent full debug log to Rainer * On Dec 18, 2008, at 4:44 PM, Rainer Gerhards wrote: > would it be possible that you mail me (privately) a full debug log > from > such a run. Not sure if there is too much confidential data in it... > > Rainer > > On Thu, 2008-12-18 at 16:35 +0100, William Tis?ter wrote: >> Yes, I could reproduce my error, the same way as described earlier. >> >> My /etc/rsyslog.conf: >> >> $template DirByTagFileByDay,"/log/%programname%/%timegenerated: >> 1:10:date-rfc3339%.log" >> $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" >> >> # Discard all but local0 >> :syslogfacility-text, !isequal, "local0" ~ >> >> $umask 0000 >> $FileCreateMode 0644 >> $DirCreateMode 0755 >> $CreateDirs off >> >> # Discard strange tags >> :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ >> badtags.log >> :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ >> >> # Collect all crits in one log >> local0.crit ?CritByDay >> :msg, contains, "(CRIT)" ?CritByDay >> >> # All local0's buffered to their own dirs >> local0.* -?DirByTagFileByDay >> >> >> / William >> >> On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: >> >>> mmhhhh... I can not reproduce the issue with my state of the code. >> Can >>> you verify that it still occurs? A git snapshot is available here: >>> >>> >> http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz >>> >>> Thanks, >>> Rainer >>> >>> On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: >>>> I've debugged some more, this only happens when you trigger the >>>> repeat- >>>> loop to the missing directory. >>>> >>>> E.g: >>>> >>>> # logger -t non_existent_dir -p local0.crit test >>>> # logger -t non_existent_dir -p local0.crit test >>>> # logger -t existing_dir -p local0.crit test >>>> >>>> Debug output from last command: >>>> >>>> Filter: check for property 'programname' (value 'existing_dir') NOT >>>> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE >>>> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >>>> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >>>> Filter: check for property 'msg' (value ' test') contains '(CRIT)': >>>> FALSE >>>> 1092925760: Called fprintlog, logging to builtin-file >>>> (DirByTagFileByDay) >>>> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ >>>> non_existent_dir/2008-12-18.log' - discarding message >>>> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not >>>> open dynamic file '/log/non_existent_dir/2008-12-18.log' - >> discarding >>>> message: No such file or directory >>>> 1092925760: Message has legacy syslog format. >>>> 1092925760: EnqueueMsg signaled condition (0) >>>> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from >> dynaCache. >>>> 1092925760: Lone worker is running... >>>> 1092925760: msg repeated 1 times, 10 sec of 30. >>>> Filter: check for property 'syslogfacility-text' (value 'syslog') >> NOT >>>> isequal 'local0': TRUE >>>> 1092925760: Called fprintlog, logging to >> builtin-discardsingleWorker: >>>> queue EMPTY, waiting for next message. >>>> >>>> >>>> William Tis?ter >>>> >>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>> http://www.blocket.se/ >>>> >>>> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: >>>> >>>>> Hi, >>>>> >>>>> Lets see if I can get this right: >>>>> >>>>> My modification in prepareFile() will return with (pData->fd = -1) >>>>> if the log file can't be created. >>>>> In prepareDynFile() we run prepareFile() and return with -1 if >>>> pData- >>>>>> fd is set to -1. >>>>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if >>>>> prepareDynFile() is not returned with 0. >>>>> >>>>> writeFile() is wrapped in doAction(). >>>>> >>>>> doAction() is exectued in fprintlog() where RS_RET_ERR never will >>>> be >>>>> catched. I discard the log message and sets the error flag to tell >>>>> the "msg repeated"-check to not log this message ("msg repeated" >> is >>>>> executed before we try to open the file if the message content >>>> match >>>>> the previous message). >>>>> >>>>> I tried without this catch in the first attempt, but I could see >>>> the >>>>> message stuck in the loop, every action to rsyslog tried to open >>>> the >>>>> file. This and some traffic volume caused rsyslog to hang (and use >>>> a >>>>> lot of i/o). >>>>> >>>>> >>>>> William Tis?ter >>>>> >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>> http://www.blocket.se/ >>>>> >>>>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: >>>>> >>>>>> Hi William, >>>>>> >>>>>> thanks for the bug report and the patch. I have now looked into >> the >>>>>> issue and could reproduce the situation. The patch obviously >>>> resolves >>>>>> it. >>>>>> >>>>>> There is one thing, though: I do not fully understand why you >> have >>>>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >>>>>> provide >>>>>> a bit more insight into the problem you intend to solve. So far, >> I >>>>>> can >>>>>> not see why this part of the patch is actually needed. >>>>>> >>>>>> I'll integrate the omfile part now. I will also see that I apply >>>>>> this to >>>>>> the other official branches. >>>>>> >>>>>> Thanks, >>>>>> Rainer >>>>>> >>>>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I don't know if this is an intended feature or not, but when >>>>>>> CreateDirs is set to off, rsyslog can no longer create files >> (time >>>>>>> stamp file names). Lets say you have a template for writing >>>> files: / >>>>>>> log//.log, this will let anybody to create >> directories >>>>>>> in / >>>>>>> log when CreateDirs is on. This may cause a lot of directories >>>> and >>>>>>> can >>>>>>> be abused by regular users. >>>>>>> >>>>>>> I've patched 2.0.2 to separate file and directory creation here: >>>>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>>>>>> >>>>>>> And steps to reproduce this can be found here: >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>>>>>> >>>>>>> Comments anyone? >>>>>>> >>>>>>> >>>>>>> William Tis?ter >>>>>>> >>>>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>>>> http://www.blocket.se/ >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> >> William Tis?ter >> >> Cell: +46 (0)76 3377182 >> Mail: william at blocket.se >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com William Tis?ter Cell: +46 (0)76 3377182 Mail: william at blocket.se Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From bakers at web-ster.com Thu Dec 18 20:59:27 2008 From: bakers at web-ster.com (Scott Baker) Date: Thu, 18 Dec 2008 11:59:27 -0800 Subject: [rsyslog] Troubleshooting missing log entries Message-ID: <494AAB9F.9060005@web-ster.com> I have the following entry in my rsyslog conf, to match entries based on IP address. Somehow it's not matching any entries. # Switches $FileCreateMode 0644 :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea :FROMHOST, isequal, "65.182.224.202" -?switches :FROMHOST, isequal, "66.206.80.60" -?switches If I do a tcpdump I see syslog hitting the box, it's just rsyslog isn't handling it right. 11:58:20.722867 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 121 11:58:23.962613 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 130 11:58:41.242621 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 108 11:58:45.874064 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 130 This box gets about 500 lines of syslog a minute so I can't really turn on debug. How else can I troubleshoot this? This is a Fedora 8 box running: rsyslog-2.0.2-3.fc8 - Scott From david at lang.hm Fri Dec 19 06:38:38 2008 From: david at lang.hm (david at lang.hm) Date: Thu, 18 Dec 2008 21:38:38 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog Message-ID: with messages appearing out-of-order the 'last message repeated X times' is pretty useless. I think I remember reading about an option to disable this, but another work-around to the problem is to change the output so that it becomes 'last message repeated X times %msg%', so you can see (most, if not all) of the message being repeated in the line telling you that it was repeated. David Lang From rgerhards at hq.adiscon.com Thu Dec 18 20:01:47 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:01:47 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: Message-ID: <1229626907.12594.19.camel@localhost.localdomain> Hi David, On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > with messages appearing out-of-order the 'last message repeated X times' > is pretty useless. Not really. Please note that form the perspective of the output module, messages do not appear out of order. The output module receives a stream of messages, and the "last message repeated x times" logic works on that stream. So no matter if messages are re-ordered by async queues (or UDP or whatever), the "last ... times" correctly reflects the way things were handed to the output module. But I concur that this feature, in its current state, is very questionable. I've talked about this on the mailing list quite some time, I think there is also at least one blog post about it. > > I think I remember reading about an option to disable this, but another > work-around to the problem is to change the output so that it becomes > 'last message repeated X times %msg%', so you can see (most, if not all) > of the message being repeated in the line telling you that it was > repeated. As I said, the message in front of this message is either another repeat message or the message that is being repeated. So you can always trace back to what was repeated (but it is painful). Given the feedback I received on this feature (use cases), it should probably (and v4 would be an appropriate version to do so) be redisigned to be a rate-limiting feature on the input side. That would pretty much simplify code, too. Rainer From david at lang.hm Fri Dec 19 12:46:10 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 03:46:10 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: <1229626907.12594.19.camel@localhost.localdomain> References: <1229626907.12594.19.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > Hi David, > > On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: >> with messages appearing out-of-order the 'last message repeated X times' >> is pretty useless. > > Not really. Please note that form the perspective of the output module, > messages do not appear out of order. The output module receives a stream > of messages, and the "last message repeated x times" logic works on that > stream. So no matter if messages are re-ordered by async queues (or UDP > or whatever), the "last ... times" correctly reflects the way things > were handed to the output module. but if the output module is then relaying to another system, that other system (or the transport between them) can also re-order the messages > But I concur that this feature, in its current state, is very > questionable. I've talked about this on the mailing list quite some > time, I think there is also at least one blog post about it. > >> >> I think I remember reading about an option to disable this, but another >> work-around to the problem is to change the output so that it becomes >> 'last message repeated X times %msg%', so you can see (most, if not all) >> of the message being repeated in the line telling you that it was >> repeated. > > As I said, the message in front of this message is either another repeat > message or the message that is being repeated. So you can always trace > back to what was repeated (but it is painful). > > Given the feedback I received on this feature (use cases), it should > probably (and v4 would be an appropriate version to do so) be redisigned > to be a rate-limiting feature on the input side. That would pretty much > simplify code, too. but since things can be re-ordered after the input module is done with them (multiple threads pulling from the queue, or relaying) it is still useful for the 'message repeated' message to have more info about what it is referring to. David Lang > Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Thu Dec 18 20:15:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:15:51 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain> Message-ID: <1229627751.12594.23.camel@localhost.localdomain> On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: > On Thu, 18 Dec 2008, Rainer Gerhards wrote: > > > Hi David, > > > > On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > >> with messages appearing out-of-order the 'last message repeated X times' > >> is pretty useless. > > > > Not really. Please note that form the perspective of the output module, > > messages do not appear out of order. The output module receives a stream > > of messages, and the "last message repeated x times" logic works on that > > stream. So no matter if messages are re-ordered by async queues (or UDP > > or whatever), the "last ... times" correctly reflects the way things > > were handed to the output module. > > but if the output module is then relaying to another system, that other > system (or the transport between them) can also re-order the messages ah, ok, remote scenario. Of course, that can happen. But that's "just" one use case where it is problematic. > > But I concur that this feature, in its current state, is very > > questionable. I've talked about this on the mailing list quite some > > time, I think there is also at least one blog post about it. > > > >> > >> I think I remember reading about an option to disable this, but another > >> work-around to the problem is to change the output so that it becomes > >> 'last message repeated X times %msg%', so you can see (most, if not all) > >> of the message being repeated in the line telling you that it was > >> repeated. > > > > As I said, the message in front of this message is either another repeat > > message or the message that is being repeated. So you can always trace > > back to what was repeated (but it is painful). > > > > Given the feedback I received on this feature (use cases), it should > > probably (and v4 would be an appropriate version to do so) be redisigned > > to be a rate-limiting feature on the input side. That would pretty much > > simplify code, too. > > but since things can be re-ordered after the input module is done with > them (multiple threads pulling from the queue, or relaying) it is still > useful for the 'message repeated' message to have more info about what it > is referring to. That makes sense, at least for scenarios where it is expected behaviour. For others, it would probably break analysers. So it needs to be optional. And probably on a per-action basis. I'll see if it is sufficiently simple, but the whole "last message repeated" thing is broken anyhow (IMO), I don't like the idea to invest any more than maybe an hour into that feature, especially in the light that the whole idea must be replaced in the not so distant future... Rainer > > David Lang > > > Rainer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 18 20:19:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:19:54 +0100 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <494AAB9F.9060005@web-ster.com> References: <494AAB9F.9060005@web-ster.com> Message-ID: <1229627994.12594.26.camel@localhost.localdomain> On Thu, 2008-12-18 at 11:59 -0800, Scott Baker wrote: > I have the following entry in my rsyslog conf, to match entries based on IP > address. Somehow it's not matching any entries. > > # Switches > $FileCreateMode 0644 > :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea > :FROMHOST, isequal, "65.182.224.202" -?switches > :FROMHOST, isequal, "66.206.80.60" -?switches > > If I do a tcpdump I see syslog hitting the box, it's just rsyslog isn't > handling it right. > > 11:58:20.722867 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 121 > 11:58:23.962613 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 130 > 11:58:41.242621 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 108 > 11:58:45.874064 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 130 > > This box gets about 500 lines of syslog a minute so I can't really turn on > debug. How else can I troubleshoot this? This is a Fedora 8 box running: > rsyslog-2.0.2-3.fc8 I'd still go for debug mode. You don't need to run it very long. We just need to see how a few of these messages are fully processed. A proper test setup would be to start up in debug mode with the network cable pulled, then plug it in for a second or two, then unplug it again. Once rsyslogd is finished processing, stop it. That should lead to useful info in the debug log. Oh - and are you sure that fromhost has the proper IP addresses? If not 100% sure, verify it by putting something like '%FROMHOST%' into a debug template (note that there is also FROMHOST-IP, which will have the IP address no matter if names are resolved or not). HTH, Rainer > > - Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Fri Dec 19 13:03:40 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 04:03:40 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: <1229627751.12594.23.camel@localhost.localdomain> References: <1229626907.12594.19.camel@localhost.localdomain> <1229627751.12594.23.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: >> >>> Hi David, >>> >>> On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: >>>> with messages appearing out-of-order the 'last message repeated X times' >>>> is pretty useless. >>> >>> Not really. Please note that form the perspective of the output module, >>> messages do not appear out of order. The output module receives a stream >>> of messages, and the "last message repeated x times" logic works on that >>> stream. So no matter if messages are re-ordered by async queues (or UDP >>> or whatever), the "last ... times" correctly reflects the way things >>> were handed to the output module. >> >> but if the output module is then relaying to another system, that other >> system (or the transport between them) can also re-order the messages > > ah, ok, remote scenario. Of course, that can happen. But that's "just" > one use case where it is problematic. > >>> But I concur that this feature, in its current state, is very >>> questionable. I've talked about this on the mailing list quite some >>> time, I think there is also at least one blog post about it. >>> >>>> >>>> I think I remember reading about an option to disable this, but another >>>> work-around to the problem is to change the output so that it becomes >>>> 'last message repeated X times %msg%', so you can see (most, if not all) >>>> of the message being repeated in the line telling you that it was >>>> repeated. >>> >>> As I said, the message in front of this message is either another repeat >>> message or the message that is being repeated. So you can always trace >>> back to what was repeated (but it is painful). >>> >>> Given the feedback I received on this feature (use cases), it should >>> probably (and v4 would be an appropriate version to do so) be redisigned >>> to be a rate-limiting feature on the input side. That would pretty much >>> simplify code, too. >> >> but since things can be re-ordered after the input module is done with >> them (multiple threads pulling from the queue, or relaying) it is still >> useful for the 'message repeated' message to have more info about what it >> is referring to. > > That makes sense, at least for scenarios where it is expected behaviour. > For others, it would probably break analysers. So it needs to be > optional. And probably on a per-action basis. I'll see if it is > sufficiently simple, but the whole "last message repeated" thing is > broken anyhow (IMO), I don't like the idea to invest any more than maybe > an hour into that feature, especially in the light that the whole idea > must be replaced in the not so distant future... it's definantly a pain for analysers (a _lot_ of them really only look at one line at a time), that's why I started tweaking this on sysklogd, and while in theory just disabling it is the right answer, the ability to take a flood of inbound messages and reduce them to a much smaller number can save your logging system when things go wrong I'll watch to see what happens. David Lang From rgerhards at hq.adiscon.com Fri Dec 19 12:05:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:05:24 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain><1229627751.12594.23.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F915@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, December 19, 2008 1:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] suggested tweak to rsyslog > it's definantly a pain for analysers Full ack > (a _lot_ of them really only look > at > one line at a time), that's why I started tweaking this on sysklogd, > and > while in theory just disabling it is the right answer, the ability to > take > a flood of inbound messages and reduce them to a much smaller number > can > save your logging system when things go wrong > > I'll watch to see what happens. I just dug out a the relevant link, please follow also to the mailing list discussion: http://kb.monitorware.com/post5789.html Rainer From david at lang.hm Fri Dec 19 13:07:53 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 04:07:53 -0800 (PST) Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <1229627994.12594.26.camel@localhost.localdomain> References: <494AAB9F.9060005@web-ster.com> <1229627994.12594.26.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Thu, 2008-12-18 at 11:59 -0800, Scott Baker wrote: >> I have the following entry in my rsyslog conf, to match entries based on IP >> address. Somehow it's not matching any entries. >> >> # Switches >> $FileCreateMode 0644 >> :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea >> :FROMHOST, isequal, "65.182.224.202" -?switches >> :FROMHOST, isequal, "66.206.80.60" -?switches > > Oh - and are you sure that fromhost has the proper IP addresses? If not > 100% sure, verify it by putting something like '%FROMHOST%' into a debug > template (note that there is also FROMHOST-IP, which will have the IP > address no matter if names are resolved or not). I was seeing some issues where the fromhost was not getting set properly, I'll have to go back and dig up the details, but I think I was seeing it use the localhost as the fromhost and putting the real fromhost information in the message. I found it by creating an output format that I could tweak and playing with it to see what was actually showing up in the various parameters. David Lang From rgerhards at hq.adiscon.com Fri Dec 19 12:24:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:24:22 +0100 Subject: [rsyslog] Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? In-Reply-To: References: Message-ID: <1229685862.12594.33.camel@localhost.localdomain> Hi Craig, I don't see this in my lab, everything works correct. May it be that you did not include some of the (automatic) initial messages in your picture? In any case, a full debug log would be useful to track this down. After my sig, you find the relevant part of my lab debug FYI. Relevant part of rsyslog.conf: $ActionExecOnlyEveryNthTime 3 :msg, contains, "test" -/rsyslog/logfile Commands issued: $ logger 1 $ logger 2 $ logger 3 $ logger 4 $ logger 5 $ logger 6 And the output file had: 2008-12-19T12:17:44.548207+01:00 rf10up rger: test 3 2008-12-19T12:17:53.522265+01:00 rf10up rger: test 6 Note that in this setup I made sure that no other messages than those with the string "test" went into the log file. Otherwise, the counters would have been affected by some startup messages. Rainer 5455.946426832:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5460.051992823:imuxsock.c: Message from UNIX socket: #4 5460.066209819:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:40 rger: test 1 5460.066408445:imuxsock.c: Message has legacy syslog format. 5460.095581070:imuxsock.c: main queue: entry added, size now 1 entries 5460.096265506:imuxsock.c: wtpAdviseMaxWorkers signals busy 5460.096810820:imuxsock.c: main queue: EnqueueMsg advised worker start 5460.097417035:main queue:Reg/w0: main queue: entering rate limiter 5460.097797246:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5460.098493974:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 1') contains 'test': TRUE 5460.105295873:main queue:Reg/w0: action 0x4c64438 passed 1 times to execution - less than neded - discarding 5460.109592735:main queue:Reg/w0: main queue: entering rate limiter 5460.110192245:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5460.115576662:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5462.317878786:imuxsock.c: Message from UNIX socket: #4 5462.318410132:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:42 rger: test 2 5462.318585850:imuxsock.c: Message has legacy syslog format. 5462.319012436:imuxsock.c: main queue: entry added, size now 1 entries 5462.319271125:imuxsock.c: wtpAdviseMaxWorkers signals busy 5462.319537636:imuxsock.c: main queue: EnqueueMsg advised worker start 5462.319871473:main queue:Reg/w0: main queue: entering rate limiter 5462.320174580:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5462.320470145:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 2') contains 'test': TRUE 5462.321508812:main queue:Reg/w0: action 0x4c64438 passed 2 times to execution - less than neded - discarding 5462.321825608:main queue:Reg/w0: main queue: entering rate limiter 5462.322078710:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5462.322350529:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5464.547937451:imuxsock.c: Message from UNIX socket: #4 5464.548420747:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:44 rger: test 3 5464.548613227:imuxsock.c: Message has legacy syslog format. 5464.549042885:imuxsock.c: main queue: entry added, size now 1 entries 5464.549287885:imuxsock.c: wtpAdviseMaxWorkers signals busy 5464.549571158:imuxsock.c: main queue: EnqueueMsg advised worker start 5464.549888793:main queue:Reg/w0: main queue: entering rate limiter 5464.550211176:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5464.550500036:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 3') contains 'test': TRUE 5464.551797950:main queue:Reg/w0: Called action, logging to builtin-file 5464.605776776:main queue:Reg/w0: (/rsyslog/logfile) 5464.606371817:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5464.627358305:main queue:Reg/w0: main queue: entering rate limiter 5464.628009218:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5469.671232071:imuxsock.c: Message from UNIX socket: #4 5469.671772915:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:49 rger: test 4 5469.671950031:imuxsock.c: Message has legacy syslog format. 5469.672340578:imuxsock.c: main queue: entry added, size now 1 entries 5469.672597591:imuxsock.c: wtpAdviseMaxWorkers signals busy 5469.672931428:imuxsock.c: main queue: EnqueueMsg advised worker start 5469.673234256:main queue:Reg/w0: main queue: entering rate limiter 5469.673563065:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5469.673893829:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 4') contains 'test': TRUE 5469.674471829:main queue:Reg/w0: action 0x4c64438 passed 1 times to execution - less than neded - discarding 5469.674788066:main queue:Reg/w0: main queue: entering rate limiter 5469.675042006:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5469.675316618:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5471.443424737:imuxsock.c: Message from UNIX socket: #4 5471.443973403:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:51 rger: test 5 5471.444148283:imuxsock.c: Message has legacy syslog format. 5471.444550564:imuxsock.c: main queue: entry added, size now 1 entries 5471.444845291:imuxsock.c: wtpAdviseMaxWorkers signals busy 5471.445187788:imuxsock.c: main queue: EnqueueMsg advised worker start 5471.445489499:main queue:Reg/w0: main queue: entering rate limiter 5471.445842612:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5471.446121136:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 5') contains 'test': TRUE 5471.446527886:main queue:Reg/w0: action 0x4c64438 passed 2 times to execution - less than neded - discarding 5471.446837140:main queue:Reg/w0: main queue: entering rate limiter 5471.447091079:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5471.447366530:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5473.522029412:imuxsock.c: Message from UNIX socket: #4 5473.522550422:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:53 rger: test 6 5473.522766368:imuxsock.c: Message has legacy syslog format. 5473.523162224:imuxsock.c: main queue: entry added, size now 1 entries 5473.523404989:imuxsock.c: wtpAdviseMaxWorkers signals busy 5473.523727931:imuxsock.c: main queue: EnqueueMsg advised worker start 5473.524011204:main queue:Reg/w0: main queue: entering rate limiter 5473.524328279:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5473.524621330:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 6') contains 'test': TRUE 5473.525016627:main queue:Reg/w0: Called action, logging to builtin-file 5473.525803589:main queue:Reg/w0: (/rsyslog/logfile) 5473.526170949:main queue:Reg/w0: main queue: entering rate limiter 5473.526423213:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5473.526753139:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 On Thu, 2008-12-18 at 12:51 -0500, Becker, Craig (DOB) wrote: > The action $ActionExecOnlyEveryNthTime is not sending the Nth message > but the first message, then the Nth + 1 message. For example, if I > set the $ActionExecOnlyEveryNthTime 3 The 1st, 4th, 7th, etc. message > are sent instead of the 3rd, 6th, 9th, etc. Has this bug (if it is a > bug) been fixed in a later version? > > > > I?m enjoying learning this product and appreciate all the hard work > that has gone into it. > > > > Thanks, > > > > Craig E. Becker > IT Specialist II > System Administrator > Division of the Budget > 518-473-1322 > craig.becker at budget.state.ny.us > > > > > > > > HTML: > ______________________________________________________________________ > This e-mail, including any attachments, may be confidential, > privileged or otherwise legally protected. If you have received this > e-mail in error, or from someone who was not authorized to send it to > you, do not disseminate, copy or otherwise use this e-mail or its > attachments. Please notify the sender immediately if you have received > this e-mail by mistake, and delete it from your system. From rgerhards at hq.adiscon.com Fri Dec 19 12:57:16 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:57:16 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain><1229627751.12594.23.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F916@grfint2.intern.adiscon.com> David, one thing I can do rather quickly. Maybe it's good enough. I've done a tester, which lacks proper configuration, but I would appreciate feedback on it: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=a185665be4cf6997525 89d81ef6e396dd61f68b6 Details in git commit comment. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, December 19, 2008 1:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] suggested tweak to rsyslog > > On Thu, 18 Dec 2008, Rainer Gerhards wrote: > > > On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: > >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: > >> > >>> Hi David, > >>> > >>> On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > >>>> with messages appearing out-of-order the 'last message repeated X > times' > >>>> is pretty useless. > >>> > >>> Not really. Please note that form the perspective of the output > module, > >>> messages do not appear out of order. The output module receives a > stream > >>> of messages, and the "last message repeated x times" logic works on > that > >>> stream. So no matter if messages are re-ordered by async queues (or > UDP > >>> or whatever), the "last ... times" correctly reflects the way > things > >>> were handed to the output module. > >> > >> but if the output module is then relaying to another system, that > other > >> system (or the transport between them) can also re-order the > messages > > > > ah, ok, remote scenario. Of course, that can happen. But that's > "just" > > one use case where it is problematic. > > > >>> But I concur that this feature, in its current state, is very > >>> questionable. I've talked about this on the mailing list quite some > >>> time, I think there is also at least one blog post about it. > >>> > >>>> > >>>> I think I remember reading about an option to disable this, but > another > >>>> work-around to the problem is to change the output so that it > becomes > >>>> 'last message repeated X times %msg%', so you can see (most, if > not all) > >>>> of the message being repeated in the line telling you that it was > >>>> repeated. > >>> > >>> As I said, the message in front of this message is either another > repeat > >>> message or the message that is being repeated. So you can always > trace > >>> back to what was repeated (but it is painful). > >>> > >>> Given the feedback I received on this feature (use cases), it > should > >>> probably (and v4 would be an appropriate version to do so) be > redisigned > >>> to be a rate-limiting feature on the input side. That would pretty > much > >>> simplify code, too. > >> > >> but since things can be re-ordered after the input module is done > with > >> them (multiple threads pulling from the queue, or relaying) it is > still > >> useful for the 'message repeated' message to have more info about > what it > >> is referring to. > > > > That makes sense, at least for scenarios where it is expected > behaviour. > > For others, it would probably break analysers. So it needs to be > > optional. And probably on a per-action basis. I'll see if it is > > sufficiently simple, but the whole "last message repeated" thing is > > broken anyhow (IMO), I don't like the idea to invest any more than > maybe > > an hour into that feature, especially in the light that the whole > idea > > must be replaced in the not so distant future... > > it's definantly a pain for analysers (a _lot_ of them really only look > at > one line at a time), that's why I started tweaking this on sysklogd, > and > while in theory just disabling it is the right answer, the ability to > take > a flood of inbound messages and reduce them to a much smaller number > can > save your logging system when things go wrong > > I'll watch to see what happens. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From bakers at web-ster.com Fri Dec 19 17:38:41 2008 From: bakers at web-ster.com (Scott Baker) Date: Fri, 19 Dec 2008 08:38:41 -0800 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <1229627994.12594.26.camel@localhost.localdomain> References: <494AAB9F.9060005@web-ster.com> <1229627994.12594.26.camel@localhost.localdomain> Message-ID: <494BCE11.7000300@web-ster.com> Rainer Gerhards wrote: > I'd still go for debug mode. You don't need to run it very long. We just > need to see how a few of these messages are fully processed. A proper > test setup would be to start up in debug mode with the network cable > pulled, then plug it in for a second or two, then unplug it again. Once > rsyslogd is finished processing, stop it. That should lead to useful > info in the debug log. > > Oh - and are you sure that fromhost has the proper IP addresses? If not > 100% sure, verify it by putting something like '%FROMHOST%' into a debug > template (note that there is also FROMHOST-IP, which will have the IP > address no matter if names are resolved or not). I like the debug template idea, that's genius. Is there a way to have a bunch of filters to catch assorted things, and then an "everything leftover" filter? ------------------------------------------------------------------------ # Mail servers log to their special section $FileCreateMode 0644 :FROMHOST, isequal, "magenta" -?magic-mail :FROMHOST, isequal, "cyan" -?magic-mail :FROMHOST, isequal, "orange" -?magic-mail # Firewalls :FROMHOST, isequal, "yin" -?firewall :FROMHOST, isequal, "yang" -?firewall # Everything that didn't get caught by one of the above filters (I have no idea what the syntax would be) From rgerhards at hq.adiscon.com Fri Dec 19 17:40:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 17:40:32 +0100 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <494BCE11.7000300@web-ster.com> References: <494AAB9F.9060005@web-ster.com><1229627994.12594.26.camel@localhost.localdomain> <494BCE11.7000300@web-ster.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F924@grfint2.intern.adiscon.com> Without verification, but should work: # Mail servers log to their special section $FileCreateMode 0644 :FROMHOST, isequal, "magenta" -?magic-mail & ~ :FROMHOST, isequal, "cyan" -?magic-mail & ~ :FROMHOST, isequal, "orange" -?magic-mail & ~ # Firewalls :FROMHOST, isequal, "yin" -?firewall & ~ :FROMHOST, isequal, "yang" -?firewall & ~ *.* /var/log/catchrest & ~ discards the message after it is written to the file in question. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Baker > Sent: Friday, December 19, 2008 5:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] Troubleshooting missing log entries > > Rainer Gerhards wrote: > > I'd still go for debug mode. You don't need to run it very long. We > just > > need to see how a few of these messages are fully processed. A proper > > test setup would be to start up in debug mode with the network cable > > pulled, then plug it in for a second or two, then unplug it again. > Once > > rsyslogd is finished processing, stop it. That should lead to useful > > info in the debug log. > > > > Oh - and are you sure that fromhost has the proper IP addresses? If > not > > 100% sure, verify it by putting something like '%FROMHOST%' into a > debug > > template (note that there is also FROMHOST-IP, which will have the IP > > address no matter if names are resolved or not). > > > I like the debug template idea, that's genius. Is there a way to have a > bunch of filters to catch assorted things, and then an "everything > leftover" filter? > > ----------------------------------------------------------------------- > - > > # Mail servers log to their special section > $FileCreateMode 0644 > :FROMHOST, isequal, "magenta" -?magic-mail > :FROMHOST, isequal, "cyan" -?magic-mail > :FROMHOST, isequal, "orange" -?magic-mail > > # Firewalls > :FROMHOST, isequal, "yin" -?firewall > :FROMHOST, isequal, "yang" -?firewall > > # Everything that didn't get caught by one of the above filters > (I have no idea what the syntax would be) > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Dec 19 17:52:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 17:52:26 +0100 Subject: [rsyslog] Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? In-Reply-To: References: <1229685862.12594.33.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F926@grfint2.intern.adiscon.com> --sorry, my mailer didn't reply to the list for some reason-- Sounds like a too-old version. Jepp, you mention 3.21.2 in the subject, I think the directives were introduced some time later. Rainer > -----Original Message----- > From: Becker, Craig (DOB) [mailto:Craig.Becker at budget.state.ny.us] > Sent: Friday, December 19, 2008 5:34 PM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: RE: Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? > > Thanks for the help. When I start Rsyslog in Debug mode, I get the > following errors: > > > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 404 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 405 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 698 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 699 > > These errors correspond to the actions: > > $ActionExecOnlyEveryNthTime 50 #Lines 404 & 698 > $ActionExecOnlyEveryNthTimeTimeout 1800 #Lines 405 & 699 > > These are the Modules loaded: > > #______All Modules loaded here__________ > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging > (e.g. via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by > rklogd) > $ModLoad ommail # This mod enables sending E-mail on alerts > $ModLoad imudp.so # provides UDP syslog reception > $ModLoad imtcp.so # provides TCP syslog reception > > I tried to add all the modules I have access to without fixing the > issue. Is there a module missing or should I look to another version? > The strange thing is that the command actually works after the first > email is sent (i.e. The 1st, 51st, 101st messages are sent). > > Any help would be appreciated. > > Thanks again, > Craig > > > > -----Original Message----- > From: Rainer Gerhards [mailto:rgerhards at hq.adiscon.com] > Sent: Friday, December 19, 2008 6:24 AM > To: rsyslog at lists.adiscon.com > Cc: Becker, Craig (DOB) > Subject: Re: Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? > > Hi Craig, > > I don't see this in my lab, everything works correct. May it be that > you > did not include some of the (automatic) initial messages in your > picture? In any case, a full debug log would be useful to track this > down. After my sig, you find the relevant part of my lab debug FYI. > > Relevant part of rsyslog.conf: > $ActionExecOnlyEveryNthTime 3 > :msg, contains, "test" -/rsyslog/logfile > > Commands issued: > $ logger 1 > $ logger 2 > $ logger 3 > $ logger 4 > $ logger 5 > $ logger 6 > > And the output file had: > 2008-12-19T12:17:44.548207+01:00 rf10up rger: test 3 > 2008-12-19T12:17:53.522265+01:00 rf10up rger: test 6 > > Note that in this setup I made sure that no other messages than those > with the string "test" went into the log file. Otherwise, the counters > would have been affected by some startup messages. > > Rainer > > 5455.946426832:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5460.051992823:imuxsock.c: Message from UNIX socket: #4 > 5460.066209819:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:40 rger: test 1 > 5460.066408445:imuxsock.c: Message has legacy syslog format. > 5460.095581070:imuxsock.c: main queue: entry added, size now 1 entries > 5460.096265506:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5460.096810820:imuxsock.c: main queue: EnqueueMsg advised worker start > 5460.097417035:main queue:Reg/w0: main queue: entering rate limiter > 5460.097797246:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5460.098493974:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 1') contains 'test': TRUE > 5460.105295873:main queue:Reg/w0: action 0x4c64438 passed 1 times to > execution - less than neded - discarding > 5460.109592735:main queue:Reg/w0: main queue: entering rate limiter > 5460.110192245:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5460.115576662:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5462.317878786:imuxsock.c: Message from UNIX socket: #4 > 5462.318410132:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:42 rger: test 2 > 5462.318585850:imuxsock.c: Message has legacy syslog format. > 5462.319012436:imuxsock.c: main queue: entry added, size now 1 entries > 5462.319271125:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5462.319537636:imuxsock.c: main queue: EnqueueMsg advised worker start > 5462.319871473:main queue:Reg/w0: main queue: entering rate limiter > 5462.320174580:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5462.320470145:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 2') contains 'test': TRUE > 5462.321508812:main queue:Reg/w0: action 0x4c64438 passed 2 times to > execution - less than neded - discarding > 5462.321825608:main queue:Reg/w0: main queue: entering rate limiter > 5462.322078710:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5462.322350529:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5464.547937451:imuxsock.c: Message from UNIX socket: #4 > 5464.548420747:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:44 rger: test 3 > 5464.548613227:imuxsock.c: Message has legacy syslog format. > 5464.549042885:imuxsock.c: main queue: entry added, size now 1 entries > 5464.549287885:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5464.549571158:imuxsock.c: main queue: EnqueueMsg advised worker start > 5464.549888793:main queue:Reg/w0: main queue: entering rate limiter > 5464.550211176:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5464.550500036:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 3') contains 'test': TRUE > 5464.551797950:main queue:Reg/w0: Called action, logging to builtin- > file > 5464.605776776:main queue:Reg/w0: (/rsyslog/logfile) > 5464.606371817:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5464.627358305:main queue:Reg/w0: main queue: entering rate limiter > 5464.628009218:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5469.671232071:imuxsock.c: Message from UNIX socket: #4 > 5469.671772915:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:49 rger: test 4 > 5469.671950031:imuxsock.c: Message has legacy syslog format. > 5469.672340578:imuxsock.c: main queue: entry added, size now 1 entries > 5469.672597591:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5469.672931428:imuxsock.c: main queue: EnqueueMsg advised worker start > 5469.673234256:main queue:Reg/w0: main queue: entering rate limiter > 5469.673563065:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5469.673893829:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 4') contains 'test': TRUE > 5469.674471829:main queue:Reg/w0: action 0x4c64438 passed 1 times to > execution - less than neded - discarding > 5469.674788066:main queue:Reg/w0: main queue: entering rate limiter > 5469.675042006:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5469.675316618:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5471.443424737:imuxsock.c: Message from UNIX socket: #4 > 5471.443973403:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:51 rger: test 5 > 5471.444148283:imuxsock.c: Message has legacy syslog format. > 5471.444550564:imuxsock.c: main queue: entry added, size now 1 entries > 5471.444845291:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5471.445187788:imuxsock.c: main queue: EnqueueMsg advised worker start > 5471.445489499:main queue:Reg/w0: main queue: entering rate limiter > 5471.445842612:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5471.446121136:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 5') contains 'test': TRUE > 5471.446527886:main queue:Reg/w0: action 0x4c64438 passed 2 times to > execution - less than neded - discarding > 5471.446837140:main queue:Reg/w0: main queue: entering rate limiter > 5471.447091079:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5471.447366530:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5473.522029412:imuxsock.c: Message from UNIX socket: #4 > 5473.522550422:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:53 rger: test 6 > 5473.522766368:imuxsock.c: Message has legacy syslog format. > 5473.523162224:imuxsock.c: main queue: entry added, size now 1 entries > 5473.523404989:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5473.523727931:imuxsock.c: main queue: EnqueueMsg advised worker start > 5473.524011204:main queue:Reg/w0: main queue: entering rate limiter > 5473.524328279:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5473.524621330:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 6') contains 'test': TRUE > 5473.525016627:main queue:Reg/w0: Called action, logging to builtin- > file > 5473.525803589:main queue:Reg/w0: (/rsyslog/logfile) > 5473.526170949:main queue:Reg/w0: main queue: entering rate limiter > 5473.526423213:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5473.526753139:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > > > > On Thu, 2008-12-18 at 12:51 -0500, Becker, Craig (DOB) wrote: > > The action $ActionExecOnlyEveryNthTime is not sending the Nth message > > but the first message, then the Nth + 1 message. For example, if I > > set the $ActionExecOnlyEveryNthTime 3 The 1st, 4th, 7th, etc. message > > are sent instead of the 3rd, 6th, 9th, etc. Has this bug (if it is a > > bug) been fixed in a later version? > > > > > > > > I?m enjoying learning this product and appreciate all the hard work > > that has gone into it. > > > > > > > > Thanks, > > > > > > > > Craig E. Becker > > IT Specialist II > > System Administrator > > Division of the Budget > > 518-473-1322 > > craig.becker at budget.state.ny.us > > > > > > > > > > > > > > > > HTML: > > > ______________________________________________________________________ > > This e-mail, including any attachments, may be confidential, > > privileged or otherwise legally protected. If you have received this > > e-mail in error, or from someone who was not authorized to send it to > > you, do not disseminate, copy or otherwise use this e-mail or its > > attachments. Please notify the sender immediately if you have > received > > this e-mail by mistake, and delete it from your system. > > > > ----------------------------------------------------------------------- > --------- > This e-mail, including any attachments, may be confidential, privileged > or otherwise legally protected. If you have received this e-mail in > error, or from someone who was not authorized to send it to you, do not > disseminate, copy or otherwise use this e-mail or its attachments. > Please notify the sender immediately if you have received this e-mail > by mistake, and delete it from your system. From rgerhards at hq.adiscon.com Wed Dec 24 11:13:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Dec 2008 11:13:41 +0100 Subject: [rsyslog] Merry Xmas and a happy new year! Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F949@grfint2.intern.adiscon.com> Hi all, I wanted to whish all of you a merry xmas and a happy new year. At the same time, I'd like to thank all of you for your qualified comments, encouragement and ideas, which helped make rsyslog become a premier logging solution. Without an active community, no project can survive and I am very happy to see such a great community backing this project. Special thanks go to all those who also participate in the forum and dispense knowledge here on the mailing list. I can promise that 2009 will become a very exciting year for rsyslog, too. There is still a lot of things in stock and even though the budget cut has forced me to work at a somewhat slower pace, I am sure we will see great enhancements. A number of companies have also asked about sponsoring, so we may even be able to get back to full speed development in 2009. Please note that I take a break from the project from end of December to January, 10th. So my response will be sluggish in that period. My apologies for that. I hope you, too, will enjoy a great holiday season. My your wishes come true and your health be good! If we don't "talk" before the end of the year, I also wanted to wish you a great end of the year. Please keep your comments, ideas and feedback floating in 2009. This is a great source of inspiration and much appreciated. My warmest regards, Rainer Gerhards From rgerhards at hq.adiscon.com Mon Dec 1 10:02:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 10:02:01 +0100 Subject: [rsyslog] rsyslog doc licenses Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Hi all, Michael Biebl made me aware that the doc files have inconsistent license information (thanks!). Some state GPL, some FDL, some even nothing. I would like to unify them to use GPLv3, which is consistent with the rest of this project. Does anybody object this move? If so what are the reasons and alternatives? Feedback is appreciated. Many thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 1 10:55:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 10:55:12 +0100 Subject: [rsyslog] security issue in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> Hi folks, thanks to a bug report, I found out that the $AllowedSender directive does not work in all releases. The bug in question is: http://bugzilla.adiscon.com/show_bug.cgi?id=111 Im am currently working on the bug. Obviously, this can lead to messages being received from systems that are not permitted so. As a work-around, proper firewalling should be set up on the vulnerable hosts. Until further note, I would assume that all versions of rsyslog are affected (I will provide more detail during my analysis). Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 1 11:26:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 11:26:56 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> Version v2-stable is NOT vulnerable. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 10:55 AM > To: rsyslog-users > Subject: [rsyslog] security issue in rsyslog > > Hi folks, > > thanks to a bug report, I found out that the $AllowedSender directive > does not work in all releases. The bug in question is: > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > Im am currently working on the bug. Obviously, this can lead to > messages > being received from systems that are not permitted so. As a work- > around, > proper firewalling should be set up on the vulnerable hosts. Until > further note, I would assume that all versions of rsyslog are affected > (I will provide more detail during my analysis). > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 15:31:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 15:31:36 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> Hi all, this is patch for v3-stable: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae6 d9bbf6b07e2f06c4dd676 I have not tried yet, but I think it will work on almost all other versions, too. I keep you posted on the progress. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 11:27 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Version v2-stable is NOT vulnerable. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 10:55 AM > > To: rsyslog-users > > Subject: [rsyslog] security issue in rsyslog > > > > Hi folks, > > > > thanks to a bug report, I found out that the $AllowedSender directive > > does not work in all releases. The bug in question is: > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > Im am currently working on the bug. Obviously, this can lead to > > messages > > being received from systems that are not permitted so. As a work- > > around, > > proper firewalling should be set up on the vulnerable hosts. Until > > further note, I would assume that all versions of rsyslog are > affected > > (I will provide more detail during my analysis). > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 16:30:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 16:30:36 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> I now clarified the affected versions. Affected are 3.12.2 and above. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 3:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Hi all, > > this is patch for v3-stable: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > 6 > d9bbf6b07e2f06c4dd676 > > I have not tried yet, but I think it will work on almost all other > versions, too. I keep you posted on the progress. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 11:27 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Version v2-stable is NOT vulnerable. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 10:55 AM > > > To: rsyslog-users > > > Subject: [rsyslog] security issue in rsyslog > > > > > > Hi folks, > > > > > > thanks to a bug report, I found out that the $AllowedSender > directive > > > does not work in all releases. The bug in question is: > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > messages > > > being received from systems that are not permitted so. As a work- > > > around, > > > proper firewalling should be set up on the vulnerable hosts. Until > > > further note, I would assume that all versions of rsyslog are > > affected > > > (I will provide more detail during my analysis). > > > > > > Thanks, > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 16:37:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 16:37:23 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> ... and the patch will not work on all of these version, due to the introduction of the netstream driver functionality. Please note that anything older than current v3-stable is outdated, so the proper way to replace the faulty code is to upgrade to the current v3-stable and apply the patch. I will also release a new v3-stable soon, hopefully today (but I'd like to conduct some more tests). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 4:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > I now clarified the affected versions. Affected are 3.12.2 and above. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 3:32 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Hi all, > > > > this is patch for v3-stable: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > 6 > > d9bbf6b07e2f06c4dd676 > > > > I have not tried yet, but I think it will work on almost all other > > versions, too. I keep you posted on the progress. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 11:27 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Version v2-stable is NOT vulnerable. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > To: rsyslog-users > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > Hi folks, > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > directive > > > > does not work in all releases. The bug in question is: > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > > messages > > > > being received from systems that are not permitted so. As a work- > > > > around, > > > > proper firewalling should be set up on the vulnerable hosts. > Until > > > > further note, I would assume that all versions of rsyslog are > > > affected > > > > (I will provide more detail during my analysis). > > > > > > > > Thanks, > > > > Rainer > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 17:08:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 17:08:05 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> The issue also exists in TCP mode, but analysis shows this is not a trial fix. The design overlooked the situation. In theory, a whole new access control feature would be needed. I am checking out if it is possible to "just" enhance the interface. With the current netstreams defined that should be possible. I am tempted to release the UDP-fixed version and release the next version with the TCP fix. Feedback from packagers is appreciated. The TCP fix may take a day or two, depending on how smart a way I find. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 4:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > ... and the patch will not work on all of these version, due to the > introduction of the netstream driver functionality. Please note that > anything older than current v3-stable is outdated, so the proper way to > replace the faulty code is to upgrade to the current v3-stable and > apply > the patch. I will also release a new v3-stable soon, hopefully today > (but I'd like to conduct some more tests). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 4:31 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > I now clarified the affected versions. Affected are 3.12.2 and above. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 3:32 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Hi all, > > > > > > this is patch for v3-stable: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > 6 > > > d9bbf6b07e2f06c4dd676 > > > > > > I have not tried yet, but I think it will work on almost all other > > > versions, too. I keep you posted on the progress. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > To: rsyslog-users > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > Hi folks, > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > directive > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > > > messages > > > > > being received from systems that are not permitted so. As a > work- > > > > > around, > > > > > proper firewalling should be set up on the vulnerable hosts. > > Until > > > > > further note, I would assume that all versions of rsyslog are > > > > affected > > > > > (I will provide more detail during my analysis). > > > > > > > > > > Thanks, > > > > > Rainer > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 17:51:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 17:51:38 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> Ok, looks like I found a work-around. Not that elegant, but seems to work quite well. Patch for TCP is here: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9e 18747b55d701e360d5aac Please note that this effectively disables GSS functionality. I'll updated the GSS drivers in the next step. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 5:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The issue also exists in TCP mode, but analysis shows this is not a > trial fix. The design overlooked the situation. In theory, a whole new > access control feature would be needed. I am checking out if it is > possible to "just" enhance the interface. With the current netstreams > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 4:37 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > ... and the patch will not work on all of these version, due to the > > introduction of the netstream driver functionality. Please note that > > anything older than current v3-stable is outdated, so the proper way > to > > replace the faulty code is to upgrade to the current v3-stable and > > apply > > the patch. I will also release a new v3-stable soon, hopefully today > > (but I'd like to conduct some more tests). > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 4:31 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > above. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Hi all, > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > 6 > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > I have not tried yet, but I think it will work on almost all > other > > > > versions, too. I keep you posted on the progress. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > To: rsyslog-users > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > > directive > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can lead > to > > > > > > messages > > > > > > being received from systems that are not permitted so. As a > > work- > > > > > > around, > > > > > > proper firewalling should be set up on the vulnerable hosts. > > > Until > > > > > > further note, I would assume that all versions of rsyslog are > > > > > affected > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > Thanks, > > > > > > Rainer > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Mon Dec 1 18:23:47 2008 From: david at lang.hm (david at lang.hm) Date: Mon, 1 Dec 2008 09:23:47 -0800 (PST) Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: On Mon, 1 Dec 2008, Rainer Gerhards wrote: > The issue also exists in TCP mode, but analysis shows this is not a > trial fix. The design overlooked the situation. In theory, a whole new > access control feature would be needed. I am checking out if it is > possible to "just" enhance the interface. With the current netstreams > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. for UDP it's trivial to forge the source IP address anyway, so the 'security' gained by this feature in that mode is questionable to start with. that being said, I'm very pleased to see how you are handling this. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >> Sent: Monday, December 01, 2008 4:37 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] security issue in rsyslog >> >> ... and the patch will not work on all of these version, due to the >> introduction of the netstream driver functionality. Please note that >> anything older than current v3-stable is outdated, so the proper way > to >> replace the faulty code is to upgrade to the current v3-stable and >> apply >> the patch. I will also release a new v3-stable soon, hopefully today >> (but I'd like to conduct some more tests). >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>> Sent: Monday, December 01, 2008 4:31 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] security issue in rsyslog >>> >>> I now clarified the affected versions. Affected are 3.12.2 and > above. >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>> Sent: Monday, December 01, 2008 3:32 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] security issue in rsyslog >>>> >>>> Hi all, >>>> >>>> this is patch for v3-stable: >>>> >>>> >>> >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae >>>> 6 >>>> d9bbf6b07e2f06c4dd676 >>>> >>>> I have not tried yet, but I think it will work on almost all other >>>> versions, too. I keep you posted on the progress. >>>> >>>> Rainer >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>>> Sent: Monday, December 01, 2008 11:27 AM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] security issue in rsyslog >>>>> >>>>> Version v2-stable is NOT vulnerable. >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>>>> Sent: Monday, December 01, 2008 10:55 AM >>>>>> To: rsyslog-users >>>>>> Subject: [rsyslog] security issue in rsyslog >>>>>> >>>>>> Hi folks, >>>>>> >>>>>> thanks to a bug report, I found out that the $AllowedSender >>>> directive >>>>>> does not work in all releases. The bug in question is: >>>>>> >>>>>> http://bugzilla.adiscon.com/show_bug.cgi?id=111 >>>>>> >>>>>> Im am currently working on the bug. Obviously, this can lead > to >>>>>> messages >>>>>> being received from systems that are not permitted so. As a >> work- >>>>>> around, >>>>>> proper firewalling should be set up on the vulnerable hosts. >>> Until >>>>>> further note, I would assume that all versions of rsyslog are >>>>> affected >>>>>> (I will provide more detail during my analysis). >>>>>> >>>>>> Thanks, >>>>>> Rainer >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From mbiebl at gmail.com Mon Dec 1 18:17:31 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 18:17:31 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: 2008/12/1 Rainer Gerhards : > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. I'd say, take the time for proper fixing and testing, even if it takes a day or two and release afterwards. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Dec 1 18:47:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 18:47:00 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> And now there is an *untested* fix for the TLS driver: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec063 83fc5969178887d00abfc Testing takes a bit more of time, I need to set up the test environment for TLS again (looks like it would really pay to have a fixed test suite for all those cases - also the issue here would have never occurred...). Please note that I mistook GSSAPI with TLS in my previous mail. The TLS part should not be really affected by the problem: there are so much better access control features in TLS... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 5:52 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Ok, looks like I found a work-around. Not that elegant, but seems to > work quite well. Patch for TCP is here: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > e > 18747b55d701e360d5aac > > Please note that this effectively disables GSS functionality. I'll > updated the GSS drivers in the next step. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 5:08 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > The issue also exists in TCP mode, but analysis shows this is not a > > trial fix. The design overlooked the situation. In theory, a whole > new > > access control feature would be needed. I am checking out if it is > > possible to "just" enhance the interface. With the current netstreams > > defined that should be possible. I am tempted to release the UDP- > fixed > > version and release the next version with the TCP fix. Feedback from > > packagers is appreciated. The TCP fix may take a day or two, > depending > > on how smart a way I find. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 4:37 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > ... and the patch will not work on all of these version, due to the > > > introduction of the netstream driver functionality. Please note > that > > > anything older than current v3-stable is outdated, so the proper > way > > to > > > replace the faulty code is to upgrade to the current v3-stable and > > > apply > > > the patch. I will also release a new v3-stable soon, hopefully > today > > > (but I'd like to conduct some more tests). > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > > above. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Hi all, > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > 6 > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > I have not tried yet, but I think it will work on almost all > > other > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > To: rsyslog-users > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > > > directive > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > lead > > to > > > > > > > messages > > > > > > > being received from systems that are not permitted so. As a > > > work- > > > > > > > around, > > > > > > > proper firewalling should be set up on the vulnerable > hosts. > > > > Until > > > > > > > further note, I would assume that all versions of rsyslog > are > > > > > > affected > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > Thanks, > > > > > > > Rainer > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Mon Dec 1 19:30:57 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 19:30:57 +0100 Subject: [rsyslog] rsyslog doc licenses In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Message-ID: 2008/12/1 Rainer Gerhards : > Hi all, > > Michael Biebl made me aware that the doc files have inconsistent license > information (thanks!). Some state GPL, some FDL, some even nothing. > > I would like to unify them to use GPLv3, which is consistent with the > rest of this project. Does anybody object this move? My contributions to the documentation where minuscule. But anyways, you have my ok for the relicensing. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From aoz.syn at gmail.com Mon Dec 1 20:00:07 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 1 Dec 2008 12:00:07 -0700 Subject: [rsyslog] rsyslog doc licenses In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Message-ID: <4255c2570812011100l7183e18g4999de9bdddf5955@mail.gmail.com> On Mon, Dec 1, 2008 at 02:02, Rainer Gerhards wrote: > I would like to unify them to use GPLv3, which is consistent with the > rest of this project. Does anybody object this move? If so what are the > reasons and alternatives? One of the more documentation-focused licenses (e.g. GFDL or one of the Creative Commons family) may be better-suited to protect the IP concerns particular to documentation. I'm not sure I perfectly understand why separate documentation & source licenses exist, but the fact that legal minds greater than mine (doesn't take much) decided the split was necessary causes me to at least pause and consider them. A particularly interesting problem some have thrown at GFDL is that it is incompatible with the GPL - you cannot convert in either direction without re-licensing from the originator. Feh. Licensing is a sticky subject. From peter.matulis at canonical.com Mon Dec 1 20:48:41 2008 From: peter.matulis at canonical.com (Peter Matulis) Date: Mon, 01 Dec 2008 14:48:41 -0500 Subject: [rsyslog] TLS certificates Message-ID: <49343F99.4030607@canonical.com> Why does the documentation state that a client does not need to include its private key and certificate in its configuration? It says to only use the CA public certificate (ca.pem). Peter From mbiebl at gmail.com Mon Dec 1 20:53:19 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 20:53:19 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <49343F99.4030607@canonical.com> References: <49343F99.4030607@canonical.com> Message-ID: 2008/12/1 Peter Matulis : > Why does the documentation state that a client does not need to include > its private key and certificate in its configuration? It says to only > use the CA public certificate (ca.pem). That depends on what you want to do. If you only want to verify, that a client is sending it's messages to the right server, then it only needs the CA to verify that. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From peter.matulis at canonical.com Mon Dec 1 21:04:29 2008 From: peter.matulis at canonical.com (Peter Matulis) Date: Mon, 01 Dec 2008 15:04:29 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: References: <49343F99.4030607@canonical.com> Message-ID: <4934434D.4070703@canonical.com> Michael Biebl wrote: > 2008/12/1 Peter Matulis : >> Why does the documentation state that a client does not need to include >> its private key and certificate in its configuration? It says to only >> use the CA public certificate (ca.pem). > > That depends on what you want to do. > > If you only want to verify, that a client is sending it's messages to > the right server, then it only needs the CA to verify that. > > Michael > Right. I want to encrypt. Peter From aoz.syn at gmail.com Mon Dec 1 21:08:29 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 1 Dec 2008 13:08:29 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: <4934434D.4070703@canonical.com> References: <49343F99.4030607@canonical.com> <4934434D.4070703@canonical.com> Message-ID: <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> On Mon, Dec 1, 2008 at 13:04, Peter Matulis wrote: > Right. I want to encrypt. Encryption is still possible without a client certificate. Generally speaking, in SSL all a client cert adds is the ability for the server to verify the client's identity at the transport layer. RB From rgerhards at hq.adiscon.com Mon Dec 1 21:10:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 21:10:24 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> References: <49343F99.4030607@canonical.com><4934434D.4070703@canonical.com> <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F77A@grfint2.intern.adiscon.com> Jupp. I'd also like to mention the more elaborate guide. It is part of the doc set, but also available online: http://www.rsyslog.com/doc-rsyslog_secure_tls.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of RB > Sent: Monday, December 01, 2008 9:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > On Mon, Dec 1, 2008 at 13:04, Peter Matulis > wrote: > > Right. I want to encrypt. > > Encryption is still possible without a client certificate. Generally > speaking, in SSL all a client cert adds is the ability for the server > to verify the client's identity at the transport layer. > > > RB > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jmiscaro at gmail.com Tue Dec 2 14:55:56 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 08:55:56 -0500 Subject: [rsyslog] TLS certificates Message-ID: I has reading the docs [1] and this confusd me to. It says "neither the client nor the server are authenticated. So while the message transfer is encrypted, you can not be sure which peer you are talking to" Also, how can client encrypt without having any keys specified in its config? Example for the client shows: $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem $ActionSendStreamDriverAuthMode anon # server is NOT authenticated 2nd question: Why is the server not authenticated? /juan [1] http://www.rsyslog.com/doc-rsyslog_tls.html From aoz.syn at gmail.com Tue Dec 2 16:56:37 2008 From: aoz.syn at gmail.com (RB) Date: Tue, 2 Dec 2008 08:56:37 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: References: Message-ID: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: > "neither the client nor the server are authenticated. So while the > message transfer is encrypted, you can not be sure which peer you are > talking to" I'm hoping Rainer will jump in and clarify precisely how much handshake validation he's implemented. The fact that the client must have a copy of the CA's public material seems to indicate he is at least verifying that the server's certificate was issued by the CA. It's possible to not do so, but the result is rather susceptible to MITM. > Also, how can client encrypt without having any keys specified in its config? This isn't the forum to discuss the particulars of the SSL handshake, but suffice it to say that SSL incorporates a challenge/response mechanism (using the server's presented certificate) followed by negotiation of an ephemeral session key. See also: public-key cryptography. > $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > > 2nd question: Why is the server not authenticated? Without looking at the code, I presume the 'anon' AuthMode is the switch used to tell the SSL library whether or not to check the server certificate against the CA. If so, it should make specifying the CA public key redundant - the client just accepts whatever certificate the server (or MITM) presents and starts encrypting to it. From rgerhards at hq.adiscon.com Tue Dec 2 17:00:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:00:43 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: Tuesday, December 02, 2008 4:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: > > "neither the client nor the server are authenticated. So while the > > message transfer is encrypted, you can not be sure which peer you are > > talking to" > > I'm hoping Rainer will jump in and clarify precisely how much > handshake validation he's implemented. The fact that the client must > have a copy of the CA's public material seems to indicate he is at > least verifying that the server's certificate was issued by the CA. > It's possible to not do so, but the result is rather susceptible to > MITM. Just a quick note, I am quite busy at the moment (guess what ;)). If the auth is set to "anon" nothing at all is validated and MITM *is* absolutely possible. That's why the doc does not recommend to use that mode. I posted a link to the long TLS setup guide, which creates a fairly safe scenario (but your milage may vary... ;)). > > > Also, how can client encrypt without having any keys specified in its > config? > > This isn't the forum to discuss the particulars of the SSL handshake, > but suffice it to say that SSL incorporates a challenge/response > mechanism (using the server's presented certificate) followed by > negotiation of an ephemeral session key. See also: public-key > cryptography. > > > $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > > $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > > > > 2nd question: Why is the server not authenticated? > > Without looking at the code, I presume the 'anon' AuthMode is the > switch used to tell the SSL library whether or not to check the server > certificate against the CA. If so, it should make specifying the CA > public key redundant - the client just accepts whatever certificate > the server (or MITM) presents and starts encrypting to it. The modes are mostly rooted in the upcoming RFC5225 (or 5226, don't remember correctly). Anon is an insecure extension. While being insecure, it is a mode that allows low end devices deployed in no-knowledge environmebnt (hopefully read: home users) to have at least the benefit of encryption (obfuscation would be more precise) but nothing (nothing!) above that. Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Tue Dec 2 17:18:30 2008 From: aoz.syn at gmail.com (RB) Date: Tue, 2 Dec 2008 09:18:30 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> Message-ID: <4255c2570812020818w50a7a8b7k1c506ab8455b511e@mail.gmail.com> On Tue, Dec 2, 2008 at 09:00, Rainer Gerhards wrote: > Just a quick note, I am quite busy at the moment (guess what ;)). If the > auth is set to "anon" nothing at all is validated and MITM *is* > absolutely possible. That's why the doc does not recommend to use that > mode. I posted a link to the long TLS setup guide, which creates a > fairly safe scenario (but your milage may vary... ;)). Understood. For everyone else's edification, here is the comment in the related code, outlining what modes are used: /* Set the authentication mode. For us, the following is supported: * anon - no certificate checks whatsoever (discouraged, but supported) * x509/certvalid - (just) check certificate validity * x509/fingerprint - certificate fingerprint * x509/name - cerfificate name check * mode == NULL is valid and defaults to x509/name * rgerhards, 2008-05-16 */ From jmiscaro at gmail.com Tue Dec 2 17:30:42 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 11:30:42 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: 2008/12/2 RB : > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: >> "neither the client nor the server are authenticated. So while the >> message transfer is encrypted, you can not be sure which peer you are >> talking to" > > I'm hoping Rainer will jump in and clarify precisely how much > handshake validation he's implemented. The fact that the client must > have a copy of the CA's public material seems to indicate he is at > least verifying that the server's certificate was issued by the CA. > It's possible to not do so, but the result is rather susceptible to > MITM. > >> Also, how can client encrypt without having any keys specified in its config? > > This isn't the forum to discuss the particulars of the SSL handshake, > but suffice it to say that SSL incorporates a challenge/response > mechanism (using the server's presented certificate) followed by > negotiation of an ephemeral session key. See also: public-key > cryptography. > >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated >> >> 2nd question: Why is the server not authenticated? > > Without looking at the code, I presume the 'anon' AuthMode is the > switch used to tell the SSL library whether or not to check the server > certificate against the CA. If so, it should make specifying the CA > public key redundant - the client just accepts whatever certificate > the server (or MITM) presents and starts encrypting to it. Thank you. I change my config and logging is hapenning on the server end. However, I get such lines in the logs on the server end when I restart the client system: invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 42 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 45 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 46 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 47 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 49 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 51 This happens for each TLS line in my client config (comments removed): $DefaultNetstreamDriver gtls $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client-cert.pem $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client-key.pem $ActionSendStreamDriverAuthMode x509/name $ActionSendStreamDriverMode 1 *.* @@192.168.4.102:10514 /juan From rgerhards at hq.adiscon.com Tue Dec 2 17:31:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:31:13 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Too old version? > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > Sent: Tuesday, December 02, 2008 5:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > 2008/12/2 RB : > > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro > wrote: > >> "neither the client nor the server are authenticated. So while the > >> message transfer is encrypted, you can not be sure which peer you > are > >> talking to" > > > > I'm hoping Rainer will jump in and clarify precisely how much > > handshake validation he's implemented. The fact that the client must > > have a copy of the CA's public material seems to indicate he is at > > least verifying that the server's certificate was issued by the CA. > > It's possible to not do so, but the result is rather susceptible to > > MITM. > > > >> Also, how can client encrypt without having any keys specified in > its config? > > > > This isn't the forum to discuss the particulars of the SSL handshake, > > but suffice it to say that SSL incorporates a challenge/response > > mechanism (using the server's presented certificate) followed by > > negotiation of an ephemeral session key. See also: public-key > > cryptography. > > > >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > >> > >> 2nd question: Why is the server not authenticated? > > > > Without looking at the code, I presume the 'anon' AuthMode is the > > switch used to tell the SSL library whether or not to check the > server > > certificate against the CA. If so, it should make specifying the CA > > public key redundant - the client just accepts whatever certificate > > the server (or MITM) presents and starts encrypting to it. > > Thank you. I change my config and logging is hapenning on the server > end. However, I get such lines in the logs on the server end when I > restart the client system: > > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 42 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 45 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 46 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 47 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 49 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 51 > > > This happens for each TLS line in my client config (comments removed): > > $DefaultNetstreamDriver gtls > $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem > $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- > cert.pem > $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- > key.pem > $ActionSendStreamDriverAuthMode x509/name > $ActionSendStreamDriverMode 1 > *.* @@192.168.4.102:10514 > > > /juan > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jmiscaro at gmail.com Tue Dec 2 17:39:49 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 11:39:49 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Message-ID: My boxes are running 3.18.1 /juan 2008/12/2 Rainer Gerhards : > Too old version? > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Juan Miscaro >> Sent: Tuesday, December 02, 2008 5:31 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] TLS certificates >> >> 2008/12/2 RB : >> > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro >> wrote: >> >> "neither the client nor the server are authenticated. So while the >> >> message transfer is encrypted, you can not be sure which peer you >> are >> >> talking to" >> > >> > I'm hoping Rainer will jump in and clarify precisely how much >> > handshake validation he's implemented. The fact that the client > must >> > have a copy of the CA's public material seems to indicate he is at >> > least verifying that the server's certificate was issued by the CA. >> > It's possible to not do so, but the result is rather susceptible to >> > MITM. >> > >> >> Also, how can client encrypt without having any keys specified in >> its config? >> > >> > This isn't the forum to discuss the particulars of the SSL > handshake, >> > but suffice it to say that SSL incorporates a challenge/response >> > mechanism (using the server's presented certificate) followed by >> > negotiation of an ephemeral session key. See also: public-key >> > cryptography. >> > >> >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem >> >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated >> >> >> >> 2nd question: Why is the server not authenticated? >> > >> > Without looking at the code, I presume the 'anon' AuthMode is the >> > switch used to tell the SSL library whether or not to check the >> server >> > certificate against the CA. If so, it should make specifying the CA >> > public key redundant - the client just accepts whatever certificate >> > the server (or MITM) presents and starts encrypting to it. >> >> Thank you. I change my config and logging is hapenning on the server >> end. However, I get such lines in the logs on the server end when I >> restart the client system: >> >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 42 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 45 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 46 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 47 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 49 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 51 >> >> >> This happens for each TLS line in my client config (comments removed): >> >> $DefaultNetstreamDriver gtls >> $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem >> $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- >> cert.pem >> $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- >> key.pem >> $ActionSendStreamDriverAuthMode x509/name >> $ActionSendStreamDriverMode 1 >> *.* @@192.168.4.102:10514 >> >> >> /juan >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Tue Dec 2 17:52:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:52:25 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7B5@grfint2.intern.adiscon.com> Jup, that's the problem. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > Sent: Tuesday, December 02, 2008 5:40 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > My boxes are running 3.18.1 > > /juan > > 2008/12/2 Rainer Gerhards : > > Too old version? > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > >> Sent: Tuesday, December 02, 2008 5:31 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] TLS certificates > >> > >> 2008/12/2 RB : > >> > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro > >> wrote: > >> >> "neither the client nor the server are authenticated. So while > the > >> >> message transfer is encrypted, you can not be sure which peer you > >> are > >> >> talking to" > >> > > >> > I'm hoping Rainer will jump in and clarify precisely how much > >> > handshake validation he's implemented. The fact that the client > > must > >> > have a copy of the CA's public material seems to indicate he is at > >> > least verifying that the server's certificate was issued by the > CA. > >> > It's possible to not do so, but the result is rather susceptible > to > >> > MITM. > >> > > >> >> Also, how can client encrypt without having any keys specified in > >> its config? > >> > > >> > This isn't the forum to discuss the particulars of the SSL > > handshake, > >> > but suffice it to say that SSL incorporates a challenge/response > >> > mechanism (using the server's presented certificate) followed by > >> > negotiation of an ephemeral session key. See also: public-key > >> > cryptography. > >> > > >> >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > >> >> $ActionSendStreamDriverAuthMode anon # server is NOT > authenticated > >> >> > >> >> 2nd question: Why is the server not authenticated? > >> > > >> > Without looking at the code, I presume the 'anon' AuthMode is the > >> > switch used to tell the SSL library whether or not to check the > >> server > >> > certificate against the CA. If so, it should make specifying the > CA > >> > public key redundant - the client just accepts whatever > certificate > >> > the server (or MITM) presents and starts encrypting to it. > >> > >> Thank you. I change my config and logging is hapenning on the > server > >> end. However, I get such lines in the logs on the server end when I > >> restart the client system: > >> > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 42 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 45 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 46 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 47 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 49 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 51 > >> > >> > >> This happens for each TLS line in my client config (comments > removed): > >> > >> $DefaultNetstreamDriver gtls > >> $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem > >> $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- > >> cert.pem > >> $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- > >> key.pem > >> $ActionSendStreamDriverAuthMode x509/name > >> $ActionSendStreamDriverMode 1 > >> *.* @@192.168.4.102:10514 > >> > >> > >> /juan > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 10:54:20 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 10:54:20 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> Ok, I ran this fix through a couple of tests yesterday. It looks well for TLS, too. Note that there is an implication that $AllowedSender TCP,... applies to TLS to (because it is TCP). I'd consider this to be a side-effect, but I do not think it is worth fixing. With TLS, there is much finer and better control. An issue may only exists if someone decides to run non-tls tcp and tls tcp together AND use $AllowedSender. Workaround in that case is to use the firewall, so I don't consider it is worth fixing now. Please note that my testing revealed a potential memory leak as side-effect of the fixes. This could be abused to a remote DoS, so I will investigate that before releasing. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 6:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > And now there is an *untested* fix for the TLS driver: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > 3 > 83fc5969178887d00abfc > > Testing takes a bit more of time, I need to set up the test environment > for TLS again (looks like it would really pay to have a fixed test > suite > for all those cases - also the issue here would have never > occurred...). > > Please note that I mistook GSSAPI with TLS in my previous mail. The TLS > part should not be really affected by the problem: there are so much > better access control features in TLS... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 5:52 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, looks like I found a work-around. Not that elegant, but seems to > > work quite well. Patch for TCP is here: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > e > > 18747b55d701e360d5aac > > > > Please note that this effectively disables GSS functionality. I'll > > updated the GSS drivers in the next step. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 5:08 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > The issue also exists in TCP mode, but analysis shows this is not a > > > trial fix. The design overlooked the situation. In theory, a whole > > new > > > access control feature would be needed. I am checking out if it is > > > possible to "just" enhance the interface. With the current > netstreams > > > defined that should be possible. I am tempted to release the UDP- > > fixed > > > version and release the next version with the TCP fix. Feedback > from > > > packagers is appreciated. The TCP fix may take a day or two, > > depending > > > on how smart a way I find. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > ... and the patch will not work on all of these version, due to > the > > > > introduction of the netstream driver functionality. Please note > > that > > > > anything older than current v3-stable is outdated, so the proper > > way > > > to > > > > replace the faulty code is to upgrade to the current v3-stable > and > > > > apply > > > > the patch. I will also release a new v3-stable soon, hopefully > > today > > > > (but I'd like to conduct some more tests). > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > > > above. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Hi all, > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > 6 > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > I have not tried yet, but I think it will work on almost all > > > other > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > $AllowedSender > > > > > > directive > > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > > lead > > > to > > > > > > > > messages > > > > > > > > being received from systems that are not permitted so. As > a > > > > work- > > > > > > > > around, > > > > > > > > proper firewalling should be set up on the vulnerable > > hosts. > > > > > Until > > > > > > > > further note, I would assume that all versions of rsyslog > > are > > > > > > > affected > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Rainer > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 11:30:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 11:30:29 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> The memory leak is now also fixed, I just quickly re-run some TLS tests to make sure nothing is broken and it works there, too. Patch (on top of the others): http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54ddd cb8703560c750f04a6370 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 10:54 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Ok, I ran this fix through a couple of tests yesterday. It looks well > for TLS, too. Note that there is an implication that $AllowedSender > TCP,... applies to TLS to (because it is TCP). I'd consider this to be > a > side-effect, but I do not think it is worth fixing. With TLS, there is > much finer and better control. An issue may only exists if someone > decides to run non-tls tcp and tls tcp together AND use $AllowedSender. > Workaround in that case is to use the firewall, so I don't consider it > is worth fixing now. > > Please note that my testing revealed a potential memory leak as > side-effect of the fixes. This could be abused to a remote DoS, so I > will investigate that before releasing. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 6:47 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > And now there is an *untested* fix for the TLS driver: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > 3 > > 83fc5969178887d00abfc > > > > Testing takes a bit more of time, I need to set up the test > environment > > for TLS again (looks like it would really pay to have a fixed test > > suite > > for all those cases - also the issue here would have never > > occurred...). > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > TLS > > part should not be really affected by the problem: there are so much > > better access control features in TLS... > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 5:52 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > to > > > work quite well. Patch for TCP is here: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > e > > > 18747b55d701e360d5aac > > > > > > Please note that this effectively disables GSS functionality. I'll > > > updated the GSS drivers in the next step. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > The issue also exists in TCP mode, but analysis shows this is not > a > > > > trial fix. The design overlooked the situation. In theory, a > whole > > > new > > > > access control feature would be needed. I am checking out if it > is > > > > possible to "just" enhance the interface. With the current > > netstreams > > > > defined that should be possible. I am tempted to release the UDP- > > > fixed > > > > version and release the next version with the TCP fix. Feedback > > from > > > > packagers is appreciated. The TCP fix may take a day or two, > > > depending > > > > on how smart a way I find. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > ... and the patch will not work on all of these version, due to > > the > > > > > introduction of the netstream driver functionality. Please note > > > that > > > > > anything older than current v3-stable is outdated, so the > proper > > > way > > > > to > > > > > replace the faulty code is to upgrade to the current v3-stable > > and > > > > > apply > > > > > the patch. I will also release a new v3-stable soon, hopefully > > > today > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > and > > > > above. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > 6 > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > all > > > > other > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > $AllowedSender > > > > > > > directive > > > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > > > lead > > > > to > > > > > > > > > messages > > > > > > > > > being received from systems that are not permitted so. > As > > a > > > > > work- > > > > > > > > > around, > > > > > > > > > proper firewalling should be set up on the vulnerable > > > hosts. > > > > > > Until > > > > > > > > > further note, I would assume that all versions of > rsyslog > > > are > > > > > > > > affected > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Rainer > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 16:47:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 16:47:45 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7D5@grfint2.intern.adiscon.com> ...TLS testing always takes an awful lot of time... But I was also able to identify another memory leak, which is nice. I think I have now finished the release (less some doc update, maybe). I'd appreciate if some of you could give it a try. I would then do the actual release tomorrow. Download is: http://download.rsyslog.com/rsyslog/rsyslog-3.20.1.tar.gz md5sum: 2786d0d8de85fc9e6e83ff4ed9f468a7 If you try it out, please let me know the results. As I said, I don't expect anything bad, so it should be suitable for production use. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 11:30 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The memory leak is now also fixed, I just quickly re-run some TLS tests > to make sure nothing is broken and it works there, too. > > Patch (on top of the others): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > d > cb8703560c750f04a6370 > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 10:54 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, I ran this fix through a couple of tests yesterday. It looks well > > for TLS, too. Note that there is an implication that $AllowedSender > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > be > > a > > side-effect, but I do not think it is worth fixing. With TLS, there > is > > much finer and better control. An issue may only exists if someone > > decides to run non-tls tcp and tls tcp together AND use > $AllowedSender. > > Workaround in that case is to use the firewall, so I don't consider > it > > is worth fixing now. > > > > Please note that my testing revealed a potential memory leak as > > side-effect of the fixes. This could be abused to a remote DoS, so I > > will investigate that before releasing. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 6:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > 3 > > > 83fc5969178887d00abfc > > > > > > Testing takes a bit more of time, I need to set up the test > > environment > > > for TLS again (looks like it would really pay to have a fixed test > > > suite > > > for all those cases - also the issue here would have never > > > occurred...). > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > > TLS > > > part should not be really affected by the problem: there are so > much > > > better access control features in TLS... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > > to > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > e > > > > 18747b55d701e360d5aac > > > > > > > > Please note that this effectively disables GSS functionality. > I'll > > > > updated the GSS drivers in the next step. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > not > > a > > > > > trial fix. The design overlooked the situation. In theory, a > > whole > > > > new > > > > > access control feature would be needed. I am checking out if it > > is > > > > > possible to "just" enhance the interface. With the current > > > netstreams > > > > > defined that should be possible. I am tempted to release the > UDP- > > > > fixed > > > > > version and release the next version with the TCP fix. Feedback > > > from > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > depending > > > > > on how smart a way I find. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > ... and the patch will not work on all of these version, due > to > > > the > > > > > > introduction of the netstream driver functionality. Please > note > > > > that > > > > > > anything older than current v3-stable is outdated, so the > > proper > > > > way > > > > > to > > > > > > replace the faulty code is to upgrade to the current v3- > stable > > > and > > > > > > apply > > > > > > the patch. I will also release a new v3-stable soon, > hopefully > > > > today > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > > and > > > > > above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > 6 > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > > all > > > > > other > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > $AllowedSender > > > > > > > > directive > > > > > > > > > > does not work in all releases. The bug in question > is: > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > can > > > > lead > > > > > to > > > > > > > > > > messages > > > > > > > > > > being received from systems that are not permitted > so. > > As > > > a > > > > > > work- > > > > > > > > > > around, > > > > > > > > > > proper firewalling should be set up on the vulnerable > > > > hosts. > > > > > > > Until > > > > > > > > > > further note, I would assume that all versions of > > rsyslog > > > > are > > > > > > > > > affected > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Rainer > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Dec 4 12:30:58 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Dec 2008 12:30:58 +0100 Subject: [rsyslog] rsyslog 3.20.1 (v3-stable) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E3@grfint2.intern.adiscon.com> Hello, We have just released rsyslog 3.20.1, a member of the v3-stable branch. Most importantly, this release addresses a security vulnerability that renders the $AllowedSender directive useless. This issue has already been discussed here on the list. In addition to this, the release also contains some bug fixes, including a memory leak that could cause real trouble in environments that use TLS, as well as some very small enhancements (that were considered small enough to be a applied to the stable branch) Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-141.phtml Change Log: http://www.rsyslog.com/Article323.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. Releases for the beta and devel branches will hopefully be posted later today. The git archive has all relevant patches if someone has an urgent need. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Thu Dec 4 13:34:08 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Dec 2008 13:34:08 +0100 Subject: [rsyslog] rsyslog 3.21.8 (v3-beta) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 3.21.8, a member of the v3-beta branch. Most importantly, this release addresses a security vulnerability the renders the $AllowedSender directive useless. This issue has already been discussed here on the list. In addition to this, the release also contains all the bug fixes and enhancements from the stable release 3.20.1. Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-142.phtml Change Log: http://www.rsyslog.com/Article326.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. Releases for the devel branch will hopefully be posted later today. The git archive has all relevant patches if someone has an urgent need. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Thu Dec 4 13:38:17 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 13:38:17 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> Grrr... One more issue. I noticed that while I resolved some conflicts on the devel branch integration. There is an option that a log message is emitted by rsyslog itself, when a remote machine's message is discarded due to no permission. This was requested so that people know when something goes wrong. This is only in the UDP code. HOWEVER, this is not rate-limited so if someone carries out a heavy attack, he can still flood the local disk by these messages. I'll change it so that the message is emited only once every minute and will then re-release what already has been released... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 11:30 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The memory leak is now also fixed, I just quickly re-run some TLS tests > to make sure nothing is broken and it works there, too. > > Patch (on top of the others): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > d > cb8703560c750f04a6370 > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 10:54 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, I ran this fix through a couple of tests yesterday. It looks well > > for TLS, too. Note that there is an implication that $AllowedSender > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > be > > a > > side-effect, but I do not think it is worth fixing. With TLS, there > is > > much finer and better control. An issue may only exists if someone > > decides to run non-tls tcp and tls tcp together AND use > $AllowedSender. > > Workaround in that case is to use the firewall, so I don't consider > it > > is worth fixing now. > > > > Please note that my testing revealed a potential memory leak as > > side-effect of the fixes. This could be abused to a remote DoS, so I > > will investigate that before releasing. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 6:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > 3 > > > 83fc5969178887d00abfc > > > > > > Testing takes a bit more of time, I need to set up the test > > environment > > > for TLS again (looks like it would really pay to have a fixed test > > > suite > > > for all those cases - also the issue here would have never > > > occurred...). > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > > TLS > > > part should not be really affected by the problem: there are so > much > > > better access control features in TLS... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > > to > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > e > > > > 18747b55d701e360d5aac > > > > > > > > Please note that this effectively disables GSS functionality. > I'll > > > > updated the GSS drivers in the next step. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > not > > a > > > > > trial fix. The design overlooked the situation. In theory, a > > whole > > > > new > > > > > access control feature would be needed. I am checking out if it > > is > > > > > possible to "just" enhance the interface. With the current > > > netstreams > > > > > defined that should be possible. I am tempted to release the > UDP- > > > > fixed > > > > > version and release the next version with the TCP fix. Feedback > > > from > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > depending > > > > > on how smart a way I find. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > ... and the patch will not work on all of these version, due > to > > > the > > > > > > introduction of the netstream driver functionality. Please > note > > > > that > > > > > > anything older than current v3-stable is outdated, so the > > proper > > > > way > > > > > to > > > > > > replace the faulty code is to upgrade to the current v3- > stable > > > and > > > > > > apply > > > > > > the patch. I will also release a new v3-stable soon, > hopefully > > > > today > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > > and > > > > > above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > 6 > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > > all > > > > > other > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > $AllowedSender > > > > > > > > directive > > > > > > > > > > does not work in all releases. The bug in question > is: > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > can > > > > lead > > > > > to > > > > > > > > > > messages > > > > > > > > > > being received from systems that are not permitted > so. > > As > > > a > > > > > > work- > > > > > > > > > > around, > > > > > > > > > > proper firewalling should be set up on the vulnerable > > > > hosts. > > > > > > > Until > > > > > > > > > > further note, I would assume that all versions of > > rsyslog > > > > are > > > > > > > > > affected > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Rainer > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 4 14:48:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 14:48:03 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> OK, 3.20.1 is now re-released as 3.20.2 (there were a few downloads...). The download link is still correct, it is updated (including the md5sum ;)). 3.21.8 is pulled and I'll restore it next. Sorry for the hassle. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, December 04, 2008 1:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Grrr... One more issue. I noticed that while I resolved some conflicts > on the devel branch integration. There is an option that a log message > is emitted by rsyslog itself, when a remote machine's message is > discarded due to no permission. This was requested so that people know > when something goes wrong. This is only in the UDP code. > > HOWEVER, this is not rate-limited so if someone carries out a heavy > attack, he can still flood the local disk by these messages. I'll > change > it so that the message is emited only once every minute and will then > re-release what already has been released... > > Rainer > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 11:30 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > The memory leak is now also fixed, I just quickly re-run some TLS > tests > > to make sure nothing is broken and it works there, too. > > > > Patch (on top of the others): > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > > d > > cb8703560c750f04a6370 > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Wednesday, December 03, 2008 10:54 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Ok, I ran this fix through a couple of tests yesterday. It looks > well > > > for TLS, too. Note that there is an implication that $AllowedSender > > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > > be > > > a > > > side-effect, but I do not think it is worth fixing. With TLS, there > > is > > > much finer and better control. An issue may only exists if someone > > > decides to run non-tls tcp and tls tcp together AND use > > $AllowedSender. > > > Workaround in that case is to use the firewall, so I don't consider > > it > > > is worth fixing now. > > > > > > Please note that my testing revealed a potential memory leak as > > > side-effect of the fixes. This could be abused to a remote DoS, so > I > > > will investigate that before releasing. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 6:47 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > > 3 > > > > 83fc5969178887d00abfc > > > > > > > > Testing takes a bit more of time, I need to set up the test > > > environment > > > > for TLS again (looks like it would really pay to have a fixed > test > > > > suite > > > > for all those cases - also the issue here would have never > > > > occurred...). > > > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. > The > > > TLS > > > > part should not be really affected by the problem: there are so > > much > > > > better access control features in TLS... > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but > seems > > > to > > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > > e > > > > > 18747b55d701e360d5aac > > > > > > > > > > Please note that this effectively disables GSS functionality. > > I'll > > > > > updated the GSS drivers in the next step. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > > not > > > a > > > > > > trial fix. The design overlooked the situation. In theory, a > > > whole > > > > > new > > > > > > access control feature would be needed. I am checking out if > it > > > is > > > > > > possible to "just" enhance the interface. With the current > > > > netstreams > > > > > > defined that should be possible. I am tempted to release the > > UDP- > > > > > fixed > > > > > > version and release the next version with the TCP fix. > Feedback > > > > from > > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > > depending > > > > > > on how smart a way I find. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > ... and the patch will not work on all of these version, > due > > to > > > > the > > > > > > > introduction of the netstream driver functionality. Please > > note > > > > > that > > > > > > > anything older than current v3-stable is outdated, so the > > > proper > > > > > way > > > > > > to > > > > > > > replace the faulty code is to upgrade to the current v3- > > stable > > > > and > > > > > > > apply > > > > > > > the patch. I will also release a new v3-stable soon, > > hopefully > > > > > today > > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > I now clarified the affected versions. Affected are > 3.12.2 > > > and > > > > > > above. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > > 6 > > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on > almost > > > all > > > > > > other > > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog- > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > Gerhards > > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > > $AllowedSender > > > > > > > > > directive > > > > > > > > > > > does not work in all releases. The bug in question > > is: > > > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > > can > > > > > lead > > > > > > to > > > > > > > > > > > messages > > > > > > > > > > > being received from systems that are not permitted > > so. > > > As > > > > a > > > > > > > work- > > > > > > > > > > > around, > > > > > > > > > > > proper firewalling should be set up on the > vulnerable > > > > > hosts. > > > > > > > > Until > > > > > > > > > > > further note, I would assume that all versions of > > > rsyslog > > > > > are > > > > > > > > > > affected > > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Rainer > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Thu Dec 4 16:03:30 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 4 Dec 2008 08:03:30 -0700 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> Message-ID: <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> > Sorry for the hassle. It's not a hassle when you're being open and honest about issues. Too few projects call an apple an apple, so it's pleasing to be able to understand precisely what issues are. From Gerhardus.Geldenhuis at gta-travel.com Thu Dec 4 16:09:07 2008 From: Gerhardus.Geldenhuis at gta-travel.com (Gerhardus.Geldenhuis at gta-travel.com) Date: Thu, 4 Dec 2008 15:09:07 -0000 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> Message-ID: <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> I must concur, I am not very active on this mailing list in either form, but rsyslog does represent to me what open source was always about. The openness, speed and intelligence with which bugs/issues are addressed are exemplary. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: 04 December 2008 15:04 > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > > Sorry for the hassle. > > It's not a hassle when you're being open and honest about issues. Too > few projects call an apple an apple, so it's pleasing to be able to > understand precisely what issues are. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ From rgerhards at hq.adiscon.com Thu Dec 4 16:39:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 16:39:45 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com><4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7F1@grfint2.intern.adiscon.com> Thanks folks for the nice statements. Obviously much appreciated ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gerhardus.Geldenhuis at gta- > travel.com > Sent: Thursday, December 04, 2008 4:09 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] security issue in rsyslog > > I must concur, I am not very active on this mailing list in either > form, > but rsyslog does represent to me what open source was always about. The > openness, speed and intelligence with which bugs/issues are addressed > are exemplary. > > Regards > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of RB > > Sent: 04 December 2008 15:04 > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > Sorry for the hassle. > > > > It's not a hassle when you're being open and honest about issues. > Too > > few projects call an apple an apple, so it's pleasing to be able to > > understand precisely what issues are. > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 4 17:40:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 17:40:43 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7FF@grfint2.intern.adiscon.com> 3.21.8 has now also been replaced by 3.21.9. As with 3.20.2, links remain intact. 3.21.8 has probably never been downloaded, but I thought it is saver to use a new version number, especially as it is a security issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, December 04, 2008 2:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > OK, 3.20.1 is now re-released as 3.20.2 (there were a few > downloads...). > The download link is still correct, it is updated (including the md5sum > ;)). 3.21.8 is pulled and I'll restore it next. > > Sorry for the hassle. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Thursday, December 04, 2008 1:38 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Grrr... One more issue. I noticed that while I resolved some > conflicts > > on the devel branch integration. There is an option that a log > message > > is emitted by rsyslog itself, when a remote machine's message is > > discarded due to no permission. This was requested so that people > know > > when something goes wrong. This is only in the UDP code. > > > > HOWEVER, this is not rate-limited so if someone carries out a heavy > > attack, he can still flood the local disk by these messages. I'll > > change > > it so that the message is emited only once every minute and will then > > re-release what already has been released... > > > > Rainer > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Wednesday, December 03, 2008 11:30 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > The memory leak is now also fixed, I just quickly re-run some TLS > > tests > > > to make sure nothing is broken and it works there, too. > > > > > > Patch (on top of the others): > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > > > d > > > cb8703560c750f04a6370 > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Wednesday, December 03, 2008 10:54 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, I ran this fix through a couple of tests yesterday. It looks > > well > > > > for TLS, too. Note that there is an implication that > $AllowedSender > > > > TCP,... applies to TLS to (because it is TCP). I'd consider this > to > > > be > > > > a > > > > side-effect, but I do not think it is worth fixing. With TLS, > there > > > is > > > > much finer and better control. An issue may only exists if > someone > > > > decides to run non-tls tcp and tls tcp together AND use > > > $AllowedSender. > > > > Workaround in that case is to use the firewall, so I don't > consider > > > it > > > > is worth fixing now. > > > > > > > > Please note that my testing revealed a potential memory leak as > > > > side-effect of the fixes. This could be abused to a remote DoS, > so > > I > > > > will investigate that before releasing. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 6:47 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > > > 3 > > > > > 83fc5969178887d00abfc > > > > > > > > > > Testing takes a bit more of time, I need to set up the test > > > > environment > > > > > for TLS again (looks like it would really pay to have a fixed > > test > > > > > suite > > > > > for all those cases - also the issue here would have never > > > > > occurred...). > > > > > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. > > The > > > > TLS > > > > > part should not be really affected by the problem: there are so > > > much > > > > > better access control features in TLS... > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but > > seems > > > > to > > > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > > > e > > > > > > 18747b55d701e360d5aac > > > > > > > > > > > > Please note that this effectively disables GSS functionality. > > > I'll > > > > > > updated the GSS drivers in the next step. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this > is > > > not > > > > a > > > > > > > trial fix. The design overlooked the situation. In theory, > a > > > > whole > > > > > > new > > > > > > > access control feature would be needed. I am checking out > if > > it > > > > is > > > > > > > possible to "just" enhance the interface. With the current > > > > > netstreams > > > > > > > defined that should be possible. I am tempted to release > the > > > UDP- > > > > > > fixed > > > > > > > version and release the next version with the TCP fix. > > Feedback > > > > > from > > > > > > > packagers is appreciated. The TCP fix may take a day or > two, > > > > > > depending > > > > > > > on how smart a way I find. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > ... and the patch will not work on all of these version, > > due > > > to > > > > > the > > > > > > > > introduction of the netstream driver functionality. > Please > > > note > > > > > > that > > > > > > > > anything older than current v3-stable is outdated, so the > > > > proper > > > > > > way > > > > > > > to > > > > > > > > replace the faulty code is to upgrade to the current v3- > > > stable > > > > > and > > > > > > > > apply > > > > > > > > the patch. I will also release a new v3-stable soon, > > > hopefully > > > > > > today > > > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > I now clarified the affected versions. Affected are > > 3.12.2 > > > > and > > > > > > > above. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > > > 6 > > > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on > > almost > > > > all > > > > > > > other > > > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog- > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > Gerhards > > > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > > [mailto:rsyslog- > > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > > Gerhards > > > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > > > $AllowedSender > > > > > > > > > > directive > > > > > > > > > > > > does not work in all releases. The bug in > question > > > is: > > > > > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, > this > > > can > > > > > > lead > > > > > > > to > > > > > > > > > > > > messages > > > > > > > > > > > > being received from systems that are not > permitted > > > so. > > > > As > > > > > a > > > > > > > > work- > > > > > > > > > > > > around, > > > > > > > > > > > > proper firewalling should be set up on the > > vulnerable > > > > > > hosts. > > > > > > > > > Until > > > > > > > > > > > > further note, I would assume that all versions of > > > > rsyslog > > > > > > are > > > > > > > > > > > affected > > > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Rainer > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Fri Dec 5 15:59:29 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 5 Dec 2008 15:59:29 +0100 Subject: [rsyslog] rsyslog 4.1.2 (v4-devel) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F81C@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 4.1.2, a member of the v4-development branch. Most importantly, this release addresses a security vulnerability that renders the $AllowedSender directive useless. This release has another security fix, which addresses a imudp which emitted a message each time a non-permitted sender tried to send a message. This could have filled the disk. Now it only emits a message once per minute. Further, all fixes and changes from 3.21.9 (beta) and 3.20.2 (stable) have been included in this release. Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-143.phtml Change Log: http://www.rsyslog.com/Article331.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From samuel at dragonboricua.net Mon Dec 8 06:25:30 2008 From: samuel at dragonboricua.net (Elisamuel Resto) Date: Mon, 8 Dec 2008 01:25:30 -0400 Subject: [rsyslog] rsyslog 3.21.8 (v3-beta) released - IMPORTANT SECURITY RELEASE In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> Message-ID: <20081208012530.20571aa8@zeus.dragonboricua.com> On Thu, 4 Dec 2008 13:34:08 +0100, Florian Riedl wrote: > Hi all, > > We have just released rsyslog 3.21.8, a member of the v3-beta branch. > Most importantly, this release addresses a security vulnerability the > renders the $AllowedSender directive useless. This issue has already > been discussed here on the list. In addition to this, the release also > contains all the bug fixes and enhancements from the stable release > 3.20.1. I believe the following is worthy of note in regards to this. On Thu, 4 Dec 2008 17:40:43 +0100, Rainer Gerhards wrote: > 3.21.8 has now also been replaced by 3.21.9. As with 3.20.2, links > remain intact. 3.21.8 has probably never been downloaded, but I thought > it is saver to use a new version number, especially as it is a security > issue. > > Rainer -- Elisamuel Resto Source Mage Tome Lead / http://sourcemage.org GPG ID: 18615F19/1024D / http://simplysam.us From nyerup at one.com Mon Dec 8 09:06:22 2008 From: nyerup at one.com (Jesper Dahl Nyerup) Date: Mon, 08 Dec 2008 09:06:22 +0100 Subject: [rsyslog] Cleanup after ugly shutdown Message-ID: <1228723582.3628.26.camel@iapetus.one.com> Hi all. I recently encountered an incident where rsyslogd died in an "ugly" manner. This was due to no fault of its own, but simply a matter of running out of disk space, because an instance, running with a disk-assisted memory queue, failed to reconnect to the syslog server, and therefore didn't have anywhere to put the log data, except on its own hard drive. When rsyslog dies like this (or in any other way that prevents a decent shutdown), it obviously do not write a .qi-file, indicating where it left off, and thus all the data written to disk is not handled when rsyslogd is started again. This behaviour is correct, as I see it, as at least some of the data written to disk, almost certainly will be inconsistent. But still I would like the opportunity to manually process these data, so that they eventually will be sent to the syslog server. What is the smartest way to properly do this? Has it been considered to let rsyslogd take a command line argument invoking such behaviour, or has someone written a seperate tool to do something like this? Any ideas are welcome. Kind regards, Jesper Nyerup. From rgerhards at hq.adiscon.com Mon Dec 8 10:42:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 10:42:46 +0100 Subject: [rsyslog] analysis of security issue in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F82E@grfint2.intern.adiscon.com> Hi all, I have done a bit of analysis of the issue and blogged about it: http://blog.gerhards.net/2008/12/root-cause-of-security-issue-in-rsyslog .html I hope this is useful information. As I wrote, I will try to focus on getting at basic test suite using DejaGNU (or anything else quickly suggested ;)) up and running. Any help would be appreciated. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 8 10:47:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 10:47:57 +0100 Subject: [rsyslog] make distcheck - multiple configure options possible? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F82F@grfint2.intern.adiscon.com> Hi all, I thought I ask this list before I search further. One thing that is annoying is that from time to time I have some minor nits because not all code pathes are even compiled due to the many configure options that exists (obviously even less are executed). I regularly run make distcheck, but as far as I know, I can only supply one set of configure options to it. Does anybody know a way to have multiple "make distcheck"s running, all with different set of configure options? All feedback is appreciated. Thanks. Rainer From rgerhards at hq.adiscon.com Mon Dec 8 12:33:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 12:33:30 +0100 Subject: [rsyslog] Security Fix for 3.18.5 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F83C@grfint2.intern.adiscon.com> Hi all, Michael Biebl made me aware that Debian Lenny uses rsyslog 3.18.5 and is unable to move to 3.20.x due to release freeze. I have now created a debian_lenny branch in git and backported the fixes to that branch. A quick test succeeded. So if anyone is interested in that version, he or she may pull it from git. If anyone finds a problem with that backport, please let me know. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 8 12:37:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 12:37:42 +0100 Subject: [rsyslog] Cleanup after ugly shutdown In-Reply-To: <1228723582.3628.26.camel@iapetus.one.com> References: <1228723582.3628.26.camel@iapetus.one.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F83D@grfint2.intern.adiscon.com> Hi Jesper, one solution is to use the checkpoint interval. This ensures a queue file is written every n-th time a message is enqueued. This is inside the queue doc. On the "restart queue" option, I agree that it would be useful, but I unfortunately currently neither have time nor funding to implement it. You may want to add this as a feature request to the bugzilla, so that it is not forgotten if time arises... Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jesper Dahl Nyerup > Sent: Monday, December 08, 2008 9:06 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Cleanup after ugly shutdown > > Hi all. > > I recently encountered an incident where rsyslogd died in an "ugly" > manner. This was due to no fault of its own, but simply a matter of > running out of disk space, because an instance, running with a > disk-assisted memory queue, failed to reconnect to the syslog server, > and therefore didn't have anywhere to put the log data, except on its > own hard drive. > > When rsyslog dies like this (or in any other way that prevents a decent > shutdown), it obviously do not write a .qi-file, indicating where it > left off, and thus all the data written to disk is not handled when > rsyslogd is started again. This behaviour is correct, as I see it, as > at > least some of the data written to disk, almost certainly will be > inconsistent. > > But still I would like the opportunity to manually process these data, > so that they eventually will be sent to the syslog server. What is the > smartest way to properly do this? Has it been considered to let > rsyslogd > take a command line argument invoking such behaviour, or has someone > written a seperate tool to do something like this? > > Any ideas are welcome. > > > Kind regards, > > Jesper Nyerup. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Fri Dec 12 07:48:48 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Fri, 12 Dec 2008 07:48:48 +0100 Subject: [rsyslog] Disk error when using rsyslog Message-ID: <49420950.2030904@net-m.de> Hi Rainer, We're using rsyslog(3.20.0) as our central logging server and clients. We experienced an DISK ERROR on server last month. At that time, we were using TCP to transport logs from client to server. And we also setup the configuration just like [1]. But unfortunately our central logging server got DISK error for one hour. So we lost logfiles of that period of time. I've a look at doc [1] carefully, I guess "RELIABLE" only means when server got offline or rsyslogd on it isn't running, then clients will save logs in buffer or write to a file on disk. If server is still online and rsyslogd is running, but with IO/Error or Disk Full, then client will still transfer logs to server even with RELP, coz I guess RELP only protects logs could be transferred via network successfully, it doesn't care the logs are written successfully to file on server. Am I right? So I guess if we need to prevent this, we need do some work on server? Do we have some "directives" options that we could transfer logs to a failover server if local disk fails or buffer in memory before disk got corrected? [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html Thanks, -- Patrick Shen Operations Engineer From hks.private at gmail.com Fri Dec 12 16:09:42 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 12 Dec 2008 10:09:42 -0500 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <49420950.2030904@net-m.de> References: <49420950.2030904@net-m.de> Message-ID: On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen wrote: > Hi Rainer, > > We're using rsyslog(3.20.0) as our central logging server and clients. > > We experienced an DISK ERROR on server last month. At that time, we were > using TCP to transport logs from client to server. And we also setup the > configuration just like [1]. But unfortunately our central logging > server got DISK error for one hour. So we lost logfiles of that period > of time. > > I've a look at doc [1] carefully, I guess "RELIABLE" only means when > server got offline or rsyslogd on it isn't running, then clients will > save logs in buffer or write to a file on disk. If server is still > online and rsyslogd is running, but with IO/Error or Disk Full, then > client will still transfer logs to server even with RELP, coz I guess > RELP only protects logs could be transferred via network successfully, > it doesn't care the logs are written successfully to file on server. Am > I right? Yes. RELP Is a protocol for the reliable exchange of event logs over a network. What the destination daemon does once it has the logs is no concern of the client's. > So I guess if we need to prevent this, we need do some work on server? > Do we have some "directives" options that we could transfer logs to a > failover server if local disk fails or buffer in memory before disk got > corrected? Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer -HKS > [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html > > Thanks, > > -- > Patrick Shen > Operations Engineer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Dec 12 16:16:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Dec 2008 16:16:21 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: References: <49420950.2030904@net-m.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> I seem to have overlooked the initial message (spam filter maybe...). HKS is right (thx), but I think this looks like a bug in that the output write does not care about the write failure (what it should). The output writer is pretty old legacy code, so that's quite possible. I'll look into it ASAP, but I got a new machine (hopefully fast enough to disply some troubles) today and currently I am happy that at least mail does work again (so far it's a mess). So... some time next week ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Friday, December 12, 2008 4:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Disk error when using rsyslog > > On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen > wrote: > > Hi Rainer, > > > > We're using rsyslog(3.20.0) as our central logging server and > clients. > > > > We experienced an DISK ERROR on server last month. At that time, we > were > > using TCP to transport logs from client to server. And we also setup > the > > configuration just like [1]. But unfortunately our central logging > > server got DISK error for one hour. So we lost logfiles of that > period > > of time. > > > > I've a look at doc [1] carefully, I guess "RELIABLE" only means when > > server got offline or rsyslogd on it isn't running, then clients will > > save logs in buffer or write to a file on disk. If server is still > > online and rsyslogd is running, but with IO/Error or Disk Full, then > > client will still transfer logs to server even with RELP, coz I guess > > RELP only protects logs could be transferred via network > successfully, > > it doesn't care the logs are written successfully to file on server. > Am > > I right? > > Yes. RELP Is a protocol for the reliable exchange of event logs over a > network. What the destination daemon does once it has the logs is no > concern of the client's. > > > > So I guess if we need to prevent this, we need do some work on > server? > > Do we have some "directives" options that we could transfer logs to a > > failover server if local disk fails or buffer in memory before disk > got > > corrected? > > Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > -HKS > > > [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html > > > > Thanks, > > > > -- > > Patrick Shen > > Operations Engineer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Sun Dec 14 04:46:38 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Sun, 14 Dec 2008 04:46:38 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: References: <49420950.2030904@net-m.de> Message-ID: <4944819E.90101@net-m.de> Hi HKS, Thanks for your good advice. It's helpful. Patrick (private) HKS wrote: > On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen wrote: >> Hi Rainer, >> >> We're using rsyslog(3.20.0) as our central logging server and clients. >> >> We experienced an DISK ERROR on server last month. At that time, we were >> using TCP to transport logs from client to server. And we also setup the >> configuration just like [1]. But unfortunately our central logging >> server got DISK error for one hour. So we lost logfiles of that period >> of time. >> >> I've a look at doc [1] carefully, I guess "RELIABLE" only means when >> server got offline or rsyslogd on it isn't running, then clients will >> save logs in buffer or write to a file on disk. If server is still >> online and rsyslogd is running, but with IO/Error or Disk Full, then >> client will still transfer logs to server even with RELP, coz I guess >> RELP only protects logs could be transferred via network successfully, >> it doesn't care the logs are written successfully to file on server. Am >> I right? > > Yes. RELP Is a protocol for the reliable exchange of event logs over a > network. What the destination daemon does once it has the logs is no > concern of the client's. > > >> So I guess if we need to prevent this, we need do some work on server? >> Do we have some "directives" options that we could transfer logs to a >> failover server if local disk fails or buffer in memory before disk got >> corrected? > > Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > -HKS > >> [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html >> >> Thanks, >> >> -- >> Patrick Shen >> Operations Engineer >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Sun Dec 14 04:56:16 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Sun, 14 Dec 2008 04:56:16 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> Message-ID: <494483E0.3030702@net-m.de> Hi Rainer, Rainer Gerhards wrote: > I seem to have overlooked the initial message (spam filter maybe...). Bad luck for my mail account :-( > HKS is right (thx), but I think this looks like a bug in that the output > write does not care about the write failure (what it should). The output > writer is pretty old legacy code, so that's quite possible. I'll look > into it ASAP, but I got a new machine (hopefully fast enough to disply > some troubles) today and currently I am happy that at least mail does > work again (so far it's a mess). So... some time next week ;) Quite appreciate if you have a look at write failure. I think it's quite Enterprise demand feature :-) Many thanks, Patrick From rgerhards at hq.adiscon.com Mon Dec 15 12:03:53 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 15 Dec 2008 12:03:53 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <494483E0.3030702@net-m.de> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> <494483E0.3030702@net-m.de> Message-ID: <1229339033.2482.3.camel@localhost.localdomain> On Sun, 2008-12-14 at 04:56 +0100, Patrick Shen wrote: > Hi Rainer, > > Rainer Gerhards wrote: > > I seem to have overlooked the initial message (spam filter maybe...). > > Bad luck for my mail account :-( > > > HKS is right (thx), but I think this looks like a bug in that the output > > write does not care about the write failure (what it should). The output > > writer is pretty old legacy code, so that's quite possible. I'll look > > into it ASAP, but I got a new machine (hopefully fast enough to disply > > some troubles) today and currently I am happy that at least mail does > > work again (so far it's a mess). So... some time next week ;) > > Quite appreciate if you have a look at write failure. I think it's quite > Enterprise demand feature :-) I have now verified that the code (by intension) ignores write errors. That, of course, is legacy from a long gone era. However, I need to think a bit about how to handle this most intelligently. The problem is partial writes. Maybe I just try to write a LF after a failure and, if that succeeds, simply continue. This results in a partial record begin written and then the same record being "duplicated" (actually, the partial part being duplicated). Does anyone have a suggestion on how to best handle such a case? Or I could try to write what could not yet be written. Maybe this is better, but it wont' be able to survive a daemon restart... Feedback appreciated. Rainer From david at lang.hm Tue Dec 16 10:23:37 2008 From: david at lang.hm (david at lang.hm) Date: Tue, 16 Dec 2008 01:23:37 -0800 (PST) Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <1229339033.2482.3.camel@localhost.localdomain> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> <494483E0.3030702@net-m.de> <1229339033.2482.3.camel@localhost.localdomain> Message-ID: On Mon, 15 Dec 2008, Rainer Gerhards wrote: > On Sun, 2008-12-14 at 04:56 +0100, Patrick Shen wrote: >> Hi Rainer, >> >> Rainer Gerhards wrote: >>> I seem to have overlooked the initial message (spam filter maybe...). >> >> Bad luck for my mail account :-( >> >>> HKS is right (thx), but I think this looks like a bug in that the output >>> write does not care about the write failure (what it should). The output >>> writer is pretty old legacy code, so that's quite possible. I'll look >>> into it ASAP, but I got a new machine (hopefully fast enough to disply >>> some troubles) today and currently I am happy that at least mail does >>> work again (so far it's a mess). So... some time next week ;) >> >> Quite appreciate if you have a look at write failure. I think it's quite >> Enterprise demand feature :-) > > I have now verified that the code (by intension) ignores write errors. > That, of course, is legacy from a long gone era. However, I need to > think a bit about how to handle this most intelligently. The problem is > partial writes. Maybe I just try to write a LF after a failure and, if > that succeeds, simply continue. This results in a partial record begin > written and then the same record being "duplicated" (actually, the > partial part being duplicated). > > Does anyone have a suggestion on how to best handle such a case? Or I > could try to write what could not yet be written. Maybe this is better, > but it wont' be able to survive a daemon restart... > > Feedback appreciated. this sounds like a smaller portion of the problem we were discussing when we were talking about de-queuing multiple messages. this approach would work, but if you know you have a file you could also truncate the file to the end of the previous record (the beginning of the record you are trying to write). David Lang From william at blocket.se Tue Dec 16 17:39:53 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Tue, 16 Dec 2008 17:39:53 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs Message-ID: Hi, I don't know if this is an intended feature or not, but when CreateDirs is set to off, rsyslog can no longer create files (time stamp file names). Lets say you have a template for writing files: / log//.log, this will let anybody to create directories in / log when CreateDirs is on. This may cause a lot of directories and can be abused by regular users. I've patched 2.0.2 to separate file and directory creation here: https://bugzilla.redhat.com/attachment.cgi?id=327100 And steps to reproduce this can be found here: https://bugzilla.redhat.com/show_bug.cgi?id=473419 Comments anyone? William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From rgerhards at hq.adiscon.com Thu Dec 18 12:01:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 12:01:25 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: References: Message-ID: <1229598085.7471.2.camel@localhost.localdomain> Hi William, thanks for the bug report and the patch. I have now looked into the issue and could reproduce the situation. The patch obviously resolves it. There is one thing, though: I do not fully understand why you have patched syslogd.c (RS_RET_ERR). I would appreciate if you could provide a bit more insight into the problem you intend to solve. So far, I can not see why this part of the patch is actually needed. I'll integrate the omfile part now. I will also see that I apply this to the other official branches. Thanks, Rainer On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > Hi, > > I don't know if this is an intended feature or not, but when > CreateDirs is set to off, rsyslog can no longer create files (time > stamp file names). Lets say you have a template for writing files: / > log//.log, this will let anybody to create directories in / > log when CreateDirs is on. This may cause a lot of directories and can > be abused by regular users. > > I've patched 2.0.2 to separate file and directory creation here: > https://bugzilla.redhat.com/attachment.cgi?id=327100 > > And steps to reproduce this can be found here: > https://bugzilla.redhat.com/show_bug.cgi?id=473419 > > Comments anyone? > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Dec 18 13:03:48 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 18 Dec 2008 13:03:48 +0100 Subject: [rsyslog] rsyslog 4.1.3 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F8F9@grfint2.intern.adiscon.com> Hi all, rsyslog 4.1.3, a member of the development branch, has been released today. Most importantly, it offers a bugfix for some situations where the imudp input module could run in a tight loop. The release also offers enhanced functionality, among others a work-around to handle the invalid syslog/tcp framing used by NetScreen. There are two more configuration-options which can be used for fine-tuning functionality (see changelog for details). We would like to express my gratitude to BlinkMind, Inc. (http://www.blinkmind.com), who sponsored the development of the $PreserveFQDN configuration option. Download http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-145.phtml Changelog http://www.rsyslog.com/Article335.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From william at blocket.se Thu Dec 18 14:35:00 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 14:35:00 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229598085.7471.2.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> Message-ID: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Hi, Lets see if I can get this right: My modification in prepareFile() will return with (pData->fd = -1) if the log file can't be created. In prepareDynFile() we run prepareFile() and return with -1 if pData- >fd is set to -1. In writeFile() we run prepareDynFile() and return RS_RET_ERR if prepareDynFile() is not returned with 0. writeFile() is wrapped in doAction(). doAction() is exectued in fprintlog() where RS_RET_ERR never will be catched. I discard the log message and sets the error flag to tell the "msg repeated"-check to not log this message ("msg repeated" is executed before we try to open the file if the message content match the previous message). I tried without this catch in the first attempt, but I could see the message stuck in the loop, every action to rsyslog tried to open the file. This and some traffic volume caused rsyslog to hang (and use a lot of i/o). William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > Hi William, > > thanks for the bug report and the patch. I have now looked into the > issue and could reproduce the situation. The patch obviously resolves > it. > > There is one thing, though: I do not fully understand why you have > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > provide > a bit more insight into the problem you intend to solve. So far, I can > not see why this part of the patch is actually needed. > > I'll integrate the omfile part now. I will also see that I apply > this to > the other official branches. > > Thanks, > Rainer > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >> Hi, >> >> I don't know if this is an intended feature or not, but when >> CreateDirs is set to off, rsyslog can no longer create files (time >> stamp file names). Lets say you have a template for writing files: / >> log//.log, this will let anybody to create directories >> in / >> log when CreateDirs is on. This may cause a lot of directories and >> can >> be abused by regular users. >> >> I've patched 2.0.2 to separate file and directory creation here: >> https://bugzilla.redhat.com/attachment.cgi?id=327100 >> >> And steps to reproduce this can be found here: >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >> >> Comments anyone? >> >> >> William Tis?ter >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From william at blocket.se Thu Dec 18 15:22:53 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 15:22:53 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Message-ID: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> I've debugged some more, this only happens when you trigger the repeat- loop to the missing directory. E.g: # logger -t non_existent_dir -p local0.crit test # logger -t non_existent_dir -p local0.crit test # logger -t existing_dir -p local0.crit test Debug output from last command: Filter: check for property 'programname' (value 'existing_dir') NOT regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE 1092925760: Called fprintlog, logging to builtin-file (CritByDay) 1092925760: Called fprintlog, logging to builtin-file (CritByDay) Filter: check for property 'msg' (value ' test') contains '(CRIT)': FALSE 1092925760: Called fprintlog, logging to builtin-file (DirByTagFileByDay) 1092925760: Called logerr, msg: Could not open dynamic file '/log/ non_existent_dir/2008-12-18.log' - discarding message 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding message: No such file or directory 1092925760: Message has legacy syslog format. 1092925760: EnqueueMsg signaled condition (0) 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. 1092925760: Lone worker is running... 1092925760: msg repeated 1 times, 10 sec of 30. Filter: check for property 'syslogfacility-text' (value 'syslog') NOT isequal 'local0': TRUE 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: queue EMPTY, waiting for next message. William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > Hi, > > Lets see if I can get this right: > > My modification in prepareFile() will return with (pData->fd = -1) > if the log file can't be created. > In prepareDynFile() we run prepareFile() and return with -1 if pData- > >fd is set to -1. > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > prepareDynFile() is not returned with 0. > > writeFile() is wrapped in doAction(). > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > catched. I discard the log message and sets the error flag to tell > the "msg repeated"-check to not log this message ("msg repeated" is > executed before we try to open the file if the message content match > the previous message). > > I tried without this catch in the first attempt, but I could see the > message stuck in the loop, every action to rsyslog tried to open the > file. This and some traffic volume caused rsyslog to hang (and use a > lot of i/o). > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > >> Hi William, >> >> thanks for the bug report and the patch. I have now looked into the >> issue and could reproduce the situation. The patch obviously resolves >> it. >> >> There is one thing, though: I do not fully understand why you have >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >> provide >> a bit more insight into the problem you intend to solve. So far, I >> can >> not see why this part of the patch is actually needed. >> >> I'll integrate the omfile part now. I will also see that I apply >> this to >> the other official branches. >> >> Thanks, >> Rainer >> >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>> Hi, >>> >>> I don't know if this is an intended feature or not, but when >>> CreateDirs is set to off, rsyslog can no longer create files (time >>> stamp file names). Lets say you have a template for writing files: / >>> log//.log, this will let anybody to create directories >>> in / >>> log when CreateDirs is on. This may cause a lot of directories and >>> can >>> be abused by regular users. >>> >>> I've patched 2.0.2 to separate file and directory creation here: >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>> >>> And steps to reproduce this can be found here: >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>> >>> Comments anyone? >>> >>> >>> William Tis?ter >>> >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>> http://www.blocket.se/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 18 15:58:50 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 15:58:50 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Message-ID: <1229612330.12594.8.camel@localhost.localdomain> Hi William, I had another look at the code. I am not picky, I just want to make sure I understand the failure scenario. The v3 versions have a different approach at this, and I need to be sure I understand the issue so that I can check the newer releases for it. more inline below... On Thu, 2008-12-18 at 14:35 +0100, William Tis?ter wrote: > Hi, > > Lets see if I can get this right: > > My modification in prepareFile() will return with (pData->fd = -1) if > the log file can't be created. > In prepareDynFile() we run prepareFile() and return with -1 if pData- > >fd is set to -1. > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > prepareDynFile() is not returned with 0. > > writeFile() is wrapped in doAction(). > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > catched. Well.. kind of. I do not check for a specific error state, but I check if all went well (if iRet == RS_RE_OK). Only then the previous count is reset, otherwise it is left as is. An output module may return a large number of errors, so it is not sufficient to check for a specific error case. That's also why I am trying to understand the issue you still see. I am sure it can occur under other circumstances, too (as I said, there are various error codes). > I discard the log message and sets the error flag to tell the > "msg repeated"-check to not log this message ("msg repeated" is > executed before we try to open the file if the message content match > the previous message). ... but this is done via fprintlog(), so the logic to try open the file is still included. > > I tried without this catch in the first attempt, but I could see the > message stuck in the loop, As of my understanding, it should not be the very same message, because that is discarded. It can be the next message, which of course will trigger the "last message repeated n times" text. > every action to rsyslog tried to open the > file. This and some traffic volume caused rsyslog to hang (and use a > lot of i/o). Do you have a procedure to reproduce that? Would be most interesting. As a side-Note, I think there is a memory leak in the patch, you will probably want to correct it. The return in line 3392 does not do the necessary cleanup (the return further UP is a different case, see its comments). Use "ABORT_FINALIZE(RS_RET_DISCARDMSG);" instead, it will invoke the finalizer, which will free some string space. > Rainer > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > Hi William, > > > > thanks for the bug report and the patch. I have now looked into the > > issue and could reproduce the situation. The patch obviously > resolves > > it. > > > > There is one thing, though: I do not fully understand why you have > > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > > provide > > a bit more insight into the problem you intend to solve. So far, I > can > > not see why this part of the patch is actually needed. > > > > I'll integrate the omfile part now. I will also see that I apply > > this to > > the other official branches. > > > > Thanks, > > Rainer > > > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >> Hi, > >> > >> I don't know if this is an intended feature or not, but when > >> CreateDirs is set to off, rsyslog can no longer create files (time > >> stamp file names). Lets say you have a template for writing > files: / > >> log//.log, this will let anybody to create directories > >> in / > >> log when CreateDirs is on. This may cause a lot of directories and > >> can > >> be abused by regular users. > >> > >> I've patched 2.0.2 to separate file and directory creation here: > >> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >> > >> And steps to reproduce this can be found here: > >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >> > >> Comments anyone? > >> > >> > >> William Tis?ter > >> > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >> http://www.blocket.se/ > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:03:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:03:24 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229612330.12594.8.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <1229612330.12594.8.camel@localhost.localdomain> Message-ID: <1229612604.12594.11.camel@localhost.localdomain> oops, one thing that occured to me just now. RS_RET_DISCARDMSG is probably not what you want. This totally *discards* the message from the queue. This state was introduced for the discard action. For example, if your config ist *.* ?dynafile *.* file1 *.* file2 and you get in issue during ?dynafile processing, the message will never be written to file1 and file2. So it is better to leave it at RS_RET_ERR. This, however, brings us close to the code as it currently is (except for the additional flag, which I am trying to understand ;)) Rainer On Thu, 2008-12-18 at 15:58 +0100, Rainer Gerhards wrote: > Hi William, > > I had another look at the code. I am not picky, I just want to make > sure > I understand the failure scenario. The v3 versions have a different > approach at this, and I need to be sure I understand the issue so that > I > can check the newer releases for it. > > more inline below... > > On Thu, 2008-12-18 at 14:35 +0100, William Tis?ter wrote: > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > if > > the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > > catched. > > Well.. kind of. I do not check for a specific error state, but I check > if all went well (if iRet == RS_RE_OK). Only then the previous count > is > reset, otherwise it is left as is. An output module may return a large > number of errors, so it is not sufficient to check for a specific > error > case. That's also why I am trying to understand the issue you still > see. > I am sure it can occur under other circumstances, too (as I said, > there > are various error codes). > > > I discard the log message and sets the error flag to tell the > > "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content match > > the previous message). > > ... but this is done via fprintlog(), so the logic to try open the > file > is still included. > > > > I tried without this catch in the first attempt, but I could see the > > message stuck in the loop, > > As of my understanding, it should not be the very same message, > because > that is discarded. It can be the next message, which of course will > trigger the "last message repeated n times" text. > > > every action to rsyslog tried to open the > > file. This and some traffic volume caused rsyslog to hang (and use a > > lot of i/o). > > Do you have a procedure to reproduce that? Would be most interesting. > > As a side-Note, I think there is a memory leak in the patch, you will > probably want to correct it. The return in line 3392 does not do the > necessary cleanup (the return further UP is a different case, see its > comments). Use "ABORT_FINALIZE(RS_RET_DISCARDMSG);" instead, it will > invoke the finalizer, which will free some string space. > > > > Rainer > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > > > Hi William, > > > > > > thanks for the bug report and the patch. I have now looked into > the > > > issue and could reproduce the situation. The patch obviously > > resolves > > > it. > > > > > > There is one thing, though: I do not fully understand why you have > > > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > > > provide > > > a bit more insight into the problem you intend to solve. So far, I > > can > > > not see why this part of the patch is actually needed. > > > > > > I'll integrate the omfile part now. I will also see that I apply > > > this to > > > the other official branches. > > > > > > Thanks, > > > Rainer > > > > > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > > >> Hi, > > >> > > >> I don't know if this is an intended feature or not, but when > > >> CreateDirs is set to off, rsyslog can no longer create files > (time > > >> stamp file names). Lets say you have a template for writing > > files: / > > >> log//.log, this will let anybody to create directories > > >> in / > > >> log when CreateDirs is on. This may cause a lot of directories > and > > >> can > > >> be abused by regular users. > > >> > > >> I've patched 2.0.2 to separate file and directory creation here: > > >> https://bugzilla.redhat.com/attachment.cgi?id=327100 > > >> > > >> And steps to reproduce this can be found here: > > >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > > >> > > >> Comments anyone? > > >> > > >> > > >> William Tis?ter > > >> > > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > >> http://www.blocket.se/ > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:04:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:04:19 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> Message-ID: <1229612659.12594.12.camel@localhost.localdomain> The spam filter caught this message - sorry. I'll test this scenario. Rainer On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > I've debugged some more, this only happens when you trigger the > repeat- > loop to the missing directory. > > E.g: > > # logger -t non_existent_dir -p local0.crit test > # logger -t non_existent_dir -p local0.crit test > # logger -t existing_dir -p local0.crit test > > Debug output from last command: > > Filter: check for property 'programname' (value 'existing_dir') NOT > regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > Filter: check for property 'msg' (value ' test') contains '(CRIT)': > FALSE > 1092925760: Called fprintlog, logging to builtin-file > (DirByTagFileByDay) > 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > non_existent_dir/2008-12-18.log' - discarding message > 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding > message: No such file or directory > 1092925760: Message has legacy syslog format. > 1092925760: EnqueueMsg signaled condition (0) > 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. > 1092925760: Lone worker is running... > 1092925760: msg repeated 1 times, 10 sec of 30. > Filter: check for property 'syslogfacility-text' (value 'syslog') NOT > isequal 'local0': TRUE > 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: > queue EMPTY, waiting for next message. > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > > if the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will > be > > catched. I discard the log message and sets the error flag to tell > > the "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content > match > > the previous message). > > > > I tried without this catch in the first attempt, but I could see > the > > message stuck in the loop, every action to rsyslog tried to open > the > > file. This and some traffic volume caused rsyslog to hang (and use > a > > lot of i/o). > > > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > >> Hi William, > >> > >> thanks for the bug report and the patch. I have now looked into the > >> issue and could reproduce the situation. The patch obviously > resolves > >> it. > >> > >> There is one thing, though: I do not fully understand why you have > >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >> provide > >> a bit more insight into the problem you intend to solve. So far, I > >> can > >> not see why this part of the patch is actually needed. > >> > >> I'll integrate the omfile part now. I will also see that I apply > >> this to > >> the other official branches. > >> > >> Thanks, > >> Rainer > >> > >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>> Hi, > >>> > >>> I don't know if this is an intended feature or not, but when > >>> CreateDirs is set to off, rsyslog can no longer create files (time > >>> stamp file names). Lets say you have a template for writing > files: / > >>> log//.log, this will let anybody to create directories > >>> in / > >>> log when CreateDirs is on. This may cause a lot of directories > and > >>> can > >>> be abused by regular users. > >>> > >>> I've patched 2.0.2 to separate file and directory creation here: > >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>> > >>> And steps to reproduce this can be found here: > >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>> > >>> Comments anyone? > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:12:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:12:34 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> Message-ID: <1229613154.12594.13.camel@localhost.localdomain> mmhhhh... I can not reproduce the issue with my state of the code. Can you verify that it still occurs? A git snapshot is available here: http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz Thanks, Rainer On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > I've debugged some more, this only happens when you trigger the > repeat- > loop to the missing directory. > > E.g: > > # logger -t non_existent_dir -p local0.crit test > # logger -t non_existent_dir -p local0.crit test > # logger -t existing_dir -p local0.crit test > > Debug output from last command: > > Filter: check for property 'programname' (value 'existing_dir') NOT > regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > Filter: check for property 'msg' (value ' test') contains '(CRIT)': > FALSE > 1092925760: Called fprintlog, logging to builtin-file > (DirByTagFileByDay) > 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > non_existent_dir/2008-12-18.log' - discarding message > 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding > message: No such file or directory > 1092925760: Message has legacy syslog format. > 1092925760: EnqueueMsg signaled condition (0) > 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. > 1092925760: Lone worker is running... > 1092925760: msg repeated 1 times, 10 sec of 30. > Filter: check for property 'syslogfacility-text' (value 'syslog') NOT > isequal 'local0': TRUE > 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: > queue EMPTY, waiting for next message. > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > > if the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will > be > > catched. I discard the log message and sets the error flag to tell > > the "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content > match > > the previous message). > > > > I tried without this catch in the first attempt, but I could see > the > > message stuck in the loop, every action to rsyslog tried to open > the > > file. This and some traffic volume caused rsyslog to hang (and use > a > > lot of i/o). > > > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > >> Hi William, > >> > >> thanks for the bug report and the patch. I have now looked into the > >> issue and could reproduce the situation. The patch obviously > resolves > >> it. > >> > >> There is one thing, though: I do not fully understand why you have > >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >> provide > >> a bit more insight into the problem you intend to solve. So far, I > >> can > >> not see why this part of the patch is actually needed. > >> > >> I'll integrate the omfile part now. I will also see that I apply > >> this to > >> the other official branches. > >> > >> Thanks, > >> Rainer > >> > >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>> Hi, > >>> > >>> I don't know if this is an intended feature or not, but when > >>> CreateDirs is set to off, rsyslog can no longer create files (time > >>> stamp file names). Lets say you have a template for writing > files: / > >>> log//.log, this will let anybody to create directories > >>> in / > >>> log when CreateDirs is on. This may cause a lot of directories > and > >>> can > >>> be abused by regular users. > >>> > >>> I've patched 2.0.2 to separate file and directory creation here: > >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>> > >>> And steps to reproduce this can be found here: > >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>> > >>> Comments anyone? > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From william at blocket.se Thu Dec 18 16:35:08 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 16:35:08 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229613154.12594.13.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> Message-ID: <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> Yes, I could reproduce my error, the same way as described earlier. My /etc/rsyslog.conf: $template DirByTagFileByDay,"/log/%programname%/%timegenerated: 1:10:date-rfc3339%.log" $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" # Discard all but local0 :syslogfacility-text, !isequal, "local0" ~ $umask 0000 $FileCreateMode 0644 $DirCreateMode 0755 $CreateDirs off # Discard strange tags :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ badtags.log :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ # Collect all crits in one log local0.crit ?CritByDay :msg, contains, "(CRIT)" ?CritByDay # All local0's buffered to their own dirs local0.* -?DirByTagFileByDay / William On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: > mmhhhh... I can not reproduce the issue with my state of the code. Can > you verify that it still occurs? A git snapshot is available here: > > http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz > > Thanks, > Rainer > > On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: >> I've debugged some more, this only happens when you trigger the >> repeat- >> loop to the missing directory. >> >> E.g: >> >> # logger -t non_existent_dir -p local0.crit test >> # logger -t non_existent_dir -p local0.crit test >> # logger -t existing_dir -p local0.crit test >> >> Debug output from last command: >> >> Filter: check for property 'programname' (value 'existing_dir') NOT >> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >> Filter: check for property 'msg' (value ' test') contains '(CRIT)': >> FALSE >> 1092925760: Called fprintlog, logging to builtin-file >> (DirByTagFileByDay) >> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ >> non_existent_dir/2008-12-18.log' - discarding message >> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not >> open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding >> message: No such file or directory >> 1092925760: Message has legacy syslog format. >> 1092925760: EnqueueMsg signaled condition (0) >> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. >> 1092925760: Lone worker is running... >> 1092925760: msg repeated 1 times, 10 sec of 30. >> Filter: check for property 'syslogfacility-text' (value 'syslog') NOT >> isequal 'local0': TRUE >> 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: >> queue EMPTY, waiting for next message. >> >> >> William Tis?ter >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> >> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: >> >>> Hi, >>> >>> Lets see if I can get this right: >>> >>> My modification in prepareFile() will return with (pData->fd = -1) >>> if the log file can't be created. >>> In prepareDynFile() we run prepareFile() and return with -1 if >> pData- >>>> fd is set to -1. >>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if >>> prepareDynFile() is not returned with 0. >>> >>> writeFile() is wrapped in doAction(). >>> >>> doAction() is exectued in fprintlog() where RS_RET_ERR never will >> be >>> catched. I discard the log message and sets the error flag to tell >>> the "msg repeated"-check to not log this message ("msg repeated" is >>> executed before we try to open the file if the message content >> match >>> the previous message). >>> >>> I tried without this catch in the first attempt, but I could see >> the >>> message stuck in the loop, every action to rsyslog tried to open >> the >>> file. This and some traffic volume caused rsyslog to hang (and use >> a >>> lot of i/o). >>> >>> >>> William Tis?ter >>> >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>> http://www.blocket.se/ >>> >>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: >>> >>>> Hi William, >>>> >>>> thanks for the bug report and the patch. I have now looked into the >>>> issue and could reproduce the situation. The patch obviously >> resolves >>>> it. >>>> >>>> There is one thing, though: I do not fully understand why you have >>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >>>> provide >>>> a bit more insight into the problem you intend to solve. So far, I >>>> can >>>> not see why this part of the patch is actually needed. >>>> >>>> I'll integrate the omfile part now. I will also see that I apply >>>> this to >>>> the other official branches. >>>> >>>> Thanks, >>>> Rainer >>>> >>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>>>> Hi, >>>>> >>>>> I don't know if this is an intended feature or not, but when >>>>> CreateDirs is set to off, rsyslog can no longer create files (time >>>>> stamp file names). Lets say you have a template for writing >> files: / >>>>> log//.log, this will let anybody to create directories >>>>> in / >>>>> log when CreateDirs is on. This may cause a lot of directories >> and >>>>> can >>>>> be abused by regular users. >>>>> >>>>> I've patched 2.0.2 to separate file and directory creation here: >>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>>>> >>>>> And steps to reproduce this can be found here: >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>>>> >>>>> Comments anyone? >>>>> >>>>> >>>>> William Tis?ter >>>>> >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>> http://www.blocket.se/ >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >> >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com William Tis?ter Cell: +46 (0)76 3377182 Mail: william at blocket.se Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From rgerhards at hq.adiscon.com Thu Dec 18 16:44:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:44:04 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> Message-ID: <1229615044.12594.14.camel@localhost.localdomain> would it be possible that you mail me (privately) a full debug log from such a run. Not sure if there is too much confidential data in it... Rainer On Thu, 2008-12-18 at 16:35 +0100, William Tis?ter wrote: > Yes, I could reproduce my error, the same way as described earlier. > > My /etc/rsyslog.conf: > > $template DirByTagFileByDay,"/log/%programname%/%timegenerated: > 1:10:date-rfc3339%.log" > $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" > > # Discard all but local0 > :syslogfacility-text, !isequal, "local0" ~ > > $umask 0000 > $FileCreateMode 0644 > $DirCreateMode 0755 > $CreateDirs off > > # Discard strange tags > :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ > badtags.log > :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ > > # Collect all crits in one log > local0.crit ?CritByDay > :msg, contains, "(CRIT)" ?CritByDay > > # All local0's buffered to their own dirs > local0.* -?DirByTagFileByDay > > > / William > > On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: > > > mmhhhh... I can not reproduce the issue with my state of the code. > Can > > you verify that it still occurs? A git snapshot is available here: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz > > > > Thanks, > > Rainer > > > > On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > >> I've debugged some more, this only happens when you trigger the > >> repeat- > >> loop to the missing directory. > >> > >> E.g: > >> > >> # logger -t non_existent_dir -p local0.crit test > >> # logger -t non_existent_dir -p local0.crit test > >> # logger -t existing_dir -p local0.crit test > >> > >> Debug output from last command: > >> > >> Filter: check for property 'programname' (value 'existing_dir') NOT > >> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > >> Filter: check for property 'msg' (value ' test') contains '(CRIT)': > >> FALSE > >> 1092925760: Called fprintlog, logging to builtin-file > >> (DirByTagFileByDay) > >> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > >> non_existent_dir/2008-12-18.log' - discarding message > >> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > >> open dynamic file '/log/non_existent_dir/2008-12-18.log' - > discarding > >> message: No such file or directory > >> 1092925760: Message has legacy syslog format. > >> 1092925760: EnqueueMsg signaled condition (0) > >> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from > dynaCache. > >> 1092925760: Lone worker is running... > >> 1092925760: msg repeated 1 times, 10 sec of 30. > >> Filter: check for property 'syslogfacility-text' (value 'syslog') > NOT > >> isequal 'local0': TRUE > >> 1092925760: Called fprintlog, logging to > builtin-discardsingleWorker: > >> queue EMPTY, waiting for next message. > >> > >> > >> William Tis?ter > >> > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >> http://www.blocket.se/ > >> > >> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > >> > >>> Hi, > >>> > >>> Lets see if I can get this right: > >>> > >>> My modification in prepareFile() will return with (pData->fd = -1) > >>> if the log file can't be created. > >>> In prepareDynFile() we run prepareFile() and return with -1 if > >> pData- > >>>> fd is set to -1. > >>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if > >>> prepareDynFile() is not returned with 0. > >>> > >>> writeFile() is wrapped in doAction(). > >>> > >>> doAction() is exectued in fprintlog() where RS_RET_ERR never will > >> be > >>> catched. I discard the log message and sets the error flag to tell > >>> the "msg repeated"-check to not log this message ("msg repeated" > is > >>> executed before we try to open the file if the message content > >> match > >>> the previous message). > >>> > >>> I tried without this catch in the first attempt, but I could see > >> the > >>> message stuck in the loop, every action to rsyslog tried to open > >> the > >>> file. This and some traffic volume caused rsyslog to hang (and use > >> a > >>> lot of i/o). > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> > >>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > >>> > >>>> Hi William, > >>>> > >>>> thanks for the bug report and the patch. I have now looked into > the > >>>> issue and could reproduce the situation. The patch obviously > >> resolves > >>>> it. > >>>> > >>>> There is one thing, though: I do not fully understand why you > have > >>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >>>> provide > >>>> a bit more insight into the problem you intend to solve. So far, > I > >>>> can > >>>> not see why this part of the patch is actually needed. > >>>> > >>>> I'll integrate the omfile part now. I will also see that I apply > >>>> this to > >>>> the other official branches. > >>>> > >>>> Thanks, > >>>> Rainer > >>>> > >>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>>>> Hi, > >>>>> > >>>>> I don't know if this is an intended feature or not, but when > >>>>> CreateDirs is set to off, rsyslog can no longer create files > (time > >>>>> stamp file names). Lets say you have a template for writing > >> files: / > >>>>> log//.log, this will let anybody to create > directories > >>>>> in / > >>>>> log when CreateDirs is on. This may cause a lot of directories > >> and > >>>>> can > >>>>> be abused by regular users. > >>>>> > >>>>> I've patched 2.0.2 to separate file and directory creation here: > >>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>>>> > >>>>> And steps to reproduce this can be found here: > >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>>>> > >>>>> Comments anyone? > >>>>> > >>>>> > >>>>> William Tis?ter > >>>>> > >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>>>> http://www.blocket.se/ > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >> > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > William Tis?ter > > Cell: +46 (0)76 3377182 > Mail: william at blocket.se > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From william at blocket.se Thu Dec 18 16:47:28 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 16:47:28 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229615044.12594.14.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> <1229615044.12594.14.camel@localhost.localdomain> Message-ID: * Sent full debug log to Rainer * On Dec 18, 2008, at 4:44 PM, Rainer Gerhards wrote: > would it be possible that you mail me (privately) a full debug log > from > such a run. Not sure if there is too much confidential data in it... > > Rainer > > On Thu, 2008-12-18 at 16:35 +0100, William Tis?ter wrote: >> Yes, I could reproduce my error, the same way as described earlier. >> >> My /etc/rsyslog.conf: >> >> $template DirByTagFileByDay,"/log/%programname%/%timegenerated: >> 1:10:date-rfc3339%.log" >> $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" >> >> # Discard all but local0 >> :syslogfacility-text, !isequal, "local0" ~ >> >> $umask 0000 >> $FileCreateMode 0644 >> $DirCreateMode 0755 >> $CreateDirs off >> >> # Discard strange tags >> :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ >> badtags.log >> :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ >> >> # Collect all crits in one log >> local0.crit ?CritByDay >> :msg, contains, "(CRIT)" ?CritByDay >> >> # All local0's buffered to their own dirs >> local0.* -?DirByTagFileByDay >> >> >> / William >> >> On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: >> >>> mmhhhh... I can not reproduce the issue with my state of the code. >> Can >>> you verify that it still occurs? A git snapshot is available here: >>> >>> >> http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz >>> >>> Thanks, >>> Rainer >>> >>> On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: >>>> I've debugged some more, this only happens when you trigger the >>>> repeat- >>>> loop to the missing directory. >>>> >>>> E.g: >>>> >>>> # logger -t non_existent_dir -p local0.crit test >>>> # logger -t non_existent_dir -p local0.crit test >>>> # logger -t existing_dir -p local0.crit test >>>> >>>> Debug output from last command: >>>> >>>> Filter: check for property 'programname' (value 'existing_dir') NOT >>>> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE >>>> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >>>> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >>>> Filter: check for property 'msg' (value ' test') contains '(CRIT)': >>>> FALSE >>>> 1092925760: Called fprintlog, logging to builtin-file >>>> (DirByTagFileByDay) >>>> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ >>>> non_existent_dir/2008-12-18.log' - discarding message >>>> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not >>>> open dynamic file '/log/non_existent_dir/2008-12-18.log' - >> discarding >>>> message: No such file or directory >>>> 1092925760: Message has legacy syslog format. >>>> 1092925760: EnqueueMsg signaled condition (0) >>>> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from >> dynaCache. >>>> 1092925760: Lone worker is running... >>>> 1092925760: msg repeated 1 times, 10 sec of 30. >>>> Filter: check for property 'syslogfacility-text' (value 'syslog') >> NOT >>>> isequal 'local0': TRUE >>>> 1092925760: Called fprintlog, logging to >> builtin-discardsingleWorker: >>>> queue EMPTY, waiting for next message. >>>> >>>> >>>> William Tis?ter >>>> >>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>> http://www.blocket.se/ >>>> >>>> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: >>>> >>>>> Hi, >>>>> >>>>> Lets see if I can get this right: >>>>> >>>>> My modification in prepareFile() will return with (pData->fd = -1) >>>>> if the log file can't be created. >>>>> In prepareDynFile() we run prepareFile() and return with -1 if >>>> pData- >>>>>> fd is set to -1. >>>>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if >>>>> prepareDynFile() is not returned with 0. >>>>> >>>>> writeFile() is wrapped in doAction(). >>>>> >>>>> doAction() is exectued in fprintlog() where RS_RET_ERR never will >>>> be >>>>> catched. I discard the log message and sets the error flag to tell >>>>> the "msg repeated"-check to not log this message ("msg repeated" >> is >>>>> executed before we try to open the file if the message content >>>> match >>>>> the previous message). >>>>> >>>>> I tried without this catch in the first attempt, but I could see >>>> the >>>>> message stuck in the loop, every action to rsyslog tried to open >>>> the >>>>> file. This and some traffic volume caused rsyslog to hang (and use >>>> a >>>>> lot of i/o). >>>>> >>>>> >>>>> William Tis?ter >>>>> >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>> http://www.blocket.se/ >>>>> >>>>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: >>>>> >>>>>> Hi William, >>>>>> >>>>>> thanks for the bug report and the patch. I have now looked into >> the >>>>>> issue and could reproduce the situation. The patch obviously >>>> resolves >>>>>> it. >>>>>> >>>>>> There is one thing, though: I do not fully understand why you >> have >>>>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >>>>>> provide >>>>>> a bit more insight into the problem you intend to solve. So far, >> I >>>>>> can >>>>>> not see why this part of the patch is actually needed. >>>>>> >>>>>> I'll integrate the omfile part now. I will also see that I apply >>>>>> this to >>>>>> the other official branches. >>>>>> >>>>>> Thanks, >>>>>> Rainer >>>>>> >>>>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I don't know if this is an intended feature or not, but when >>>>>>> CreateDirs is set to off, rsyslog can no longer create files >> (time >>>>>>> stamp file names). Lets say you have a template for writing >>>> files: / >>>>>>> log//.log, this will let anybody to create >> directories >>>>>>> in / >>>>>>> log when CreateDirs is on. This may cause a lot of directories >>>> and >>>>>>> can >>>>>>> be abused by regular users. >>>>>>> >>>>>>> I've patched 2.0.2 to separate file and directory creation here: >>>>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>>>>>> >>>>>>> And steps to reproduce this can be found here: >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>>>>>> >>>>>>> Comments anyone? >>>>>>> >>>>>>> >>>>>>> William Tis?ter >>>>>>> >>>>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>>>> http://www.blocket.se/ >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> >> William Tis?ter >> >> Cell: +46 (0)76 3377182 >> Mail: william at blocket.se >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com William Tis?ter Cell: +46 (0)76 3377182 Mail: william at blocket.se Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From bakers at web-ster.com Thu Dec 18 20:59:27 2008 From: bakers at web-ster.com (Scott Baker) Date: Thu, 18 Dec 2008 11:59:27 -0800 Subject: [rsyslog] Troubleshooting missing log entries Message-ID: <494AAB9F.9060005@web-ster.com> I have the following entry in my rsyslog conf, to match entries based on IP address. Somehow it's not matching any entries. # Switches $FileCreateMode 0644 :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea :FROMHOST, isequal, "65.182.224.202" -?switches :FROMHOST, isequal, "66.206.80.60" -?switches If I do a tcpdump I see syslog hitting the box, it's just rsyslog isn't handling it right. 11:58:20.722867 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 121 11:58:23.962613 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 130 11:58:41.242621 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 108 11:58:45.874064 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 130 This box gets about 500 lines of syslog a minute so I can't really turn on debug. How else can I troubleshoot this? This is a Fedora 8 box running: rsyslog-2.0.2-3.fc8 - Scott From david at lang.hm Fri Dec 19 06:38:38 2008 From: david at lang.hm (david at lang.hm) Date: Thu, 18 Dec 2008 21:38:38 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog Message-ID: with messages appearing out-of-order the 'last message repeated X times' is pretty useless. I think I remember reading about an option to disable this, but another work-around to the problem is to change the output so that it becomes 'last message repeated X times %msg%', so you can see (most, if not all) of the message being repeated in the line telling you that it was repeated. David Lang From rgerhards at hq.adiscon.com Thu Dec 18 20:01:47 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:01:47 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: Message-ID: <1229626907.12594.19.camel@localhost.localdomain> Hi David, On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > with messages appearing out-of-order the 'last message repeated X times' > is pretty useless. Not really. Please note that form the perspective of the output module, messages do not appear out of order. The output module receives a stream of messages, and the "last message repeated x times" logic works on that stream. So no matter if messages are re-ordered by async queues (or UDP or whatever), the "last ... times" correctly reflects the way things were handed to the output module. But I concur that this feature, in its current state, is very questionable. I've talked about this on the mailing list quite some time, I think there is also at least one blog post about it. > > I think I remember reading about an option to disable this, but another > work-around to the problem is to change the output so that it becomes > 'last message repeated X times %msg%', so you can see (most, if not all) > of the message being repeated in the line telling you that it was > repeated. As I said, the message in front of this message is either another repeat message or the message that is being repeated. So you can always trace back to what was repeated (but it is painful). Given the feedback I received on this feature (use cases), it should probably (and v4 would be an appropriate version to do so) be redisigned to be a rate-limiting feature on the input side. That would pretty much simplify code, too. Rainer From david at lang.hm Fri Dec 19 12:46:10 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 03:46:10 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: <1229626907.12594.19.camel@localhost.localdomain> References: <1229626907.12594.19.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > Hi David, > > On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: >> with messages appearing out-of-order the 'last message repeated X times' >> is pretty useless. > > Not really. Please note that form the perspective of the output module, > messages do not appear out of order. The output module receives a stream > of messages, and the "last message repeated x times" logic works on that > stream. So no matter if messages are re-ordered by async queues (or UDP > or whatever), the "last ... times" correctly reflects the way things > were handed to the output module. but if the output module is then relaying to another system, that other system (or the transport between them) can also re-order the messages > But I concur that this feature, in its current state, is very > questionable. I've talked about this on the mailing list quite some > time, I think there is also at least one blog post about it. > >> >> I think I remember reading about an option to disable this, but another >> work-around to the problem is to change the output so that it becomes >> 'last message repeated X times %msg%', so you can see (most, if not all) >> of the message being repeated in the line telling you that it was >> repeated. > > As I said, the message in front of this message is either another repeat > message or the message that is being repeated. So you can always trace > back to what was repeated (but it is painful). > > Given the feedback I received on this feature (use cases), it should > probably (and v4 would be an appropriate version to do so) be redisigned > to be a rate-limiting feature on the input side. That would pretty much > simplify code, too. but since things can be re-ordered after the input module is done with them (multiple threads pulling from the queue, or relaying) it is still useful for the 'message repeated' message to have more info about what it is referring to. David Lang > Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Thu Dec 18 20:15:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:15:51 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain> Message-ID: <1229627751.12594.23.camel@localhost.localdomain> On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: > On Thu, 18 Dec 2008, Rainer Gerhards wrote: > > > Hi David, > > > > On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > >> with messages appearing out-of-order the 'last message repeated X times' > >> is pretty useless. > > > > Not really. Please note that form the perspective of the output module, > > messages do not appear out of order. The output module receives a stream > > of messages, and the "last message repeated x times" logic works on that > > stream. So no matter if messages are re-ordered by async queues (or UDP > > or whatever), the "last ... times" correctly reflects the way things > > were handed to the output module. > > but if the output module is then relaying to another system, that other > system (or the transport between them) can also re-order the messages ah, ok, remote scenario. Of course, that can happen. But that's "just" one use case where it is problematic. > > But I concur that this feature, in its current state, is very > > questionable. I've talked about this on the mailing list quite some > > time, I think there is also at least one blog post about it. > > > >> > >> I think I remember reading about an option to disable this, but another > >> work-around to the problem is to change the output so that it becomes > >> 'last message repeated X times %msg%', so you can see (most, if not all) > >> of the message being repeated in the line telling you that it was > >> repeated. > > > > As I said, the message in front of this message is either another repeat > > message or the message that is being repeated. So you can always trace > > back to what was repeated (but it is painful). > > > > Given the feedback I received on this feature (use cases), it should > > probably (and v4 would be an appropriate version to do so) be redisigned > > to be a rate-limiting feature on the input side. That would pretty much > > simplify code, too. > > but since things can be re-ordered after the input module is done with > them (multiple threads pulling from the queue, or relaying) it is still > useful for the 'message repeated' message to have more info about what it > is referring to. That makes sense, at least for scenarios where it is expected behaviour. For others, it would probably break analysers. So it needs to be optional. And probably on a per-action basis. I'll see if it is sufficiently simple, but the whole "last message repeated" thing is broken anyhow (IMO), I don't like the idea to invest any more than maybe an hour into that feature, especially in the light that the whole idea must be replaced in the not so distant future... Rainer > > David Lang > > > Rainer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 18 20:19:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:19:54 +0100 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <494AAB9F.9060005@web-ster.com> References: <494AAB9F.9060005@web-ster.com> Message-ID: <1229627994.12594.26.camel@localhost.localdomain> On Thu, 2008-12-18 at 11:59 -0800, Scott Baker wrote: > I have the following entry in my rsyslog conf, to match entries based on IP > address. Somehow it's not matching any entries. > > # Switches > $FileCreateMode 0644 > :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea > :FROMHOST, isequal, "65.182.224.202" -?switches > :FROMHOST, isequal, "66.206.80.60" -?switches > > If I do a tcpdump I see syslog hitting the box, it's just rsyslog isn't > handling it right. > > 11:58:20.722867 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 121 > 11:58:23.962613 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 130 > 11:58:41.242621 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 108 > 11:58:45.874064 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 130 > > This box gets about 500 lines of syslog a minute so I can't really turn on > debug. How else can I troubleshoot this? This is a Fedora 8 box running: > rsyslog-2.0.2-3.fc8 I'd still go for debug mode. You don't need to run it very long. We just need to see how a few of these messages are fully processed. A proper test setup would be to start up in debug mode with the network cable pulled, then plug it in for a second or two, then unplug it again. Once rsyslogd is finished processing, stop it. That should lead to useful info in the debug log. Oh - and are you sure that fromhost has the proper IP addresses? If not 100% sure, verify it by putting something like '%FROMHOST%' into a debug template (note that there is also FROMHOST-IP, which will have the IP address no matter if names are resolved or not). HTH, Rainer > > - Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Fri Dec 19 13:03:40 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 04:03:40 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: <1229627751.12594.23.camel@localhost.localdomain> References: <1229626907.12594.19.camel@localhost.localdomain> <1229627751.12594.23.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: >> >>> Hi David, >>> >>> On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: >>>> with messages appearing out-of-order the 'last message repeated X times' >>>> is pretty useless. >>> >>> Not really. Please note that form the perspective of the output module, >>> messages do not appear out of order. The output module receives a stream >>> of messages, and the "last message repeated x times" logic works on that >>> stream. So no matter if messages are re-ordered by async queues (or UDP >>> or whatever), the "last ... times" correctly reflects the way things >>> were handed to the output module. >> >> but if the output module is then relaying to another system, that other >> system (or the transport between them) can also re-order the messages > > ah, ok, remote scenario. Of course, that can happen. But that's "just" > one use case where it is problematic. > >>> But I concur that this feature, in its current state, is very >>> questionable. I've talked about this on the mailing list quite some >>> time, I think there is also at least one blog post about it. >>> >>>> >>>> I think I remember reading about an option to disable this, but another >>>> work-around to the problem is to change the output so that it becomes >>>> 'last message repeated X times %msg%', so you can see (most, if not all) >>>> of the message being repeated in the line telling you that it was >>>> repeated. >>> >>> As I said, the message in front of this message is either another repeat >>> message or the message that is being repeated. So you can always trace >>> back to what was repeated (but it is painful). >>> >>> Given the feedback I received on this feature (use cases), it should >>> probably (and v4 would be an appropriate version to do so) be redisigned >>> to be a rate-limiting feature on the input side. That would pretty much >>> simplify code, too. >> >> but since things can be re-ordered after the input module is done with >> them (multiple threads pulling from the queue, or relaying) it is still >> useful for the 'message repeated' message to have more info about what it >> is referring to. > > That makes sense, at least for scenarios where it is expected behaviour. > For others, it would probably break analysers. So it needs to be > optional. And probably on a per-action basis. I'll see if it is > sufficiently simple, but the whole "last message repeated" thing is > broken anyhow (IMO), I don't like the idea to invest any more than maybe > an hour into that feature, especially in the light that the whole idea > must be replaced in the not so distant future... it's definantly a pain for analysers (a _lot_ of them really only look at one line at a time), that's why I started tweaking this on sysklogd, and while in theory just disabling it is the right answer, the ability to take a flood of inbound messages and reduce them to a much smaller number can save your logging system when things go wrong I'll watch to see what happens. David Lang From rgerhards at hq.adiscon.com Fri Dec 19 12:05:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:05:24 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain><1229627751.12594.23.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F915@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, December 19, 2008 1:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] suggested tweak to rsyslog > it's definantly a pain for analysers Full ack > (a _lot_ of them really only look > at > one line at a time), that's why I started tweaking this on sysklogd, > and > while in theory just disabling it is the right answer, the ability to > take > a flood of inbound messages and reduce them to a much smaller number > can > save your logging system when things go wrong > > I'll watch to see what happens. I just dug out a the relevant link, please follow also to the mailing list discussion: http://kb.monitorware.com/post5789.html Rainer From david at lang.hm Fri Dec 19 13:07:53 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 04:07:53 -0800 (PST) Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <1229627994.12594.26.camel@localhost.localdomain> References: <494AAB9F.9060005@web-ster.com> <1229627994.12594.26.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Thu, 2008-12-18 at 11:59 -0800, Scott Baker wrote: >> I have the following entry in my rsyslog conf, to match entries based on IP >> address. Somehow it's not matching any entries. >> >> # Switches >> $FileCreateMode 0644 >> :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea >> :FROMHOST, isequal, "65.182.224.202" -?switches >> :FROMHOST, isequal, "66.206.80.60" -?switches > > Oh - and are you sure that fromhost has the proper IP addresses? If not > 100% sure, verify it by putting something like '%FROMHOST%' into a debug > template (note that there is also FROMHOST-IP, which will have the IP > address no matter if names are resolved or not). I was seeing some issues where the fromhost was not getting set properly, I'll have to go back and dig up the details, but I think I was seeing it use the localhost as the fromhost and putting the real fromhost information in the message. I found it by creating an output format that I could tweak and playing with it to see what was actually showing up in the various parameters. David Lang From rgerhards at hq.adiscon.com Fri Dec 19 12:24:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:24:22 +0100 Subject: [rsyslog] Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? In-Reply-To: References: Message-ID: <1229685862.12594.33.camel@localhost.localdomain> Hi Craig, I don't see this in my lab, everything works correct. May it be that you did not include some of the (automatic) initial messages in your picture? In any case, a full debug log would be useful to track this down. After my sig, you find the relevant part of my lab debug FYI. Relevant part of rsyslog.conf: $ActionExecOnlyEveryNthTime 3 :msg, contains, "test" -/rsyslog/logfile Commands issued: $ logger 1 $ logger 2 $ logger 3 $ logger 4 $ logger 5 $ logger 6 And the output file had: 2008-12-19T12:17:44.548207+01:00 rf10up rger: test 3 2008-12-19T12:17:53.522265+01:00 rf10up rger: test 6 Note that in this setup I made sure that no other messages than those with the string "test" went into the log file. Otherwise, the counters would have been affected by some startup messages. Rainer 5455.946426832:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5460.051992823:imuxsock.c: Message from UNIX socket: #4 5460.066209819:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:40 rger: test 1 5460.066408445:imuxsock.c: Message has legacy syslog format. 5460.095581070:imuxsock.c: main queue: entry added, size now 1 entries 5460.096265506:imuxsock.c: wtpAdviseMaxWorkers signals busy 5460.096810820:imuxsock.c: main queue: EnqueueMsg advised worker start 5460.097417035:main queue:Reg/w0: main queue: entering rate limiter 5460.097797246:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5460.098493974:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 1') contains 'test': TRUE 5460.105295873:main queue:Reg/w0: action 0x4c64438 passed 1 times to execution - less than neded - discarding 5460.109592735:main queue:Reg/w0: main queue: entering rate limiter 5460.110192245:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5460.115576662:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5462.317878786:imuxsock.c: Message from UNIX socket: #4 5462.318410132:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:42 rger: test 2 5462.318585850:imuxsock.c: Message has legacy syslog format. 5462.319012436:imuxsock.c: main queue: entry added, size now 1 entries 5462.319271125:imuxsock.c: wtpAdviseMaxWorkers signals busy 5462.319537636:imuxsock.c: main queue: EnqueueMsg advised worker start 5462.319871473:main queue:Reg/w0: main queue: entering rate limiter 5462.320174580:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5462.320470145:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 2') contains 'test': TRUE 5462.321508812:main queue:Reg/w0: action 0x4c64438 passed 2 times to execution - less than neded - discarding 5462.321825608:main queue:Reg/w0: main queue: entering rate limiter 5462.322078710:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5462.322350529:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5464.547937451:imuxsock.c: Message from UNIX socket: #4 5464.548420747:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:44 rger: test 3 5464.548613227:imuxsock.c: Message has legacy syslog format. 5464.549042885:imuxsock.c: main queue: entry added, size now 1 entries 5464.549287885:imuxsock.c: wtpAdviseMaxWorkers signals busy 5464.549571158:imuxsock.c: main queue: EnqueueMsg advised worker start 5464.549888793:main queue:Reg/w0: main queue: entering rate limiter 5464.550211176:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5464.550500036:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 3') contains 'test': TRUE 5464.551797950:main queue:Reg/w0: Called action, logging to builtin-file 5464.605776776:main queue:Reg/w0: (/rsyslog/logfile) 5464.606371817:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5464.627358305:main queue:Reg/w0: main queue: entering rate limiter 5464.628009218:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5469.671232071:imuxsock.c: Message from UNIX socket: #4 5469.671772915:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:49 rger: test 4 5469.671950031:imuxsock.c: Message has legacy syslog format. 5469.672340578:imuxsock.c: main queue: entry added, size now 1 entries 5469.672597591:imuxsock.c: wtpAdviseMaxWorkers signals busy 5469.672931428:imuxsock.c: main queue: EnqueueMsg advised worker start 5469.673234256:main queue:Reg/w0: main queue: entering rate limiter 5469.673563065:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5469.673893829:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 4') contains 'test': TRUE 5469.674471829:main queue:Reg/w0: action 0x4c64438 passed 1 times to execution - less than neded - discarding 5469.674788066:main queue:Reg/w0: main queue: entering rate limiter 5469.675042006:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5469.675316618:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5471.443424737:imuxsock.c: Message from UNIX socket: #4 5471.443973403:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:51 rger: test 5 5471.444148283:imuxsock.c: Message has legacy syslog format. 5471.444550564:imuxsock.c: main queue: entry added, size now 1 entries 5471.444845291:imuxsock.c: wtpAdviseMaxWorkers signals busy 5471.445187788:imuxsock.c: main queue: EnqueueMsg advised worker start 5471.445489499:main queue:Reg/w0: main queue: entering rate limiter 5471.445842612:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5471.446121136:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 5') contains 'test': TRUE 5471.446527886:main queue:Reg/w0: action 0x4c64438 passed 2 times to execution - less than neded - discarding 5471.446837140:main queue:Reg/w0: main queue: entering rate limiter 5471.447091079:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5471.447366530:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5473.522029412:imuxsock.c: Message from UNIX socket: #4 5473.522550422:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:53 rger: test 6 5473.522766368:imuxsock.c: Message has legacy syslog format. 5473.523162224:imuxsock.c: main queue: entry added, size now 1 entries 5473.523404989:imuxsock.c: wtpAdviseMaxWorkers signals busy 5473.523727931:imuxsock.c: main queue: EnqueueMsg advised worker start 5473.524011204:main queue:Reg/w0: main queue: entering rate limiter 5473.524328279:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5473.524621330:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 6') contains 'test': TRUE 5473.525016627:main queue:Reg/w0: Called action, logging to builtin-file 5473.525803589:main queue:Reg/w0: (/rsyslog/logfile) 5473.526170949:main queue:Reg/w0: main queue: entering rate limiter 5473.526423213:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5473.526753139:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 On Thu, 2008-12-18 at 12:51 -0500, Becker, Craig (DOB) wrote: > The action $ActionExecOnlyEveryNthTime is not sending the Nth message > but the first message, then the Nth + 1 message. For example, if I > set the $ActionExecOnlyEveryNthTime 3 The 1st, 4th, 7th, etc. message > are sent instead of the 3rd, 6th, 9th, etc. Has this bug (if it is a > bug) been fixed in a later version? > > > > I?m enjoying learning this product and appreciate all the hard work > that has gone into it. > > > > Thanks, > > > > Craig E. Becker > IT Specialist II > System Administrator > Division of the Budget > 518-473-1322 > craig.becker at budget.state.ny.us > > > > > > > > HTML: > ______________________________________________________________________ > This e-mail, including any attachments, may be confidential, > privileged or otherwise legally protected. If you have received this > e-mail in error, or from someone who was not authorized to send it to > you, do not disseminate, copy or otherwise use this e-mail or its > attachments. Please notify the sender immediately if you have received > this e-mail by mistake, and delete it from your system. From rgerhards at hq.adiscon.com Fri Dec 19 12:57:16 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:57:16 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain><1229627751.12594.23.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F916@grfint2.intern.adiscon.com> David, one thing I can do rather quickly. Maybe it's good enough. I've done a tester, which lacks proper configuration, but I would appreciate feedback on it: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=a185665be4cf6997525 89d81ef6e396dd61f68b6 Details in git commit comment. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, December 19, 2008 1:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] suggested tweak to rsyslog > > On Thu, 18 Dec 2008, Rainer Gerhards wrote: > > > On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: > >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: > >> > >>> Hi David, > >>> > >>> On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > >>>> with messages appearing out-of-order the 'last message repeated X > times' > >>>> is pretty useless. > >>> > >>> Not really. Please note that form the perspective of the output > module, > >>> messages do not appear out of order. The output module receives a > stream > >>> of messages, and the "last message repeated x times" logic works on > that > >>> stream. So no matter if messages are re-ordered by async queues (or > UDP > >>> or whatever), the "last ... times" correctly reflects the way > things > >>> were handed to the output module. > >> > >> but if the output module is then relaying to another system, that > other > >> system (or the transport between them) can also re-order the > messages > > > > ah, ok, remote scenario. Of course, that can happen. But that's > "just" > > one use case where it is problematic. > > > >>> But I concur that this feature, in its current state, is very > >>> questionable. I've talked about this on the mailing list quite some > >>> time, I think there is also at least one blog post about it. > >>> > >>>> > >>>> I think I remember reading about an option to disable this, but > another > >>>> work-around to the problem is to change the output so that it > becomes > >>>> 'last message repeated X times %msg%', so you can see (most, if > not all) > >>>> of the message being repeated in the line telling you that it was > >>>> repeated. > >>> > >>> As I said, the message in front of this message is either another > repeat > >>> message or the message that is being repeated. So you can always > trace > >>> back to what was repeated (but it is painful). > >>> > >>> Given the feedback I received on this feature (use cases), it > should > >>> probably (and v4 would be an appropriate version to do so) be > redisigned > >>> to be a rate-limiting feature on the input side. That would pretty > much > >>> simplify code, too. > >> > >> but since things can be re-ordered after the input module is done > with > >> them (multiple threads pulling from the queue, or relaying) it is > still > >> useful for the 'message repeated' message to have more info about > what it > >> is referring to. > > > > That makes sense, at least for scenarios where it is expected > behaviour. > > For others, it would probably break analysers. So it needs to be > > optional. And probably on a per-action basis. I'll see if it is > > sufficiently simple, but the whole "last message repeated" thing is > > broken anyhow (IMO), I don't like the idea to invest any more than > maybe > > an hour into that feature, especially in the light that the whole > idea > > must be replaced in the not so distant future... > > it's definantly a pain for analysers (a _lot_ of them really only look > at > one line at a time), that's why I started tweaking this on sysklogd, > and > while in theory just disabling it is the right answer, the ability to > take > a flood of inbound messages and reduce them to a much smaller number > can > save your logging system when things go wrong > > I'll watch to see what happens. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From bakers at web-ster.com Fri Dec 19 17:38:41 2008 From: bakers at web-ster.com (Scott Baker) Date: Fri, 19 Dec 2008 08:38:41 -0800 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <1229627994.12594.26.camel@localhost.localdomain> References: <494AAB9F.9060005@web-ster.com> <1229627994.12594.26.camel@localhost.localdomain> Message-ID: <494BCE11.7000300@web-ster.com> Rainer Gerhards wrote: > I'd still go for debug mode. You don't need to run it very long. We just > need to see how a few of these messages are fully processed. A proper > test setup would be to start up in debug mode with the network cable > pulled, then plug it in for a second or two, then unplug it again. Once > rsyslogd is finished processing, stop it. That should lead to useful > info in the debug log. > > Oh - and are you sure that fromhost has the proper IP addresses? If not > 100% sure, verify it by putting something like '%FROMHOST%' into a debug > template (note that there is also FROMHOST-IP, which will have the IP > address no matter if names are resolved or not). I like the debug template idea, that's genius. Is there a way to have a bunch of filters to catch assorted things, and then an "everything leftover" filter? ------------------------------------------------------------------------ # Mail servers log to their special section $FileCreateMode 0644 :FROMHOST, isequal, "magenta" -?magic-mail :FROMHOST, isequal, "cyan" -?magic-mail :FROMHOST, isequal, "orange" -?magic-mail # Firewalls :FROMHOST, isequal, "yin" -?firewall :FROMHOST, isequal, "yang" -?firewall # Everything that didn't get caught by one of the above filters (I have no idea what the syntax would be) From rgerhards at hq.adiscon.com Fri Dec 19 17:40:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 17:40:32 +0100 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <494BCE11.7000300@web-ster.com> References: <494AAB9F.9060005@web-ster.com><1229627994.12594.26.camel@localhost.localdomain> <494BCE11.7000300@web-ster.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F924@grfint2.intern.adiscon.com> Without verification, but should work: # Mail servers log to their special section $FileCreateMode 0644 :FROMHOST, isequal, "magenta" -?magic-mail & ~ :FROMHOST, isequal, "cyan" -?magic-mail & ~ :FROMHOST, isequal, "orange" -?magic-mail & ~ # Firewalls :FROMHOST, isequal, "yin" -?firewall & ~ :FROMHOST, isequal, "yang" -?firewall & ~ *.* /var/log/catchrest & ~ discards the message after it is written to the file in question. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Baker > Sent: Friday, December 19, 2008 5:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] Troubleshooting missing log entries > > Rainer Gerhards wrote: > > I'd still go for debug mode. You don't need to run it very long. We > just > > need to see how a few of these messages are fully processed. A proper > > test setup would be to start up in debug mode with the network cable > > pulled, then plug it in for a second or two, then unplug it again. > Once > > rsyslogd is finished processing, stop it. That should lead to useful > > info in the debug log. > > > > Oh - and are you sure that fromhost has the proper IP addresses? If > not > > 100% sure, verify it by putting something like '%FROMHOST%' into a > debug > > template (note that there is also FROMHOST-IP, which will have the IP > > address no matter if names are resolved or not). > > > I like the debug template idea, that's genius. Is there a way to have a > bunch of filters to catch assorted things, and then an "everything > leftover" filter? > > ----------------------------------------------------------------------- > - > > # Mail servers log to their special section > $FileCreateMode 0644 > :FROMHOST, isequal, "magenta" -?magic-mail > :FROMHOST, isequal, "cyan" -?magic-mail > :FROMHOST, isequal, "orange" -?magic-mail > > # Firewalls > :FROMHOST, isequal, "yin" -?firewall > :FROMHOST, isequal, "yang" -?firewall > > # Everything that didn't get caught by one of the above filters > (I have no idea what the syntax would be) > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Dec 19 17:52:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 17:52:26 +0100 Subject: [rsyslog] Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? In-Reply-To: References: <1229685862.12594.33.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F926@grfint2.intern.adiscon.com> --sorry, my mailer didn't reply to the list for some reason-- Sounds like a too-old version. Jepp, you mention 3.21.2 in the subject, I think the directives were introduced some time later. Rainer > -----Original Message----- > From: Becker, Craig (DOB) [mailto:Craig.Becker at budget.state.ny.us] > Sent: Friday, December 19, 2008 5:34 PM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: RE: Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? > > Thanks for the help. When I start Rsyslog in Debug mode, I get the > following errors: > > > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 404 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 405 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 698 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 699 > > These errors correspond to the actions: > > $ActionExecOnlyEveryNthTime 50 #Lines 404 & 698 > $ActionExecOnlyEveryNthTimeTimeout 1800 #Lines 405 & 699 > > These are the Modules loaded: > > #______All Modules loaded here__________ > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging > (e.g. via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by > rklogd) > $ModLoad ommail # This mod enables sending E-mail on alerts > $ModLoad imudp.so # provides UDP syslog reception > $ModLoad imtcp.so # provides TCP syslog reception > > I tried to add all the modules I have access to without fixing the > issue. Is there a module missing or should I look to another version? > The strange thing is that the command actually works after the first > email is sent (i.e. The 1st, 51st, 101st messages are sent). > > Any help would be appreciated. > > Thanks again, > Craig > > > > -----Original Message----- > From: Rainer Gerhards [mailto:rgerhards at hq.adiscon.com] > Sent: Friday, December 19, 2008 6:24 AM > To: rsyslog at lists.adiscon.com > Cc: Becker, Craig (DOB) > Subject: Re: Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? > > Hi Craig, > > I don't see this in my lab, everything works correct. May it be that > you > did not include some of the (automatic) initial messages in your > picture? In any case, a full debug log would be useful to track this > down. After my sig, you find the relevant part of my lab debug FYI. > > Relevant part of rsyslog.conf: > $ActionExecOnlyEveryNthTime 3 > :msg, contains, "test" -/rsyslog/logfile > > Commands issued: > $ logger 1 > $ logger 2 > $ logger 3 > $ logger 4 > $ logger 5 > $ logger 6 > > And the output file had: > 2008-12-19T12:17:44.548207+01:00 rf10up rger: test 3 > 2008-12-19T12:17:53.522265+01:00 rf10up rger: test 6 > > Note that in this setup I made sure that no other messages than those > with the string "test" went into the log file. Otherwise, the counters > would have been affected by some startup messages. > > Rainer > > 5455.946426832:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5460.051992823:imuxsock.c: Message from UNIX socket: #4 > 5460.066209819:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:40 rger: test 1 > 5460.066408445:imuxsock.c: Message has legacy syslog format. > 5460.095581070:imuxsock.c: main queue: entry added, size now 1 entries > 5460.096265506:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5460.096810820:imuxsock.c: main queue: EnqueueMsg advised worker start > 5460.097417035:main queue:Reg/w0: main queue: entering rate limiter > 5460.097797246:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5460.098493974:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 1') contains 'test': TRUE > 5460.105295873:main queue:Reg/w0: action 0x4c64438 passed 1 times to > execution - less than neded - discarding > 5460.109592735:main queue:Reg/w0: main queue: entering rate limiter > 5460.110192245:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5460.115576662:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5462.317878786:imuxsock.c: Message from UNIX socket: #4 > 5462.318410132:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:42 rger: test 2 > 5462.318585850:imuxsock.c: Message has legacy syslog format. > 5462.319012436:imuxsock.c: main queue: entry added, size now 1 entries > 5462.319271125:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5462.319537636:imuxsock.c: main queue: EnqueueMsg advised worker start > 5462.319871473:main queue:Reg/w0: main queue: entering rate limiter > 5462.320174580:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5462.320470145:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 2') contains 'test': TRUE > 5462.321508812:main queue:Reg/w0: action 0x4c64438 passed 2 times to > execution - less than neded - discarding > 5462.321825608:main queue:Reg/w0: main queue: entering rate limiter > 5462.322078710:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5462.322350529:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5464.547937451:imuxsock.c: Message from UNIX socket: #4 > 5464.548420747:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:44 rger: test 3 > 5464.548613227:imuxsock.c: Message has legacy syslog format. > 5464.549042885:imuxsock.c: main queue: entry added, size now 1 entries > 5464.549287885:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5464.549571158:imuxsock.c: main queue: EnqueueMsg advised worker start > 5464.549888793:main queue:Reg/w0: main queue: entering rate limiter > 5464.550211176:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5464.550500036:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 3') contains 'test': TRUE > 5464.551797950:main queue:Reg/w0: Called action, logging to builtin- > file > 5464.605776776:main queue:Reg/w0: (/rsyslog/logfile) > 5464.606371817:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5464.627358305:main queue:Reg/w0: main queue: entering rate limiter > 5464.628009218:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5469.671232071:imuxsock.c: Message from UNIX socket: #4 > 5469.671772915:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:49 rger: test 4 > 5469.671950031:imuxsock.c: Message has legacy syslog format. > 5469.672340578:imuxsock.c: main queue: entry added, size now 1 entries > 5469.672597591:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5469.672931428:imuxsock.c: main queue: EnqueueMsg advised worker start > 5469.673234256:main queue:Reg/w0: main queue: entering rate limiter > 5469.673563065:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5469.673893829:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 4') contains 'test': TRUE > 5469.674471829:main queue:Reg/w0: action 0x4c64438 passed 1 times to > execution - less than neded - discarding > 5469.674788066:main queue:Reg/w0: main queue: entering rate limiter > 5469.675042006:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5469.675316618:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5471.443424737:imuxsock.c: Message from UNIX socket: #4 > 5471.443973403:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:51 rger: test 5 > 5471.444148283:imuxsock.c: Message has legacy syslog format. > 5471.444550564:imuxsock.c: main queue: entry added, size now 1 entries > 5471.444845291:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5471.445187788:imuxsock.c: main queue: EnqueueMsg advised worker start > 5471.445489499:main queue:Reg/w0: main queue: entering rate limiter > 5471.445842612:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5471.446121136:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 5') contains 'test': TRUE > 5471.446527886:main queue:Reg/w0: action 0x4c64438 passed 2 times to > execution - less than neded - discarding > 5471.446837140:main queue:Reg/w0: main queue: entering rate limiter > 5471.447091079:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5471.447366530:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5473.522029412:imuxsock.c: Message from UNIX socket: #4 > 5473.522550422:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:53 rger: test 6 > 5473.522766368:imuxsock.c: Message has legacy syslog format. > 5473.523162224:imuxsock.c: main queue: entry added, size now 1 entries > 5473.523404989:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5473.523727931:imuxsock.c: main queue: EnqueueMsg advised worker start > 5473.524011204:main queue:Reg/w0: main queue: entering rate limiter > 5473.524328279:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5473.524621330:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 6') contains 'test': TRUE > 5473.525016627:main queue:Reg/w0: Called action, logging to builtin- > file > 5473.525803589:main queue:Reg/w0: (/rsyslog/logfile) > 5473.526170949:main queue:Reg/w0: main queue: entering rate limiter > 5473.526423213:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5473.526753139:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > > > > On Thu, 2008-12-18 at 12:51 -0500, Becker, Craig (DOB) wrote: > > The action $ActionExecOnlyEveryNthTime is not sending the Nth message > > but the first message, then the Nth + 1 message. For example, if I > > set the $ActionExecOnlyEveryNthTime 3 The 1st, 4th, 7th, etc. message > > are sent instead of the 3rd, 6th, 9th, etc. Has this bug (if it is a > > bug) been fixed in a later version? > > > > > > > > I?m enjoying learning this product and appreciate all the hard work > > that has gone into it. > > > > > > > > Thanks, > > > > > > > > Craig E. Becker > > IT Specialist II > > System Administrator > > Division of the Budget > > 518-473-1322 > > craig.becker at budget.state.ny.us > > > > > > > > > > > > > > > > HTML: > > > ______________________________________________________________________ > > This e-mail, including any attachments, may be confidential, > > privileged or otherwise legally protected. If you have received this > > e-mail in error, or from someone who was not authorized to send it to > > you, do not disseminate, copy or otherwise use this e-mail or its > > attachments. Please notify the sender immediately if you have > received > > this e-mail by mistake, and delete it from your system. > > > > ----------------------------------------------------------------------- > --------- > This e-mail, including any attachments, may be confidential, privileged > or otherwise legally protected. If you have received this e-mail in > error, or from someone who was not authorized to send it to you, do not > disseminate, copy or otherwise use this e-mail or its attachments. > Please notify the sender immediately if you have received this e-mail > by mistake, and delete it from your system. From rgerhards at hq.adiscon.com Wed Dec 24 11:13:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Dec 2008 11:13:41 +0100 Subject: [rsyslog] Merry Xmas and a happy new year! Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F949@grfint2.intern.adiscon.com> Hi all, I wanted to whish all of you a merry xmas and a happy new year. At the same time, I'd like to thank all of you for your qualified comments, encouragement and ideas, which helped make rsyslog become a premier logging solution. Without an active community, no project can survive and I am very happy to see such a great community backing this project. Special thanks go to all those who also participate in the forum and dispense knowledge here on the mailing list. I can promise that 2009 will become a very exciting year for rsyslog, too. There is still a lot of things in stock and even though the budget cut has forced me to work at a somewhat slower pace, I am sure we will see great enhancements. A number of companies have also asked about sponsoring, so we may even be able to get back to full speed development in 2009. Please note that I take a break from the project from end of December to January, 10th. So my response will be sluggish in that period. My apologies for that. I hope you, too, will enjoy a great holiday season. My your wishes come true and your health be good! If we don't "talk" before the end of the year, I also wanted to wish you a great end of the year. Please keep your comments, ideas and feedback floating in 2009. This is a great source of inspiration and much appreciated. My warmest regards, Rainer Gerhards From rgerhards at hq.adiscon.com Mon Dec 1 10:02:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 10:02:01 +0100 Subject: [rsyslog] rsyslog doc licenses Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Hi all, Michael Biebl made me aware that the doc files have inconsistent license information (thanks!). Some state GPL, some FDL, some even nothing. I would like to unify them to use GPLv3, which is consistent with the rest of this project. Does anybody object this move? If so what are the reasons and alternatives? Feedback is appreciated. Many thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 1 10:55:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 10:55:12 +0100 Subject: [rsyslog] security issue in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> Hi folks, thanks to a bug report, I found out that the $AllowedSender directive does not work in all releases. The bug in question is: http://bugzilla.adiscon.com/show_bug.cgi?id=111 Im am currently working on the bug. Obviously, this can lead to messages being received from systems that are not permitted so. As a work-around, proper firewalling should be set up on the vulnerable hosts. Until further note, I would assume that all versions of rsyslog are affected (I will provide more detail during my analysis). Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 1 11:26:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 11:26:56 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> Version v2-stable is NOT vulnerable. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 10:55 AM > To: rsyslog-users > Subject: [rsyslog] security issue in rsyslog > > Hi folks, > > thanks to a bug report, I found out that the $AllowedSender directive > does not work in all releases. The bug in question is: > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > Im am currently working on the bug. Obviously, this can lead to > messages > being received from systems that are not permitted so. As a work- > around, > proper firewalling should be set up on the vulnerable hosts. Until > further note, I would assume that all versions of rsyslog are affected > (I will provide more detail during my analysis). > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 15:31:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 15:31:36 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> Hi all, this is patch for v3-stable: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae6 d9bbf6b07e2f06c4dd676 I have not tried yet, but I think it will work on almost all other versions, too. I keep you posted on the progress. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 11:27 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Version v2-stable is NOT vulnerable. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 10:55 AM > > To: rsyslog-users > > Subject: [rsyslog] security issue in rsyslog > > > > Hi folks, > > > > thanks to a bug report, I found out that the $AllowedSender directive > > does not work in all releases. The bug in question is: > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > Im am currently working on the bug. Obviously, this can lead to > > messages > > being received from systems that are not permitted so. As a work- > > around, > > proper firewalling should be set up on the vulnerable hosts. Until > > further note, I would assume that all versions of rsyslog are > affected > > (I will provide more detail during my analysis). > > > > Thanks, > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 16:30:36 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 16:30:36 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> I now clarified the affected versions. Affected are 3.12.2 and above. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 3:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Hi all, > > this is patch for v3-stable: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > 6 > d9bbf6b07e2f06c4dd676 > > I have not tried yet, but I think it will work on almost all other > versions, too. I keep you posted on the progress. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 11:27 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Version v2-stable is NOT vulnerable. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 10:55 AM > > > To: rsyslog-users > > > Subject: [rsyslog] security issue in rsyslog > > > > > > Hi folks, > > > > > > thanks to a bug report, I found out that the $AllowedSender > directive > > > does not work in all releases. The bug in question is: > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > messages > > > being received from systems that are not permitted so. As a work- > > > around, > > > proper firewalling should be set up on the vulnerable hosts. Until > > > further note, I would assume that all versions of rsyslog are > > affected > > > (I will provide more detail during my analysis). > > > > > > Thanks, > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 16:37:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 16:37:23 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> ... and the patch will not work on all of these version, due to the introduction of the netstream driver functionality. Please note that anything older than current v3-stable is outdated, so the proper way to replace the faulty code is to upgrade to the current v3-stable and apply the patch. I will also release a new v3-stable soon, hopefully today (but I'd like to conduct some more tests). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 4:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > I now clarified the affected versions. Affected are 3.12.2 and above. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 3:32 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Hi all, > > > > this is patch for v3-stable: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > 6 > > d9bbf6b07e2f06c4dd676 > > > > I have not tried yet, but I think it will work on almost all other > > versions, too. I keep you posted on the progress. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 11:27 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Version v2-stable is NOT vulnerable. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > To: rsyslog-users > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > Hi folks, > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > directive > > > > does not work in all releases. The bug in question is: > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > > messages > > > > being received from systems that are not permitted so. As a work- > > > > around, > > > > proper firewalling should be set up on the vulnerable hosts. > Until > > > > further note, I would assume that all versions of rsyslog are > > > affected > > > > (I will provide more detail during my analysis). > > > > > > > > Thanks, > > > > Rainer > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 17:08:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 17:08:05 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> The issue also exists in TCP mode, but analysis shows this is not a trial fix. The design overlooked the situation. In theory, a whole new access control feature would be needed. I am checking out if it is possible to "just" enhance the interface. With the current netstreams defined that should be possible. I am tempted to release the UDP-fixed version and release the next version with the TCP fix. Feedback from packagers is appreciated. The TCP fix may take a day or two, depending on how smart a way I find. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 4:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > ... and the patch will not work on all of these version, due to the > introduction of the netstream driver functionality. Please note that > anything older than current v3-stable is outdated, so the proper way to > replace the faulty code is to upgrade to the current v3-stable and > apply > the patch. I will also release a new v3-stable soon, hopefully today > (but I'd like to conduct some more tests). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 4:31 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > I now clarified the affected versions. Affected are 3.12.2 and above. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 3:32 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Hi all, > > > > > > this is patch for v3-stable: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > 6 > > > d9bbf6b07e2f06c4dd676 > > > > > > I have not tried yet, but I think it will work on almost all other > > > versions, too. I keep you posted on the progress. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > To: rsyslog-users > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > Hi folks, > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > directive > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > Im am currently working on the bug. Obviously, this can lead to > > > > > messages > > > > > being received from systems that are not permitted so. As a > work- > > > > > around, > > > > > proper firewalling should be set up on the vulnerable hosts. > > Until > > > > > further note, I would assume that all versions of rsyslog are > > > > affected > > > > > (I will provide more detail during my analysis). > > > > > > > > > > Thanks, > > > > > Rainer > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Dec 1 17:51:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 17:51:38 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> Ok, looks like I found a work-around. Not that elegant, but seems to work quite well. Patch for TCP is here: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9e 18747b55d701e360d5aac Please note that this effectively disables GSS functionality. I'll updated the GSS drivers in the next step. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 5:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The issue also exists in TCP mode, but analysis shows this is not a > trial fix. The design overlooked the situation. In theory, a whole new > access control feature would be needed. I am checking out if it is > possible to "just" enhance the interface. With the current netstreams > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 4:37 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > ... and the patch will not work on all of these version, due to the > > introduction of the netstream driver functionality. Please note that > > anything older than current v3-stable is outdated, so the proper way > to > > replace the faulty code is to upgrade to the current v3-stable and > > apply > > the patch. I will also release a new v3-stable soon, hopefully today > > (but I'd like to conduct some more tests). > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 4:31 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > above. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Hi all, > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > 6 > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > I have not tried yet, but I think it will work on almost all > other > > > > versions, too. I keep you posted on the progress. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > To: rsyslog-users > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > > directive > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can lead > to > > > > > > messages > > > > > > being received from systems that are not permitted so. As a > > work- > > > > > > around, > > > > > > proper firewalling should be set up on the vulnerable hosts. > > > Until > > > > > > further note, I would assume that all versions of rsyslog are > > > > > affected > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > Thanks, > > > > > > Rainer > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Mon Dec 1 18:23:47 2008 From: david at lang.hm (david at lang.hm) Date: Mon, 1 Dec 2008 09:23:47 -0800 (PST) Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: On Mon, 1 Dec 2008, Rainer Gerhards wrote: > The issue also exists in TCP mode, but analysis shows this is not a > trial fix. The design overlooked the situation. In theory, a whole new > access control feature would be needed. I am checking out if it is > possible to "just" enhance the interface. With the current netstreams > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. for UDP it's trivial to forge the source IP address anyway, so the 'security' gained by this feature in that mode is questionable to start with. that being said, I'm very pleased to see how you are handling this. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >> Sent: Monday, December 01, 2008 4:37 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] security issue in rsyslog >> >> ... and the patch will not work on all of these version, due to the >> introduction of the netstream driver functionality. Please note that >> anything older than current v3-stable is outdated, so the proper way > to >> replace the faulty code is to upgrade to the current v3-stable and >> apply >> the patch. I will also release a new v3-stable soon, hopefully today >> (but I'd like to conduct some more tests). >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>> Sent: Monday, December 01, 2008 4:31 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] security issue in rsyslog >>> >>> I now clarified the affected versions. Affected are 3.12.2 and > above. >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>> Sent: Monday, December 01, 2008 3:32 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] security issue in rsyslog >>>> >>>> Hi all, >>>> >>>> this is patch for v3-stable: >>>> >>>> >>> >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae >>>> 6 >>>> d9bbf6b07e2f06c4dd676 >>>> >>>> I have not tried yet, but I think it will work on almost all other >>>> versions, too. I keep you posted on the progress. >>>> >>>> Rainer >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>>> Sent: Monday, December 01, 2008 11:27 AM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] security issue in rsyslog >>>>> >>>>> Version v2-stable is NOT vulnerable. >>>>> >>>>> Rainer >>>>> >>>>>> -----Original Message----- >>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >>>>>> Sent: Monday, December 01, 2008 10:55 AM >>>>>> To: rsyslog-users >>>>>> Subject: [rsyslog] security issue in rsyslog >>>>>> >>>>>> Hi folks, >>>>>> >>>>>> thanks to a bug report, I found out that the $AllowedSender >>>> directive >>>>>> does not work in all releases. The bug in question is: >>>>>> >>>>>> http://bugzilla.adiscon.com/show_bug.cgi?id=111 >>>>>> >>>>>> Im am currently working on the bug. Obviously, this can lead > to >>>>>> messages >>>>>> being received from systems that are not permitted so. As a >> work- >>>>>> around, >>>>>> proper firewalling should be set up on the vulnerable hosts. >>> Until >>>>>> further note, I would assume that all versions of rsyslog are >>>>> affected >>>>>> (I will provide more detail during my analysis). >>>>>> >>>>>> Thanks, >>>>>> Rainer >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From mbiebl at gmail.com Mon Dec 1 18:17:31 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 18:17:31 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> Message-ID: 2008/12/1 Rainer Gerhards : > defined that should be possible. I am tempted to release the UDP-fixed > version and release the next version with the TCP fix. Feedback from > packagers is appreciated. The TCP fix may take a day or two, depending > on how smart a way I find. I'd say, take the time for proper fixing and testing, even if it takes a day or two and release afterwards. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Dec 1 18:47:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 18:47:00 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> And now there is an *untested* fix for the TLS driver: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec063 83fc5969178887d00abfc Testing takes a bit more of time, I need to set up the test environment for TLS again (looks like it would really pay to have a fixed test suite for all those cases - also the issue here would have never occurred...). Please note that I mistook GSSAPI with TLS in my previous mail. The TLS part should not be really affected by the problem: there are so much better access control features in TLS... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 5:52 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Ok, looks like I found a work-around. Not that elegant, but seems to > work quite well. Patch for TCP is here: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > e > 18747b55d701e360d5aac > > Please note that this effectively disables GSS functionality. I'll > updated the GSS drivers in the next step. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 5:08 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > The issue also exists in TCP mode, but analysis shows this is not a > > trial fix. The design overlooked the situation. In theory, a whole > new > > access control feature would be needed. I am checking out if it is > > possible to "just" enhance the interface. With the current netstreams > > defined that should be possible. I am tempted to release the UDP- > fixed > > version and release the next version with the TCP fix. Feedback from > > packagers is appreciated. The TCP fix may take a day or two, > depending > > on how smart a way I find. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 4:37 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > ... and the patch will not work on all of these version, due to the > > > introduction of the netstream driver functionality. Please note > that > > > anything older than current v3-stable is outdated, so the proper > way > > to > > > replace the faulty code is to upgrade to the current v3-stable and > > > apply > > > the patch. I will also release a new v3-stable soon, hopefully > today > > > (but I'd like to conduct some more tests). > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > > above. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Hi all, > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > 6 > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > I have not tried yet, but I think it will work on almost all > > other > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > To: rsyslog-users > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > thanks to a bug report, I found out that the $AllowedSender > > > > > directive > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > lead > > to > > > > > > > messages > > > > > > > being received from systems that are not permitted so. As a > > > work- > > > > > > > around, > > > > > > > proper firewalling should be set up on the vulnerable > hosts. > > > > Until > > > > > > > further note, I would assume that all versions of rsyslog > are > > > > > > affected > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > Thanks, > > > > > > > Rainer > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Mon Dec 1 19:30:57 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 19:30:57 +0100 Subject: [rsyslog] rsyslog doc licenses In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Message-ID: 2008/12/1 Rainer Gerhards : > Hi all, > > Michael Biebl made me aware that the doc files have inconsistent license > information (thanks!). Some state GPL, some FDL, some even nothing. > > I would like to unify them to use GPLv3, which is consistent with the > rest of this project. Does anybody object this move? My contributions to the documentation where minuscule. But anyways, you have my ok for the relicensing. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From aoz.syn at gmail.com Mon Dec 1 20:00:07 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 1 Dec 2008 12:00:07 -0700 Subject: [rsyslog] rsyslog doc licenses In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F763@grfint2.intern.adiscon.com> Message-ID: <4255c2570812011100l7183e18g4999de9bdddf5955@mail.gmail.com> On Mon, Dec 1, 2008 at 02:02, Rainer Gerhards wrote: > I would like to unify them to use GPLv3, which is consistent with the > rest of this project. Does anybody object this move? If so what are the > reasons and alternatives? One of the more documentation-focused licenses (e.g. GFDL or one of the Creative Commons family) may be better-suited to protect the IP concerns particular to documentation. I'm not sure I perfectly understand why separate documentation & source licenses exist, but the fact that legal minds greater than mine (doesn't take much) decided the split was necessary causes me to at least pause and consider them. A particularly interesting problem some have thrown at GFDL is that it is incompatible with the GPL - you cannot convert in either direction without re-licensing from the originator. Feh. Licensing is a sticky subject. From peter.matulis at canonical.com Mon Dec 1 20:48:41 2008 From: peter.matulis at canonical.com (Peter Matulis) Date: Mon, 01 Dec 2008 14:48:41 -0500 Subject: [rsyslog] TLS certificates Message-ID: <49343F99.4030607@canonical.com> Why does the documentation state that a client does not need to include its private key and certificate in its configuration? It says to only use the CA public certificate (ca.pem). Peter From mbiebl at gmail.com Mon Dec 1 20:53:19 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 1 Dec 2008 20:53:19 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <49343F99.4030607@canonical.com> References: <49343F99.4030607@canonical.com> Message-ID: 2008/12/1 Peter Matulis : > Why does the documentation state that a client does not need to include > its private key and certificate in its configuration? It says to only > use the CA public certificate (ca.pem). That depends on what you want to do. If you only want to verify, that a client is sending it's messages to the right server, then it only needs the CA to verify that. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From peter.matulis at canonical.com Mon Dec 1 21:04:29 2008 From: peter.matulis at canonical.com (Peter Matulis) Date: Mon, 01 Dec 2008 15:04:29 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: References: <49343F99.4030607@canonical.com> Message-ID: <4934434D.4070703@canonical.com> Michael Biebl wrote: > 2008/12/1 Peter Matulis : >> Why does the documentation state that a client does not need to include >> its private key and certificate in its configuration? It says to only >> use the CA public certificate (ca.pem). > > That depends on what you want to do. > > If you only want to verify, that a client is sending it's messages to > the right server, then it only needs the CA to verify that. > > Michael > Right. I want to encrypt. Peter From aoz.syn at gmail.com Mon Dec 1 21:08:29 2008 From: aoz.syn at gmail.com (RB) Date: Mon, 1 Dec 2008 13:08:29 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: <4934434D.4070703@canonical.com> References: <49343F99.4030607@canonical.com> <4934434D.4070703@canonical.com> Message-ID: <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> On Mon, Dec 1, 2008 at 13:04, Peter Matulis wrote: > Right. I want to encrypt. Encryption is still possible without a client certificate. Generally speaking, in SSL all a client cert adds is the ability for the server to verify the client's identity at the transport layer. RB From rgerhards at hq.adiscon.com Mon Dec 1 21:10:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 1 Dec 2008 21:10:24 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> References: <49343F99.4030607@canonical.com><4934434D.4070703@canonical.com> <4255c2570812011208heacf9bbx975332f0e575dbd3@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F77A@grfint2.intern.adiscon.com> Jupp. I'd also like to mention the more elaborate guide. It is part of the doc set, but also available online: http://www.rsyslog.com/doc-rsyslog_secure_tls.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of RB > Sent: Monday, December 01, 2008 9:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > On Mon, Dec 1, 2008 at 13:04, Peter Matulis > wrote: > > Right. I want to encrypt. > > Encryption is still possible without a client certificate. Generally > speaking, in SSL all a client cert adds is the ability for the server > to verify the client's identity at the transport layer. > > > RB > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jmiscaro at gmail.com Tue Dec 2 14:55:56 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 08:55:56 -0500 Subject: [rsyslog] TLS certificates Message-ID: I has reading the docs [1] and this confusd me to. It says "neither the client nor the server are authenticated. So while the message transfer is encrypted, you can not be sure which peer you are talking to" Also, how can client encrypt without having any keys specified in its config? Example for the client shows: $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem $ActionSendStreamDriverAuthMode anon # server is NOT authenticated 2nd question: Why is the server not authenticated? /juan [1] http://www.rsyslog.com/doc-rsyslog_tls.html From aoz.syn at gmail.com Tue Dec 2 16:56:37 2008 From: aoz.syn at gmail.com (RB) Date: Tue, 2 Dec 2008 08:56:37 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: References: Message-ID: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: > "neither the client nor the server are authenticated. So while the > message transfer is encrypted, you can not be sure which peer you are > talking to" I'm hoping Rainer will jump in and clarify precisely how much handshake validation he's implemented. The fact that the client must have a copy of the CA's public material seems to indicate he is at least verifying that the server's certificate was issued by the CA. It's possible to not do so, but the result is rather susceptible to MITM. > Also, how can client encrypt without having any keys specified in its config? This isn't the forum to discuss the particulars of the SSL handshake, but suffice it to say that SSL incorporates a challenge/response mechanism (using the server's presented certificate) followed by negotiation of an ephemeral session key. See also: public-key cryptography. > $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > > 2nd question: Why is the server not authenticated? Without looking at the code, I presume the 'anon' AuthMode is the switch used to tell the SSL library whether or not to check the server certificate against the CA. If so, it should make specifying the CA public key redundant - the client just accepts whatever certificate the server (or MITM) presents and starts encrypting to it. From rgerhards at hq.adiscon.com Tue Dec 2 17:00:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:00:43 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: Tuesday, December 02, 2008 4:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: > > "neither the client nor the server are authenticated. So while the > > message transfer is encrypted, you can not be sure which peer you are > > talking to" > > I'm hoping Rainer will jump in and clarify precisely how much > handshake validation he's implemented. The fact that the client must > have a copy of the CA's public material seems to indicate he is at > least verifying that the server's certificate was issued by the CA. > It's possible to not do so, but the result is rather susceptible to > MITM. Just a quick note, I am quite busy at the moment (guess what ;)). If the auth is set to "anon" nothing at all is validated and MITM *is* absolutely possible. That's why the doc does not recommend to use that mode. I posted a link to the long TLS setup guide, which creates a fairly safe scenario (but your milage may vary... ;)). > > > Also, how can client encrypt without having any keys specified in its > config? > > This isn't the forum to discuss the particulars of the SSL handshake, > but suffice it to say that SSL incorporates a challenge/response > mechanism (using the server's presented certificate) followed by > negotiation of an ephemeral session key. See also: public-key > cryptography. > > > $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > > $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > > > > 2nd question: Why is the server not authenticated? > > Without looking at the code, I presume the 'anon' AuthMode is the > switch used to tell the SSL library whether or not to check the server > certificate against the CA. If so, it should make specifying the CA > public key redundant - the client just accepts whatever certificate > the server (or MITM) presents and starts encrypting to it. The modes are mostly rooted in the upcoming RFC5225 (or 5226, don't remember correctly). Anon is an insecure extension. While being insecure, it is a mode that allows low end devices deployed in no-knowledge environmebnt (hopefully read: home users) to have at least the benefit of encryption (obfuscation would be more precise) but nothing (nothing!) above that. Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Tue Dec 2 17:18:30 2008 From: aoz.syn at gmail.com (RB) Date: Tue, 2 Dec 2008 09:18:30 -0700 Subject: [rsyslog] TLS certificates In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44F7AF@grfint2.intern.adiscon.com> Message-ID: <4255c2570812020818w50a7a8b7k1c506ab8455b511e@mail.gmail.com> On Tue, Dec 2, 2008 at 09:00, Rainer Gerhards wrote: > Just a quick note, I am quite busy at the moment (guess what ;)). If the > auth is set to "anon" nothing at all is validated and MITM *is* > absolutely possible. That's why the doc does not recommend to use that > mode. I posted a link to the long TLS setup guide, which creates a > fairly safe scenario (but your milage may vary... ;)). Understood. For everyone else's edification, here is the comment in the related code, outlining what modes are used: /* Set the authentication mode. For us, the following is supported: * anon - no certificate checks whatsoever (discouraged, but supported) * x509/certvalid - (just) check certificate validity * x509/fingerprint - certificate fingerprint * x509/name - cerfificate name check * mode == NULL is valid and defaults to x509/name * rgerhards, 2008-05-16 */ From jmiscaro at gmail.com Tue Dec 2 17:30:42 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 11:30:42 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: 2008/12/2 RB : > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro wrote: >> "neither the client nor the server are authenticated. So while the >> message transfer is encrypted, you can not be sure which peer you are >> talking to" > > I'm hoping Rainer will jump in and clarify precisely how much > handshake validation he's implemented. The fact that the client must > have a copy of the CA's public material seems to indicate he is at > least verifying that the server's certificate was issued by the CA. > It's possible to not do so, but the result is rather susceptible to > MITM. > >> Also, how can client encrypt without having any keys specified in its config? > > This isn't the forum to discuss the particulars of the SSL handshake, > but suffice it to say that SSL incorporates a challenge/response > mechanism (using the server's presented certificate) followed by > negotiation of an ephemeral session key. See also: public-key > cryptography. > >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated >> >> 2nd question: Why is the server not authenticated? > > Without looking at the code, I presume the 'anon' AuthMode is the > switch used to tell the SSL library whether or not to check the server > certificate against the CA. If so, it should make specifying the CA > public key redundant - the client just accepts whatever certificate > the server (or MITM) presents and starts encrypting to it. Thank you. I change my config and logging is hapenning on the server end. However, I get such lines in the logs on the server end when I restart the client system: invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 42 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 45 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 46 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 47 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 49 invalid or yet-unknown config file command - have you forgotten to load a module? the last error occured in /etc/rsyslog.conf, line 51 This happens for each TLS line in my client config (comments removed): $DefaultNetstreamDriver gtls $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client-cert.pem $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client-key.pem $ActionSendStreamDriverAuthMode x509/name $ActionSendStreamDriverMode 1 *.* @@192.168.4.102:10514 /juan From rgerhards at hq.adiscon.com Tue Dec 2 17:31:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:31:13 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Too old version? > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > Sent: Tuesday, December 02, 2008 5:31 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > 2008/12/2 RB : > > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro > wrote: > >> "neither the client nor the server are authenticated. So while the > >> message transfer is encrypted, you can not be sure which peer you > are > >> talking to" > > > > I'm hoping Rainer will jump in and clarify precisely how much > > handshake validation he's implemented. The fact that the client must > > have a copy of the CA's public material seems to indicate he is at > > least verifying that the server's certificate was issued by the CA. > > It's possible to not do so, but the result is rather susceptible to > > MITM. > > > >> Also, how can client encrypt without having any keys specified in > its config? > > > > This isn't the forum to discuss the particulars of the SSL handshake, > > but suffice it to say that SSL incorporates a challenge/response > > mechanism (using the server's presented certificate) followed by > > negotiation of an ephemeral session key. See also: public-key > > cryptography. > > > >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated > >> > >> 2nd question: Why is the server not authenticated? > > > > Without looking at the code, I presume the 'anon' AuthMode is the > > switch used to tell the SSL library whether or not to check the > server > > certificate against the CA. If so, it should make specifying the CA > > public key redundant - the client just accepts whatever certificate > > the server (or MITM) presents and starts encrypting to it. > > Thank you. I change my config and logging is hapenning on the server > end. However, I get such lines in the logs on the server end when I > restart the client system: > > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 42 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 45 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 46 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 47 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 49 > invalid or yet-unknown config file command - have you forgotten to > load a module? > the last error occured in /etc/rsyslog.conf, line 51 > > > This happens for each TLS line in my client config (comments removed): > > $DefaultNetstreamDriver gtls > $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem > $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- > cert.pem > $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- > key.pem > $ActionSendStreamDriverAuthMode x509/name > $ActionSendStreamDriverMode 1 > *.* @@192.168.4.102:10514 > > > /juan > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jmiscaro at gmail.com Tue Dec 2 17:39:49 2008 From: jmiscaro at gmail.com (Juan Miscaro) Date: Tue, 2 Dec 2008 11:39:49 -0500 Subject: [rsyslog] TLS certificates In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Message-ID: My boxes are running 3.18.1 /juan 2008/12/2 Rainer Gerhards : > Too old version? > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Juan Miscaro >> Sent: Tuesday, December 02, 2008 5:31 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] TLS certificates >> >> 2008/12/2 RB : >> > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro >> wrote: >> >> "neither the client nor the server are authenticated. So while the >> >> message transfer is encrypted, you can not be sure which peer you >> are >> >> talking to" >> > >> > I'm hoping Rainer will jump in and clarify precisely how much >> > handshake validation he's implemented. The fact that the client > must >> > have a copy of the CA's public material seems to indicate he is at >> > least verifying that the server's certificate was issued by the CA. >> > It's possible to not do so, but the result is rather susceptible to >> > MITM. >> > >> >> Also, how can client encrypt without having any keys specified in >> its config? >> > >> > This isn't the forum to discuss the particulars of the SSL > handshake, >> > but suffice it to say that SSL incorporates a challenge/response >> > mechanism (using the server's presented certificate) followed by >> > negotiation of an ephemeral session key. See also: public-key >> > cryptography. >> > >> >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem >> >> $ActionSendStreamDriverAuthMode anon # server is NOT authenticated >> >> >> >> 2nd question: Why is the server not authenticated? >> > >> > Without looking at the code, I presume the 'anon' AuthMode is the >> > switch used to tell the SSL library whether or not to check the >> server >> > certificate against the CA. If so, it should make specifying the CA >> > public key redundant - the client just accepts whatever certificate >> > the server (or MITM) presents and starts encrypting to it. >> >> Thank you. I change my config and logging is hapenning on the server >> end. However, I get such lines in the logs on the server end when I >> restart the client system: >> >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 42 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 45 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 46 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 47 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 49 >> invalid or yet-unknown config file command - have you forgotten to >> load a module? >> the last error occured in /etc/rsyslog.conf, line 51 >> >> >> This happens for each TLS line in my client config (comments removed): >> >> $DefaultNetstreamDriver gtls >> $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem >> $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- >> cert.pem >> $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- >> key.pem >> $ActionSendStreamDriverAuthMode x509/name >> $ActionSendStreamDriverMode 1 >> *.* @@192.168.4.102:10514 >> >> >> /juan >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Tue Dec 2 17:52:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 2 Dec 2008 17:52:25 +0100 Subject: [rsyslog] TLS certificates In-Reply-To: References: <4255c2570812020756l3d4f97b3p268cd5f90680d81b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44F7B2@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7B5@grfint2.intern.adiscon.com> Jup, that's the problem. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > Sent: Tuesday, December 02, 2008 5:40 PM > To: rsyslog-users > Subject: Re: [rsyslog] TLS certificates > > My boxes are running 3.18.1 > > /juan > > 2008/12/2 Rainer Gerhards : > > Too old version? > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Juan Miscaro > >> Sent: Tuesday, December 02, 2008 5:31 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] TLS certificates > >> > >> 2008/12/2 RB : > >> > On Tue, Dec 2, 2008 at 06:55, Juan Miscaro > >> wrote: > >> >> "neither the client nor the server are authenticated. So while > the > >> >> message transfer is encrypted, you can not be sure which peer you > >> are > >> >> talking to" > >> > > >> > I'm hoping Rainer will jump in and clarify precisely how much > >> > handshake validation he's implemented. The fact that the client > > must > >> > have a copy of the CA's public material seems to indicate he is at > >> > least verifying that the server's certificate was issued by the > CA. > >> > It's possible to not do so, but the result is rather susceptible > to > >> > MITM. > >> > > >> >> Also, how can client encrypt without having any keys specified in > >> its config? > >> > > >> > This isn't the forum to discuss the particulars of the SSL > > handshake, > >> > but suffice it to say that SSL incorporates a challenge/response > >> > mechanism (using the server's presented certificate) followed by > >> > negotiation of an ephemeral session key. See also: public-key > >> > cryptography. > >> > > >> >> $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem > >> >> $ActionSendStreamDriverAuthMode anon # server is NOT > authenticated > >> >> > >> >> 2nd question: Why is the server not authenticated? > >> > > >> > Without looking at the code, I presume the 'anon' AuthMode is the > >> > switch used to tell the SSL library whether or not to check the > >> server > >> > certificate against the CA. If so, it should make specifying the > CA > >> > public key redundant - the client just accepts whatever > certificate > >> > the server (or MITM) presents and starts encrypting to it. > >> > >> Thank you. I change my config and logging is hapenning on the > server > >> end. However, I get such lines in the logs on the server end when I > >> restart the client system: > >> > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 42 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 45 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 46 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 47 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 49 > >> invalid or yet-unknown config file command - have you forgotten to > >> load a module? > >> the last error occured in /etc/rsyslog.conf, line 51 > >> > >> > >> This happens for each TLS line in my client config (comments > removed): > >> > >> $DefaultNetstreamDriver gtls > >> $DefaultNetstreamDriverCAFile /home/client/Data/tls/ca/ca.pem > >> $DefaultNetstreamDriverCertFile /home/client/Data/tls/client/client- > >> cert.pem > >> $DefaultNetstreamDriverKeyFile /home/client/Data/tls/client/client- > >> key.pem > >> $ActionSendStreamDriverAuthMode x509/name > >> $ActionSendStreamDriverMode 1 > >> *.* @@192.168.4.102:10514 > >> > >> > >> /juan > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 10:54:20 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 10:54:20 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> Ok, I ran this fix through a couple of tests yesterday. It looks well for TLS, too. Note that there is an implication that $AllowedSender TCP,... applies to TLS to (because it is TCP). I'd consider this to be a side-effect, but I do not think it is worth fixing. With TLS, there is much finer and better control. An issue may only exists if someone decides to run non-tls tcp and tls tcp together AND use $AllowedSender. Workaround in that case is to use the firewall, so I don't consider it is worth fixing now. Please note that my testing revealed a potential memory leak as side-effect of the fixes. This could be abused to a remote DoS, so I will investigate that before releasing. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, December 01, 2008 6:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > And now there is an *untested* fix for the TLS driver: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > 3 > 83fc5969178887d00abfc > > Testing takes a bit more of time, I need to set up the test environment > for TLS again (looks like it would really pay to have a fixed test > suite > for all those cases - also the issue here would have never > occurred...). > > Please note that I mistook GSSAPI with TLS in my previous mail. The TLS > part should not be really affected by the problem: there are so much > better access control features in TLS... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 5:52 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, looks like I found a work-around. Not that elegant, but seems to > > work quite well. Patch for TCP is here: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > e > > 18747b55d701e360d5aac > > > > Please note that this effectively disables GSS functionality. I'll > > updated the GSS drivers in the next step. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 5:08 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > The issue also exists in TCP mode, but analysis shows this is not a > > > trial fix. The design overlooked the situation. In theory, a whole > > new > > > access control feature would be needed. I am checking out if it is > > > possible to "just" enhance the interface. With the current > netstreams > > > defined that should be possible. I am tempted to release the UDP- > > fixed > > > version and release the next version with the TCP fix. Feedback > from > > > packagers is appreciated. The TCP fix may take a day or two, > > depending > > > on how smart a way I find. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > ... and the patch will not work on all of these version, due to > the > > > > introduction of the netstream driver functionality. Please note > > that > > > > anything older than current v3-stable is outdated, so the proper > > way > > > to > > > > replace the faulty code is to upgrade to the current v3-stable > and > > > > apply > > > > the patch. I will also release a new v3-stable soon, hopefully > > today > > > > (but I'd like to conduct some more tests). > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 and > > > above. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Hi all, > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > 6 > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > I have not tried yet, but I think it will work on almost all > > > other > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > $AllowedSender > > > > > > directive > > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > > lead > > > to > > > > > > > > messages > > > > > > > > being received from systems that are not permitted so. As > a > > > > work- > > > > > > > > around, > > > > > > > > proper firewalling should be set up on the vulnerable > > hosts. > > > > > Until > > > > > > > > further note, I would assume that all versions of rsyslog > > are > > > > > > > affected > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Rainer > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 11:30:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 11:30:29 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> The memory leak is now also fixed, I just quickly re-run some TLS tests to make sure nothing is broken and it works there, too. Patch (on top of the others): http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54ddd cb8703560c750f04a6370 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 10:54 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Ok, I ran this fix through a couple of tests yesterday. It looks well > for TLS, too. Note that there is an implication that $AllowedSender > TCP,... applies to TLS to (because it is TCP). I'd consider this to be > a > side-effect, but I do not think it is worth fixing. With TLS, there is > much finer and better control. An issue may only exists if someone > decides to run non-tls tcp and tls tcp together AND use $AllowedSender. > Workaround in that case is to use the firewall, so I don't consider it > is worth fixing now. > > Please note that my testing revealed a potential memory leak as > side-effect of the fixes. This could be abused to a remote DoS, so I > will investigate that before releasing. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Monday, December 01, 2008 6:47 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > And now there is an *untested* fix for the TLS driver: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > 3 > > 83fc5969178887d00abfc > > > > Testing takes a bit more of time, I need to set up the test > environment > > for TLS again (looks like it would really pay to have a fixed test > > suite > > for all those cases - also the issue here would have never > > occurred...). > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > TLS > > part should not be really affected by the problem: there are so much > > better access control features in TLS... > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 5:52 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > to > > > work quite well. Patch for TCP is here: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > e > > > 18747b55d701e360d5aac > > > > > > Please note that this effectively disables GSS functionality. I'll > > > updated the GSS drivers in the next step. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > The issue also exists in TCP mode, but analysis shows this is not > a > > > > trial fix. The design overlooked the situation. In theory, a > whole > > > new > > > > access control feature would be needed. I am checking out if it > is > > > > possible to "just" enhance the interface. With the current > > netstreams > > > > defined that should be possible. I am tempted to release the UDP- > > > fixed > > > > version and release the next version with the TCP fix. Feedback > > from > > > > packagers is appreciated. The TCP fix may take a day or two, > > > depending > > > > on how smart a way I find. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > ... and the patch will not work on all of these version, due to > > the > > > > > introduction of the netstream driver functionality. Please note > > > that > > > > > anything older than current v3-stable is outdated, so the > proper > > > way > > > > to > > > > > replace the faulty code is to upgrade to the current v3-stable > > and > > > > > apply > > > > > the patch. I will also release a new v3-stable soon, hopefully > > > today > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > and > > > > above. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > 6 > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > all > > > > other > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > $AllowedSender > > > > > > > directive > > > > > > > > > does not work in all releases. The bug in question is: > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this can > > > lead > > > > to > > > > > > > > > messages > > > > > > > > > being received from systems that are not permitted so. > As > > a > > > > > work- > > > > > > > > > around, > > > > > > > > > proper firewalling should be set up on the vulnerable > > > hosts. > > > > > > Until > > > > > > > > > further note, I would assume that all versions of > rsyslog > > > are > > > > > > > > affected > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Rainer > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Dec 3 16:47:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 3 Dec 2008 16:47:45 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7D5@grfint2.intern.adiscon.com> ...TLS testing always takes an awful lot of time... But I was also able to identify another memory leak, which is nice. I think I have now finished the release (less some doc update, maybe). I'd appreciate if some of you could give it a try. I would then do the actual release tomorrow. Download is: http://download.rsyslog.com/rsyslog/rsyslog-3.20.1.tar.gz md5sum: 2786d0d8de85fc9e6e83ff4ed9f468a7 If you try it out, please let me know the results. As I said, I don't expect anything bad, so it should be suitable for production use. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 11:30 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The memory leak is now also fixed, I just quickly re-run some TLS tests > to make sure nothing is broken and it works there, too. > > Patch (on top of the others): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > d > cb8703560c750f04a6370 > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 10:54 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, I ran this fix through a couple of tests yesterday. It looks well > > for TLS, too. Note that there is an implication that $AllowedSender > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > be > > a > > side-effect, but I do not think it is worth fixing. With TLS, there > is > > much finer and better control. An issue may only exists if someone > > decides to run non-tls tcp and tls tcp together AND use > $AllowedSender. > > Workaround in that case is to use the firewall, so I don't consider > it > > is worth fixing now. > > > > Please note that my testing revealed a potential memory leak as > > side-effect of the fixes. This could be abused to a remote DoS, so I > > will investigate that before releasing. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 6:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > 3 > > > 83fc5969178887d00abfc > > > > > > Testing takes a bit more of time, I need to set up the test > > environment > > > for TLS again (looks like it would really pay to have a fixed test > > > suite > > > for all those cases - also the issue here would have never > > > occurred...). > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > > TLS > > > part should not be really affected by the problem: there are so > much > > > better access control features in TLS... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > > to > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > e > > > > 18747b55d701e360d5aac > > > > > > > > Please note that this effectively disables GSS functionality. > I'll > > > > updated the GSS drivers in the next step. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > not > > a > > > > > trial fix. The design overlooked the situation. In theory, a > > whole > > > > new > > > > > access control feature would be needed. I am checking out if it > > is > > > > > possible to "just" enhance the interface. With the current > > > netstreams > > > > > defined that should be possible. I am tempted to release the > UDP- > > > > fixed > > > > > version and release the next version with the TCP fix. Feedback > > > from > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > depending > > > > > on how smart a way I find. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > ... and the patch will not work on all of these version, due > to > > > the > > > > > > introduction of the netstream driver functionality. Please > note > > > > that > > > > > > anything older than current v3-stable is outdated, so the > > proper > > > > way > > > > > to > > > > > > replace the faulty code is to upgrade to the current v3- > stable > > > and > > > > > > apply > > > > > > the patch. I will also release a new v3-stable soon, > hopefully > > > > today > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > > and > > > > > above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > 6 > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > > all > > > > > other > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > $AllowedSender > > > > > > > > directive > > > > > > > > > > does not work in all releases. The bug in question > is: > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > can > > > > lead > > > > > to > > > > > > > > > > messages > > > > > > > > > > being received from systems that are not permitted > so. > > As > > > a > > > > > > work- > > > > > > > > > > around, > > > > > > > > > > proper firewalling should be set up on the vulnerable > > > > hosts. > > > > > > > Until > > > > > > > > > > further note, I would assume that all versions of > > rsyslog > > > > are > > > > > > > > > affected > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Rainer > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Dec 4 12:30:58 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Dec 2008 12:30:58 +0100 Subject: [rsyslog] rsyslog 3.20.1 (v3-stable) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E3@grfint2.intern.adiscon.com> Hello, We have just released rsyslog 3.20.1, a member of the v3-stable branch. Most importantly, this release addresses a security vulnerability that renders the $AllowedSender directive useless. This issue has already been discussed here on the list. In addition to this, the release also contains some bug fixes, including a memory leak that could cause real trouble in environments that use TLS, as well as some very small enhancements (that were considered small enough to be a applied to the stable branch) Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-141.phtml Change Log: http://www.rsyslog.com/Article323.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. Releases for the beta and devel branches will hopefully be posted later today. The git archive has all relevant patches if someone has an urgent need. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From friedl at hq.adiscon.com Thu Dec 4 13:34:08 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 4 Dec 2008 13:34:08 +0100 Subject: [rsyslog] rsyslog 3.21.8 (v3-beta) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 3.21.8, a member of the v3-beta branch. Most importantly, this release addresses a security vulnerability the renders the $AllowedSender directive useless. This issue has already been discussed here on the list. In addition to this, the release also contains all the bug fixes and enhancements from the stable release 3.20.1. Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-142.phtml Change Log: http://www.rsyslog.com/Article326.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. Releases for the devel branch will hopefully be posted later today. The git archive has all relevant patches if someone has an urgent need. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Thu Dec 4 13:38:17 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 13:38:17 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> Grrr... One more issue. I noticed that while I resolved some conflicts on the devel branch integration. There is an option that a log message is emitted by rsyslog itself, when a remote machine's message is discarded due to no permission. This was requested so that people know when something goes wrong. This is only in the UDP code. HOWEVER, this is not rate-limited so if someone carries out a heavy attack, he can still flood the local disk by these messages. I'll change it so that the message is emited only once every minute and will then re-release what already has been released... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, December 03, 2008 11:30 AM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > The memory leak is now also fixed, I just quickly re-run some TLS tests > to make sure nothing is broken and it works there, too. > > Patch (on top of the others): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > d > cb8703560c750f04a6370 > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 10:54 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Ok, I ran this fix through a couple of tests yesterday. It looks well > > for TLS, too. Note that there is an implication that $AllowedSender > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > be > > a > > side-effect, but I do not think it is worth fixing. With TLS, there > is > > much finer and better control. An issue may only exists if someone > > decides to run non-tls tcp and tls tcp together AND use > $AllowedSender. > > Workaround in that case is to use the firewall, so I don't consider > it > > is worth fixing now. > > > > Please note that my testing revealed a potential memory leak as > > side-effect of the fixes. This could be abused to a remote DoS, so I > > will investigate that before releasing. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Monday, December 01, 2008 6:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > 3 > > > 83fc5969178887d00abfc > > > > > > Testing takes a bit more of time, I need to set up the test > > environment > > > for TLS again (looks like it would really pay to have a fixed test > > > suite > > > for all those cases - also the issue here would have never > > > occurred...). > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. The > > TLS > > > part should not be really affected by the problem: there are so > much > > > better access control features in TLS... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but seems > > to > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > e > > > > 18747b55d701e360d5aac > > > > > > > > Please note that this effectively disables GSS functionality. > I'll > > > > updated the GSS drivers in the next step. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > not > > a > > > > > trial fix. The design overlooked the situation. In theory, a > > whole > > > > new > > > > > access control feature would be needed. I am checking out if it > > is > > > > > possible to "just" enhance the interface. With the current > > > netstreams > > > > > defined that should be possible. I am tempted to release the > UDP- > > > > fixed > > > > > version and release the next version with the TCP fix. Feedback > > > from > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > depending > > > > > on how smart a way I find. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > ... and the patch will not work on all of these version, due > to > > > the > > > > > > introduction of the netstream driver functionality. Please > note > > > > that > > > > > > anything older than current v3-stable is outdated, so the > > proper > > > > way > > > > > to > > > > > > replace the faulty code is to upgrade to the current v3- > stable > > > and > > > > > > apply > > > > > > the patch. I will also release a new v3-stable soon, > hopefully > > > > today > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > I now clarified the affected versions. Affected are 3.12.2 > > and > > > > > above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > 6 > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on almost > > all > > > > > other > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > $AllowedSender > > > > > > > > directive > > > > > > > > > > does not work in all releases. The bug in question > is: > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > can > > > > lead > > > > > to > > > > > > > > > > messages > > > > > > > > > > being received from systems that are not permitted > so. > > As > > > a > > > > > > work- > > > > > > > > > > around, > > > > > > > > > > proper firewalling should be set up on the vulnerable > > > > hosts. > > > > > > > Until > > > > > > > > > > further note, I would assume that all versions of > > rsyslog > > > > are > > > > > > > > > affected > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Rainer > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 4 14:48:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 14:48:03 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> OK, 3.20.1 is now re-released as 3.20.2 (there were a few downloads...). The download link is still correct, it is updated (including the md5sum ;)). 3.21.8 is pulled and I'll restore it next. Sorry for the hassle. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, December 04, 2008 1:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > Grrr... One more issue. I noticed that while I resolved some conflicts > on the devel branch integration. There is an option that a log message > is emitted by rsyslog itself, when a remote machine's message is > discarded due to no permission. This was requested so that people know > when something goes wrong. This is only in the UDP code. > > HOWEVER, this is not rate-limited so if someone carries out a heavy > attack, he can still flood the local disk by these messages. I'll > change > it so that the message is emited only once every minute and will then > re-release what already has been released... > > Rainer > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Wednesday, December 03, 2008 11:30 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > The memory leak is now also fixed, I just quickly re-run some TLS > tests > > to make sure nothing is broken and it works there, too. > > > > Patch (on top of the others): > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > > d > > cb8703560c750f04a6370 > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Wednesday, December 03, 2008 10:54 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > Ok, I ran this fix through a couple of tests yesterday. It looks > well > > > for TLS, too. Note that there is an implication that $AllowedSender > > > TCP,... applies to TLS to (because it is TCP). I'd consider this to > > be > > > a > > > side-effect, but I do not think it is worth fixing. With TLS, there > > is > > > much finer and better control. An issue may only exists if someone > > > decides to run non-tls tcp and tls tcp together AND use > > $AllowedSender. > > > Workaround in that case is to use the firewall, so I don't consider > > it > > > is worth fixing now. > > > > > > Please note that my testing revealed a potential memory leak as > > > side-effect of the fixes. This could be abused to a remote DoS, so > I > > > will investigate that before releasing. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Monday, December 01, 2008 6:47 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > > 3 > > > > 83fc5969178887d00abfc > > > > > > > > Testing takes a bit more of time, I need to set up the test > > > environment > > > > for TLS again (looks like it would really pay to have a fixed > test > > > > suite > > > > for all those cases - also the issue here would have never > > > > occurred...). > > > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. > The > > > TLS > > > > part should not be really affected by the problem: there are so > > much > > > > better access control features in TLS... > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but > seems > > > to > > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > > e > > > > > 18747b55d701e360d5aac > > > > > > > > > > Please note that this effectively disables GSS functionality. > > I'll > > > > > updated the GSS drivers in the next step. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this is > > not > > > a > > > > > > trial fix. The design overlooked the situation. In theory, a > > > whole > > > > > new > > > > > > access control feature would be needed. I am checking out if > it > > > is > > > > > > possible to "just" enhance the interface. With the current > > > > netstreams > > > > > > defined that should be possible. I am tempted to release the > > UDP- > > > > > fixed > > > > > > version and release the next version with the TCP fix. > Feedback > > > > from > > > > > > packagers is appreciated. The TCP fix may take a day or two, > > > > > depending > > > > > > on how smart a way I find. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > ... and the patch will not work on all of these version, > due > > to > > > > the > > > > > > > introduction of the netstream driver functionality. Please > > note > > > > > that > > > > > > > anything older than current v3-stable is outdated, so the > > > proper > > > > > way > > > > > > to > > > > > > > replace the faulty code is to upgrade to the current v3- > > stable > > > > and > > > > > > > apply > > > > > > > the patch. I will also release a new v3-stable soon, > > hopefully > > > > > today > > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > I now clarified the affected versions. Affected are > 3.12.2 > > > and > > > > > > above. > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > > 6 > > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on > almost > > > all > > > > > > other > > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog- > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > Gerhards > > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > > $AllowedSender > > > > > > > > > directive > > > > > > > > > > > does not work in all releases. The bug in question > > is: > > > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, this > > can > > > > > lead > > > > > > to > > > > > > > > > > > messages > > > > > > > > > > > being received from systems that are not permitted > > so. > > > As > > > > a > > > > > > > work- > > > > > > > > > > > around, > > > > > > > > > > > proper firewalling should be set up on the > vulnerable > > > > > hosts. > > > > > > > > Until > > > > > > > > > > > further note, I would assume that all versions of > > > rsyslog > > > > > are > > > > > > > > > > affected > > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Rainer > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Thu Dec 4 16:03:30 2008 From: aoz.syn at gmail.com (RB) Date: Thu, 4 Dec 2008 08:03:30 -0700 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> Message-ID: <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> > Sorry for the hassle. It's not a hassle when you're being open and honest about issues. Too few projects call an apple an apple, so it's pleasing to be able to understand precisely what issues are. From Gerhardus.Geldenhuis at gta-travel.com Thu Dec 4 16:09:07 2008 From: Gerhardus.Geldenhuis at gta-travel.com (Gerhardus.Geldenhuis at gta-travel.com) Date: Thu, 4 Dec 2008 15:09:07 -0000 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> <4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> Message-ID: <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> I must concur, I am not very active on this mailing list in either form, but rsyslog does represent to me what open source was always about. The openness, speed and intelligence with which bugs/issues are addressed are exemplary. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: 04 December 2008 15:04 > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > > Sorry for the hassle. > > It's not a hassle when you're being open and honest about issues. Too > few projects call an apple an apple, so it's pleasing to be able to > understand precisely what issues are. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ From rgerhards at hq.adiscon.com Thu Dec 4 16:39:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 16:39:45 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com><4255c2570812040703p2882426fs9c8e9fb9ef79b27@mail.gmail.com> <1BCA52CF5845E543B4B81AAFEF2AFD7904AD25D6@LONSEXC01.gta.travel.lcl> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7F1@grfint2.intern.adiscon.com> Thanks folks for the nice statements. Obviously much appreciated ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gerhardus.Geldenhuis at gta- > travel.com > Sent: Thursday, December 04, 2008 4:09 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] security issue in rsyslog > > I must concur, I am not very active on this mailing list in either > form, > but rsyslog does represent to me what open source was always about. The > openness, speed and intelligence with which bugs/issues are addressed > are exemplary. > > Regards > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of RB > > Sent: 04 December 2008 15:04 > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > Sorry for the hassle. > > > > It's not a hassle when you're being open and honest about issues. > Too > > few projects call an apple an apple, so it's pleasing to be able to > > understand precisely what issues are. > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 4 17:40:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 4 Dec 2008 17:40:43 +0100 Subject: [rsyslog] security issue in rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F764@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F765@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F767@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F768@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F769@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F76E@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F776@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F778@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7C5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F7E6@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44F7E9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F7FF@grfint2.intern.adiscon.com> 3.21.8 has now also been replaced by 3.21.9. As with 3.20.2, links remain intact. 3.21.8 has probably never been downloaded, but I thought it is saver to use a new version number, especially as it is a security issue. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, December 04, 2008 2:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] security issue in rsyslog > > OK, 3.20.1 is now re-released as 3.20.2 (there were a few > downloads...). > The download link is still correct, it is updated (including the md5sum > ;)). 3.21.8 is pulled and I'll restore it next. > > Sorry for the hassle. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: Thursday, December 04, 2008 1:38 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] security issue in rsyslog > > > > Grrr... One more issue. I noticed that while I resolved some > conflicts > > on the devel branch integration. There is an option that a log > message > > is emitted by rsyslog itself, when a remote machine's message is > > discarded due to no permission. This was requested so that people > know > > when something goes wrong. This is only in the UDP code. > > > > HOWEVER, this is not rate-limited so if someone carries out a heavy > > attack, he can still flood the local disk by these messages. I'll > > change > > it so that the message is emited only once every minute and will then > > re-release what already has been released... > > > > Rainer > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: Wednesday, December 03, 2008 11:30 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > The memory leak is now also fixed, I just quickly re-run some TLS > > tests > > > to make sure nothing is broken and it works there, too. > > > > > > Patch (on top of the others): > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=b41bdeff56ad9d54dd > > > d > > > cb8703560c750f04a6370 > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: Wednesday, December 03, 2008 10:54 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > Ok, I ran this fix through a couple of tests yesterday. It looks > > well > > > > for TLS, too. Note that there is an implication that > $AllowedSender > > > > TCP,... applies to TLS to (because it is TCP). I'd consider this > to > > > be > > > > a > > > > side-effect, but I do not think it is worth fixing. With TLS, > there > > > is > > > > much finer and better control. An issue may only exists if > someone > > > > decides to run non-tls tcp and tls tcp together AND use > > > $AllowedSender. > > > > Workaround in that case is to use the firewall, so I don't > consider > > > it > > > > is worth fixing now. > > > > > > > > Please note that my testing revealed a potential memory leak as > > > > side-effect of the fixes. This could be abused to a remote DoS, > so > > I > > > > will investigate that before releasing. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: Monday, December 01, 2008 6:47 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > And now there is an *untested* fix for the TLS driver: > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=61b59a78c6b558ec06 > > > > > 3 > > > > > 83fc5969178887d00abfc > > > > > > > > > > Testing takes a bit more of time, I need to set up the test > > > > environment > > > > > for TLS again (looks like it would really pay to have a fixed > > test > > > > > suite > > > > > for all those cases - also the issue here would have never > > > > > occurred...). > > > > > > > > > > Please note that I mistook GSSAPI with TLS in my previous mail. > > The > > > > TLS > > > > > part should not be really affected by the problem: there are so > > > much > > > > > better access control features in TLS... > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > Sent: Monday, December 01, 2008 5:52 PM > > > > > > To: rsyslog-users > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > Ok, looks like I found a work-around. Not that elegant, but > > seems > > > > to > > > > > > work quite well. Patch for TCP is here: > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=97b89435aad77bd6d9 > > > > > > e > > > > > > 18747b55d701e360d5aac > > > > > > > > > > > > Please note that this effectively disables GSS functionality. > > > I'll > > > > > > updated the GSS drivers in the next step. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > Sent: Monday, December 01, 2008 5:08 PM > > > > > > > To: rsyslog-users > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > The issue also exists in TCP mode, but analysis shows this > is > > > not > > > > a > > > > > > > trial fix. The design overlooked the situation. In theory, > a > > > > whole > > > > > > new > > > > > > > access control feature would be needed. I am checking out > if > > it > > > > is > > > > > > > possible to "just" enhance the interface. With the current > > > > > netstreams > > > > > > > defined that should be possible. I am tempted to release > the > > > UDP- > > > > > > fixed > > > > > > > version and release the next version with the TCP fix. > > Feedback > > > > > from > > > > > > > packagers is appreciated. The TCP fix may take a day or > two, > > > > > > depending > > > > > > > on how smart a way I find. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > Sent: Monday, December 01, 2008 4:37 PM > > > > > > > > To: rsyslog-users > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > ... and the patch will not work on all of these version, > > due > > > to > > > > > the > > > > > > > > introduction of the netstream driver functionality. > Please > > > note > > > > > > that > > > > > > > > anything older than current v3-stable is outdated, so the > > > > proper > > > > > > way > > > > > > > to > > > > > > > > replace the faulty code is to upgrade to the current v3- > > > stable > > > > > and > > > > > > > > apply > > > > > > > > the patch. I will also release a new v3-stable soon, > > > hopefully > > > > > > today > > > > > > > > (but I'd like to conduct some more tests). > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog- > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > > > > > Sent: Monday, December 01, 2008 4:31 PM > > > > > > > > > To: rsyslog-users > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > I now clarified the affected versions. Affected are > > 3.12.2 > > > > and > > > > > > > above. > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog- > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > Gerhards > > > > > > > > > > Sent: Monday, December 01, 2008 3:32 PM > > > > > > > > > > To: rsyslog-users > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > this is patch for v3-stable: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=f0ddbed44c332391ae > > > > > > > > > > 6 > > > > > > > > > > d9bbf6b07e2f06c4dd676 > > > > > > > > > > > > > > > > > > > > I have not tried yet, but I think it will work on > > almost > > > > all > > > > > > > other > > > > > > > > > > versions, too. I keep you posted on the progress. > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog- > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > Gerhards > > > > > > > > > > > Sent: Monday, December 01, 2008 11:27 AM > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > Subject: Re: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > Version v2-stable is NOT vulnerable. > > > > > > > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com > > > > [mailto:rsyslog- > > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer > > > Gerhards > > > > > > > > > > > > Sent: Monday, December 01, 2008 10:55 AM > > > > > > > > > > > > To: rsyslog-users > > > > > > > > > > > > Subject: [rsyslog] security issue in rsyslog > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > thanks to a bug report, I found out that the > > > > > $AllowedSender > > > > > > > > > > directive > > > > > > > > > > > > does not work in all releases. The bug in > question > > > is: > > > > > > > > > > > > > > > > > > > > > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=111 > > > > > > > > > > > > > > > > > > > > > > > > Im am currently working on the bug. Obviously, > this > > > can > > > > > > lead > > > > > > > to > > > > > > > > > > > > messages > > > > > > > > > > > > being received from systems that are not > permitted > > > so. > > > > As > > > > > a > > > > > > > > work- > > > > > > > > > > > > around, > > > > > > > > > > > > proper firewalling should be set up on the > > vulnerable > > > > > > hosts. > > > > > > > > > Until > > > > > > > > > > > > further note, I would assume that all versions of > > > > rsyslog > > > > > > are > > > > > > > > > > > affected > > > > > > > > > > > > (I will provide more detail during my analysis). > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Rainer > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > rsyslog mailing list > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > > _______________________________________________ > > > > > > > > > > rsyslog mailing list > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > http://www.rsyslog.com > > > > > > > > > _______________________________________________ > > > > > > > > > rsyslog mailing list > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > http://www.rsyslog.com > > > > > > > > _______________________________________________ > > > > > > > > rsyslog mailing list > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > > > > > rsyslog mailing list > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > http://www.rsyslog.com > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Fri Dec 5 15:59:29 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 5 Dec 2008 15:59:29 +0100 Subject: [rsyslog] rsyslog 4.1.2 (v4-devel) released - IMPORTANT SECURITY RELEASE Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F81C@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 4.1.2, a member of the v4-development branch. Most importantly, this release addresses a security vulnerability that renders the $AllowedSender directive useless. This release has another security fix, which addresses a imudp which emitted a message each time a non-permitted sender tried to send a message. This could have filled the disk. Now it only emits a message once per minute. Further, all fixes and changes from 3.21.9 (beta) and 3.20.2 (stable) have been included in this release. Security Advisory: http://www.rsyslog.com/Article322.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-143.phtml Change Log: http://www.rsyslog.com/Article331.phtml All users are advised to update to this release. It is urgently recommended not only for those that would be vulnerable to the security issue but also to anyone using TLS-based communications. As always, feedback is appreciated. We hope this release will be useful. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From samuel at dragonboricua.net Mon Dec 8 06:25:30 2008 From: samuel at dragonboricua.net (Elisamuel Resto) Date: Mon, 8 Dec 2008 01:25:30 -0400 Subject: [rsyslog] rsyslog 3.21.8 (v3-beta) released - IMPORTANT SECURITY RELEASE In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44F7E5@grfint2.intern.adiscon.com> Message-ID: <20081208012530.20571aa8@zeus.dragonboricua.com> On Thu, 4 Dec 2008 13:34:08 +0100, Florian Riedl wrote: > Hi all, > > We have just released rsyslog 3.21.8, a member of the v3-beta branch. > Most importantly, this release addresses a security vulnerability the > renders the $AllowedSender directive useless. This issue has already > been discussed here on the list. In addition to this, the release also > contains all the bug fixes and enhancements from the stable release > 3.20.1. I believe the following is worthy of note in regards to this. On Thu, 4 Dec 2008 17:40:43 +0100, Rainer Gerhards wrote: > 3.21.8 has now also been replaced by 3.21.9. As with 3.20.2, links > remain intact. 3.21.8 has probably never been downloaded, but I thought > it is saver to use a new version number, especially as it is a security > issue. > > Rainer -- Elisamuel Resto Source Mage Tome Lead / http://sourcemage.org GPG ID: 18615F19/1024D / http://simplysam.us From nyerup at one.com Mon Dec 8 09:06:22 2008 From: nyerup at one.com (Jesper Dahl Nyerup) Date: Mon, 08 Dec 2008 09:06:22 +0100 Subject: [rsyslog] Cleanup after ugly shutdown Message-ID: <1228723582.3628.26.camel@iapetus.one.com> Hi all. I recently encountered an incident where rsyslogd died in an "ugly" manner. This was due to no fault of its own, but simply a matter of running out of disk space, because an instance, running with a disk-assisted memory queue, failed to reconnect to the syslog server, and therefore didn't have anywhere to put the log data, except on its own hard drive. When rsyslog dies like this (or in any other way that prevents a decent shutdown), it obviously do not write a .qi-file, indicating where it left off, and thus all the data written to disk is not handled when rsyslogd is started again. This behaviour is correct, as I see it, as at least some of the data written to disk, almost certainly will be inconsistent. But still I would like the opportunity to manually process these data, so that they eventually will be sent to the syslog server. What is the smartest way to properly do this? Has it been considered to let rsyslogd take a command line argument invoking such behaviour, or has someone written a seperate tool to do something like this? Any ideas are welcome. Kind regards, Jesper Nyerup. From rgerhards at hq.adiscon.com Mon Dec 8 10:42:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 10:42:46 +0100 Subject: [rsyslog] analysis of security issue in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F82E@grfint2.intern.adiscon.com> Hi all, I have done a bit of analysis of the issue and blogged about it: http://blog.gerhards.net/2008/12/root-cause-of-security-issue-in-rsyslog .html I hope this is useful information. As I wrote, I will try to focus on getting at basic test suite using DejaGNU (or anything else quickly suggested ;)) up and running. Any help would be appreciated. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 8 10:47:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 10:47:57 +0100 Subject: [rsyslog] make distcheck - multiple configure options possible? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F82F@grfint2.intern.adiscon.com> Hi all, I thought I ask this list before I search further. One thing that is annoying is that from time to time I have some minor nits because not all code pathes are even compiled due to the many configure options that exists (obviously even less are executed). I regularly run make distcheck, but as far as I know, I can only supply one set of configure options to it. Does anybody know a way to have multiple "make distcheck"s running, all with different set of configure options? All feedback is appreciated. Thanks. Rainer From rgerhards at hq.adiscon.com Mon Dec 8 12:33:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 12:33:30 +0100 Subject: [rsyslog] Security Fix for 3.18.5 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F83C@grfint2.intern.adiscon.com> Hi all, Michael Biebl made me aware that Debian Lenny uses rsyslog 3.18.5 and is unable to move to 3.20.x due to release freeze. I have now created a debian_lenny branch in git and backported the fixes to that branch. A quick test succeeded. So if anyone is interested in that version, he or she may pull it from git. If anyone finds a problem with that backport, please let me know. Thanks, Rainer From rgerhards at hq.adiscon.com Mon Dec 8 12:37:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 8 Dec 2008 12:37:42 +0100 Subject: [rsyslog] Cleanup after ugly shutdown In-Reply-To: <1228723582.3628.26.camel@iapetus.one.com> References: <1228723582.3628.26.camel@iapetus.one.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F83D@grfint2.intern.adiscon.com> Hi Jesper, one solution is to use the checkpoint interval. This ensures a queue file is written every n-th time a message is enqueued. This is inside the queue doc. On the "restart queue" option, I agree that it would be useful, but I unfortunately currently neither have time nor funding to implement it. You may want to add this as a feature request to the bugzilla, so that it is not forgotten if time arises... Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jesper Dahl Nyerup > Sent: Monday, December 08, 2008 9:06 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Cleanup after ugly shutdown > > Hi all. > > I recently encountered an incident where rsyslogd died in an "ugly" > manner. This was due to no fault of its own, but simply a matter of > running out of disk space, because an instance, running with a > disk-assisted memory queue, failed to reconnect to the syslog server, > and therefore didn't have anywhere to put the log data, except on its > own hard drive. > > When rsyslog dies like this (or in any other way that prevents a decent > shutdown), it obviously do not write a .qi-file, indicating where it > left off, and thus all the data written to disk is not handled when > rsyslogd is started again. This behaviour is correct, as I see it, as > at > least some of the data written to disk, almost certainly will be > inconsistent. > > But still I would like the opportunity to manually process these data, > so that they eventually will be sent to the syslog server. What is the > smartest way to properly do this? Has it been considered to let > rsyslogd > take a command line argument invoking such behaviour, or has someone > written a seperate tool to do something like this? > > Any ideas are welcome. > > > Kind regards, > > Jesper Nyerup. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Fri Dec 12 07:48:48 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Fri, 12 Dec 2008 07:48:48 +0100 Subject: [rsyslog] Disk error when using rsyslog Message-ID: <49420950.2030904@net-m.de> Hi Rainer, We're using rsyslog(3.20.0) as our central logging server and clients. We experienced an DISK ERROR on server last month. At that time, we were using TCP to transport logs from client to server. And we also setup the configuration just like [1]. But unfortunately our central logging server got DISK error for one hour. So we lost logfiles of that period of time. I've a look at doc [1] carefully, I guess "RELIABLE" only means when server got offline or rsyslogd on it isn't running, then clients will save logs in buffer or write to a file on disk. If server is still online and rsyslogd is running, but with IO/Error or Disk Full, then client will still transfer logs to server even with RELP, coz I guess RELP only protects logs could be transferred via network successfully, it doesn't care the logs are written successfully to file on server. Am I right? So I guess if we need to prevent this, we need do some work on server? Do we have some "directives" options that we could transfer logs to a failover server if local disk fails or buffer in memory before disk got corrected? [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html Thanks, -- Patrick Shen Operations Engineer From hks.private at gmail.com Fri Dec 12 16:09:42 2008 From: hks.private at gmail.com ((private) HKS) Date: Fri, 12 Dec 2008 10:09:42 -0500 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <49420950.2030904@net-m.de> References: <49420950.2030904@net-m.de> Message-ID: On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen wrote: > Hi Rainer, > > We're using rsyslog(3.20.0) as our central logging server and clients. > > We experienced an DISK ERROR on server last month. At that time, we were > using TCP to transport logs from client to server. And we also setup the > configuration just like [1]. But unfortunately our central logging > server got DISK error for one hour. So we lost logfiles of that period > of time. > > I've a look at doc [1] carefully, I guess "RELIABLE" only means when > server got offline or rsyslogd on it isn't running, then clients will > save logs in buffer or write to a file on disk. If server is still > online and rsyslogd is running, but with IO/Error or Disk Full, then > client will still transfer logs to server even with RELP, coz I guess > RELP only protects logs could be transferred via network successfully, > it doesn't care the logs are written successfully to file on server. Am > I right? Yes. RELP Is a protocol for the reliable exchange of event logs over a network. What the destination daemon does once it has the logs is no concern of the client's. > So I guess if we need to prevent this, we need do some work on server? > Do we have some "directives" options that we could transfer logs to a > failover server if local disk fails or buffer in memory before disk got > corrected? Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer -HKS > [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html > > Thanks, > > -- > Patrick Shen > Operations Engineer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Dec 12 16:16:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 12 Dec 2008 16:16:21 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: References: <49420950.2030904@net-m.de> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> I seem to have overlooked the initial message (spam filter maybe...). HKS is right (thx), but I think this looks like a bug in that the output write does not care about the write failure (what it should). The output writer is pretty old legacy code, so that's quite possible. I'll look into it ASAP, but I got a new machine (hopefully fast enough to disply some troubles) today and currently I am happy that at least mail does work again (so far it's a mess). So... some time next week ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of (private) HKS > Sent: Friday, December 12, 2008 4:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Disk error when using rsyslog > > On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen > wrote: > > Hi Rainer, > > > > We're using rsyslog(3.20.0) as our central logging server and > clients. > > > > We experienced an DISK ERROR on server last month. At that time, we > were > > using TCP to transport logs from client to server. And we also setup > the > > configuration just like [1]. But unfortunately our central logging > > server got DISK error for one hour. So we lost logfiles of that > period > > of time. > > > > I've a look at doc [1] carefully, I guess "RELIABLE" only means when > > server got offline or rsyslogd on it isn't running, then clients will > > save logs in buffer or write to a file on disk. If server is still > > online and rsyslogd is running, but with IO/Error or Disk Full, then > > client will still transfer logs to server even with RELP, coz I guess > > RELP only protects logs could be transferred via network > successfully, > > it doesn't care the logs are written successfully to file on server. > Am > > I right? > > Yes. RELP Is a protocol for the reliable exchange of event logs over a > network. What the destination daemon does once it has the logs is no > concern of the client's. > > > > So I guess if we need to prevent this, we need do some work on > server? > > Do we have some "directives" options that we could transfer logs to a > > failover server if local disk fails or buffer in memory before disk > got > > corrected? > > Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > -HKS > > > [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html > > > > Thanks, > > > > -- > > Patrick Shen > > Operations Engineer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Sun Dec 14 04:46:38 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Sun, 14 Dec 2008 04:46:38 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: References: <49420950.2030904@net-m.de> Message-ID: <4944819E.90101@net-m.de> Hi HKS, Thanks for your good advice. It's helpful. Patrick (private) HKS wrote: > On Fri, Dec 12, 2008 at 1:48 AM, Patrick Shen wrote: >> Hi Rainer, >> >> We're using rsyslog(3.20.0) as our central logging server and clients. >> >> We experienced an DISK ERROR on server last month. At that time, we were >> using TCP to transport logs from client to server. And we also setup the >> configuration just like [1]. But unfortunately our central logging >> server got DISK error for one hour. So we lost logfiles of that period >> of time. >> >> I've a look at doc [1] carefully, I guess "RELIABLE" only means when >> server got offline or rsyslogd on it isn't running, then clients will >> save logs in buffer or write to a file on disk. If server is still >> online and rsyslogd is running, but with IO/Error or Disk Full, then >> client will still transfer logs to server even with RELP, coz I guess >> RELP only protects logs could be transferred via network successfully, >> it doesn't care the logs are written successfully to file on server. Am >> I right? > > Yes. RELP Is a protocol for the reliable exchange of event logs over a > network. What the destination daemon does once it has the logs is no > concern of the client's. > > >> So I guess if we need to prevent this, we need do some work on server? >> Do we have some "directives" options that we could transfer logs to a >> failover server if local disk fails or buffer in memory before disk got >> corrected? > > Yes: http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > -HKS > >> [1]: http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html >> >> Thanks, >> >> -- >> Patrick Shen >> Operations Engineer >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From patrick.shen at net-m.de Sun Dec 14 04:56:16 2008 From: patrick.shen at net-m.de (Patrick Shen) Date: Sun, 14 Dec 2008 04:56:16 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> Message-ID: <494483E0.3030702@net-m.de> Hi Rainer, Rainer Gerhards wrote: > I seem to have overlooked the initial message (spam filter maybe...). Bad luck for my mail account :-( > HKS is right (thx), but I think this looks like a bug in that the output > write does not care about the write failure (what it should). The output > writer is pretty old legacy code, so that's quite possible. I'll look > into it ASAP, but I got a new machine (hopefully fast enough to disply > some troubles) today and currently I am happy that at least mail does > work again (so far it's a mess). So... some time next week ;) Quite appreciate if you have a look at write failure. I think it's quite Enterprise demand feature :-) Many thanks, Patrick From rgerhards at hq.adiscon.com Mon Dec 15 12:03:53 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 15 Dec 2008 12:03:53 +0100 Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <494483E0.3030702@net-m.de> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> <494483E0.3030702@net-m.de> Message-ID: <1229339033.2482.3.camel@localhost.localdomain> On Sun, 2008-12-14 at 04:56 +0100, Patrick Shen wrote: > Hi Rainer, > > Rainer Gerhards wrote: > > I seem to have overlooked the initial message (spam filter maybe...). > > Bad luck for my mail account :-( > > > HKS is right (thx), but I think this looks like a bug in that the output > > write does not care about the write failure (what it should). The output > > writer is pretty old legacy code, so that's quite possible. I'll look > > into it ASAP, but I got a new machine (hopefully fast enough to disply > > some troubles) today and currently I am happy that at least mail does > > work again (so far it's a mess). So... some time next week ;) > > Quite appreciate if you have a look at write failure. I think it's quite > Enterprise demand feature :-) I have now verified that the code (by intension) ignores write errors. That, of course, is legacy from a long gone era. However, I need to think a bit about how to handle this most intelligently. The problem is partial writes. Maybe I just try to write a LF after a failure and, if that succeeds, simply continue. This results in a partial record begin written and then the same record being "duplicated" (actually, the partial part being duplicated). Does anyone have a suggestion on how to best handle such a case? Or I could try to write what could not yet be written. Maybe this is better, but it wont' be able to survive a daemon restart... Feedback appreciated. Rainer From david at lang.hm Tue Dec 16 10:23:37 2008 From: david at lang.hm (david at lang.hm) Date: Tue, 16 Dec 2008 01:23:37 -0800 (PST) Subject: [rsyslog] Disk error when using rsyslog In-Reply-To: <1229339033.2482.3.camel@localhost.localdomain> References: <49420950.2030904@net-m.de> <577465F99B41C842AAFBE9ED71E70ABA44F8BB@grfint2.intern.adiscon.com> <494483E0.3030702@net-m.de> <1229339033.2482.3.camel@localhost.localdomain> Message-ID: On Mon, 15 Dec 2008, Rainer Gerhards wrote: > On Sun, 2008-12-14 at 04:56 +0100, Patrick Shen wrote: >> Hi Rainer, >> >> Rainer Gerhards wrote: >>> I seem to have overlooked the initial message (spam filter maybe...). >> >> Bad luck for my mail account :-( >> >>> HKS is right (thx), but I think this looks like a bug in that the output >>> write does not care about the write failure (what it should). The output >>> writer is pretty old legacy code, so that's quite possible. I'll look >>> into it ASAP, but I got a new machine (hopefully fast enough to disply >>> some troubles) today and currently I am happy that at least mail does >>> work again (so far it's a mess). So... some time next week ;) >> >> Quite appreciate if you have a look at write failure. I think it's quite >> Enterprise demand feature :-) > > I have now verified that the code (by intension) ignores write errors. > That, of course, is legacy from a long gone era. However, I need to > think a bit about how to handle this most intelligently. The problem is > partial writes. Maybe I just try to write a LF after a failure and, if > that succeeds, simply continue. This results in a partial record begin > written and then the same record being "duplicated" (actually, the > partial part being duplicated). > > Does anyone have a suggestion on how to best handle such a case? Or I > could try to write what could not yet be written. Maybe this is better, > but it wont' be able to survive a daemon restart... > > Feedback appreciated. this sounds like a smaller portion of the problem we were discussing when we were talking about de-queuing multiple messages. this approach would work, but if you know you have a file you could also truncate the file to the end of the previous record (the beginning of the record you are trying to write). David Lang From william at blocket.se Tue Dec 16 17:39:53 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Tue, 16 Dec 2008 17:39:53 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs Message-ID: Hi, I don't know if this is an intended feature or not, but when CreateDirs is set to off, rsyslog can no longer create files (time stamp file names). Lets say you have a template for writing files: / log//.log, this will let anybody to create directories in / log when CreateDirs is on. This may cause a lot of directories and can be abused by regular users. I've patched 2.0.2 to separate file and directory creation here: https://bugzilla.redhat.com/attachment.cgi?id=327100 And steps to reproduce this can be found here: https://bugzilla.redhat.com/show_bug.cgi?id=473419 Comments anyone? William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From rgerhards at hq.adiscon.com Thu Dec 18 12:01:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 12:01:25 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: References: Message-ID: <1229598085.7471.2.camel@localhost.localdomain> Hi William, thanks for the bug report and the patch. I have now looked into the issue and could reproduce the situation. The patch obviously resolves it. There is one thing, though: I do not fully understand why you have patched syslogd.c (RS_RET_ERR). I would appreciate if you could provide a bit more insight into the problem you intend to solve. So far, I can not see why this part of the patch is actually needed. I'll integrate the omfile part now. I will also see that I apply this to the other official branches. Thanks, Rainer On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > Hi, > > I don't know if this is an intended feature or not, but when > CreateDirs is set to off, rsyslog can no longer create files (time > stamp file names). Lets say you have a template for writing files: / > log//.log, this will let anybody to create directories in / > log when CreateDirs is on. This may cause a lot of directories and can > be abused by regular users. > > I've patched 2.0.2 to separate file and directory creation here: > https://bugzilla.redhat.com/attachment.cgi?id=327100 > > And steps to reproduce this can be found here: > https://bugzilla.redhat.com/show_bug.cgi?id=473419 > > Comments anyone? > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From friedl at hq.adiscon.com Thu Dec 18 13:03:48 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 18 Dec 2008 13:03:48 +0100 Subject: [rsyslog] rsyslog 4.1.3 (devel) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F8F9@grfint2.intern.adiscon.com> Hi all, rsyslog 4.1.3, a member of the development branch, has been released today. Most importantly, it offers a bugfix for some situations where the imudp input module could run in a tight loop. The release also offers enhanced functionality, among others a work-around to handle the invalid syslog/tcp framing used by NetScreen. There are two more configuration-options which can be used for fine-tuning functionality (see changelog for details). We would like to express my gratitude to BlinkMind, Inc. (http://www.blinkmind.com), who sponsored the development of the $PreserveFQDN configuration option. Download http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-145.phtml Changelog http://www.rsyslog.com/Article335.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From william at blocket.se Thu Dec 18 14:35:00 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 14:35:00 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229598085.7471.2.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> Message-ID: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Hi, Lets see if I can get this right: My modification in prepareFile() will return with (pData->fd = -1) if the log file can't be created. In prepareDynFile() we run prepareFile() and return with -1 if pData- >fd is set to -1. In writeFile() we run prepareDynFile() and return RS_RET_ERR if prepareDynFile() is not returned with 0. writeFile() is wrapped in doAction(). doAction() is exectued in fprintlog() where RS_RET_ERR never will be catched. I discard the log message and sets the error flag to tell the "msg repeated"-check to not log this message ("msg repeated" is executed before we try to open the file if the message content match the previous message). I tried without this catch in the first attempt, but I could see the message stuck in the loop, every action to rsyslog tried to open the file. This and some traffic volume caused rsyslog to hang (and use a lot of i/o). William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > Hi William, > > thanks for the bug report and the patch. I have now looked into the > issue and could reproduce the situation. The patch obviously resolves > it. > > There is one thing, though: I do not fully understand why you have > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > provide > a bit more insight into the problem you intend to solve. So far, I can > not see why this part of the patch is actually needed. > > I'll integrate the omfile part now. I will also see that I apply > this to > the other official branches. > > Thanks, > Rainer > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >> Hi, >> >> I don't know if this is an intended feature or not, but when >> CreateDirs is set to off, rsyslog can no longer create files (time >> stamp file names). Lets say you have a template for writing files: / >> log//.log, this will let anybody to create directories >> in / >> log when CreateDirs is on. This may cause a lot of directories and >> can >> be abused by regular users. >> >> I've patched 2.0.2 to separate file and directory creation here: >> https://bugzilla.redhat.com/attachment.cgi?id=327100 >> >> And steps to reproduce this can be found here: >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >> >> Comments anyone? >> >> >> William Tis?ter >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From william at blocket.se Thu Dec 18 15:22:53 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 15:22:53 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Message-ID: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> I've debugged some more, this only happens when you trigger the repeat- loop to the missing directory. E.g: # logger -t non_existent_dir -p local0.crit test # logger -t non_existent_dir -p local0.crit test # logger -t existing_dir -p local0.crit test Debug output from last command: Filter: check for property 'programname' (value 'existing_dir') NOT regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE 1092925760: Called fprintlog, logging to builtin-file (CritByDay) 1092925760: Called fprintlog, logging to builtin-file (CritByDay) Filter: check for property 'msg' (value ' test') contains '(CRIT)': FALSE 1092925760: Called fprintlog, logging to builtin-file (DirByTagFileByDay) 1092925760: Called logerr, msg: Could not open dynamic file '/log/ non_existent_dir/2008-12-18.log' - discarding message 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding message: No such file or directory 1092925760: Message has legacy syslog format. 1092925760: EnqueueMsg signaled condition (0) 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. 1092925760: Lone worker is running... 1092925760: msg repeated 1 times, 10 sec of 30. Filter: check for property 'syslogfacility-text' (value 'syslog') NOT isequal 'local0': TRUE 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: queue EMPTY, waiting for next message. William Tis?ter Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > Hi, > > Lets see if I can get this right: > > My modification in prepareFile() will return with (pData->fd = -1) > if the log file can't be created. > In prepareDynFile() we run prepareFile() and return with -1 if pData- > >fd is set to -1. > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > prepareDynFile() is not returned with 0. > > writeFile() is wrapped in doAction(). > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > catched. I discard the log message and sets the error flag to tell > the "msg repeated"-check to not log this message ("msg repeated" is > executed before we try to open the file if the message content match > the previous message). > > I tried without this catch in the first attempt, but I could see the > message stuck in the loop, every action to rsyslog tried to open the > file. This and some traffic volume caused rsyslog to hang (and use a > lot of i/o). > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > >> Hi William, >> >> thanks for the bug report and the patch. I have now looked into the >> issue and could reproduce the situation. The patch obviously resolves >> it. >> >> There is one thing, though: I do not fully understand why you have >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >> provide >> a bit more insight into the problem you intend to solve. So far, I >> can >> not see why this part of the patch is actually needed. >> >> I'll integrate the omfile part now. I will also see that I apply >> this to >> the other official branches. >> >> Thanks, >> Rainer >> >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>> Hi, >>> >>> I don't know if this is an intended feature or not, but when >>> CreateDirs is set to off, rsyslog can no longer create files (time >>> stamp file names). Lets say you have a template for writing files: / >>> log//.log, this will let anybody to create directories >>> in / >>> log when CreateDirs is on. This may cause a lot of directories and >>> can >>> be abused by regular users. >>> >>> I've patched 2.0.2 to separate file and directory creation here: >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>> >>> And steps to reproduce this can be found here: >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>> >>> Comments anyone? >>> >>> >>> William Tis?ter >>> >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>> http://www.blocket.se/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 18 15:58:50 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 15:58:50 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> Message-ID: <1229612330.12594.8.camel@localhost.localdomain> Hi William, I had another look at the code. I am not picky, I just want to make sure I understand the failure scenario. The v3 versions have a different approach at this, and I need to be sure I understand the issue so that I can check the newer releases for it. more inline below... On Thu, 2008-12-18 at 14:35 +0100, William Tis?ter wrote: > Hi, > > Lets see if I can get this right: > > My modification in prepareFile() will return with (pData->fd = -1) if > the log file can't be created. > In prepareDynFile() we run prepareFile() and return with -1 if pData- > >fd is set to -1. > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > prepareDynFile() is not returned with 0. > > writeFile() is wrapped in doAction(). > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > catched. Well.. kind of. I do not check for a specific error state, but I check if all went well (if iRet == RS_RE_OK). Only then the previous count is reset, otherwise it is left as is. An output module may return a large number of errors, so it is not sufficient to check for a specific error case. That's also why I am trying to understand the issue you still see. I am sure it can occur under other circumstances, too (as I said, there are various error codes). > I discard the log message and sets the error flag to tell the > "msg repeated"-check to not log this message ("msg repeated" is > executed before we try to open the file if the message content match > the previous message). ... but this is done via fprintlog(), so the logic to try open the file is still included. > > I tried without this catch in the first attempt, but I could see the > message stuck in the loop, As of my understanding, it should not be the very same message, because that is discarded. It can be the next message, which of course will trigger the "last message repeated n times" text. > every action to rsyslog tried to open the > file. This and some traffic volume caused rsyslog to hang (and use a > lot of i/o). Do you have a procedure to reproduce that? Would be most interesting. As a side-Note, I think there is a memory leak in the patch, you will probably want to correct it. The return in line 3392 does not do the necessary cleanup (the return further UP is a different case, see its comments). Use "ABORT_FINALIZE(RS_RET_DISCARDMSG);" instead, it will invoke the finalizer, which will free some string space. > Rainer > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > Hi William, > > > > thanks for the bug report and the patch. I have now looked into the > > issue and could reproduce the situation. The patch obviously > resolves > > it. > > > > There is one thing, though: I do not fully understand why you have > > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > > provide > > a bit more insight into the problem you intend to solve. So far, I > can > > not see why this part of the patch is actually needed. > > > > I'll integrate the omfile part now. I will also see that I apply > > this to > > the other official branches. > > > > Thanks, > > Rainer > > > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >> Hi, > >> > >> I don't know if this is an intended feature or not, but when > >> CreateDirs is set to off, rsyslog can no longer create files (time > >> stamp file names). Lets say you have a template for writing > files: / > >> log//.log, this will let anybody to create directories > >> in / > >> log when CreateDirs is on. This may cause a lot of directories and > >> can > >> be abused by regular users. > >> > >> I've patched 2.0.2 to separate file and directory creation here: > >> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >> > >> And steps to reproduce this can be found here: > >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >> > >> Comments anyone? > >> > >> > >> William Tis?ter > >> > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >> http://www.blocket.se/ > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:03:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:03:24 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229612330.12594.8.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <1229612330.12594.8.camel@localhost.localdomain> Message-ID: <1229612604.12594.11.camel@localhost.localdomain> oops, one thing that occured to me just now. RS_RET_DISCARDMSG is probably not what you want. This totally *discards* the message from the queue. This state was introduced for the discard action. For example, if your config ist *.* ?dynafile *.* file1 *.* file2 and you get in issue during ?dynafile processing, the message will never be written to file1 and file2. So it is better to leave it at RS_RET_ERR. This, however, brings us close to the code as it currently is (except for the additional flag, which I am trying to understand ;)) Rainer On Thu, 2008-12-18 at 15:58 +0100, Rainer Gerhards wrote: > Hi William, > > I had another look at the code. I am not picky, I just want to make > sure > I understand the failure scenario. The v3 versions have a different > approach at this, and I need to be sure I understand the issue so that > I > can check the newer releases for it. > > more inline below... > > On Thu, 2008-12-18 at 14:35 +0100, William Tis?ter wrote: > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > if > > the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will be > > catched. > > Well.. kind of. I do not check for a specific error state, but I check > if all went well (if iRet == RS_RE_OK). Only then the previous count > is > reset, otherwise it is left as is. An output module may return a large > number of errors, so it is not sufficient to check for a specific > error > case. That's also why I am trying to understand the issue you still > see. > I am sure it can occur under other circumstances, too (as I said, > there > are various error codes). > > > I discard the log message and sets the error flag to tell the > > "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content match > > the previous message). > > ... but this is done via fprintlog(), so the logic to try open the > file > is still included. > > > > I tried without this catch in the first attempt, but I could see the > > message stuck in the loop, > > As of my understanding, it should not be the very same message, > because > that is discarded. It can be the next message, which of course will > trigger the "last message repeated n times" text. > > > every action to rsyslog tried to open the > > file. This and some traffic volume caused rsyslog to hang (and use a > > lot of i/o). > > Do you have a procedure to reproduce that? Would be most interesting. > > As a side-Note, I think there is a memory leak in the patch, you will > probably want to correct it. The return in line 3392 does not do the > necessary cleanup (the return further UP is a different case, see its > comments). Use "ABORT_FINALIZE(RS_RET_DISCARDMSG);" instead, it will > invoke the finalizer, which will free some string space. > > > > Rainer > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > > > Hi William, > > > > > > thanks for the bug report and the patch. I have now looked into > the > > > issue and could reproduce the situation. The patch obviously > > resolves > > > it. > > > > > > There is one thing, though: I do not fully understand why you have > > > patched syslogd.c (RS_RET_ERR). I would appreciate if you could > > > provide > > > a bit more insight into the problem you intend to solve. So far, I > > can > > > not see why this part of the patch is actually needed. > > > > > > I'll integrate the omfile part now. I will also see that I apply > > > this to > > > the other official branches. > > > > > > Thanks, > > > Rainer > > > > > > On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > > >> Hi, > > >> > > >> I don't know if this is an intended feature or not, but when > > >> CreateDirs is set to off, rsyslog can no longer create files > (time > > >> stamp file names). Lets say you have a template for writing > > files: / > > >> log//.log, this will let anybody to create directories > > >> in / > > >> log when CreateDirs is on. This may cause a lot of directories > and > > >> can > > >> be abused by regular users. > > >> > > >> I've patched 2.0.2 to separate file and directory creation here: > > >> https://bugzilla.redhat.com/attachment.cgi?id=327100 > > >> > > >> And steps to reproduce this can be found here: > > >> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > > >> > > >> Comments anyone? > > >> > > >> > > >> William Tis?ter > > >> > > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > >> http://www.blocket.se/ > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:04:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:04:19 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> Message-ID: <1229612659.12594.12.camel@localhost.localdomain> The spam filter caught this message - sorry. I'll test this scenario. Rainer On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > I've debugged some more, this only happens when you trigger the > repeat- > loop to the missing directory. > > E.g: > > # logger -t non_existent_dir -p local0.crit test > # logger -t non_existent_dir -p local0.crit test > # logger -t existing_dir -p local0.crit test > > Debug output from last command: > > Filter: check for property 'programname' (value 'existing_dir') NOT > regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > Filter: check for property 'msg' (value ' test') contains '(CRIT)': > FALSE > 1092925760: Called fprintlog, logging to builtin-file > (DirByTagFileByDay) > 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > non_existent_dir/2008-12-18.log' - discarding message > 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding > message: No such file or directory > 1092925760: Message has legacy syslog format. > 1092925760: EnqueueMsg signaled condition (0) > 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. > 1092925760: Lone worker is running... > 1092925760: msg repeated 1 times, 10 sec of 30. > Filter: check for property 'syslogfacility-text' (value 'syslog') NOT > isequal 'local0': TRUE > 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: > queue EMPTY, waiting for next message. > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > > if the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will > be > > catched. I discard the log message and sets the error flag to tell > > the "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content > match > > the previous message). > > > > I tried without this catch in the first attempt, but I could see > the > > message stuck in the loop, every action to rsyslog tried to open > the > > file. This and some traffic volume caused rsyslog to hang (and use > a > > lot of i/o). > > > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > >> Hi William, > >> > >> thanks for the bug report and the patch. I have now looked into the > >> issue and could reproduce the situation. The patch obviously > resolves > >> it. > >> > >> There is one thing, though: I do not fully understand why you have > >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >> provide > >> a bit more insight into the problem you intend to solve. So far, I > >> can > >> not see why this part of the patch is actually needed. > >> > >> I'll integrate the omfile part now. I will also see that I apply > >> this to > >> the other official branches. > >> > >> Thanks, > >> Rainer > >> > >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>> Hi, > >>> > >>> I don't know if this is an intended feature or not, but when > >>> CreateDirs is set to off, rsyslog can no longer create files (time > >>> stamp file names). Lets say you have a template for writing > files: / > >>> log//.log, this will let anybody to create directories > >>> in / > >>> log when CreateDirs is on. This may cause a lot of directories > and > >>> can > >>> be abused by regular users. > >>> > >>> I've patched 2.0.2 to separate file and directory creation here: > >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>> > >>> And steps to reproduce this can be found here: > >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>> > >>> Comments anyone? > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From rgerhards at hq.adiscon.com Thu Dec 18 16:12:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:12:34 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> Message-ID: <1229613154.12594.13.camel@localhost.localdomain> mmhhhh... I can not reproduce the issue with my state of the code. Can you verify that it still occurs? A git snapshot is available here: http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz Thanks, Rainer On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > I've debugged some more, this only happens when you trigger the > repeat- > loop to the missing directory. > > E.g: > > # logger -t non_existent_dir -p local0.crit test > # logger -t non_existent_dir -p local0.crit test > # logger -t existing_dir -p local0.crit test > > Debug output from last command: > > Filter: check for property 'programname' (value 'existing_dir') NOT > regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > Filter: check for property 'msg' (value ' test') contains '(CRIT)': > FALSE > 1092925760: Called fprintlog, logging to builtin-file > (DirByTagFileByDay) > 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > non_existent_dir/2008-12-18.log' - discarding message > 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding > message: No such file or directory > 1092925760: Message has legacy syslog format. > 1092925760: EnqueueMsg signaled condition (0) > 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. > 1092925760: Lone worker is running... > 1092925760: msg repeated 1 times, 10 sec of 30. > Filter: check for property 'syslogfacility-text' (value 'syslog') NOT > isequal 'local0': TRUE > 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: > queue EMPTY, waiting for next message. > > > William Tis?ter > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > > > Hi, > > > > Lets see if I can get this right: > > > > My modification in prepareFile() will return with (pData->fd = -1) > > if the log file can't be created. > > In prepareDynFile() we run prepareFile() and return with -1 if > pData- > > >fd is set to -1. > > In writeFile() we run prepareDynFile() and return RS_RET_ERR if > > prepareDynFile() is not returned with 0. > > > > writeFile() is wrapped in doAction(). > > > > doAction() is exectued in fprintlog() where RS_RET_ERR never will > be > > catched. I discard the log message and sets the error flag to tell > > the "msg repeated"-check to not log this message ("msg repeated" is > > executed before we try to open the file if the message content > match > > the previous message). > > > > I tried without this catch in the first attempt, but I could see > the > > message stuck in the loop, every action to rsyslog tried to open > the > > file. This and some traffic volume caused rsyslog to hang (and use > a > > lot of i/o). > > > > > > William Tis?ter > > > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > > http://www.blocket.se/ > > > > On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > > > >> Hi William, > >> > >> thanks for the bug report and the patch. I have now looked into the > >> issue and could reproduce the situation. The patch obviously > resolves > >> it. > >> > >> There is one thing, though: I do not fully understand why you have > >> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >> provide > >> a bit more insight into the problem you intend to solve. So far, I > >> can > >> not see why this part of the patch is actually needed. > >> > >> I'll integrate the omfile part now. I will also see that I apply > >> this to > >> the other official branches. > >> > >> Thanks, > >> Rainer > >> > >> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>> Hi, > >>> > >>> I don't know if this is an intended feature or not, but when > >>> CreateDirs is set to off, rsyslog can no longer create files (time > >>> stamp file names). Lets say you have a template for writing > files: / > >>> log//.log, this will let anybody to create directories > >>> in / > >>> log when CreateDirs is on. This may cause a lot of directories > and > >>> can > >>> be abused by regular users. > >>> > >>> I've patched 2.0.2 to separate file and directory creation here: > >>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>> > >>> And steps to reproduce this can be found here: > >>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>> > >>> Comments anyone? > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From william at blocket.se Thu Dec 18 16:35:08 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 16:35:08 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229613154.12594.13.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> Message-ID: <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> Yes, I could reproduce my error, the same way as described earlier. My /etc/rsyslog.conf: $template DirByTagFileByDay,"/log/%programname%/%timegenerated: 1:10:date-rfc3339%.log" $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" # Discard all but local0 :syslogfacility-text, !isequal, "local0" ~ $umask 0000 $FileCreateMode 0644 $DirCreateMode 0755 $CreateDirs off # Discard strange tags :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ badtags.log :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ # Collect all crits in one log local0.crit ?CritByDay :msg, contains, "(CRIT)" ?CritByDay # All local0's buffered to their own dirs local0.* -?DirByTagFileByDay / William On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: > mmhhhh... I can not reproduce the issue with my state of the code. Can > you verify that it still occurs? A git snapshot is available here: > > http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz > > Thanks, > Rainer > > On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: >> I've debugged some more, this only happens when you trigger the >> repeat- >> loop to the missing directory. >> >> E.g: >> >> # logger -t non_existent_dir -p local0.crit test >> # logger -t non_existent_dir -p local0.crit test >> # logger -t existing_dir -p local0.crit test >> >> Debug output from last command: >> >> Filter: check for property 'programname' (value 'existing_dir') NOT >> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >> Filter: check for property 'msg' (value ' test') contains '(CRIT)': >> FALSE >> 1092925760: Called fprintlog, logging to builtin-file >> (DirByTagFileByDay) >> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ >> non_existent_dir/2008-12-18.log' - discarding message >> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not >> open dynamic file '/log/non_existent_dir/2008-12-18.log' - discarding >> message: No such file or directory >> 1092925760: Message has legacy syslog format. >> 1092925760: EnqueueMsg signaled condition (0) >> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from dynaCache. >> 1092925760: Lone worker is running... >> 1092925760: msg repeated 1 times, 10 sec of 30. >> Filter: check for property 'syslogfacility-text' (value 'syslog') NOT >> isequal 'local0': TRUE >> 1092925760: Called fprintlog, logging to builtin-discardsingleWorker: >> queue EMPTY, waiting for next message. >> >> >> William Tis?ter >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> >> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: >> >>> Hi, >>> >>> Lets see if I can get this right: >>> >>> My modification in prepareFile() will return with (pData->fd = -1) >>> if the log file can't be created. >>> In prepareDynFile() we run prepareFile() and return with -1 if >> pData- >>>> fd is set to -1. >>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if >>> prepareDynFile() is not returned with 0. >>> >>> writeFile() is wrapped in doAction(). >>> >>> doAction() is exectued in fprintlog() where RS_RET_ERR never will >> be >>> catched. I discard the log message and sets the error flag to tell >>> the "msg repeated"-check to not log this message ("msg repeated" is >>> executed before we try to open the file if the message content >> match >>> the previous message). >>> >>> I tried without this catch in the first attempt, but I could see >> the >>> message stuck in the loop, every action to rsyslog tried to open >> the >>> file. This and some traffic volume caused rsyslog to hang (and use >> a >>> lot of i/o). >>> >>> >>> William Tis?ter >>> >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>> http://www.blocket.se/ >>> >>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: >>> >>>> Hi William, >>>> >>>> thanks for the bug report and the patch. I have now looked into the >>>> issue and could reproduce the situation. The patch obviously >> resolves >>>> it. >>>> >>>> There is one thing, though: I do not fully understand why you have >>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >>>> provide >>>> a bit more insight into the problem you intend to solve. So far, I >>>> can >>>> not see why this part of the patch is actually needed. >>>> >>>> I'll integrate the omfile part now. I will also see that I apply >>>> this to >>>> the other official branches. >>>> >>>> Thanks, >>>> Rainer >>>> >>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>>>> Hi, >>>>> >>>>> I don't know if this is an intended feature or not, but when >>>>> CreateDirs is set to off, rsyslog can no longer create files (time >>>>> stamp file names). Lets say you have a template for writing >> files: / >>>>> log//.log, this will let anybody to create directories >>>>> in / >>>>> log when CreateDirs is on. This may cause a lot of directories >> and >>>>> can >>>>> be abused by regular users. >>>>> >>>>> I've patched 2.0.2 to separate file and directory creation here: >>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>>>> >>>>> And steps to reproduce this can be found here: >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>>>> >>>>> Comments anyone? >>>>> >>>>> >>>>> William Tis?ter >>>>> >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>> http://www.blocket.se/ >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >> >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com William Tis?ter Cell: +46 (0)76 3377182 Mail: william at blocket.se Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From rgerhards at hq.adiscon.com Thu Dec 18 16:44:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 16:44:04 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> Message-ID: <1229615044.12594.14.camel@localhost.localdomain> would it be possible that you mail me (privately) a full debug log from such a run. Not sure if there is too much confidential data in it... Rainer On Thu, 2008-12-18 at 16:35 +0100, William Tis?ter wrote: > Yes, I could reproduce my error, the same way as described earlier. > > My /etc/rsyslog.conf: > > $template DirByTagFileByDay,"/log/%programname%/%timegenerated: > 1:10:date-rfc3339%.log" > $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" > > # Discard all but local0 > :syslogfacility-text, !isequal, "local0" ~ > > $umask 0000 > $FileCreateMode 0644 > $DirCreateMode 0755 > $CreateDirs off > > # Discard strange tags > :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ > badtags.log > :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ > > # Collect all crits in one log > local0.crit ?CritByDay > :msg, contains, "(CRIT)" ?CritByDay > > # All local0's buffered to their own dirs > local0.* -?DirByTagFileByDay > > > / William > > On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: > > > mmhhhh... I can not reproduce the issue with my state of the code. > Can > > you verify that it still occurs? A git snapshot is available here: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz > > > > Thanks, > > Rainer > > > > On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: > >> I've debugged some more, this only happens when you trigger the > >> repeat- > >> loop to the missing directory. > >> > >> E.g: > >> > >> # logger -t non_existent_dir -p local0.crit test > >> # logger -t non_existent_dir -p local0.crit test > >> # logger -t existing_dir -p local0.crit test > >> > >> Debug output from last command: > >> > >> Filter: check for property 'programname' (value 'existing_dir') NOT > >> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE > >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > >> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) > >> Filter: check for property 'msg' (value ' test') contains '(CRIT)': > >> FALSE > >> 1092925760: Called fprintlog, logging to builtin-file > >> (DirByTagFileByDay) > >> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ > >> non_existent_dir/2008-12-18.log' - discarding message > >> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not > >> open dynamic file '/log/non_existent_dir/2008-12-18.log' - > discarding > >> message: No such file or directory > >> 1092925760: Message has legacy syslog format. > >> 1092925760: EnqueueMsg signaled condition (0) > >> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from > dynaCache. > >> 1092925760: Lone worker is running... > >> 1092925760: msg repeated 1 times, 10 sec of 30. > >> Filter: check for property 'syslogfacility-text' (value 'syslog') > NOT > >> isequal 'local0': TRUE > >> 1092925760: Called fprintlog, logging to > builtin-discardsingleWorker: > >> queue EMPTY, waiting for next message. > >> > >> > >> William Tis?ter > >> > >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >> http://www.blocket.se/ > >> > >> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: > >> > >>> Hi, > >>> > >>> Lets see if I can get this right: > >>> > >>> My modification in prepareFile() will return with (pData->fd = -1) > >>> if the log file can't be created. > >>> In prepareDynFile() we run prepareFile() and return with -1 if > >> pData- > >>>> fd is set to -1. > >>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if > >>> prepareDynFile() is not returned with 0. > >>> > >>> writeFile() is wrapped in doAction(). > >>> > >>> doAction() is exectued in fprintlog() where RS_RET_ERR never will > >> be > >>> catched. I discard the log message and sets the error flag to tell > >>> the "msg repeated"-check to not log this message ("msg repeated" > is > >>> executed before we try to open the file if the message content > >> match > >>> the previous message). > >>> > >>> I tried without this catch in the first attempt, but I could see > >> the > >>> message stuck in the loop, every action to rsyslog tried to open > >> the > >>> file. This and some traffic volume caused rsyslog to hang (and use > >> a > >>> lot of i/o). > >>> > >>> > >>> William Tis?ter > >>> > >>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>> http://www.blocket.se/ > >>> > >>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: > >>> > >>>> Hi William, > >>>> > >>>> thanks for the bug report and the patch. I have now looked into > the > >>>> issue and could reproduce the situation. The patch obviously > >> resolves > >>>> it. > >>>> > >>>> There is one thing, though: I do not fully understand why you > have > >>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could > >>>> provide > >>>> a bit more insight into the problem you intend to solve. So far, > I > >>>> can > >>>> not see why this part of the patch is actually needed. > >>>> > >>>> I'll integrate the omfile part now. I will also see that I apply > >>>> this to > >>>> the other official branches. > >>>> > >>>> Thanks, > >>>> Rainer > >>>> > >>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: > >>>>> Hi, > >>>>> > >>>>> I don't know if this is an intended feature or not, but when > >>>>> CreateDirs is set to off, rsyslog can no longer create files > (time > >>>>> stamp file names). Lets say you have a template for writing > >> files: / > >>>>> log//.log, this will let anybody to create > directories > >>>>> in / > >>>>> log when CreateDirs is on. This may cause a lot of directories > >> and > >>>>> can > >>>>> be abused by regular users. > >>>>> > >>>>> I've patched 2.0.2 to separate file and directory creation here: > >>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 > >>>>> > >>>>> And steps to reproduce this can be found here: > >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 > >>>>> > >>>>> Comments anyone? > >>>>> > >>>>> > >>>>> William Tis?ter > >>>>> > >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad > >>>>> http://www.blocket.se/ > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >> > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > William Tis?ter > > Cell: +46 (0)76 3377182 > Mail: william at blocket.se > > Blocket.se - Sveriges st?rsta K?p & S?lj marknad > http://www.blocket.se/ > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > From william at blocket.se Thu Dec 18 16:47:28 2008 From: william at blocket.se (=?ISO-8859-1?Q?William_Tis=E4ter?=) Date: Thu, 18 Dec 2008 16:47:28 +0100 Subject: [rsyslog] Allow files to be created without CreateDirs In-Reply-To: <1229615044.12594.14.camel@localhost.localdomain> References: <1229598085.7471.2.camel@localhost.localdomain> <8D1C7C06-26C6-4AA0-B7C7-6DE07D9DCBC6@blocket.se> <353BE160-AA7B-40D2-B921-A281164452B1@blocket.se> <1229613154.12594.13.camel@localhost.localdomain> <9CC236B9-DDB6-4382-B9E3-567C887BF18C@blocket.se> <1229615044.12594.14.camel@localhost.localdomain> Message-ID: * Sent full debug log to Rainer * On Dec 18, 2008, at 4:44 PM, Rainer Gerhards wrote: > would it be possible that you mail me (privately) a full debug log > from > such a run. Not sure if there is too much confidential data in it... > > Rainer > > On Thu, 2008-12-18 at 16:35 +0100, William Tis?ter wrote: >> Yes, I could reproduce my error, the same way as described earlier. >> >> My /etc/rsyslog.conf: >> >> $template DirByTagFileByDay,"/log/%programname%/%timegenerated: >> 1:10:date-rfc3339%.log" >> $template CritByDay,"/log/CRIT/%timegenerated:1:10:date-rfc3339%.log" >> >> # Discard all but local0 >> :syslogfacility-text, !isequal, "local0" ~ >> >> $umask 0000 >> $FileCreateMode 0644 >> $DirCreateMode 0755 >> $CreateDirs off >> >> # Discard strange tags >> :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" /log/badtags/ >> badtags.log >> :programname, !regex, "^[a-zA-Z_][a-zA-Z_]*$" ~ >> >> # Collect all crits in one log >> local0.crit ?CritByDay >> :msg, contains, "(CRIT)" ?CritByDay >> >> # All local0's buffered to their own dirs >> local0.* -?DirByTagFileByDay >> >> >> / William >> >> On Dec 18, 2008, at 4:12 PM, Rainer Gerhards wrote: >> >>> mmhhhh... I can not reproduce the issue with my state of the code. >> Can >>> you verify that it still occurs? A git snapshot is available here: >>> >>> >> http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=3c236053cf87a16dfd7449f729e477dffd6e2fae;sf=tgz >>> >>> Thanks, >>> Rainer >>> >>> On Thu, 2008-12-18 at 15:22 +0100, William Tis?ter wrote: >>>> I've debugged some more, this only happens when you trigger the >>>> repeat- >>>> loop to the missing directory. >>>> >>>> E.g: >>>> >>>> # logger -t non_existent_dir -p local0.crit test >>>> # logger -t non_existent_dir -p local0.crit test >>>> # logger -t existing_dir -p local0.crit test >>>> >>>> Debug output from last command: >>>> >>>> Filter: check for property 'programname' (value 'existing_dir') NOT >>>> regex '^[a-zA-Z_][a-zA-Z_]*$': FALSE >>>> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >>>> 1092925760: Called fprintlog, logging to builtin-file (CritByDay) >>>> Filter: check for property 'msg' (value ' test') contains '(CRIT)': >>>> FALSE >>>> 1092925760: Called fprintlog, logging to builtin-file >>>> (DirByTagFileByDay) >>>> 1092925760: Called logerr, msg: Could not open dynamic file '/log/ >>>> non_existent_dir/2008-12-18.log' - discarding message >>>> 1092925760: logmsg: syslog.err<43>, flags 5, from '', msg Could not >>>> open dynamic file '/log/non_existent_dir/2008-12-18.log' - >> discarding >>>> message: No such file or directory >>>> 1092925760: Message has legacy syslog format. >>>> 1092925760: EnqueueMsg signaled condition (0) >>>> 1092925760: Removed entry 2 for file '[OPEN FAILED]' from >> dynaCache. >>>> 1092925760: Lone worker is running... >>>> 1092925760: msg repeated 1 times, 10 sec of 30. >>>> Filter: check for property 'syslogfacility-text' (value 'syslog') >> NOT >>>> isequal 'local0': TRUE >>>> 1092925760: Called fprintlog, logging to >> builtin-discardsingleWorker: >>>> queue EMPTY, waiting for next message. >>>> >>>> >>>> William Tis?ter >>>> >>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>> http://www.blocket.se/ >>>> >>>> On Dec 18, 2008, at 2:35 PM, William Tis?ter wrote: >>>> >>>>> Hi, >>>>> >>>>> Lets see if I can get this right: >>>>> >>>>> My modification in prepareFile() will return with (pData->fd = -1) >>>>> if the log file can't be created. >>>>> In prepareDynFile() we run prepareFile() and return with -1 if >>>> pData- >>>>>> fd is set to -1. >>>>> In writeFile() we run prepareDynFile() and return RS_RET_ERR if >>>>> prepareDynFile() is not returned with 0. >>>>> >>>>> writeFile() is wrapped in doAction(). >>>>> >>>>> doAction() is exectued in fprintlog() where RS_RET_ERR never will >>>> be >>>>> catched. I discard the log message and sets the error flag to tell >>>>> the "msg repeated"-check to not log this message ("msg repeated" >> is >>>>> executed before we try to open the file if the message content >>>> match >>>>> the previous message). >>>>> >>>>> I tried without this catch in the first attempt, but I could see >>>> the >>>>> message stuck in the loop, every action to rsyslog tried to open >>>> the >>>>> file. This and some traffic volume caused rsyslog to hang (and use >>>> a >>>>> lot of i/o). >>>>> >>>>> >>>>> William Tis?ter >>>>> >>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>> http://www.blocket.se/ >>>>> >>>>> On Dec 18, 2008, at 12:01 PM, Rainer Gerhards wrote: >>>>> >>>>>> Hi William, >>>>>> >>>>>> thanks for the bug report and the patch. I have now looked into >> the >>>>>> issue and could reproduce the situation. The patch obviously >>>> resolves >>>>>> it. >>>>>> >>>>>> There is one thing, though: I do not fully understand why you >> have >>>>>> patched syslogd.c (RS_RET_ERR). I would appreciate if you could >>>>>> provide >>>>>> a bit more insight into the problem you intend to solve. So far, >> I >>>>>> can >>>>>> not see why this part of the patch is actually needed. >>>>>> >>>>>> I'll integrate the omfile part now. I will also see that I apply >>>>>> this to >>>>>> the other official branches. >>>>>> >>>>>> Thanks, >>>>>> Rainer >>>>>> >>>>>> On Tue, 2008-12-16 at 17:39 +0100, William Tis?ter wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I don't know if this is an intended feature or not, but when >>>>>>> CreateDirs is set to off, rsyslog can no longer create files >> (time >>>>>>> stamp file names). Lets say you have a template for writing >>>> files: / >>>>>>> log//.log, this will let anybody to create >> directories >>>>>>> in / >>>>>>> log when CreateDirs is on. This may cause a lot of directories >>>> and >>>>>>> can >>>>>>> be abused by regular users. >>>>>>> >>>>>>> I've patched 2.0.2 to separate file and directory creation here: >>>>>>> https://bugzilla.redhat.com/attachment.cgi?id=327100 >>>>>>> >>>>>>> And steps to reproduce this can be found here: >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=473419 >>>>>>> >>>>>>> Comments anyone? >>>>>>> >>>>>>> >>>>>>> William Tis?ter >>>>>>> >>>>>>> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >>>>>>> http://www.blocket.se/ >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com >>>>>> >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com >>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> >> >> William Tis?ter >> >> Cell: +46 (0)76 3377182 >> Mail: william at blocket.se >> >> Blocket.se - Sveriges st?rsta K?p & S?lj marknad >> http://www.blocket.se/ >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com William Tis?ter Cell: +46 (0)76 3377182 Mail: william at blocket.se Blocket.se - Sveriges st?rsta K?p & S?lj marknad http://www.blocket.se/ From bakers at web-ster.com Thu Dec 18 20:59:27 2008 From: bakers at web-ster.com (Scott Baker) Date: Thu, 18 Dec 2008 11:59:27 -0800 Subject: [rsyslog] Troubleshooting missing log entries Message-ID: <494AAB9F.9060005@web-ster.com> I have the following entry in my rsyslog conf, to match entries based on IP address. Somehow it's not matching any entries. # Switches $FileCreateMode 0644 :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea :FROMHOST, isequal, "65.182.224.202" -?switches :FROMHOST, isequal, "66.206.80.60" -?switches If I do a tcpdump I see syslog hitting the box, it's just rsyslog isn't handling it right. 11:58:20.722867 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 121 11:58:23.962613 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 130 11:58:41.242621 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 108 11:58:45.874064 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG local4.info, length: 130 This box gets about 500 lines of syslog a minute so I can't really turn on debug. How else can I troubleshoot this? This is a Fedora 8 box running: rsyslog-2.0.2-3.fc8 - Scott From david at lang.hm Fri Dec 19 06:38:38 2008 From: david at lang.hm (david at lang.hm) Date: Thu, 18 Dec 2008 21:38:38 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog Message-ID: with messages appearing out-of-order the 'last message repeated X times' is pretty useless. I think I remember reading about an option to disable this, but another work-around to the problem is to change the output so that it becomes 'last message repeated X times %msg%', so you can see (most, if not all) of the message being repeated in the line telling you that it was repeated. David Lang From rgerhards at hq.adiscon.com Thu Dec 18 20:01:47 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:01:47 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: Message-ID: <1229626907.12594.19.camel@localhost.localdomain> Hi David, On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > with messages appearing out-of-order the 'last message repeated X times' > is pretty useless. Not really. Please note that form the perspective of the output module, messages do not appear out of order. The output module receives a stream of messages, and the "last message repeated x times" logic works on that stream. So no matter if messages are re-ordered by async queues (or UDP or whatever), the "last ... times" correctly reflects the way things were handed to the output module. But I concur that this feature, in its current state, is very questionable. I've talked about this on the mailing list quite some time, I think there is also at least one blog post about it. > > I think I remember reading about an option to disable this, but another > work-around to the problem is to change the output so that it becomes > 'last message repeated X times %msg%', so you can see (most, if not all) > of the message being repeated in the line telling you that it was > repeated. As I said, the message in front of this message is either another repeat message or the message that is being repeated. So you can always trace back to what was repeated (but it is painful). Given the feedback I received on this feature (use cases), it should probably (and v4 would be an appropriate version to do so) be redisigned to be a rate-limiting feature on the input side. That would pretty much simplify code, too. Rainer From david at lang.hm Fri Dec 19 12:46:10 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 03:46:10 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: <1229626907.12594.19.camel@localhost.localdomain> References: <1229626907.12594.19.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > Hi David, > > On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: >> with messages appearing out-of-order the 'last message repeated X times' >> is pretty useless. > > Not really. Please note that form the perspective of the output module, > messages do not appear out of order. The output module receives a stream > of messages, and the "last message repeated x times" logic works on that > stream. So no matter if messages are re-ordered by async queues (or UDP > or whatever), the "last ... times" correctly reflects the way things > were handed to the output module. but if the output module is then relaying to another system, that other system (or the transport between them) can also re-order the messages > But I concur that this feature, in its current state, is very > questionable. I've talked about this on the mailing list quite some > time, I think there is also at least one blog post about it. > >> >> I think I remember reading about an option to disable this, but another >> work-around to the problem is to change the output so that it becomes >> 'last message repeated X times %msg%', so you can see (most, if not all) >> of the message being repeated in the line telling you that it was >> repeated. > > As I said, the message in front of this message is either another repeat > message or the message that is being repeated. So you can always trace > back to what was repeated (but it is painful). > > Given the feedback I received on this feature (use cases), it should > probably (and v4 would be an appropriate version to do so) be redisigned > to be a rate-limiting feature on the input side. That would pretty much > simplify code, too. but since things can be re-ordered after the input module is done with them (multiple threads pulling from the queue, or relaying) it is still useful for the 'message repeated' message to have more info about what it is referring to. David Lang > Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Thu Dec 18 20:15:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:15:51 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain> Message-ID: <1229627751.12594.23.camel@localhost.localdomain> On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: > On Thu, 18 Dec 2008, Rainer Gerhards wrote: > > > Hi David, > > > > On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > >> with messages appearing out-of-order the 'last message repeated X times' > >> is pretty useless. > > > > Not really. Please note that form the perspective of the output module, > > messages do not appear out of order. The output module receives a stream > > of messages, and the "last message repeated x times" logic works on that > > stream. So no matter if messages are re-ordered by async queues (or UDP > > or whatever), the "last ... times" correctly reflects the way things > > were handed to the output module. > > but if the output module is then relaying to another system, that other > system (or the transport between them) can also re-order the messages ah, ok, remote scenario. Of course, that can happen. But that's "just" one use case where it is problematic. > > But I concur that this feature, in its current state, is very > > questionable. I've talked about this on the mailing list quite some > > time, I think there is also at least one blog post about it. > > > >> > >> I think I remember reading about an option to disable this, but another > >> work-around to the problem is to change the output so that it becomes > >> 'last message repeated X times %msg%', so you can see (most, if not all) > >> of the message being repeated in the line telling you that it was > >> repeated. > > > > As I said, the message in front of this message is either another repeat > > message or the message that is being repeated. So you can always trace > > back to what was repeated (but it is painful). > > > > Given the feedback I received on this feature (use cases), it should > > probably (and v4 would be an appropriate version to do so) be redisigned > > to be a rate-limiting feature on the input side. That would pretty much > > simplify code, too. > > but since things can be re-ordered after the input module is done with > them (multiple threads pulling from the queue, or relaying) it is still > useful for the 'message repeated' message to have more info about what it > is referring to. That makes sense, at least for scenarios where it is expected behaviour. For others, it would probably break analysers. So it needs to be optional. And probably on a per-action basis. I'll see if it is sufficiently simple, but the whole "last message repeated" thing is broken anyhow (IMO), I don't like the idea to invest any more than maybe an hour into that feature, especially in the light that the whole idea must be replaced in the not so distant future... Rainer > > David Lang > > > Rainer > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Dec 18 20:19:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 18 Dec 2008 20:19:54 +0100 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <494AAB9F.9060005@web-ster.com> References: <494AAB9F.9060005@web-ster.com> Message-ID: <1229627994.12594.26.camel@localhost.localdomain> On Thu, 2008-12-18 at 11:59 -0800, Scott Baker wrote: > I have the following entry in my rsyslog conf, to match entries based on IP > address. Somehow it's not matching any entries. > > # Switches > $FileCreateMode 0644 > :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea > :FROMHOST, isequal, "65.182.224.202" -?switches > :FROMHOST, isequal, "66.206.80.60" -?switches > > If I do a tcpdump I see syslog hitting the box, it's just rsyslog isn't > handling it right. > > 11:58:20.722867 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 121 > 11:58:23.962613 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 130 > 11:58:41.242621 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 108 > 11:58:45.874064 IP 65.182.224.13.8888 > 65.182.224.26.514: SYSLOG > local4.info, length: 130 > > This box gets about 500 lines of syslog a minute so I can't really turn on > debug. How else can I troubleshoot this? This is a Fedora 8 box running: > rsyslog-2.0.2-3.fc8 I'd still go for debug mode. You don't need to run it very long. We just need to see how a few of these messages are fully processed. A proper test setup would be to start up in debug mode with the network cable pulled, then plug it in for a second or two, then unplug it again. Once rsyslogd is finished processing, stop it. That should lead to useful info in the debug log. Oh - and are you sure that fromhost has the proper IP addresses? If not 100% sure, verify it by putting something like '%FROMHOST%' into a debug template (note that there is also FROMHOST-IP, which will have the IP address no matter if names are resolved or not). HTH, Rainer > > - Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Fri Dec 19 13:03:40 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 04:03:40 -0800 (PST) Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: <1229627751.12594.23.camel@localhost.localdomain> References: <1229626907.12594.19.camel@localhost.localdomain> <1229627751.12594.23.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: >> >>> Hi David, >>> >>> On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: >>>> with messages appearing out-of-order the 'last message repeated X times' >>>> is pretty useless. >>> >>> Not really. Please note that form the perspective of the output module, >>> messages do not appear out of order. The output module receives a stream >>> of messages, and the "last message repeated x times" logic works on that >>> stream. So no matter if messages are re-ordered by async queues (or UDP >>> or whatever), the "last ... times" correctly reflects the way things >>> were handed to the output module. >> >> but if the output module is then relaying to another system, that other >> system (or the transport between them) can also re-order the messages > > ah, ok, remote scenario. Of course, that can happen. But that's "just" > one use case where it is problematic. > >>> But I concur that this feature, in its current state, is very >>> questionable. I've talked about this on the mailing list quite some >>> time, I think there is also at least one blog post about it. >>> >>>> >>>> I think I remember reading about an option to disable this, but another >>>> work-around to the problem is to change the output so that it becomes >>>> 'last message repeated X times %msg%', so you can see (most, if not all) >>>> of the message being repeated in the line telling you that it was >>>> repeated. >>> >>> As I said, the message in front of this message is either another repeat >>> message or the message that is being repeated. So you can always trace >>> back to what was repeated (but it is painful). >>> >>> Given the feedback I received on this feature (use cases), it should >>> probably (and v4 would be an appropriate version to do so) be redisigned >>> to be a rate-limiting feature on the input side. That would pretty much >>> simplify code, too. >> >> but since things can be re-ordered after the input module is done with >> them (multiple threads pulling from the queue, or relaying) it is still >> useful for the 'message repeated' message to have more info about what it >> is referring to. > > That makes sense, at least for scenarios where it is expected behaviour. > For others, it would probably break analysers. So it needs to be > optional. And probably on a per-action basis. I'll see if it is > sufficiently simple, but the whole "last message repeated" thing is > broken anyhow (IMO), I don't like the idea to invest any more than maybe > an hour into that feature, especially in the light that the whole idea > must be replaced in the not so distant future... it's definantly a pain for analysers (a _lot_ of them really only look at one line at a time), that's why I started tweaking this on sysklogd, and while in theory just disabling it is the right answer, the ability to take a flood of inbound messages and reduce them to a much smaller number can save your logging system when things go wrong I'll watch to see what happens. David Lang From rgerhards at hq.adiscon.com Fri Dec 19 12:05:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:05:24 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain><1229627751.12594.23.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F915@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, December 19, 2008 1:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] suggested tweak to rsyslog > it's definantly a pain for analysers Full ack > (a _lot_ of them really only look > at > one line at a time), that's why I started tweaking this on sysklogd, > and > while in theory just disabling it is the right answer, the ability to > take > a flood of inbound messages and reduce them to a much smaller number > can > save your logging system when things go wrong > > I'll watch to see what happens. I just dug out a the relevant link, please follow also to the mailing list discussion: http://kb.monitorware.com/post5789.html Rainer From david at lang.hm Fri Dec 19 13:07:53 2008 From: david at lang.hm (david at lang.hm) Date: Fri, 19 Dec 2008 04:07:53 -0800 (PST) Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <1229627994.12594.26.camel@localhost.localdomain> References: <494AAB9F.9060005@web-ster.com> <1229627994.12594.26.camel@localhost.localdomain> Message-ID: On Thu, 18 Dec 2008, Rainer Gerhards wrote: > On Thu, 2008-12-18 at 11:59 -0800, Scott Baker wrote: >> I have the following entry in my rsyslog conf, to match entries based on IP >> address. Somehow it's not matching any entries. >> >> # Switches >> $FileCreateMode 0644 >> :FROMHOST, isequal, "65.182.224.13" -?switches # Necalea >> :FROMHOST, isequal, "65.182.224.202" -?switches >> :FROMHOST, isequal, "66.206.80.60" -?switches > > Oh - and are you sure that fromhost has the proper IP addresses? If not > 100% sure, verify it by putting something like '%FROMHOST%' into a debug > template (note that there is also FROMHOST-IP, which will have the IP > address no matter if names are resolved or not). I was seeing some issues where the fromhost was not getting set properly, I'll have to go back and dig up the details, but I think I was seeing it use the localhost as the fromhost and putting the real fromhost information in the message. I found it by creating an output format that I could tweak and playing with it to see what was actually showing up in the various parameters. David Lang From rgerhards at hq.adiscon.com Fri Dec 19 12:24:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:24:22 +0100 Subject: [rsyslog] Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? In-Reply-To: References: Message-ID: <1229685862.12594.33.camel@localhost.localdomain> Hi Craig, I don't see this in my lab, everything works correct. May it be that you did not include some of the (automatic) initial messages in your picture? In any case, a full debug log would be useful to track this down. After my sig, you find the relevant part of my lab debug FYI. Relevant part of rsyslog.conf: $ActionExecOnlyEveryNthTime 3 :msg, contains, "test" -/rsyslog/logfile Commands issued: $ logger 1 $ logger 2 $ logger 3 $ logger 4 $ logger 5 $ logger 6 And the output file had: 2008-12-19T12:17:44.548207+01:00 rf10up rger: test 3 2008-12-19T12:17:53.522265+01:00 rf10up rger: test 6 Note that in this setup I made sure that no other messages than those with the string "test" went into the log file. Otherwise, the counters would have been affected by some startup messages. Rainer 5455.946426832:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5460.051992823:imuxsock.c: Message from UNIX socket: #4 5460.066209819:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:40 rger: test 1 5460.066408445:imuxsock.c: Message has legacy syslog format. 5460.095581070:imuxsock.c: main queue: entry added, size now 1 entries 5460.096265506:imuxsock.c: wtpAdviseMaxWorkers signals busy 5460.096810820:imuxsock.c: main queue: EnqueueMsg advised worker start 5460.097417035:main queue:Reg/w0: main queue: entering rate limiter 5460.097797246:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5460.098493974:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 1') contains 'test': TRUE 5460.105295873:main queue:Reg/w0: action 0x4c64438 passed 1 times to execution - less than neded - discarding 5460.109592735:main queue:Reg/w0: main queue: entering rate limiter 5460.110192245:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5460.115576662:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5462.317878786:imuxsock.c: Message from UNIX socket: #4 5462.318410132:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:42 rger: test 2 5462.318585850:imuxsock.c: Message has legacy syslog format. 5462.319012436:imuxsock.c: main queue: entry added, size now 1 entries 5462.319271125:imuxsock.c: wtpAdviseMaxWorkers signals busy 5462.319537636:imuxsock.c: main queue: EnqueueMsg advised worker start 5462.319871473:main queue:Reg/w0: main queue: entering rate limiter 5462.320174580:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5462.320470145:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 2') contains 'test': TRUE 5462.321508812:main queue:Reg/w0: action 0x4c64438 passed 2 times to execution - less than neded - discarding 5462.321825608:main queue:Reg/w0: main queue: entering rate limiter 5462.322078710:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5462.322350529:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5464.547937451:imuxsock.c: Message from UNIX socket: #4 5464.548420747:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:44 rger: test 3 5464.548613227:imuxsock.c: Message has legacy syslog format. 5464.549042885:imuxsock.c: main queue: entry added, size now 1 entries 5464.549287885:imuxsock.c: wtpAdviseMaxWorkers signals busy 5464.549571158:imuxsock.c: main queue: EnqueueMsg advised worker start 5464.549888793:main queue:Reg/w0: main queue: entering rate limiter 5464.550211176:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5464.550500036:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 3') contains 'test': TRUE 5464.551797950:main queue:Reg/w0: Called action, logging to builtin-file 5464.605776776:main queue:Reg/w0: (/rsyslog/logfile) 5464.606371817:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5464.627358305:main queue:Reg/w0: main queue: entering rate limiter 5464.628009218:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5469.671232071:imuxsock.c: Message from UNIX socket: #4 5469.671772915:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:49 rger: test 4 5469.671950031:imuxsock.c: Message has legacy syslog format. 5469.672340578:imuxsock.c: main queue: entry added, size now 1 entries 5469.672597591:imuxsock.c: wtpAdviseMaxWorkers signals busy 5469.672931428:imuxsock.c: main queue: EnqueueMsg advised worker start 5469.673234256:main queue:Reg/w0: main queue: entering rate limiter 5469.673563065:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5469.673893829:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 4') contains 'test': TRUE 5469.674471829:main queue:Reg/w0: action 0x4c64438 passed 1 times to execution - less than neded - discarding 5469.674788066:main queue:Reg/w0: main queue: entering rate limiter 5469.675042006:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5469.675316618:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5471.443424737:imuxsock.c: Message from UNIX socket: #4 5471.443973403:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:51 rger: test 5 5471.444148283:imuxsock.c: Message has legacy syslog format. 5471.444550564:imuxsock.c: main queue: entry added, size now 1 entries 5471.444845291:imuxsock.c: wtpAdviseMaxWorkers signals busy 5471.445187788:imuxsock.c: main queue: EnqueueMsg advised worker start 5471.445489499:main queue:Reg/w0: main queue: entering rate limiter 5471.445842612:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5471.446121136:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 5') contains 'test': TRUE 5471.446527886:main queue:Reg/w0: action 0x4c64438 passed 2 times to execution - less than neded - discarding 5471.446837140:main queue:Reg/w0: main queue: entering rate limiter 5471.447091079:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5471.447366530:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 5473.522029412:imuxsock.c: Message from UNIX socket: #4 5473.522550422:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg Dec 19 12:17:53 rger: test 6 5473.522766368:imuxsock.c: Message has legacy syslog format. 5473.523162224:imuxsock.c: main queue: entry added, size now 1 entries 5473.523404989:imuxsock.c: wtpAdviseMaxWorkers signals busy 5473.523727931:imuxsock.c: main queue: EnqueueMsg advised worker start 5473.524011204:main queue:Reg/w0: main queue: entering rate limiter 5473.524328279:main queue:Reg/w0: main queue: entry deleted, state 0, size now 0 entries 5473.524621330:main queue:Reg/w0: Filter: check for property 'msg' (value ' test 6') contains 'test': TRUE 5473.525016627:main queue:Reg/w0: Called action, logging to builtin-file 5473.525803589:main queue:Reg/w0: (/rsyslog/logfile) 5473.526170949:main queue:Reg/w0: main queue: entering rate limiter 5473.526423213:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, waiting for work. 5473.526753139:imuxsock.c: --------imuxsock calling select, active file descriptors (max 4): 4 On Thu, 2008-12-18 at 12:51 -0500, Becker, Craig (DOB) wrote: > The action $ActionExecOnlyEveryNthTime is not sending the Nth message > but the first message, then the Nth + 1 message. For example, if I > set the $ActionExecOnlyEveryNthTime 3 The 1st, 4th, 7th, etc. message > are sent instead of the 3rd, 6th, 9th, etc. Has this bug (if it is a > bug) been fixed in a later version? > > > > I?m enjoying learning this product and appreciate all the hard work > that has gone into it. > > > > Thanks, > > > > Craig E. Becker > IT Specialist II > System Administrator > Division of the Budget > 518-473-1322 > craig.becker at budget.state.ny.us > > > > > > > > HTML: > ______________________________________________________________________ > This e-mail, including any attachments, may be confidential, > privileged or otherwise legally protected. If you have received this > e-mail in error, or from someone who was not authorized to send it to > you, do not disseminate, copy or otherwise use this e-mail or its > attachments. Please notify the sender immediately if you have received > this e-mail by mistake, and delete it from your system. From rgerhards at hq.adiscon.com Fri Dec 19 12:57:16 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 12:57:16 +0100 Subject: [rsyslog] suggested tweak to rsyslog In-Reply-To: References: <1229626907.12594.19.camel@localhost.localdomain><1229627751.12594.23.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F916@grfint2.intern.adiscon.com> David, one thing I can do rather quickly. Maybe it's good enough. I've done a tester, which lacks proper configuration, but I would appreciate feedback on it: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=a185665be4cf6997525 89d81ef6e396dd61f68b6 Details in git commit comment. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, December 19, 2008 1:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] suggested tweak to rsyslog > > On Thu, 18 Dec 2008, Rainer Gerhards wrote: > > > On Fri, 2008-12-19 at 03:46 -0800, david at lang.hm wrote: > >> On Thu, 18 Dec 2008, Rainer Gerhards wrote: > >> > >>> Hi David, > >>> > >>> On Thu, 2008-12-18 at 21:38 -0800, david at lang.hm wrote: > >>>> with messages appearing out-of-order the 'last message repeated X > times' > >>>> is pretty useless. > >>> > >>> Not really. Please note that form the perspective of the output > module, > >>> messages do not appear out of order. The output module receives a > stream > >>> of messages, and the "last message repeated x times" logic works on > that > >>> stream. So no matter if messages are re-ordered by async queues (or > UDP > >>> or whatever), the "last ... times" correctly reflects the way > things > >>> were handed to the output module. > >> > >> but if the output module is then relaying to another system, that > other > >> system (or the transport between them) can also re-order the > messages > > > > ah, ok, remote scenario. Of course, that can happen. But that's > "just" > > one use case where it is problematic. > > > >>> But I concur that this feature, in its current state, is very > >>> questionable. I've talked about this on the mailing list quite some > >>> time, I think there is also at least one blog post about it. > >>> > >>>> > >>>> I think I remember reading about an option to disable this, but > another > >>>> work-around to the problem is to change the output so that it > becomes > >>>> 'last message repeated X times %msg%', so you can see (most, if > not all) > >>>> of the message being repeated in the line telling you that it was > >>>> repeated. > >>> > >>> As I said, the message in front of this message is either another > repeat > >>> message or the message that is being repeated. So you can always > trace > >>> back to what was repeated (but it is painful). > >>> > >>> Given the feedback I received on this feature (use cases), it > should > >>> probably (and v4 would be an appropriate version to do so) be > redisigned > >>> to be a rate-limiting feature on the input side. That would pretty > much > >>> simplify code, too. > >> > >> but since things can be re-ordered after the input module is done > with > >> them (multiple threads pulling from the queue, or relaying) it is > still > >> useful for the 'message repeated' message to have more info about > what it > >> is referring to. > > > > That makes sense, at least for scenarios where it is expected > behaviour. > > For others, it would probably break analysers. So it needs to be > > optional. And probably on a per-action basis. I'll see if it is > > sufficiently simple, but the whole "last message repeated" thing is > > broken anyhow (IMO), I don't like the idea to invest any more than > maybe > > an hour into that feature, especially in the light that the whole > idea > > must be replaced in the not so distant future... > > it's definantly a pain for analysers (a _lot_ of them really only look > at > one line at a time), that's why I started tweaking this on sysklogd, > and > while in theory just disabling it is the right answer, the ability to > take > a flood of inbound messages and reduce them to a much smaller number > can > save your logging system when things go wrong > > I'll watch to see what happens. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From bakers at web-ster.com Fri Dec 19 17:38:41 2008 From: bakers at web-ster.com (Scott Baker) Date: Fri, 19 Dec 2008 08:38:41 -0800 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <1229627994.12594.26.camel@localhost.localdomain> References: <494AAB9F.9060005@web-ster.com> <1229627994.12594.26.camel@localhost.localdomain> Message-ID: <494BCE11.7000300@web-ster.com> Rainer Gerhards wrote: > I'd still go for debug mode. You don't need to run it very long. We just > need to see how a few of these messages are fully processed. A proper > test setup would be to start up in debug mode with the network cable > pulled, then plug it in for a second or two, then unplug it again. Once > rsyslogd is finished processing, stop it. That should lead to useful > info in the debug log. > > Oh - and are you sure that fromhost has the proper IP addresses? If not > 100% sure, verify it by putting something like '%FROMHOST%' into a debug > template (note that there is also FROMHOST-IP, which will have the IP > address no matter if names are resolved or not). I like the debug template idea, that's genius. Is there a way to have a bunch of filters to catch assorted things, and then an "everything leftover" filter? ------------------------------------------------------------------------ # Mail servers log to their special section $FileCreateMode 0644 :FROMHOST, isequal, "magenta" -?magic-mail :FROMHOST, isequal, "cyan" -?magic-mail :FROMHOST, isequal, "orange" -?magic-mail # Firewalls :FROMHOST, isequal, "yin" -?firewall :FROMHOST, isequal, "yang" -?firewall # Everything that didn't get caught by one of the above filters (I have no idea what the syntax would be) From rgerhards at hq.adiscon.com Fri Dec 19 17:40:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 17:40:32 +0100 Subject: [rsyslog] Troubleshooting missing log entries In-Reply-To: <494BCE11.7000300@web-ster.com> References: <494AAB9F.9060005@web-ster.com><1229627994.12594.26.camel@localhost.localdomain> <494BCE11.7000300@web-ster.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F924@grfint2.intern.adiscon.com> Without verification, but should work: # Mail servers log to their special section $FileCreateMode 0644 :FROMHOST, isequal, "magenta" -?magic-mail & ~ :FROMHOST, isequal, "cyan" -?magic-mail & ~ :FROMHOST, isequal, "orange" -?magic-mail & ~ # Firewalls :FROMHOST, isequal, "yin" -?firewall & ~ :FROMHOST, isequal, "yang" -?firewall & ~ *.* /var/log/catchrest & ~ discards the message after it is written to the file in question. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Scott Baker > Sent: Friday, December 19, 2008 5:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] Troubleshooting missing log entries > > Rainer Gerhards wrote: > > I'd still go for debug mode. You don't need to run it very long. We > just > > need to see how a few of these messages are fully processed. A proper > > test setup would be to start up in debug mode with the network cable > > pulled, then plug it in for a second or two, then unplug it again. > Once > > rsyslogd is finished processing, stop it. That should lead to useful > > info in the debug log. > > > > Oh - and are you sure that fromhost has the proper IP addresses? If > not > > 100% sure, verify it by putting something like '%FROMHOST%' into a > debug > > template (note that there is also FROMHOST-IP, which will have the IP > > address no matter if names are resolved or not). > > > I like the debug template idea, that's genius. Is there a way to have a > bunch of filters to catch assorted things, and then an "everything > leftover" filter? > > ----------------------------------------------------------------------- > - > > # Mail servers log to their special section > $FileCreateMode 0644 > :FROMHOST, isequal, "magenta" -?magic-mail > :FROMHOST, isequal, "cyan" -?magic-mail > :FROMHOST, isequal, "orange" -?magic-mail > > # Firewalls > :FROMHOST, isequal, "yin" -?firewall > :FROMHOST, isequal, "yang" -?firewall > > # Everything that didn't get caught by one of the above filters > (I have no idea what the syntax would be) > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Dec 19 17:52:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 19 Dec 2008 17:52:26 +0100 Subject: [rsyslog] Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? In-Reply-To: References: <1229685862.12594.33.camel@localhost.localdomain> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F926@grfint2.intern.adiscon.com> --sorry, my mailer didn't reply to the list for some reason-- Sounds like a too-old version. Jepp, you mention 3.21.2 in the subject, I think the directives were introduced some time later. Rainer > -----Original Message----- > From: Becker, Craig (DOB) [mailto:Craig.Becker at budget.state.ny.us] > Sent: Friday, December 19, 2008 5:34 PM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: RE: Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? > > Thanks for the help. When I start Rsyslog in Debug mode, I get the > following errors: > > > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 404 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 405 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 698 > rsyslogd: invalid or yet-unknown config file command - have you > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] > rsyslogd: the last error occured in /etc/rsyslog.conf, line 699 > > These errors correspond to the actions: > > $ActionExecOnlyEveryNthTime 50 #Lines 404 & 698 > $ActionExecOnlyEveryNthTimeTimeout 1800 #Lines 405 & 699 > > These are the Modules loaded: > > #______All Modules loaded here__________ > $ModLoad immark.so # provides --MARK-- message capability > $ModLoad imuxsock.so # provides support for local system logging > (e.g. via logger command) > $ModLoad imklog.so # kernel logging (formerly provided by > rklogd) > $ModLoad ommail # This mod enables sending E-mail on alerts > $ModLoad imudp.so # provides UDP syslog reception > $ModLoad imtcp.so # provides TCP syslog reception > > I tried to add all the modules I have access to without fixing the > issue. Is there a module missing or should I look to another version? > The strange thing is that the command actually works after the first > email is sent (i.e. The 1st, 51st, 101st messages are sent). > > Any help would be appreciated. > > Thanks again, > Craig > > > > -----Original Message----- > From: Rainer Gerhards [mailto:rgerhards at hq.adiscon.com] > Sent: Friday, December 19, 2008 6:24 AM > To: rsyslog at lists.adiscon.com > Cc: Becker, Craig (DOB) > Subject: Re: Rsyslog 3.21.2 - $ActionExecOnlyEveryNthTime bug? > > Hi Craig, > > I don't see this in my lab, everything works correct. May it be that > you > did not include some of the (automatic) initial messages in your > picture? In any case, a full debug log would be useful to track this > down. After my sig, you find the relevant part of my lab debug FYI. > > Relevant part of rsyslog.conf: > $ActionExecOnlyEveryNthTime 3 > :msg, contains, "test" -/rsyslog/logfile > > Commands issued: > $ logger 1 > $ logger 2 > $ logger 3 > $ logger 4 > $ logger 5 > $ logger 6 > > And the output file had: > 2008-12-19T12:17:44.548207+01:00 rf10up rger: test 3 > 2008-12-19T12:17:53.522265+01:00 rf10up rger: test 6 > > Note that in this setup I made sure that no other messages than those > with the string "test" went into the log file. Otherwise, the counters > would have been affected by some startup messages. > > Rainer > > 5455.946426832:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5460.051992823:imuxsock.c: Message from UNIX socket: #4 > 5460.066209819:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:40 rger: test 1 > 5460.066408445:imuxsock.c: Message has legacy syslog format. > 5460.095581070:imuxsock.c: main queue: entry added, size now 1 entries > 5460.096265506:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5460.096810820:imuxsock.c: main queue: EnqueueMsg advised worker start > 5460.097417035:main queue:Reg/w0: main queue: entering rate limiter > 5460.097797246:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5460.098493974:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 1') contains 'test': TRUE > 5460.105295873:main queue:Reg/w0: action 0x4c64438 passed 1 times to > execution - less than neded - discarding > 5460.109592735:main queue:Reg/w0: main queue: entering rate limiter > 5460.110192245:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5460.115576662:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5462.317878786:imuxsock.c: Message from UNIX socket: #4 > 5462.318410132:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:42 rger: test 2 > 5462.318585850:imuxsock.c: Message has legacy syslog format. > 5462.319012436:imuxsock.c: main queue: entry added, size now 1 entries > 5462.319271125:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5462.319537636:imuxsock.c: main queue: EnqueueMsg advised worker start > 5462.319871473:main queue:Reg/w0: main queue: entering rate limiter > 5462.320174580:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5462.320470145:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 2') contains 'test': TRUE > 5462.321508812:main queue:Reg/w0: action 0x4c64438 passed 2 times to > execution - less than neded - discarding > 5462.321825608:main queue:Reg/w0: main queue: entering rate limiter > 5462.322078710:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5462.322350529:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5464.547937451:imuxsock.c: Message from UNIX socket: #4 > 5464.548420747:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:44 rger: test 3 > 5464.548613227:imuxsock.c: Message has legacy syslog format. > 5464.549042885:imuxsock.c: main queue: entry added, size now 1 entries > 5464.549287885:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5464.549571158:imuxsock.c: main queue: EnqueueMsg advised worker start > 5464.549888793:main queue:Reg/w0: main queue: entering rate limiter > 5464.550211176:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5464.550500036:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 3') contains 'test': TRUE > 5464.551797950:main queue:Reg/w0: Called action, logging to builtin- > file > 5464.605776776:main queue:Reg/w0: (/rsyslog/logfile) > 5464.606371817:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5464.627358305:main queue:Reg/w0: main queue: entering rate limiter > 5464.628009218:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5469.671232071:imuxsock.c: Message from UNIX socket: #4 > 5469.671772915:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:49 rger: test 4 > 5469.671950031:imuxsock.c: Message has legacy syslog format. > 5469.672340578:imuxsock.c: main queue: entry added, size now 1 entries > 5469.672597591:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5469.672931428:imuxsock.c: main queue: EnqueueMsg advised worker start > 5469.673234256:main queue:Reg/w0: main queue: entering rate limiter > 5469.673563065:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5469.673893829:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 4') contains 'test': TRUE > 5469.674471829:main queue:Reg/w0: action 0x4c64438 passed 1 times to > execution - less than neded - discarding > 5469.674788066:main queue:Reg/w0: main queue: entering rate limiter > 5469.675042006:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5469.675316618:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5471.443424737:imuxsock.c: Message from UNIX socket: #4 > 5471.443973403:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:51 rger: test 5 > 5471.444148283:imuxsock.c: Message has legacy syslog format. > 5471.444550564:imuxsock.c: main queue: entry added, size now 1 entries > 5471.444845291:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5471.445187788:imuxsock.c: main queue: EnqueueMsg advised worker start > 5471.445489499:main queue:Reg/w0: main queue: entering rate limiter > 5471.445842612:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5471.446121136:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 5') contains 'test': TRUE > 5471.446527886:main queue:Reg/w0: action 0x4c64438 passed 2 times to > execution - less than neded - discarding > 5471.446837140:main queue:Reg/w0: main queue: entering rate limiter > 5471.447091079:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5471.447366530:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > 5473.522029412:imuxsock.c: Message from UNIX socket: #4 > 5473.522550422:imuxsock.c: logmsg: flags 4, pri 13, from 'rf10up', msg > Dec 19 12:17:53 rger: test 6 > 5473.522766368:imuxsock.c: Message has legacy syslog format. > 5473.523162224:imuxsock.c: main queue: entry added, size now 1 entries > 5473.523404989:imuxsock.c: wtpAdviseMaxWorkers signals busy > 5473.523727931:imuxsock.c: main queue: EnqueueMsg advised worker start > 5473.524011204:main queue:Reg/w0: main queue: entering rate limiter > 5473.524328279:main queue:Reg/w0: main queue: entry deleted, state 0, > size now 0 entries > 5473.524621330:main queue:Reg/w0: Filter: check for property > 'msg' (value ' test 6') contains 'test': TRUE > 5473.525016627:main queue:Reg/w0: Called action, logging to builtin- > file > 5473.525803589:main queue:Reg/w0: (/rsyslog/logfile) > 5473.526170949:main queue:Reg/w0: main queue: entering rate limiter > 5473.526423213:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, > waiting for work. > 5473.526753139:imuxsock.c: --------imuxsock calling select, active file > descriptors (max 4): 4 > > > > On Thu, 2008-12-18 at 12:51 -0500, Becker, Craig (DOB) wrote: > > The action $ActionExecOnlyEveryNthTime is not sending the Nth message > > but the first message, then the Nth + 1 message. For example, if I > > set the $ActionExecOnlyEveryNthTime 3 The 1st, 4th, 7th, etc. message > > are sent instead of the 3rd, 6th, 9th, etc. Has this bug (if it is a > > bug) been fixed in a later version? > > > > > > > > I?m enjoying learning this product and appreciate all the hard work > > that has gone into it. > > > > > > > > Thanks, > > > > > > > > Craig E. Becker > > IT Specialist II > > System Administrator > > Division of the Budget > > 518-473-1322 > > craig.becker at budget.state.ny.us > > > > > > > > > > > > > > > > HTML: > > > ______________________________________________________________________ > > This e-mail, including any attachments, may be confidential, > > privileged or otherwise legally protected. If you have received this > > e-mail in error, or from someone who was not authorized to send it to > > you, do not disseminate, copy or otherwise use this e-mail or its > > attachments. Please notify the sender immediately if you have > received > > this e-mail by mistake, and delete it from your system. > > > > ----------------------------------------------------------------------- > --------- > This e-mail, including any attachments, may be confidential, privileged > or otherwise legally protected. If you have received this e-mail in > error, or from someone who was not authorized to send it to you, do not > disseminate, copy or otherwise use this e-mail or its attachments. > Please notify the sender immediately if you have received this e-mail > by mistake, and delete it from your system. From rgerhards at hq.adiscon.com Wed Dec 24 11:13:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 24 Dec 2008 11:13:41 +0100 Subject: [rsyslog] Merry Xmas and a happy new year! Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F949@grfint2.intern.adiscon.com> Hi all, I wanted to whish all of you a merry xmas and a happy new year. At the same time, I'd like to thank all of you for your qualified comments, encouragement and ideas, which helped make rsyslog become a premier logging solution. Without an active community, no project can survive and I am very happy to see such a great community backing this project. Special thanks go to all those who also participate in the forum and dispense knowledge here on the mailing list. I can promise that 2009 will become a very exciting year for rsyslog, too. There is still a lot of things in stock and even though the budget cut has forced me to work at a somewhat slower pace, I am sure we will see great enhancements. A number of companies have also asked about sponsoring, so we may even be able to get back to full speed development in 2009. Please note that I take a break from the project from end of December to January, 10th. So my response will be sluggish in that period. My apologies for that. I hope you, too, will enjoy a great holiday season. My your wishes come true and your health be good! If we don't "talk" before the end of the year, I also wanted to wish you a great end of the year. Please keep your comments, ideas and feedback floating in 2009. This is a great source of inspiration and much appreciated. My warmest regards, Rainer Gerhards