From lanas at securenet.net Thu May 1 13:12:28 2008 From: lanas at securenet.net (lanas) Date: Thu, 1 May 2008 07:12:28 -0400 Subject: [rsyslog] Tp place MARK msgs in another file Message-ID: <20080501071228.4c5e54d7@mistral.stie> Hello, As a simple test, I'm trying to put MARK msgs in a separate file. So I have a cfg file with: mark.* /var/log/mark The file is created when rsyslog starts, but the MARK msgs are not going there, remaining in the main log file. I start rsyslog with '-c3'. Thanks to Michael and Rainer for the recent explanations on 'rsyslog in general' ! Cheers, Al From friedl at hq.adiscon.com Fri May 2 11:23:56 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 2 May 2008 11:23:56 +0200 Subject: [rsyslog] rsyslog 3.16.1 (v3-stbale) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308F30@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 3.16.1, a release of the v3-stable branch. This version contains one important bug fix. The imklog input plugin did not correctly start up in some environments and could even cause a segfault. This is now corrected. Upgrading to 3.16.1 is strongly recommended for all users of the v3-stable branch who use imklog. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-100.phtml Changelog: http://www.rsyslog.com/Article218.phtml Florian Riedl From lanas at securenet.net Fri May 2 13:00:45 2008 From: lanas at securenet.net (lanas) Date: Fri, 2 May 2008 07:00:45 -0400 Subject: [rsyslog] MARK frequency Message-ID: <20080502070045.60d2c904@mistral.stie> Hello, I use the stable rsyslogd 3.16. I launched it using the -c3 switch. I'm trying to vary the frequency of the MARK log entries. 1) The configuration file has: $ModLoad immark.so $MarkMessagePeriod 60 But rsyslog does: 3829.182720884:main thread: wtpAdviseMaxWorkers signals busy 3829.182865900:main thread: logmsg: flags 5, from 'localhost', msg Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark 3829.183039173:main thread: Message has legacy syslog format. 3829.183203040:main thread: main queue: entry added, size now 4 entries 3829.183353576:main thread: main queue: EnqueueMsg signaled condition (0) 3829.183563839:main thread: wtpAdviseMaxWorkers signals busy 3829.183714361:main thread: logmsg: flags 5, from 'localhost', msg Warning: backward compatibility layer added to following directive to rsyslog.conf: MarkMessagePeriod 1200 3829.183890978:main thread: Message has legacy syslog format. 3829.184054851:main thread: main queue: entry added, size now 5 entries Why isn't the 60 value taken in ? 2) Is the -c3 switch mandatory or can rsyslog work naturally without compatibility mode without that switch ? Thanks. From rgerhards at hq.adiscon.com Fri May 2 13:37:38 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 2 May 2008 13:37:38 +0200 Subject: [rsyslog] MARK frequency Message-ID: <001b01c8ac49$01776713$060013ac@intern.adiscon.com> Please send me the 50 or so FIRST lines from startup. I guess there is something wrong with the option (processing). Thanks, rainer ----- Urspr?ngliche Nachricht ----- Von: "lanas" An: "rsyslog at lists.adiscon.com" Gesendet: 02.05.08 13:03 Betreff: [rsyslog] MARK frequency Hello, I use the stable rsyslogd 3.16. I launched it using the -c3 switch. I'm trying to vary the frequency of the MARK log entries. 1) The configuration file has: $ModLoad immark.so $MarkMessagePeriod 60 But rsyslog does: 3829.182720884:main thread: wtpAdviseMaxWorkers signals busy 3829.182865900:main thread: logmsg: flags 5, from 'localhost', msg Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark 3829.183039173:main thread: Message has legacy syslog format. 3829.183203040:main thread: main queue: entry added, size now 4 entries 3829.183353576:main thread: main queue: EnqueueMsg signaled condition (0) 3829.183563839:main thread: wtpAdviseMaxWorkers signals busy 3829.183714361:main thread: logmsg: flags 5, from 'localhost', msg Warning: backward compatibility layer added to following directive to rsyslog.conf: MarkMessagePeriod 1200 3829.183890978:main thread: Message has legacy syslog format. 3829.184054851:main thread: main queue: entry added, size now 5 entries Why isn't the 60 value taken in ? 2) Is the -c3 switch mandatory or can rsyslog work naturally without compatibility mode without that switch ? Thanks. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sun May 4 10:52:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 4 May 2008 10:52:37 +0200 Subject: [rsyslog] rsyslog 3.17.2 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308F40@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.17.2. This is a new major improvement of the beta branch. It brings all features of the former 3.17 devel branch to the beta, plus includes a bug fix in the imklog module. Version 3.17.2 is a recommended update for all beta branch users. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-101.phtml Change Log: http://www.rsyslog.com/Article220.phtml I hope this release is useful. As always, feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Tue May 6 12:09:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 6 May 2008 12:09:44 +0200 Subject: [rsyslog] rsyslog 3.19.0 released - world's first syslog-transport-tls implementation Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308F5A@grfint2.intern.adiscon.com> Hi all, I am very pleased to announce rsyslog 3.19.0. It is the first release that natively supports TLS for plain TCP syslog. Actually, it is the world's first implementation of the upcoming syslog-transport-tls IETF standard. As this standard is not yet finished, the implementation is obviously experimental. Native TLS is a big improvement over existing functionality. For example, rsyslog can now be used without the help of stunnel, which relieves us of some problems from those configurations. To the best of my knowledge, rsyslog is the first open-source syslogd offering TLS support natively. The current TLS functionality is limited to the bare minimum. During the next few weeks, I will improve it based on my own spec and feedback (hopefully received). My hope is to have a production-grade implementation by summer at latest. Please note rsyslog's premium and ultra-reliable RELP protocol does not yet support TLS (but can be used with stunnel without the real problems legacy tcp had with it). My plan is to let TLS mature with legacy syslog and then move it to RELP. Thus I can limit my development to one major use case, which I think will facilitate things. There is some documentation on how to use the new TLS mode: http://www.rsyslog.com/doc-rsyslog_tls.html Currently, TLS is provided via GnuTLS. As I outlined earlier on the list, GnuTLS offered much more support to getting started (documentation and sample-code wise). I will focus on GnuTLS until I am fully satisfied with the TLS implementation). I'll then see that I can also integrate NSS. Advise in this regard would be highly welcome, so if you have some knowledge in this area, please contribute. In order to support TLS (and multiple libraries!), a major rewrite of the networking components has been done. Rsyslog now supports a so-called "network streams" (netstreams) driver interface. This interface enables app-level functionality (like the legacy tcp syslog sender and receiver) to work with dynamically selectable netstream drivers (like plain (unencrypted) TCP) and TLS). This interface will enable rsyslog to utilize other TLS drivers (and even other protocols) in the future. Different drivers can even be used concurrently. Rsyslog now has been split into a runtime system and tools (with currently rsyslogd being the only tool). This design will further strengthen modularization and help make rsyslog functionality available in small parts. Finally, the RFC 3195 input has been rewritten in the form of an input plugin. It can now be build as part of the normal build procedure. Please note that there were a couple of major changes. I expect the initial 3.19.0 to be quite Unstable. I recommend it for testing environments, only. Even those parts that were not directly touched may have become a bit destabilized due to the runtime split. So please use it with care. Feedback, however, would be more than welcome, because I need to start somewhere to stabilize this release. That can only be done with your help. So please use it on test systems, try to break it and file bug reports when it fails. Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-102.phtml Changelog: http://www.rsyslog.com/Article221.phtml File your bug reports here ;) : http://bugzilla.adiscon.com/rsyslog-bugs.html I hope this release is useful. Feedback is much appreciated. Rainer From rgerhards at hq.adiscon.com Tue May 6 17:45:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 6 May 2008 17:45:32 +0200 Subject: [rsyslog] missing file in 3.19.0 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308F67@grfint2.intern.adiscon.com> Hi all, there seems to be a file missing in the root of rsyslog in 3.19.0. It is dirty.h, which can be found here: http://git.adiscon.com/?p=rsyslog.git;a=blob;f=dirty.h;h=6c4516621e75f0b 4368948d908ead91753feabea;hb=HEAD Thanks to darix for pointing this out. I will release an updated tarball tomorrow, but I wait for some more feedback which probably helps me ironing out a few other nits of this release. Thanks, Rainer From rgerhards at hq.adiscon.com Wed May 7 19:53:14 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 7 May 2008 19:53:14 +0200 Subject: [rsyslog] rsyslog 3.19.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308F8C@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.1, an update of the development branch. This version primarily fixes some problems with the build system. There were some files missing, which could result in strange warnings. Also, some minor code cleanup to get rid of some compiler warnings is included. This version is highly recommended for all users of the development branch. Note, however, that it is still not suitable for use on production machines. Changelog: http://www.rsyslog.com/Article223.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-103.phtml As always, feedback is appreciated. Rainer -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Fri May 9 09:05:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 9 May 2008 09:05:04 +0200 Subject: [rsyslog] syslog TLS use cases Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308FAE@grfint2.intern.adiscon.com> Hi folks, I am looking for use cases for TLS protected syslog. This is in an effort to both drive the relevant syslog standard into the right direction as well as make sure that rsyslog's TLS implementation will focus on the real-world needs. I have set up a wiki page for this: http://wiki.rsyslog.com/index.php/TLS_for_syslog_use_cases I would appreciate contributions, preferably by direct wiki edits. But you may also simply mail me and I can integrate it. Please provide feedback, it is extremely useful to get things correctly done in rsyslog. Rainer From rgerhards at hq.adiscon.com Thu May 15 12:38:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 15 May 2008 12:38:43 +0200 Subject: [rsyslog] rsyslog 2.0.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309005@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 2.0.5, an update for the v2-stable branch. This is primarily a bug fixing release. It solves a problem with regular expression based field-extraction and provides better compatibility with recent versions of liblogging. This update is recommended, but not absolutely necessary if you do not experience any problems with your current v2-stable install. Changelog: http://www.rsyslog.com/Article226.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-104.phtml I hope this release is useful. As always, feedback is appreciated. Rainer From friedl at hq.adiscon.com Fri May 16 16:44:50 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Fri, 16 May 2008 16:44:50 +0200 Subject: [rsyslog] rsyslog 3.19.2 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30902A@grfint2.intern.adiscon.com> Hi all, rsyslog 3.19.2, a development branch version, has just been released. This version contains a new property "fromhost-ip", which enables to obtain the IP address of the syslog sender irrelevant of the -x option setting. So now both the name and IP address are available. Other than that, the version contains a number of bug fixes, many in the TLS area. This version is a recommended update for all development branch users. It is quite stable. It is not yet suggested to use the TLS functionality outside of test environments. Changelog: http://www.rsyslog.com/Article228.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-105.phtml As always, feedback is appreciated, Florian Riedl From friedl at hq.adiscon.com Wed May 21 13:01:33 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 21 May 2008 13:01:33 +0200 Subject: [rsyslog] rsyslog 3.19.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com> Hi all, We have just released rsyslog 3.19.3. The primary new feature is support for (optional) mutual authentication in TLS mode. This is based on IETF's upcoming syslog-transport-tls standard in certificate fingerprint authentication mode. Other than that, there are a couple of bug fixes. The is a recommended update for all users of the development branch. Please note that TLS support is still quite experimental and not recommended for production use. Changelog: http://www.rsyslog.com/Article230.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-106.phtml As always, feedback is appreciated. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From mbiebl at gmail.com Wed May 21 14:10:41 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Wed, 21 May 2008 14:10:41 +0200 Subject: [rsyslog] rsyslog 3.19.3 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com> Message-ID: 2008/5/21 Florian Riedl : > Hi all, > > We have just released rsyslog 3.19.3. The primary new feature is support > for (optional) mutual authentication in TLS mode. This is based on > IETF's upcoming syslog-transport-tls standard in certificate fingerprint > authentication mode. Other than that, there are a couple of bug fixes. > The is a recommended update for all users of the development branch. > Please note that TLS support is still quite experimental and not > recommended for production use. I'm getting 100% cpu usage with this version and the daemon can only be killed with -9. Stock configuration without using TLS or RELP. I can provide more info if requested. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Wed May 21 14:13:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 21 May 2008 14:13:06 +0200 Subject: [rsyslog] rsyslog 3.19.3 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30906D@grfint2.intern.adiscon.com> Yes, pls provide more info. This seems to be a problem that I have seen (but not yet able to fix) on BSD only. This is the bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=67 Please try to disable imklog, so that we know if it is the culprit (if not, that's probably a different bug). Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Wednesday, May 21, 2008 2:11 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.19.3 released > > 2008/5/21 Florian Riedl : > > Hi all, > > > > We have just released rsyslog 3.19.3. The primary new feature is > support > > for (optional) mutual authentication in TLS mode. This is based on > > IETF's upcoming syslog-transport-tls standard in certificate > fingerprint > > authentication mode. Other than that, there are a couple of bug > fixes. > > The is a recommended update for all users of the development branch. > > Please note that TLS support is still quite experimental and not > > recommended for production use. > > I'm getting 100% cpu usage with this version and the daemon can only > be killed with -9. > Stock configuration without using TLS or RELP. > I can provide more info if requested. > > Cheers, > Michael > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Wed May 21 14:21:02 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Wed, 21 May 2008 14:21:02 +0200 Subject: [rsyslog] rsyslog 3.19.3 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30906D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA30906D@grfint2.intern.adiscon.com> Message-ID: 2008/5/21 Rainer Gerhards : > Yes, pls provide more info. This seems to be a problem that I have seen > (but not yet able to fix) on BSD only. This is the bug tracker: > > http://bugzilla.adiscon.com/show_bug.cgi?id=67 > > Please try to disable imklog, so that we know if it is the culprit (if > not, that's probably a different bug). Disabling imklog did indeed help. I'm on Linux (Debian) though. Config file attached Michael P.S: Should we continue the discussion via the BTS? -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Wed May 21 14:23:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 21 May 2008 14:23:12 +0200 Subject: [rsyslog] rsyslog 3.19.3 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA30906D@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30906E@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Wednesday, May 21, 2008 2:21 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.19.3 released > > 2008/5/21 Rainer Gerhards : > > Yes, pls provide more info. This seems to be a problem that I have > seen > > (but not yet able to fix) on BSD only. This is the bug tracker: > > > > http://bugzilla.adiscon.com/show_bug.cgi?id=67 > > > > Please try to disable imklog, so that we know if it is the culprit > (if > > not, that's probably a different bug). > > Disabling imklog did indeed help. I'm on Linux (Debian) though. > Config file attached > > Michael > > P.S: Should we continue the discussion via the BTS? That's a good idea, especially as the list processor stripped the config ;) Rainer From mbiebl at gmail.com Wed May 21 14:38:00 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Wed, 21 May 2008 14:38:00 +0200 Subject: [rsyslog] rsyslog 3.19.3 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30906E@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA309069@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA30906D@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA30906E@grfint2.intern.adiscon.com> Message-ID: 2008/5/21 Rainer Gerhards : >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl >> Sent: Wednesday, May 21, 2008 2:21 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog 3.19.3 released >> >> 2008/5/21 Rainer Gerhards : >> > Yes, pls provide more info. This seems to be a problem that I have >> seen >> > (but not yet able to fix) on BSD only. This is the bug tracker: >> > >> > http://bugzilla.adiscon.com/show_bug.cgi?id=67 >> > >> > Please try to disable imklog, so that we know if it is the culprit >> (if >> > not, that's probably a different bug). >> >> Disabling imklog did indeed help. I'm on Linux (Debian) though. >> Config file attached >> >> Michael >> >> P.S: Should we continue the discussion via the BTS? > > That's a good idea, especially as the list processor stripped the config Done -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From stephen.carville at gmail.com Sun May 25 06:23:22 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Sat, 24 May 2008 21:23:22 -0700 Subject: [rsyslog] Duplicate entries Message-ID: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com> omething weird is happening with rsyslog. The good news is I dont seem to lose anything even when the sub-morons in charge of the network load a six month old firewall config when they click the wrong button in the GUI. The bad news is that, lately, I have been seeing duplicate entries in the messages table but not in the other tables. At first I thought it was beacause there might be a few machines still running both syslog and rsyslog. However, I tracked doen and zapped the rogue syslog processes and the problem still persists. I know I'm a little down rev but I'd like to out off an upgrade unitl after the next audit. However, if this is a "known issue" I'll certainly upgrade and take whatever licks it costs. Current config: $ rsyslogd -v rsyslogd 2.0.1, compiled with: FEATURE_PTHREADS (dual-threading): Yes FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes SYSLOG_INET (Internet/remote support): Yes FEATURE_GSSAPI (GSSAPI Kerberos 5 support): No FEATURE_DEBUG (debug build, slow code): No ############### Server rsyslog.conf file ####################### $ModLoad MySQL *.info;mail.none;authpriv.none;cron.none >127.0.0.1,messages,syslogger, authpriv.* >127.0.0.1,secure,syslogger. mail.* -/var/log/maillog cron.* /var/log/cron *.emerg * uucp,news.crit /var/log/spooler local7.* /var/log/boot.log ###################### Host rsyslog.conf file ####################### *.info;mail.none;authpriv.none;cron.none /var/log/messages *.info;mail.none;authpriv.none;cron.none @@scacisys01 auth,authpriv.* /var/log/secure auth,authpriv.* @@scacisys01 # Log all the mail messages in one place. mail.* -/var/log/maillog cron.* /var/log/cron *.emerg * uucp,news.crit /var/log/spooler local7.* /var/log/boot.log -- Stephen Carville From rgerhards at hq.adiscon.com Sun May 25 10:57:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 25 May 2008 10:57:35 +0200 Subject: [rsyslog] Duplicate entries In-Reply-To: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com> References: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> Mhhh... I don't see anything bad nor have found any past reference to duplicated messages. In later versions, this may (very unlinkely) happen by intension to prevent message loss when a TCP connection breaks. This was introduced to cope with the unreliability of TCP syslog: http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.ht ml However, this is not in the version you have. Can you report anything specific on the message duplication? Also, would it be an option to upgrade to the latest v2-stable version (which is 2.0.5). Note that the difference is only bug fixes, no new functionality is being added to v2. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: Sunday, May 25, 2008 6:23 AM > To: rsyslog-users > Subject: [rsyslog] Duplicate entries > > omething weird is happening with rsyslog. > > The good news is I dont seem to lose anything even when the sub-morons > in charge of the network load a six month old firewall config when > they click the wrong button in the GUI. > > The bad news is that, lately, I have been seeing duplicate entries in > the messages table but not in the other tables. At first I thought it > was beacause there might be a few machines still running both syslog > and rsyslog. However, I tracked doen and zapped the rogue syslog > processes and the problem still persists. > > I know I'm a little down rev but I'd like to out off an upgrade unitl > after the next audit. However, if this is a "known issue" I'll > certainly upgrade and take whatever licks it costs. > > Current config: > > $ rsyslogd -v > rsyslogd 2.0.1, compiled with: > FEATURE_PTHREADS (dual-threading): Yes > FEATURE_REGEXP: Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > SYSLOG_INET (Internet/remote support): Yes > FEATURE_GSSAPI (GSSAPI Kerberos 5 support): No > FEATURE_DEBUG (debug build, slow code): No > > ############### Server rsyslog.conf file ####################### > > $ModLoad MySQL > > *.info;mail.none;authpriv.none;cron.none > >127.0.0.1,messages,syslogger, > > authpriv.* > >127.0.0.1,secure,syslogger. > > mail.* - > /var/log/maillog > > cron.* /var/log/cron > > *.emerg * > > uucp,news.crit > /var/log/spooler > > local7.* > /var/log/boot.log > > ###################### Host rsyslog.conf file ####################### > > *.info;mail.none;authpriv.none;cron.none /var/log/messages > *.info;mail.none;authpriv.none;cron.none @@scacisys01 > > auth,authpriv.* /var/log/secure > auth,authpriv.* @@scacisys01 > > # Log all the mail messages in one place. > mail.* -/var/log/maillog > > cron.* /var/log/cron > > *.emerg * > > uucp,news.crit /var/log/spooler > > local7.* /var/log/boot.log > > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From stephen.carville at gmail.com Sun May 25 18:10:00 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Sun, 25 May 2008 09:10:00 -0700 Subject: [rsyslog] Duplicate entries In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> References: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> Message-ID: <2428c0380805250910v725341bdj9b35febca3335016@mail.gmail.com> On Sun, May 25, 2008 at 1:57 AM, Rainer Gerhards wrote: > However, this is not in the version you have. Can you report anything > specific on the message duplication? I used some network tools to watch the behavior between scacitf01-d (clent) and scacisys01 (server). $ netstat -atp (domain name removed for readability); on scacisys01 tcp 0 0 scacisys01:shell scacitf01-d:57878 ESTABLISHED 26777/rsyslogd tcp 0 0 scacisys01:shell scacitf01-d:57876 ESTABLISHED 26777/rsyslogd on scacitf01-d tcp 0 0 scacitf01-d:57878 scacisys01:shell ESTABLISHED 10387/rsyslogd tcp 0 0 scacitf01-d:57876 scacisys01:shell ESTABLISHED 10387/rsyslogd Should there be two connections? For an ssh login, wireshark reports that the syslog server received the following from scacitf01-d:57876 : <38>May 25 07:35:26 scacitf01-d sshd[16155]: Connection from 10.212.166.26 port 46039 <38>May 25 07:35:26 scacitf01-d sshd[16155]: Found matching RSA key: a0:f6:2d:e7:4d:b8:c0:53:01:c6:b3:ce:63:16:05:4f <38>May 25 07:35:26 scacitf01-d sshd[16156]: Postponed publickey for stephen from 10.212.166.26 port 46039 ssh2 <38>May 25 07:35:26 scacitf01-d sshd[16155]: Found matching RSA key: a0:f6:2d:e7:4d:b8:c0:53:01:c6:b3:ce:63:16:05:4f <38>May 25 07:35:26 scacitf01-d sshd[16155]: Accepted publickey for stephen from 10.212.166.26 port 46039 ssh2 and from scacitf01-d:57878: <38>May 25 07:35:26 scacitf01-d sshd[16155]: Connection from 10.212.166.26 port 46039 <38>May 25 07:35:26 scacitf01-d sshd[16155]: Found matching RSA key: a0:f6:2d:e7:4d:b8:c0:53:01:c6:b3:ce:63:16:05:4f <38>May 25 07:35:26 scacitf01-d sshd[16156]: Postponed publickey for stephen from 10.212.166.26 port 46039 ssh2 <38>May 25 07:35:26 scacitf01-d sshd[16155]: Found matching RSA key: a0:f6:2d:e7:4d:b8:c0:53:01:c6:b3:ce:63:16:05:4f <38>May 25 07:35:26 scacitf01-d sshd[16155]: Accepted publickey for stephen from 10.212.166.26 port 46039 ssh2 <86>May 25 07:35:26 scacitf01-d sshd[16155]: pam_unix(sshd:session): session opened for user stephen by (uid=0) So some message are being sent twice. >Also, would it be an option to > upgrade to the latest v2-stable version (which is 2.0.5). Note that the > difference is only bug fixes, no new functionality is being added to v2. Probably if it's only bug fixes. -- Stephen Carville From stephen.carville at gmail.com Sun May 25 21:10:01 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Sun, 25 May 2008 12:10:01 -0700 Subject: [rsyslog] Duplicate entries In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> References: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> Message-ID: <2428c0380805251210q4b3f647q4e8d2b0de5fcf97d@mail.gmail.com> It definitely seems to the mulitple connections. I change the client configuration to funnel all the messages thru a single connection and teh duplication stopped. Before *.info;mail.none;authpriv.none;cron.none /var/log/messages *.info;mail.none;authpriv.none;cron.none @@scacisys01 authpriv.* /var/log/secure authpriv.* @@scacisys01 After *.info;mail.none;authpriv.none;cron.none /var/log/messages authpriv.* /var/log/secure *.info;auth,authpriv.*;mail.none;cron.none @@scacisys01 -- Stephen Carville From rgerhards at hq.adiscon.com Mon May 26 08:18:32 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 26 May 2008 08:18:32 +0200 Subject: [rsyslog] Duplicate entries In-Reply-To: <2428c0380805251210q4b3f647q4e8d2b0de5fcf97d@mail.gmail.com> References: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> <2428c0380805251210q4b3f647q4e8d2b0de5fcf97d@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30908D@grfint2.intern.adiscon.com> Hi Stephen, If I understand you right, there was nothing wrong with rsyslog. You instructed it to forward authpriv.none message to the remote host: > *.info;mail.none;authpriv.none;cron.none @@scacisys01 And you also instructed it to also forward all authpriv messages to the same host: > authpriv.* @@scacisys01 That, of course, leads to authpriv.none message to be forwarded twice - once by the first rule and another time by the second. Please note that rule execution is serially. There is no interdependency between rules. If a rule matches, the action is carried out. This is context-free. Rsyslog doesn't care if another rule has the same action. In this sample, it may be useful you did not intend what you configured, but in most cases it would be hard to tell what your real intension would be. So rather than trying the figure out the users intension, rsyslog simply carries out what is configured ;) The bottom line to keep in mind is that each rule is a separate unit of execution. HTH, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: Sunday, May 25, 2008 9:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Duplicate entries > > It definitely seems to the mulitple connections. I change the client > configuration to funnel all the messages thru a single connection and > teh duplication stopped. > > Before > > *.info;mail.none;authpriv.none;cron.none /var/log/messages > *.info;mail.none;authpriv.none;cron.none @@scacisys01 > > authpriv.* /var/log/secure > authpriv.* @@scacisys01 > > After > > *.info;mail.none;authpriv.none;cron.none /var/log/messages > > authpriv.* /var/log/secure > > *.info;auth,authpriv.*;mail.none;cron.none @@scacisys01 > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From erik at tengblad.net Mon May 26 09:15:06 2008 From: erik at tengblad.net (Erik Tengblad) Date: Mon, 26 May 2008 09:15:06 +0200 Subject: [rsyslog] Create log files based on host name Message-ID: <483A637A.3090009@tengblad.net> Hello all, First of all, let me apologize in advance for the somewhat newbie-ish nature of this question. I'm sure there's an easy way to solve my problem, but I've been looking for an answer for weeks now without being able to find one. Here's the deal; A friend and I am currently working as interns at a company.We've been tasked with setting up a central logging service for one of their test environments. Getting rsyslog to write to the log server is the easy part. What we're having trouble with is setting it up (both on server and client side) so that 1) All the log files are written per host. IE, we want rsyslog to write seperate log files based on the host from which the logs are being sent. Say we have 10 machines, each called host01 to host 10. We want all the log information from host01 to be written to /var/log/host01/logfile.log and so on. We've tried achieving this using templates and the :hostname, isqueal, "host01" feature, but we just can't get it to work. Most likely we've not used the correct syntax in the rsyslog.conf file. 2) We want all log information containing a certain string to be written to separate log files as well. So everything containing 'zyx' should be written to /var/log/host01/zyx.log and everything containing 'abc' should be written to /var/log/host01/abc.log and so on. As I said, I'm certain there are easy ways to achieve this using rsyslog, but we're too inexperienced at using the app to know how to do it. Hopefully someone on here can help us out a bit. :) Thanks in advance, Erik Tengblad From stephen.carville at gmail.com Tue May 27 01:02:39 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Mon, 26 May 2008 16:02:39 -0700 Subject: [rsyslog] Duplicate entries In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30908D@grfint2.intern.adiscon.com> References: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com> <2428c0380805251210q4b3f647q4e8d2b0de5fcf97d@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA30908D@grfint2.intern.adiscon.com> Message-ID: <2428c0380805261602l3bfda40fm15667d7f3195a211@mail.gmail.com> On Sun, May 25, 2008 at 11:18 PM, Rainer Gerhards wrote: > Hi Stephen, > > If I understand you right, there was nothing wrong with rsyslog. You > instructed it to forward authpriv.none message to the remote host: > >> *.info;mail.none;authpriv.none;cron.none @@scacisys01 > > And you also instructed it to also forward all authpriv messages to the > same host: > >> authpriv.* @@scacisys01 > > That, of course, leads to authpriv.none message to be forwarded twice - > once by the first rule and another time by the second. Doesn't authpriv.none mean send no authpriv messages? Since it is is after the *.info it should take precedence. At least thats how I understand the syslog rules to work. In any case, I understand what I did wrong. I had *.info in one rule and auth.* in another creating an overlap. > Please note that rule execution is serially. There is no interdependency > between rules. If a rule matches, the action is carried out. This is > context-free. Rsyslog doesn't care if another rule has the same action. > In this sample, it may be useful you did not intend what you configured, > but in most cases it would be hard to tell what your real intension > would be. So rather than trying the figure out the users intension, > rsyslog simply carries out what is configured ;) > > The bottom line to keep in mind is that each rule is a separate unit of > execution. I cannot see any downside to putting them all on the same connection so I'll leave it that way FTTB. Thank your for you help -- Stephen Carville From rgerhards at hq.adiscon.com Tue May 27 10:20:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 27 May 2008 10:20:00 +0200 Subject: [rsyslog] Duplicate entries In-Reply-To: <2428c0380805261602l3bfda40fm15667d7f3195a211@mail.gmail.com> References: <2428c0380805242123x12bf89f6jcefc205fa9f63d79@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30908B@grfint2.intern.adiscon.com><2428c0380805251210q4b3f647q4e8d2b0de5fcf97d@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30908D@grfint2.intern.adiscon.com> <2428c0380805261602l3bfda40fm15667d7f3195a211@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3090B2@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: Tuesday, May 27, 2008 1:03 AM > To: rsyslog-users > Subject: Re: [rsyslog] Duplicate entries > > On Sun, May 25, 2008 at 11:18 PM, Rainer Gerhards > wrote: > > Hi Stephen, > > > > If I understand you right, there was nothing wrong with rsyslog. You > > instructed it to forward authpriv.none message to the remote host: > > > >> *.info;mail.none;authpriv.none;cron.none @@scacisys01 > > > > And you also instructed it to also forward all authpriv messages to > the > > same host: > > > >> authpriv.* @@scacisys01 > > > > That, of course, leads to authpriv.none message to be forwarded twice > - > > once by the first rule and another time by the second. > > Doesn't authpriv.none mean send no authpriv messages? Since it is is > after the *.info it should take precedence. At least thats how I > understand the syslog rules to work. I have to admit that this is probably the last bit of sysklogd code I did never even look at ;) But you are right, this sounds like a bug. I'll review the code once I am through with TLS. > In any case, I understand what I did wrong. I had *.info in one rule > and auth.* in another creating an overlap. > > > Please note that rule execution is serially. There is no > interdependency > > between rules. If a rule matches, the action is carried out. This is > > context-free. Rsyslog doesn't care if another rule has the same > action. > > In this sample, it may be useful you did not intend what you > configured, > > but in most cases it would be hard to tell what your real intension > > would be. So rather than trying the figure out the users intension, > > rsyslog simply carries out what is configured ;) > > > > The bottom line to keep in mind is that each rule is a separate unit > of > > execution. > > I cannot see any downside to putting them all on the same connection > so I'll leave it that way FTTB. One connection is fine. The fewer rules you need, the better the performance is :) Rainer > > Thank your for you help > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue May 27 15:21:41 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 27 May 2008 15:21:41 +0200 Subject: [rsyslog] rsyslog 3.19.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3090B6@grfint2.intern.adiscon.com> Hi all, I am glad to announce rsyslog 3.19.4. Its main new feature is greatly improved authorization settings for TLS peers. It now contains a full implementation of IETF's most recent additions to the syslog-transport-tls draft. Certificates can now be verified and authentication be based on names inside certificates, including the ability to use wildcard matches. The changed fingerprint syntax is also supported (this requires a config change in existing installations). There is one important bug fix, solving a situation where imklog could cause 100% CPU utilization. This is a recommended update for all development branch users. Change Log: http://www.rsyslog.com/Article232.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-108.phtml I expect TLS support to be feature-complete with this release (except if something major is changing inside the IETF drafts). My focus will now be on polishing that release a bit and then move on to another target while this release matures in practice. I do not yet recommend placing 3.19.4 onto a production box. As always, feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Wed May 28 10:02:10 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 28 May 2008 10:02:10 +0200 Subject: [rsyslog] some background on TLS in rsyslog Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3090BE@grfint2.intern.adiscon.com> Hi all, I have written an implementation report of TLS syslog. While the prime audience is the IETF syslog working group, it may be of interest for you, too: http://blog.gerhards.net/2008/05/syslog-transport-tls-12-implementation. html Rainer From friedl at hq.adiscon.com Wed May 28 13:06:12 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Wed, 28 May 2008 13:06:12 +0200 Subject: [rsyslog] rsyslog 3.17.3 (beta) released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3090CF@grfint2.intern.adiscon.com> Hi all, We have just released a refresh of the current beta. It contains a bugfix for imklog running in a tight loop and utilizing 100% CPU in some cases. There are no other changes. This is a strongly recommended update for all beta branch users. ChangeLog: http://www.rsyslog.com/Article233.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-109.phtml Please note that we are nearing the point where 3.17.x will become the new stable. As such, we would really appreciate if you let us know any bugs and annoyances you may discover. Florian Riedl -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html . From rgerhards at hq.adiscon.com Fri May 30 17:10:14 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 30 May 2008 17:10:14 +0200 Subject: [rsyslog] rsyslog 3.19.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA309105@grfint2.intern.adiscon.com> Hi all, I have just released rsyslog 3.19.5. This is a feature-enhancement release. It offers improved regular expression support in the property replacer. Posix ERE expressions and submatches are now supported. Also, the result in a case where no match could be found can now be configured. There are no other changes in this release. This is a recommended update for all development branch users. Changelog: http://www.rsyslog.com/Article236.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-110.phtml I hope this is a useful release. As always, feedback is appreciated. Rainer