From leitstelle at bnet.li Mon Mar 3 15:07:23 2008 From: leitstelle at bnet.li (leitstelle at bnet.li) Date: Mon, 03 Mar 2008 15:07:23 +0100 Subject: [rsyslog] Selector and MySQL Message-ID: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> Hi all, i tried to write data into a DB with a selector in rsyslog.conf: ... !dhcpd $template test, "...",sql :msg, contains, "dinfo" >127.0.0.1,db,user,pw;test #eof one of the messages, which rsyslog is like: 2008-03-03 14:14.20 dhcpd : dinfo:, myipadress and mac So from my point of view the selector should write this message to the DB. BUT it doesn`t. If i replace the selector e.g. with *.info it works (but then all message are written to the DB) Thanks for your support Andre From rgerhards at hq.adiscon.com Mon Mar 3 15:17:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 3 Mar 2008 15:17:21 +0100 Subject: [rsyslog] Selector and MySQL In-Reply-To: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> References: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AC0@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of leitstelle at bnet.li > Sent: Monday, March 03, 2008 3:07 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Selector and MySQL > > Hi all, > > i tried to write data into a DB with a selector > > in rsyslog.conf: > ... > !dhcpd > $template test, "...",sql > :msg, contains, "dinfo" >127.0.0.1,db,user,pw;test > #eof > > one of the messages, which rsyslog is like: > 2008-03-03 14:14.20 dhcpd : dinfo:, myipadress and mac Umm... That message format looks strange. I guess the parser gets confused by it. Can you please run rsyslogd in interactively in debug mode (-d -n) and post the resulting log while such a message is received. Thanks, Rainer > > So from my point of view the selector should write this message to the > DB. > BUT it doesn`t. > > If i replace the selector e.g. with *.info it works (but then all > message are written to the DB) > > Thanks for your support > Andre > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 3 16:27:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 3 Mar 2008 16:27:30 +0100 Subject: [rsyslog] Solaris Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> Hi all, is there anybody on this list with (open) Solaris? I installed a system and tried to compile rsyslog on it, but I can't get the environment ready :( I somebody is in the know, advise would deeply be appreciated (if not, that's a good indication I still do not need to care about Solaris, which also solves my issue ;)). Thanks, Rainer From Mark.Capelle at pcmc.com Tue Mar 4 15:43:19 2008 From: Mark.Capelle at pcmc.com (Capelle, Mark (PCMC-GB)) Date: Tue, 4 Mar 2008 08:43:19 -0600 Subject: [rsyslog] Recording IP address of sender in logs Message-ID: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> I have been using syslogd 1.4.1 for quite some time to gather syslogs from multiple hosts which I then have a script which separates out the logs based on the source host. I just rebuilt my syslog server and took the opportunity to use rsyslogd 2.0.2 instead of the syslogd, but I noticed that my syslogs no longer record the address of the source host which will break my scripts. Is there any way to change this behavior? Examples - Syslogd: Mar 4 08:05:47 10.10.1.1 Mar 04 2008 08:05:47: %ASA-4-106023: Deny tcp src inside:x.x.x.x/3713 dst outside:x.x.x.x/1021 by access-group "inside_access_out" [0x0, 0x0] ^^^^^^^^^ Rsyslogd: Mar 4 08:41:17 Mar 04 2008 08:41:25: %ASA-4-106023: Deny tcp src inside:x.x.x.x/2125 dst outside:x.x.x.x/9018 by access-group "inside_access_out" [0x0, 0x0] Regards, Mark From rgerhards at hq.adiscon.com Tue Mar 4 16:58:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Mar 2008 16:58:23 +0100 Subject: [rsyslog] Recording IP address of sender in logs In-Reply-To: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> References: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AD4@grfint2.intern.adiscon.com> Mark, it looks like message format received is somewhat invalid. Rsyslog tries to parse according to rfc 3164 but also applies legacy legacy parsing if it finds a way to do so. If you could provide me a wireshark capture of the message, I could see if I can modify the parser. HOWEVER, that shouldn't limit to do you what you want. Nothing is fixed in rsyslog and you can modify the format via a template. It is just the default template that gives you the result below. Please see the rsyslog doc on how to define other template. If you have a hard time, let me know and I'll figure it out for you, but I am currently very busy with new work, so I can't promise on the timeline. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Capelle, Mark (PCMC-GB) > Sent: Tuesday, March 04, 2008 3:43 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Recording IP address of sender in logs > > I have been using syslogd 1.4.1 for quite some time to gather syslogs > from multiple hosts which I then have a script which separates out the > logs based on the source host. I just rebuilt my syslog server and > took > the opportunity to use rsyslogd 2.0.2 instead of the syslogd, but I > noticed that my syslogs no longer record the address of the source host > which will break my scripts. Is there any way to change this behavior? > > > > Examples - > > > > Syslogd: > > Mar 4 08:05:47 10.10.1.1 Mar 04 2008 08:05:47: > %ASA-4-106023: Deny tcp src inside:x.x.x.x/3713 dst > outside:x.x.x.x/1021 > by access-group "inside_access_out" [0x0, 0x0] > > ^^^^^^^^^ > > > > Rsyslogd: > > Mar 4 08:41:17 Mar 04 2008 08:41:25: %ASA-4-106023: Deny > tcp src inside:x.x.x.x/2125 dst outside:x.x.x.x/9018 by access-group > "inside_access_out" [0x0, 0x0] > > > > > > Regards, > > Mark > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Thu Mar 6 01:49:46 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 05 Mar 2008 19:49:46 -0500 Subject: [rsyslog] local/remote logging Message-ID: <47CF3FAA.6050302@gmail.com> I want to have logs written locally (the way they usually are) as well as sent to a central logger. The two situations that I want local logs for are: 1) in case the central logger is not available for any reason, I don't lose messages 2) if I need to troubleshoot something, I can just look at the local logs Obviously, I will rotate the local logs and have older ones discarded. Whereas the central logger will archive them for longer periods. Can I accomplish this with rsyslog? AIUI, I can, and I have to move to at least v3.11.x -- is that correct? Assuming yes, I already got the remote logging working. What do I do to get the simultaneous local logging to work too? I can't find any wiki or faq on this. If I get it working, I'd be happy to post it on the site. johnn From mbiebl at gmail.com Thu Mar 6 02:24:12 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Thu, 6 Mar 2008 02:24:12 +0100 Subject: [rsyslog] Solaris In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> Message-ID: 2008/3/3, Rainer Gerhards : > Hi all, > > is there anybody on this list with (open) Solaris? I installed a system > and tried to compile rsyslog on it, but I can't get the environment > ready :( > > I somebody is in the know, advise would deeply be appreciated (if not, > that's a good indication I still do not need to care about Solaris, > which also solves my issue ;)). http://blogs.sun.com/AngeloJ/entry/virtualbox_and_opensolaris_indiana ? -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Wed Mar 5 19:16:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 05 Mar 2008 19:16:18 +0100 Subject: [rsyslog] local/remote logging In-Reply-To: <47CF3FAA.6050302@gmail.com> References: <47CF3FAA.6050302@gmail.com> Message-ID: <1204740978.567.17.camel@localhost.localdomain> Hi Johnn, On Wed, 2008-03-05 at 19:49 -0500, Johnny Tan wrote: > I want to have logs written locally (the way they usually > are) as well as sent to a central logger. > > The two situations that I want local logs for are: > > 1) in case the central logger is not available for any > reason, I don't lose messages You don't need to lose message when the remote host is unavailable. Use queued mode. This is currently documented for the database, but works for any other action as well: http://www.rsyslog.com/doc-rsyslog_high_database_rate.html > > 2) if I need to troubleshoot something, I can just look at > the local logs > > Obviously, I will rotate the local logs and have older ones > discarded. Whereas the central logger will archive them for > longer periods. > > > Can I accomplish this with rsyslog? AIUI, I can, and I have > to move to at least v3.11.x -- is that correct? Indeed. But don't use anything earlier than 3.12.0 - if you do, you'll run into a couple of bugs. > Assuming yes, I already got the remote logging working. What > do I do to get the simultaneous local logging to work too? Rsyslog carries out whatever you configure. So just add a rule for local logging, e.g. *.* -/var/log/local-logfile That's it ;) > I can't find any wiki or faq on this. If I get it working, > I'd be happy to post it on the site. That would be much appreciated! HTH Rainer From rgerhards at hq.adiscon.com Thu Mar 6 12:24:16 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 12:24:16 +0100 Subject: [rsyslog] rsyslog on HP UX Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B02@grfint2.intern.adiscon.com> Hi all, thanks to HP's testdrive, I got rsyslog to compile and basically run on HP UX. My testing abilities on the testdrive are limited (non-root access), but I was successfully able to receive a UDP syslog message and store it to a file. So the engine and the plugins seem to run quite OK. If anybody has HP UX and is interested in trying rsyslog out, please let me know. Obviously, you'll receive great support if you try ;) The HP UX "support" will be part of 3.12.1, which I hope to release soon (today or tomorrow). During the compile, there is a set of warnings that I do not fully understand. If somebody can shed some light on them, I'd deeply appreciate that: net.c: In function `AddAllowedSender': net.c:203: warning: cast increases required alignment of target type net.c:212: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:290: warning: cast increases required alignment of target type net.c:294: warning: cast increases required alignment of target type net.c:295: warning: cast increases required alignment of target type net.c:296: warning: cast increases required alignment of target type net.c: In function `MaskCmp': net.c:491: warning: cast increases required alignment of target type net.c:492: warning: cast increases required alignment of target type net.c:502: warning: cast increases required alignment of target type net.c:503: warning: cast increases required alignment of target type net.c:512: warning: cast increases required alignment of target type net.c:513: warning: cast increases required alignment of target type net.c:513: warning: cast increases required alignment of target type net.c:516: warning: cast increases required alignment of target type net.c:517: warning: cast increases required alignment of target type net.c: In function `debugListenInfo': net.c:744: warning: cast increases required alignment of target type net.c:749: warning: cast increases required alignment of target type No suffix list. They seem to be related to casting IPv4/IPv6 structures. Feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Thu Mar 6 16:39:10 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 16:39:10 +0100 Subject: [rsyslog] rsyslog 3.12.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B0E@grfint2.intern.adiscon.com> Hi all, Rsyslog 3.12.1 is available immediately. There has been a log going on under the hood, code has been cleaned up and become more modular (you'll see more of that in the upcoming releases) and a library plugin-interface has been added (which so far is not really visible to the end user). We also worked a bit on portability and rsyslogd now compiles under HP UX. We are looking for testers, so if you have HP UX, you are invited to try it out. Feature-wise, a new syslog mib as been added, runtime debugging support enhanced and the ability to discard timestamps received via unix sockets has now been added (and is the default). This is necessary to keep time information consistent even if local programs log with wrong time information (what they do in practice). The gssapi code has greatly been changed and uses the new dynamic plugin library system. Consequently, some trouble in this regard cannot be ruled out ;). Finally, there is an important bugfix: failed actions were not properly retried, causing message loss in a destination-failure scenario. This is now corrected. Download: http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml Change Log: http://www.rsyslog.com/Article187.phtml As always, feedback is appreciated. Rainer Gerhards From stephen.carville at gmail.com Thu Mar 6 18:54:06 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Thu, 6 Mar 2008 09:54:06 -0800 Subject: [rsyslog] Log watch software Message-ID: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> I have a cenltralized repository usng rsyslogm and syslog to mirror /var/log/messages, /var/log/secure ,and information messages from cfengine. In the near future I hope to get auditd reporting to a central server. My immedate taks is to add some log analysis software on teh central server. I've started modifiying LogWatch to work with MySQL -- thats pretty straightforward -- but I'm curious what other solutions there may be out there. FOSS is preferred but a I'm not against a reasonably priced commercial product. So far everything Google has returned are commercial products for Windows sytems. -- Stephen Carville From rgerhards at hq.adiscon.com Thu Mar 6 18:55:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 18:55:51 +0100 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> I am not so involved with logwatch. Let me ask feature-wise: what capabilities do you need to do the job? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: Thursday, March 06, 2008 6:54 PM > To: rsyslog-users > Subject: [rsyslog] Log watch software > > I have a cenltralized repository usng rsyslogm and syslog to mirror > /var/log/messages, /var/log/secure ,and information messages from > cfengine. In the near future I hope to get auditd reporting to a > central server. My immedate taks is to add some log analysis software > on teh central server. I've started modifiying LogWatch to work with > MySQL -- thats pretty straightforward -- but I'm curious what other > solutions there may be out there. FOSS is preferred but a I'm not > against a reasonably priced commercial product. So far everything > Google has returned are commercial products for Windows sytems. > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From Gerrard.Geldenhuis at datacash.com Thu Mar 6 18:57:35 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Thu, 6 Mar 2008 17:57:35 -0000 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> Message-ID: Hi Stephen, Did you mean to ask a question... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: 06 March 2008 17:54 > To: rsyslog-users > Subject: [rsyslog] Log watch software > > I have a cenltralized repository usng rsyslogm and syslog to mirror > /var/log/messages, /var/log/secure ,and information messages from > cfengine. In the near future I hope to get auditd reporting to a > central server. My immedate taks is to add some log analysis software > on teh central server. I've started modifiying LogWatch to work with > MySQL -- thats pretty straightforward -- but I'm curious what other > solutions there may be out there. FOSS is preferred but a I'm not > against a reasonably priced commercial product. So far everything > Google has returned are commercial products for Windows sytems. > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From stephen.carville at gmail.com Thu Mar 6 20:44:11 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Thu, 6 Mar 2008 11:44:11 -0800 Subject: [rsyslog] Log watch software In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> Message-ID: <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> On Thu, Mar 6, 2008 at 9:55 AM, Rainer Gerhards wrote: > I am not so involved with logwatch. Let me ask feature-wise: what > capabilities do you need to do the job? About 99% of what's in messages or secure is trivia. JoeBob logged in, ran a sudo command and logged off. An authenticated mount request was received from ip.add.re.ss. That sort of thing. What I'm looking for is a parser that can pick out the (hopefully) rare messages that indicates a problem like a disk drive is reporting errors. I can modify big brother and logwatch to do this but I am curious if anyone has a favorite package I haven't heard of yet. > Rainer > > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > > Sent: Thursday, March 06, 2008 6:54 PM > > To: rsyslog-users > > Subject: [rsyslog] Log watch software > > > > I have a cenltralized repository usng rsyslogm and syslog to mirror > > /var/log/messages, /var/log/secure ,and information messages from > > cfengine. In the near future I hope to get auditd reporting to a > > central server. My immedate taks is to add some log analysis software > > on teh central server. I've started modifiying LogWatch to work with > > MySQL -- thats pretty straightforward -- but I'm curious what other > > solutions there may be out there. FOSS is preferred but a I'm not > > against a reasonably priced commercial product. So far everything > > Google has returned are commercial products for Windows sytems. > > > > -- > > Stephen Carville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Stephen Carville From rgerhards at hq.adiscon.com Thu Mar 6 21:27:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 21:27:13 +0100 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> Hi, I am seeing where you come from. That's the million-dollar question ;) I suggest you also post to the loganalysis list, that's probably a better place than over here: http://www.loganalysis.org/mailman/listinfo/loganalysis Let me hijack this thread to share an idea. Rsyslog has a lot of infrastructure in place. Once I am finished with the essentials (which will of course be in a few month...), I'd like to put that infrastructure to better use than just drive the simple outputs we currently have. One thing I have on my mind is an output plugin which stores (hashes) of all message within a timeframe (e.g. last 7 days). Then, when a new message comes in, it compares it to all previous messages and emits a special message itself if the message occured less than "n" times in the past. I think this goes into the direction of what you are looking for. But would it generally be considered to be a useful idea? Even though we are months away from an implementation, feedack would be very valuable to me as it helps me shape my mid- to long-term direction. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Stephen Carville > Sent: Thursday, March 06, 2008 8:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] Log watch software > > On Thu, Mar 6, 2008 at 9:55 AM, Rainer Gerhards > wrote: > > I am not so involved with logwatch. Let me ask feature-wise: what > > capabilities do you need to do the job? > > About 99% of what's in messages or secure is trivia. JoeBob logged > in, ran a sudo command and logged off. An authenticated mount request > was received from ip.add.re.ss. That sort of thing. What I'm looking > for is a parser that can pick out the (hopefully) rare messages that > indicates a problem like a disk drive is reporting errors. > > I can modify big brother and logwatch to do this but I am curious if > anyone has a favorite package I haven't heard of yet. > > > Rainer > > > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > > > Sent: Thursday, March 06, 2008 6:54 PM > > > To: rsyslog-users > > > Subject: [rsyslog] Log watch software > > > > > > I have a cenltralized repository usng rsyslogm and > syslog to mirror > > > /var/log/messages, /var/log/secure ,and information messages from > > > cfengine. In the near future I hope to get auditd reporting to a > > > central server. My immedate taks is to add some log > analysis software > > > on teh central server. I've started modifiying LogWatch > to work with > > > MySQL -- thats pretty straightforward -- but I'm curious > what other > > > solutions there may be out there. FOSS is preferred but a I'm not > > > against a reasonably priced commercial product. So far > everything > > > Google has returned are commercial products for Windows sytems. > > > > > > -- > > > Stephen Carville > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Thu Mar 6 21:47:47 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 06 Mar 2008 15:47:47 -0500 Subject: [rsyslog] local/remote logging In-Reply-To: <1204740978.567.17.camel@localhost.localdomain> References: <47CF3FAA.6050302@gmail.com> <1204740978.567.17.camel@localhost.localdomain> Message-ID: <47D05873.60300@gmail.com> Rainer Gerhards wrote: > Hi Johnn, > > On Wed, 2008-03-05 at 19:49 -0500, Johnny Tan wrote: >> I want to have logs written locally (the way they usually >> are) as well as sent to a central logger. >> >> The two situations that I want local logs for are: >> >> 1) in case the central logger is not available for any >> reason, I don't lose messages > > You don't need to lose message when the remote host is unavailable. Use > queued mode. This is currently documented for the database, but works > for any other action as well: Hmm... and this is doable even with 2.x, right? So I could have it queued in memory buffer long enough to bring it back up or bring up another one with same ip to accept the messages? >> Assuming yes, I already got the remote logging working. What >> do I do to get the simultaneous local logging to work too? > > Rsyslog carries out whatever you configure. So just add a rule for local > logging, e.g. > > *.* -/var/log/local-logfile Awesome! Thanks. I'll post on the wiki once I get it working. johnn From rgerhards at hq.adiscon.com Thu Mar 6 21:53:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 21:53:59 +0100 Subject: [rsyslog] local/remote logging In-Reply-To: <47D05873.60300@gmail.com> References: <47CF3FAA.6050302@gmail.com><1204740978.567.17.camel@localhost.localdomain> <47D05873.60300@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B18@grfint2.intern.adiscon.com> > >> 1) in case the central logger is not available for any > >> reason, I don't lose messages > > > > You don't need to lose message when the remote host is > unavailable. Use > > queued mode. This is currently documented for the database, > but works > > for any other action as well: > > Hmm... and this is doable even with 2.x, right? Sorry, Dude, no such queues in v2 :-( > So I could have it queued in memory buffer long enough to > bring it back up or bring up another one with same ip to > accept the messages? It's *not* a main memory buffer. If you configure it to use disk *and* you have enough disk space, it can queue for ages. It's pretty advanced, the full doc on queues is here: http://www.rsyslog.com/doc-queues.html The disk, of course, is only touched if needed. If the destination server is only offline for a brief period, messages may be buffered in memory only. But if the volume is too large or you stop rsyslogd for a while, it is written disk. So if you have enough disk space and your destination dies on Friday and you are too lazy to come in over the weekend, rsyslogd will send all queued elements after the restart on Monday. The same, of course, works if you are on vacation ;) Rainer From Gerrard.Geldenhuis at datacash.com Fri Mar 7 10:34:17 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Fri, 7 Mar 2008 09:34:17 -0000 Subject: [rsyslog] Log watch software In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com><2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> Message-ID: > Let me hijack this thread to share an idea. Rsyslog has a lot of > infrastructure in place. Once I am finished with the essentials (which > will of course be in a few month...), I'd like to put that > infrastructure to better use than just drive the simple outputs we > currently have. One thing I have on my mind is an output plugin which > stores (hashes) of all message within a timeframe (e.g. last 7 days). > Then, when a new message comes in, it compares it to all previous > messages and emits a special message itself if the message occured less > than "n" times in the past. I think this goes into the direction of what > you are looking for. > > But would it generally be considered to be a useful idea? Even though we > are months away from an implementation, feedack would be very valuable > to me as it helps me shape my mid- to long-term direction. > > Rainer Just thinking out loud... it would be very cool if one could build in some AI in such a plugin. You could then spend time "training" the plugin and buying "trained" AI's to regonize certain patterns in the logs etc. Regards From linuxweb at gmail.com Sat Mar 8 00:22:11 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 07 Mar 2008 18:22:11 -0500 Subject: [rsyslog] wiki with various application logging examples Message-ID: <47D1CE23.2010005@gmail.com> Just wondering if there is a wiki with examples of how to log to syslog/rsyslog for various apps. For my needs, I was specifically looking for apache, mysql, tomcat, puppet, etc. For apache, I already figured out error_log can just point to syslog, but access_log, you do: CustomLog "| /usr/bin/logger -p local0.info" combined Do I do something similar for any application where I can specify the logfile destination? johnn From rgerhards at hq.adiscon.com Sat Mar 8 13:54:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 8 Mar 2008 13:54:19 +0100 Subject: [rsyslog] wiki with various application logging examples Message-ID: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> I think nobody has added descriptions so far, but maybe that changes now ;) On the file logs - I suggest you look into imfule, that sound much cleaner and reliable to me than the logger trick. rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 08.03.08 00:22 Betreff: [rsyslog] wiki with various application logging examples Just wondering if there is a wiki with examples of how to log to syslog/rsyslog for various apps. For my needs, I was specifically looking for apache, mysql, tomcat, puppet, etc. For apache, I already figured out error_log can just point to syslog, but access_log, you do: CustomLog "| /usr/bin/logger -p local0.info" combined Do I do something similar for any application where I can specify the logfile destination? johnn _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 8 17:04:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 8 Mar 2008 17:04:18 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Just to facilitate things: this is what I am thinking about: http://www.rsyslog.com/doc-imfile.html It can monitor all application log files natively. Did I understand your need right? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Rainer Gerhards > Sent: Saturday, March 08, 2008 1:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > I think nobody has added descriptions so far, but maybe that > changes now ;) > > On the file logs - I suggest you look into imfule, that sound > much cleaner and reliable to me than the logger trick. > > rainer > > ----- Urspr?ngliche Nachricht ----- > Von: "Johnny Tan" > An: "rsyslog-users" > Gesendet: 08.03.08 00:22 > Betreff: [rsyslog] wiki with various application logging examples > > Just wondering if there is a wiki with examples of how to > log to syslog/rsyslog for various apps. > > For my needs, I was specifically looking for apache, mysql, > tomcat, puppet, etc. > > For apache, I already figured out error_log can just point > to syslog, but access_log, you do: > CustomLog "| /usr/bin/logger -p local0.info" combined > > Do I do something similar for any application where I can > specify the logfile destination? > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Sun Mar 9 01:14:47 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Sat, 08 Mar 2008 19:14:47 -0500 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: <47D32BF7.5030108@gmail.com> Rainer Gerhards wrote: > Just to facilitate things: this is what I am thinking > about: > > http://www.rsyslog.com/doc-imfile.html > > It can monitor all application log files natively. Did I > understand your need right? Yes, that's exactly what I was looking for. Thanks, really really awesome! One quick error -- in your Sample, you use: $InputFilePollingInterval 10 It should be $InputFilePollInterval (no -ing on Poll) Thanks again! johnn From janfrode at tanso.net Sun Mar 9 12:56:15 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Mar 2008 12:56:15 +0100 Subject: [rsyslog] wiki with various application logging examples References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: On 2008-03-08, Rainer Gerhards wrote: > Just to facilitate things: this is what I am thinking about: > > http://www.rsyslog.com/doc-imfile.html > > It can monitor all application log files natively. Did I understand your need right? > Seems like "imfile" could be trivially implemented trough: while sleep $InputFilePollInterval ; do logtail $InputFileName $InputFileStateFile | \ logger -t $InputFileTag -p $InputFileFacility.$InputFileSeverity done http://debian.stro.at/bzr-test/logtail/ Or for realtime monitoring: tail --follow=name --retry $InputFileName | \ logger -t $InputFileTag -p $InputFileFacility.$InputFileSeverity IMHO rsyslogd might be trying to do too much internally. When will it start reading mail ? That might not sound too far fetched.. :-) "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." -- jwz -jf From mbiebl at gmail.com Sun Mar 9 15:02:04 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sun, 9 Mar 2008 15:02:04 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: 2008/3/9, Jan-Frode Myklebust : > IMHO rsyslogd might be trying to do too much internally. When will it start > reading mail ? That might not sound too far fetched.. :-) > > "Every program attempts to expand until it can read mail. Those programs > which cannot so expand are replaced by ones which can." -- jwz To some extent, I share your concerns. I very much believe in the Unix philosophy of small tools, which do one job, and do that right and can be chained/combined easily. rsyslog should keeps it's focus and stay what it's primary focus is, a syslog daemon. Taking your email example: Having the functionality to send out email notifications on certain events is certainly useful. But we should not build the email sending functionality into rsyslog itself, but simply hand out (pipe) the data to external tools (mail/sendmail). The point is, what we can expect to be available on a given system or not ("logtail" probably isn't, "mail" very likely is). Also, as it's implemented as a loadable module, it doesn't clutter the main rsyslog code and so keeps the code base maintainable. But again ,I think your concern is valid and we should evaluate carefully what to implement directly into rsyslog and where it makes more sense to utilise external tools. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sun Mar 9 18:38:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 9 Mar 2008 18:38:00 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Hi all, and especially JF and Michael, Good discussion, please keep the thoughts flowing :) Let me start by tell you I am guilty of a much broader view of what a syslogd can do than most people probably have. To me, a syslogd processes *system events* and that's much more than just those pieces sent via syslog. I know that I should probably stopped the rsyslog project at some point in time and fork an eventlogd from it. But, why would this have made any sense? To me, the core point is maintainability of code and the freedom for people to use only (and exactly) what they need (and want!) to use. After all, there are also lots of folks how say that database support has nothing to do with a syslogd and thus should not be part of if (and that was the birthplace of rsyslog... ;)). So for me the natural thing was to make rsyslog pluggable AND keep only in the actual core what is ultimately needed. I got very serious about this last summer. There are a couple of posts from last August in my blog and I'd like to highlight this one: http://rgerhards.blogspot.com/2007/08/on-importance-of-plug-ins-for-rsys log.html We have now reached a point where we have a quite well implemented plugin interface and rsyslogd has become much more modular. Still, there are things that people need to run even if they never need them (like the syslog-protocol message parser and a couple of "standard" outputs that come with sysklogd). I am continuously working to remove these dependencies. For example, on Friday I moved regex support out to a separate module. So you no longer need to load regexpes if you do not need them. Now to the plugins. What is a plugin? Good question. A plugin is *a project in itself*. Just for convenience, it is part of the main tarball (we had a couple of real headaches when we tried to separate the initial plugins and came to the conclusion that it is by far more convenient to deliver them as part of the main tarball). Plugins need not be written by the rsyslog team. Maybe somebody has already written an email reception plugin and just not told us - may very well be (though I doubt ;)). Now let me take on the imfile example. The key point - at least IMHO - is that there is no single line in rsyslog core's code that has been added in support of imfile. And, more importantly, if imfile would go away, not a single line of code could be removed. So the imfile plugin (project) does neither add complexity nor code nor other overhead to rsyslog core. So can it be evil? Imfile was added in response of a user request. When I heared it, I thought it would be useful to have such a plugin. Did I know about the logger approach? Yes - and I have read some many folks complain over this or that aspect of a logger-approach that I knew I didn't want it. Maybe these were all false claims, maybe people simply did not understand to do it right. I don't know and I honestly admit that I don't care. The presence of many people feeling bugged about that approach made me not like it (if you look for actual samples, I suggest going through the loganalysis mailing list archive). Did I know about logtail? No. Had I written imfile if I did? Not sure: from the small shell script you provided, it looks like there is a problem if a) script is in sleep period b) data is appended to text file c) text file is rotated d) new lines are written to text file e) script awake for new polling loop On a quick look, it looks like the data written in b) will never make it to the syslogd. Imfile handles that. But even if logtail (or the script) handles it - as I said, I did not know about it. So I took roughly a day of my time to create an initial imfile. If logtail handles the situation described above, would be available on the platforms I was asked for and I knew about it, I'd probably refrained from implementing imfile. But, again, no harm happened. If you don't like imfile, simply don't use it. It adds nothing to the rsyslog core. You can still use the logtail approach. IMHO this is what we want: freedom of choice. And rsyslog provides this freedom. On the mail output case (though I need to be a bit brief as dinner is approaching ;)): I actually intend to add an email output plugin. It's on the list for long a time. I personally see some benefit in the ability to talk to a tightly coupled, in-memory, very performant accessible way to send mail message (just like with the SNMP traps). On the other hand, you can see that I didn't implement it for a long time - simply because I did not consider it important enough given what else needs to be done. But I now need to speak up, because I have actually begun to seriously look into implementing it before this discussion begun ;) Before I do, I will make sure I understand that other options and if there is some benefit in doing so. But if I implement it for one reason or the other, nothing bad happens to rsyslog: if you don't like it, you don't need to use it. It doesn't add any overhead to the core. And now please feedback. Am I overlooking something real evil? What is the bad thing in offering exotic functionality that only few will ever need? Thanks again for the discussion, I am *really* interested in getting more feedback on this topic. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Sunday, March 09, 2008 3:02 PM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > 2008/3/9, Jan-Frode Myklebust : > > IMHO rsyslogd might be trying to do too much internally. When will > it start > > reading mail ? That might not sound too far fetched.. :-) > > > > "Every program attempts to expand until it can read mail. > Those programs > > which cannot so expand are replaced by ones which can." -- > jwz > > To some extent, I share your concerns. > I very much believe in the Unix philosophy of small tools, which do > one job, and do that right and can be chained/combined easily. > > rsyslog should keeps it's focus and stay what it's primary focus is, a > syslog daemon. > Taking your email example: > Having the functionality to send out email notifications on certain > events is certainly useful. But we should not build the email sending > functionality into rsyslog itself, but simply hand out (pipe) the data > to external tools (mail/sendmail). > > The point is, what we can expect to be available on a given system or > not ("logtail" probably isn't, "mail" very likely is). Also, as it's > implemented as a loadable module, it doesn't clutter the main rsyslog > code and so keeps the code base maintainable. > > But again ,I think your concern is valid and we should evaluate > carefully what to implement directly into rsyslog and where it makes > more sense to utilise external tools. > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Sun Mar 9 23:05:10 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Mar 2008 23:05:10 +0100 Subject: [rsyslog] wiki with various application logging examples References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Message-ID: On 2008-03-09, Rainer Gerhards wrote: > > Now let me take on the imfile example. The key point - at least IMHO - > is that there is no single line in rsyslog core's code that has been > added in support of imfile. And, more importantly, if imfile would go > away, not a single line of code could be removed. So the imfile plugin > (project) does neither add complexity nor code nor other overhead to > rsyslog core. So can it be evil? It's evil that you're forcing me to upgrade to rsyslog v3.x to take advantage of it ;-) That's what triggered my previous post.. I want to be tracking non-syslog logfiles, and if imfile/plugins where more in the unix philosophy of small tools that chain easily.. I might have been able to pick it from v3.x and use it on stable systems. > from the small shell script you provided, it looks like there is a > problem if > > a) script is in sleep period > b) data is appended to text file > c) text file is rotated > d) new lines are written to text file > e) script awake for new polling loop > > On a quick look, it looks like the data written in b) will never make it > to the syslogd. Imfile handles that. Yes, you're right. Cool that imfile handles it. > On the mail output case (though I need to be a bit brief as dinner is > approaching ;)): I actually intend to add an email output plugin. The quote was about being able to *read* mail: "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." and I was thinking it not too far fetched since "splunk" can do it (it can download email messages via IMAP, index them and create alerts on suspicious content). It might seem like you want to take rsyslog in that direction, i.e. your complete eventlogd&alertSystem fork of rsyslogd that can read any input (syslog, other-logfile, email, snmptraps) and analyze and alert on the data. That's not what *I* want from a syslog server. I just want it to reliably collect and store the logs in an organized manner. Then I'll use other tools to read and analyze them. -jf From rgerhards at hq.adiscon.com Mon Mar 10 08:52:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Mar 2008 08:52:46 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B3B@grfint2.intern.adiscon.com> Hi JF, thanks for the note. Of course, it triggers a couple of responses (see below). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Sunday, March 09, 2008 11:05 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] wiki with various application logging examples > > On 2008-03-09, Rainer Gerhards wrote: > > > > Now let me take on the imfile example. The key point - at least IMHO > - > > is that there is no single line in rsyslog core's code that has been > > added in support of imfile. And, more importantly, if imfile would go > > away, not a single line of code could be removed. So the imfile > plugin > > (project) does neither add complexity nor code nor other overhead to > > rsyslog core. So can it be evil? > > It's evil that you're forcing me to upgrade to rsyslog v3.x to take > advantage of it ;-) That's what triggered my previous post.. I want > to be tracking non-syslog logfiles, and if imfile/plugins where more > in the unix philosophy of small tools that chain easily.. I might have > been able to pick it from v3.x and use it on stable systems. I've see the smily but let me do a couple of comments. First of all, the "philosophy of small tools that chain easily" implies that they chain via a pipe. This is great for a lot of applications, but it has its drawbacks. A plain pipe is a simplex, relatively loosely coupled IPC method. So if one part of the pipe dies, other parts will learn about it eventually, but not at the same instant and, most importantly, they do not exactly know what was processed and what not. It's pretty much the same thing as plain TCP transport, which pretends reliability but still has a few windows of exposure where message loss may occur (see http://www.monitorware.com/en/workinprogress/selp.txt section 2.4 and, yes, rsyslog is victim to this as long as we don't have a full RFC 3195 implementation). So relying on plain pipes is not exactly my premier communications method if I would like to implement a reliable syslogd (and "reliable" is the "r" in rsyslog). One can circumvent the problem by defining an app-level protocol on top of the pipe, working with app-level acks. HOWEVER, with that you would run into the pretty same situation, namely that you could not use a new version together with an old engine. One can circumvent that, too, but only as far as the old engine has implemented such methodology. And v1/2 do not have this (due to time constraints, rsyslog is still only about half finished...). There is some other reason that makes me avoid pipes. If I do an app-level ack, I need to do at least 4 system calls to pass a single message: 1. input write to pipe 2. engine read msg from pipe 3. engine post processing stat (ACK) to pipe 4. input read ACK from pipe If you do a little bit of math, you'll see with how many user/kernel space transitions you end up, plus how many cycles are needed to run the necessary housekeeping code inside the kernel and libraries. I don't sum them up now, but I am pretty sure that I can completely process the message in less time than is need just for the IPC in that case. That probably is fine if you look at a low-end workstation syslogd, but you don't want to have this overhead if you aim for an enterprise solutions capable of handling massive data input. Also, if everything goes to the system log socket, there is little you can multithread. But we need to be able to multithread as much as possible. If I write a new engine (as I currently do), I would like to see it well working for at least the next 10 years. If I think about hardware trends during that period, it is clear that a single core will become not much faster than it is today. But the number of cores will greatly increase. In order to utilize that, an application must be able to run on as many threads as possible - all with reasonable overhead, of course. So my conclusion is that rsyslog must be able to run massively multithreaded for the high-end use case. This also prohibits using primary interfaces which cannot easily be multithreaded. Finally there is the issue of flow control. Rsyslog *does* flow control, and will do more advanced flow control in the future. Especially with world-dominating UDP syslog it is vitally important to do flow control, because UDP cannot be flow controlled. Sound strange? Well... If we can not flow control UDP syslog, we need to apply intelligent and adoptable flow control that flow controls sources who can (tcp syslog, rfc 3195 and of course file data!) so that buffer space is kept for those precious UDP message which are lost if we can not buffer them at the right instant. If you think this through, you'll see that this requires different level and methods of flow control, depending on the source [so far, I see three levels: can not (UDP), can somewhat (TCP, local sockets), can easily (log files and other sources that generate data themselves)]. Having a native interface greatly reduces code complexity and thus program reliability when it comes at implementing these features. Also, it would require specialized plugins in any case, you couldn't do it with a simple "pipe me in approach" (well, ... some things yes, but at a complexity cost). Also, while I too believe in the Unix approach of small tools, I also think it is important that the average user is able to configure it. Rsyslog aims not only at being enterprise-class but at the some time aims at being easy to use for the novice. Novices don't understand complex scripting to get the job done. I think relying too much on complex glue doesn't help getting the job done. To come back to your orginal post, rsyslogd *should* of course support the Unix way of piping. It looks like there is a small plugin missing to read natively from a pipe. However, I never got a request to implement it. I guess most people use the logger trick to accomplish that task. I know syslog-ng can natively read from pipes, but have not yet considered this important enough given the lack of requests from the community. If you like such a plugin, it's probably a good idea to speak up now ;) > > > > from the small shell script you provided, it looks like there is a > > problem if > > > > a) script is in sleep period > > b) data is appended to text file > > c) text file is rotated > > d) new lines are written to text file > > e) script awake for new polling loop > > > > On a quick look, it looks like the data written in b) will never make > it > > to the syslogd. Imfile handles that. > > Yes, you're right. Cool that imfile handles it. > > > > On the mail output case (though I need to be a bit brief as dinner is > > approaching ;)): I actually intend to add an email output plugin. > > The quote was about being able to *read* mail: I was replying here to Michael Biebel. It looks I was a bit too much in hurry to point this out. Sorry... > > "Every program attempts to expand until it can read mail. > Those programs which cannot so expand are replaced by ones > which can." > > and I was thinking it not too far fetched since "splunk" can do it (it > can > download email messages via IMAP, index them and create alerts on > suspicious > content). In short: receiving email is very low on my agenda. Keep in mind that I already have architecture and at least partly written such a beast on Windows: http://www.monitorware.com/en/Product/product_comparision.php The email question never was in much demand. But if demand comes up, it for sure is not a big thing to add it... (and, of course, you already can do it today with a bit of scripting, the right mailbox rules and logger -- but that isn't appealing to most folks and is one reason I tend to write plugins ;)). > It might seem like you want to take rsyslog in that > direction, i.e. > your complete eventlogd&alertSystem fork of rsyslogd that can read any > input > (syslog, other-logfile, email, snmptraps) and analyze and alert on the > data. > > That's not what *I* want from a syslog server. I just want it to > reliably > collect and store the logs in an organized manner. Then I'll use other > tools to read and analyze them. ... and this is of course perfectly fine with me, too. However, if you look at the core engine needs, you'll see that the "do it all" eventprocessor and the "plain simple syslogd" have exactly the same needs - at least if you would like to extend the syslogd to be enterprise class. Take the queue engine in v3. Its complex. Actually horribly complex. I didn't like to include that complex beast, but it was the only clean solution to the need of being massively concurrent AND being able to queue data while a destination is down. Any other alternative IMHO would have been dirty and hackish - and in the long term much less maintainable. So I just did the right thing (hopefully), even though it was a really big effort and even though it probably will need a few more firedrills before it is really ready for prime time in all scenarios. Another alternative would have been to use one of the big enterprise class message queueing projects. But that would have created a dependency for such a system on each desktop - ouch... I hope I made the right compromise. Currently, the full queue engine is part of the core. If that turns out to be a problem, I can outsource that to a plugin, but that isn't currently very appealing to me. After all, it's "just" some memory overhead - if you don't need the advanced features, no code is executed to do that. The queue uses an internal driver model and simply configuration means simple code. Thus, the v3 queue engine is as reliable as v2, except if you use all those bells and whistles where I am sure currently a few bugs wait to be detected (even though the situation has much improved recently and will improve with each new feedback I receive). Another good example is the config file: of course, there is no need to have a scriptable configuration for a simple syslogd. But while thinking about the (necessary) expression support and a lot of user requests for a better to read config file format, I came to the conclusion that creating a scriptable format is actually the right route to take: http://rgerhards.blogspot.com/2008/02/introducing-rainerscript-and-some. html Anything else (IMHO) would again be less clean, less maintainable and, in this case, would even take longer to implement. So one might think it is evil to include a virtual machine inside a syslogd, but to me it is actually the least effort to implement things. Of course, you can rightly argue that all of this is over the board if you just want to have a plain local logger that takes messages from the local log socket, maybe UDP syslog and store it to local files. You are probably right. But in this case, you can still continue to use sysklogd. After all, rsyslog was spawned from it to create and advanced syslogd. So it comes at no surprise that I am adding features which may not be required for the simple use cases. ;) Let me conclude with two core points: - rsyslog core is as slim as possible, plugins are separate projects that extend the core; This means nobody is forced to run more code than actually required for his job - both a simple, but fairly enhanced syslogd as well as a full network Event processor share the same root engine needs Thus, rsyslog implements this core engine and I occasionally add a plugin here and there to take advantage of the core. Right now, rsyslog core is far from being finished, as are the plugins. At this time, I am working on getting the core right and doing the most requested plugins. When I am done with that, I'll look at the *real* advanced plugins for all kinds of things that users have interest in. I don't see any need to fork of a separate core project for that. In fact, I think it would be counter-productive as I would need to maintain two code bases and the newly forked project would always be able to do what rsyslogd does. So what would be the motivation to maintain another, feature-less project...? Anyhow, I may be totally wrong. Feedback on this topic is still highly appreciated (be it brief or elaborate ;)). I am probably abusing this thread to also tell you a bit about design decisions I have not yet communicated (sorry for that, so much to do, so few time...;)). Rainer > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 10 08:54:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Mar 2008 08:54:42 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <47D32BF7.5030108@gmail.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <47D32BF7.5030108@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> Hi Johnn, I'd really appreciate if you could any actual samples (even brief) to the wiki at http://wiki.rsyslog.com They would definitely be of great benefit for other users, too. As you have probably seen in my other mails, I am pretty busy with developing right now so I hope that users contribute use cases. Thanks, Raiener > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Sunday, March 09, 2008 1:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > Rainer Gerhards wrote: > > Just to facilitate things: this is what I am thinking > > about: > > > > http://www.rsyslog.com/doc-imfile.html > > > > It can monitor all application log files natively. Did I > > understand your need right? > > Yes, that's exactly what I was looking for. Thanks, really > really awesome! > > One quick error -- in your Sample, you use: > $InputFilePollingInterval 10 > > It should be $InputFilePollInterval (no -ing on Poll) > > Thanks again! > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Mon Mar 10 17:01:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 10 Mar 2008 12:01:22 -0400 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <47D32BF7.5030108@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> Message-ID: <47D55B52.7010905@gmail.com> Rainer Gerhards wrote: > Hi Johnn, > > I'd really appreciate if you could any actual samples (even brief) to > the wiki at > > http://wiki.rsyslog.com > > They would definitely be of great benefit for other users, too. As you > have probably seen in my other mails, I am pretty busy with developing > right now so I hope that users contribute use cases. Yes, absolutely! I'm a rsyslog convert :). Since we're going live this week, I've got a ton of things on my plate. But I fully intend to put up my samples (however basic they are) on the wiki soon after launch! Thanks for a great program! johnn From maurizio.rottin at gmail.com Tue Mar 11 13:01:57 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Tue, 11 Mar 2008 13:01:57 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log Message-ID: Hi all, i set up an apache webserver wich logs with directives: ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined" then syslog forwards them with local5.err @10.10.10.89 local6.info @10.10.10.89 and it works good! now on 10.10.10.89 i set up an rsyslog server with directives $template ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" local5.err -?ApacheRemoteErr $template ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Custom.log" local6.info -?ApacheRemoteCustom the result is that i get these files in /var/log/rsyslog/Apache/ ./www.mysite.com:/Custom.log with customLog ./error_www.mysite.com:/Err.log with ErroLog ./message/Err.log with lines like Mar 11 12:32:52 last message repeated 9 times Mar 11 12:32:52 last message repeated 5 times (Note that this is very annoying!!!) Now this a simple way i have found to divide logs per VHost. I don't know if there is a better way, but the real question is different: In the Custom or Error logs i have lines like: Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part do i have to use a double template setup? one for the file and one for the content? the template in the file seems to be this: $template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" but i need just $template TheOneINeed,%msg%\n" i want only the %msg%\n in my file Thanks in advance -- mr From linuxweb at gmail.com Tue Mar 11 19:08:06 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 14:08:06 -0400 Subject: [rsyslog] rsyslog v3 and selinux Message-ID: <47D6CA86.6060501@gmail.com> I took Fedora 8's rsyslog v2.0.2 SRPM and rebuilt it for v3.12.1 I'm using the same init script. When I used that init script (i.e., "/etc/init.d/rsyslog start") on v2.0.2, the only SELinux problem I had was the domain transition, as documented here: http://tanso.net/rsyslog/ However, when I rebuilt the RPM for v3.12.1, and used the same init script to run it, I get many SELinux errors. Eventually, I worked out all the things I had to allow for rsyslog v3.12.1 to run properly. It's enclosed below as a semanage module. Just curious if there was some change to rsyslog between versions 2 and 3 which would make rsyslog, even when running properly in domain syslogd_exec_t, to cause so many SELinux denials, including not being able to do TCP bind?? Thanks for any clues, johnn p.s. Here's the module that finally worked. Will document on wiki when all is done. ========== module rsyslog 1.0; require { class dir search; class file { getattr read write }; class filesystem remount; class tcp_socket { create accept read setopt bind name_bind node_bind listen }; type boot_t; type auditd_log_t; type var_log_t; type syslogd_t; type syslogd_port_t; type port_t; type mount_t; type system_map_t; type inaddr_any_node_t; type unspec_node_t; role system_r; }; allow syslogd_t boot_t:dir search; allow syslogd_t auditd_log_t:dir search; allow syslogd_t auditd_log_t:file { getattr read }; allow syslogd_t self:tcp_socket { create accept read setopt bind listen }; allow syslogd_t syslogd_port_t:tcp_socket name_bind; allow syslogd_t port_t:tcp_socket name_bind; allow syslogd_t system_map_t:file { read getattr }; allow syslogd_t inaddr_any_node_t:tcp_socket node_bind; allow syslogd_t unspec_node_t:tcp_socket node_bind; ========== From rgerhards at hq.adiscon.com Tue Mar 11 20:05:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Mar 2008 20:05:13 +0100 Subject: [rsyslog] rsyslog v3 and selinux Message-ID: <005c01c883aa$d9cecd2e$060013ac@intern.adiscon.com> I have absolutely no idea about selinux ... But: may it be that the fact that now plugins - separate libraries - are loaded by rsyslogd to do the job? rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 11.03.08 19:08 Betreff: [rsyslog] rsyslog v3 and selinux I took Fedora 8's rsyslog v2.0.2 SRPM and rebuilt it for v3.12.1 I'm using the same init script. When I used that init script (i.e., "/etc/init.d/rsyslog start") on v2.0.2, the only SELinux problem I had was the domain transition, as documented here: http://tanso.net/rsyslog/ However, when I rebuilt the RPM for v3.12.1, and used the same init script to run it, I get many SELinux errors. Eventually, I worked out all the things I had to allow for rsyslog v3.12.1 to run properly. It's enclosed below as a semanage module. Just curious if there was some change to rsyslog between versions 2 and 3 which would make rsyslog, even when running properly in domain syslogd_exec_t, to cause so many SELinux denials, including not being able to do TCP bind?? Thanks for any clues, johnn p.s. Here's the module that finally worked. Will document on wiki when all is done. ========== module rsyslog 1.0; require { class dir search; class file { getattr read write }; class filesystem remount; class tcp_socket { create accept read setopt bind name_bind node_bind listen }; type boot_t; type auditd_log_t; type var_log_t; type syslogd_t; type syslogd_port_t; type port_t; type mount_t; type system_map_t; type inaddr_any_node_t; type unspec_node_t; role system_r; }; allow syslogd_t boot_t:dir search; allow syslogd_t auditd_log_t:dir search; allow syslogd_t auditd_log_t:file { getattr read }; allow syslogd_t self:tcp_socket { create accept read setopt bind listen }; allow syslogd_t syslogd_port_t:tcp_socket name_bind; allow syslogd_t port_t:tcp_socket name_bind; allow syslogd_t system_map_t:file { read getattr }; allow syslogd_t inaddr_any_node_t:tcp_socket node_bind; allow syslogd_t unspec_node_t:tcp_socket node_bind; ========== _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 11 21:45:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Mar 2008 21:45:00 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Maurizio Rottin > Sent: Tuesday, March 11, 2008 1:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog with apache and per vhost log > > Hi all, > i set up an apache webserver wich logs with directives: > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com > " "combined" > > then syslog forwards them with > local5.err @10.10.10.89 > local6.info @10.10.10.89 > and it works good! > > now on 10.10.10.89 i set up an rsyslog server with directives > $template > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" > local5.err -?ApacheRemoteErr > $template > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C > ustom.log" > local6.info -?ApacheRemoteCustom > > the result is that i get these files in /var/log/rsyslog/Apache/ > ./www.mysite.com:/Custom.log > with customLog > > ./error_www.mysite.com:/Err.log > with ErroLog > > ./message/Err.log > with lines like > Mar 11 12:32:52 last message repeated 9 times > Mar 11 12:32:52 last message repeated 5 times > (Note that this is very annoying!!!) Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd compatibility, -e stand for "log *e*very message") > > > Now this a simple way i have found to divide logs per VHost. > I don't know if there is a better way, but the real question > is different: > In the Custom or Error logs i have lines like: > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.7.13) Gecko/20060607" > > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part > do i have to use a double template setup? one for the file and one for > the content? > This is caused by the default template, which is what most peiople expect in their syslog files. But, as you write, the soluton is quite easy, just use another template ;) > the template in the file seems to be this: > $template TraditionalFormat,%timegenerated% %HOSTNAME% > %syslogtag%%msg%\n" > but i need just > $template TheOneINeed,%msg%\n" > i want only the %msg%\n in my file > So, use local6.info -?ApacheRemoteCustom;TheOneINeed HTH, feel free to ask any follow-up questions. :) Rainer > > Thanks in advance > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 00:42:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 19:42:22 -0400 Subject: [rsyslog] segmentation fault Message-ID: <47D718DE.90603@gmail.com> Ahh, so close yet so far. I've setup a central rsyslog server and am setting up my first client to point to it. I am going over stunnel. I set it up pretty much like it is in the secure rsyslog article. There are some existing logs on the client box. I load imfile.so module to convert those logs to syslog and send them over TCP/stunnel to my central logger. Everytime I try to start it, it dies within seconds. When I turn on debug mode, I notice the client gets a segmentation fault. I have pasted the tail end of the debug output here: http://pastebin.com/m9a83946 Is it just the sheer number of messages that is causing it to segfault, or is something else going on? johnn From linuxweb at gmail.com Wed Mar 12 03:09:00 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 22:09:00 -0400 Subject: [rsyslog] segmentation fault In-Reply-To: <47D718DE.90603@gmail.com> References: <47D718DE.90603@gmail.com> Message-ID: <47D73B3C.1000509@gmail.com> After MUCH trial & error, I've found the problem. If there is a blank line in one of the log files being read by imfile, then rsyslog dies. I filed a bug: http://bugzilla.adiscon.com/show_bug.cgi?id=49 johnn From rgerhards at hq.adiscon.com Wed Mar 12 08:00:40 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 08:00:40 +0100 Subject: [rsyslog] segmentation fault In-Reply-To: <47D73B3C.1000509@gmail.com> References: <47D718DE.90603@gmail.com> <47D73B3C.1000509@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B64@grfint2.intern.adiscon.com> Thanks for finding out. Looks like we got a bug-fixing day today... Will update the bug tracker as things progress. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 3:09 AM > To: rsyslog-users > Subject: Re: [rsyslog] segmentation fault > > After MUCH trial & error, I've found the problem. If there > is a blank line in one of the log files being read by > imfile, then rsyslog dies. > > I filed a bug: > http://bugzilla.adiscon.com/show_bug.cgi?id=49 > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Wed Mar 12 10:47:31 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 10:47:31 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> Message-ID: 2008/3/11, Rainer Gerhards : > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > > Maurizio Rottin > > Sent: Tuesday, March 11, 2008 1:02 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] rsyslog with apache and per vhost log > > > > Hi all, > > i set up an apache webserver wich logs with directives: > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com > > " "combined" > > > > then syslog forwards them with > > local5.err @10.10.10.89 > > local6.info @10.10.10.89 > > and it works good! > > > > now on 10.10.10.89 i set up an rsyslog server with directives > > $template > > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" > > local5.err -?ApacheRemoteErr > > $template > > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C > > ustom.log" > > local6.info -?ApacheRemoteCustom > > > > the result is that i get these files in /var/log/rsyslog/Apache/ > > ./www.mysite.com:/Custom.log > > with customLog > > > > ./error_www.mysite.com:/Err.log > > with ErroLog > > > > ./message/Err.log > > with lines like > > Mar 11 12:32:52 last message repeated 9 times > > Mar 11 12:32:52 last message repeated 5 times > > (Note that this is very annoying!!!) > > > Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd > compatibility, -e stand for "log *e*very message") And i red about the -e options a lot of times...thank! > > > > > > > Now this a simple way i have found to divide logs per VHost. > > I don't know if there is a better way, but the real question > > is different: > > In the Custom or Error logs i have lines like: > > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - > > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 > > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; > > en-US; rv:1.7.13) Gecko/20060607" > > > > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part > > do i have to use a double template setup? one for the file and one for > > the content? > > > > This is caused by the default template, which is what most peiople > expect in their syslog files. But, as you write, the soluton is quite > easy, just use another template ;) > > > > the template in the file seems to be this: > > $template TraditionalFormat,%timegenerated% %HOSTNAME% > > %syslogtag%%msg%\n" > > but i need just > > $template TheOneINeed,%msg%\n" > > i want only the %msg%\n in my file > > > > > So, use > > local6.info -?ApacheRemoteCustom;TheOneINeed works great man! Thanks! > HTH, feel free to ask any follow-up questions. :) > just one question, do you know any way to pass the website name in "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an other program...i know only logger... -- mr From rgerhards at hq.adiscon.com Wed Mar 12 10:50:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 10:50:06 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> [happy snip] > just one question, do you know any way to pass the website name in > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > other program...i know only logger... Well, you could use the imfile plugin to read the log files directly: http://www.rsyslog.com/doc-imfile.html Then, I'd set the site name as tag and so you can process it from the tag value. HTH Rainer From maurizio.rottin at gmail.com Wed Mar 12 11:24:08 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 11:24:08 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > [happy snip] > > > just one question, do you know any way to pass the website name in > > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > > other program...i know only logger... > > > Well, you could use the imfile plugin to read the log files directly: > > http://www.rsyslog.com/doc-imfile.html > > Then, I'd set the site name as tag and so you can process it from the > tag value. > > HTH > > Rainer > mmmmm...i'll check that better but at a glance it seems a heavy process with big logs. -- mr From rgerhards at hq.adiscon.com Wed Mar 12 11:25:53 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 11:25:53 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> > mmmmm...i'll check that better but at a glance it seems a heavy > process with big logs. Sorry, I don't really understand what you mean ;) But the problem is that I probably do not really know how what you do with logger really works. It is part of the apache config, is this right? Maybe you could post a full config sample? Rainer From janfrode at tanso.net Wed Mar 12 12:39:04 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 12 Mar 2008 12:39:04 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> Message-ID: On 2008-03-11, Johnny Tan wrote: > > However, when I rebuilt the RPM for v3.12.1, and used the > same init script to run it, I get many SELinux errors. > Eventually, I worked out all the things I had to allow for > rsyslog v3.12.1 to run properly. It's enclosed below as a > semanage module. These seems very strange: > allow syslogd_t boot_t:dir search; > allow syslogd_t system_map_t:file { read getattr }; Why does it want to read /boot/System.map* ? > allow syslogd_t auditd_log_t:dir search; > allow syslogd_t auditd_log_t:file { getattr read }; And why read /var/log/audit/* ? -jf From rgerhards at hq.adiscon.com Wed Mar 12 13:48:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 13:48:42 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6D@grfint2.intern.adiscon.com> That's part of the klog(d) code I inherited from sysklogd. I've never questioned if there is another way to do it. It seems to use these to obtain kernel symbols. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Wednesday, March 12, 2008 12:39 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog v3 and selinux > > On 2008-03-11, Johnny Tan wrote: > > > > However, when I rebuilt the RPM for v3.12.1, and used the > > same init script to run it, I get many SELinux errors. > > Eventually, I worked out all the things I had to allow for > > rsyslog v3.12.1 to run properly. It's enclosed below as a > > semanage module. > > These seems very strange: > > > allow syslogd_t boot_t:dir search; > > allow syslogd_t system_map_t:file { read getattr }; > > Why does it want to read /boot/System.map* ? > > > allow syslogd_t auditd_log_t:dir search; > > allow syslogd_t auditd_log_t:file { getattr read }; > > And why read /var/log/audit/* ? > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Wed Mar 12 14:37:25 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 14:37:25 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > > mmmmm...i'll check that better but at a glance it seems a heavy > > process with big logs. > > > Sorry, I don't really understand what you mean ;) But the problem is > that I probably do not really know how what you do with logger really > works. It is part of the apache config, is this right? Maybe you could > post a full config sample? > but i did... this is the log part in the directive ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined" in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with facility "-p local*.xxx" with tag "-t www.mysite.com" then i can use the tag to split logs by website in the remote rsyslog machine ;) -- mr From rgerhards at hq.adiscon.com Wed Mar 12 14:44:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 14:44:31 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Wednesday, March 12, 2008 2:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/12, Rainer Gerhards : > > > mmmmm...i'll check that better but at a glance it seems a heavy > > > process with big logs. > > > > > > Sorry, I don't really understand what you mean ;) But the problem is > > that I probably do not really know how what you do with logger > really > > works. It is part of the apache config, is this right? Maybe you > could > > post a full config sample? > > > but i did... > this is the log part in the directive > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > "combined" > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > facility "-p local*.xxx" with tag "-t www.mysite.com" > > then i can use the tag to split logs by website in the remote rsyslog > machine ;) Ah, OK, I didn't realize this was actual Apache config. So your original question > just one question, do you know any way to pass the website name in > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > other program...i know only logger... was how you could use the same apache config command for each log, so that you don't need to have separate apache config statements for each virtual server? Please bear with me if I am getting it wrong, I probably have too much rsyslog on my mind. I am trying to help but I don't get a grip at what the real problem is... Rainer From linuxweb at gmail.com Wed Mar 12 15:08:17 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 10:08:17 -0400 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> Message-ID: <47D7E3D1.9010309@gmail.com> Jan-Frode Myklebust wrote: >> allow syslogd_t auditd_log_t:dir search; >> allow syslogd_t auditd_log_t:file { getattr read }; > > And why read /var/log/audit/* ? This is custom. I prefer to install auditd and have the SELinux logs separated out from /var/log/messages. Then I use imfile to convert the audit logs to syslog in order to send it over to the central logger. Yes, it's a waste of a syslog "local" facility, but I'm not aware of any other way unless I remove auditd and have those go back into /var/log/messages. johnn From linuxweb at gmail.com Wed Mar 12 18:16:56 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 13:16:56 -0400 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <47D81008.3090302@gmail.com> I wanted to setup my rsyslog clients such that when the rsyslog server is down and then comes back up, the clients will resume forwarding messages to it. I'm running into this same bug as another user: http://bugzilla.adiscon.com/show_bug.cgi?id=45 According to that bug, the fix is in 3.12.1, which is what I'm using on client and server. But I still can't seem to get it to work. Prior to doing this, I simply had this line: *.* @@127.0.0.1:61514 Am I supposed to replace that line entirely with the ActionQueue stanza, such as this one (the == is for clarity, not actually part of my config): == $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 30 $ActionResumeRetryCount -1 # infinite retries on insert failure & @@127.0.0.1:61514 == If I do, rsyslog dies immediately on restart. So I changed it so look like this instead: == *.* @@127.0.0.1:61514 $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 & @@127.0.0.1:61514 == This allows rsyslog to work again, but then it doesn't actually do any resume. Just as the OP states in the bug, if the rsyslog server is stopped, the client simply logs messages locally. When the rsyslog server is started again, the client once again logs locally as well as to the server. But all the messages from the client between stop and start on the server are not forwarded. I checked action.c and it does have the code-fix listed in the forum post tied to the bug. I also understand that one message WILL be lost, which is a TCP syslog protocol problem. So I sent multiple messages, none of them came through. johnn From rgerhards at hq.adiscon.com Wed Mar 12 18:22:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:22:12 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D81008.3090302@gmail.com> References: <47D81008.3090302@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B81@grfint2.intern.adiscon.com> I need to check for the restart problem (currently working on something else). Please also follow this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=48 This is the "one message lost" problem, and I have created at least some cure for it today (to be verified). Will check your bug report when I verify it. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 6:17 PM > To: rsyslog-users > Subject: [rsyslog] actionqueue in front of tcp forward > > I wanted to setup my rsyslog clients such that when the > rsyslog server is down and then comes back up, the clients > will resume forwarding messages to it. > > I'm running into this same bug as another user: > http://bugzilla.adiscon.com/show_bug.cgi?id=45 > > According to that bug, the fix is in 3.12.1, which is what > I'm using on client and server. > > > But I still can't seem to get it to work. Prior to doing > this, I simply had this line: > > *.* @@127.0.0.1:61514 > > > > Am I supposed to replace that line entirely with the > ActionQueue stanza, such as this one (the == is for clarity, > not actually part of my config): > > == > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 30 > $ActionResumeRetryCount -1 # infinite retries on insert failure > & @@127.0.0.1:61514 > == > > If I do, rsyslog dies immediately on restart. So I changed > it so look like this instead: > > == > *.* @@127.0.0.1:61514 > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 10 > $ActionResumeRetryCount -1 > & @@127.0.0.1:61514 > == > > This allows rsyslog to work again, but then it doesn't > actually do any resume. Just as the OP states in the bug, if > the rsyslog server is stopped, the client simply logs > messages locally. When the rsyslog server is started again, > the client once again logs locally as well as to the server. > But all the messages from the client between stop and start > on the server are not forwarded. > > I checked action.c and it does have the code-fix listed in > the forum post tied to the bug. > > I also understand that one message WILL be lost, which is a > TCP syslog protocol problem. So I sent multiple messages, > none of them came through. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 18:32:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:32:35 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D81008.3090302@gmail.com> References: <47D81008.3090302@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Argh, some config parsing seems to be wrong. There is a problem with the config file. The right config format is this: == $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 == & in column one is to chain multiple actions together, but it can never be at the start of the chain. Did you take it from some rsyslog documentation? If so and you still know which, please let me know, I'd like to fix that. If I think correctly, the queued action is now chained to the regular action, and probably never executed because the regular action fails ... or something along these lines. Please try the config above, I think it will work (at least it does in my lab). And of course I'll look into the config parser and co ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 6:17 PM > To: rsyslog-users > Subject: [rsyslog] actionqueue in front of tcp forward > > I wanted to setup my rsyslog clients such that when the > rsyslog server is down and then comes back up, the clients > will resume forwarding messages to it. > > I'm running into this same bug as another user: > http://bugzilla.adiscon.com/show_bug.cgi?id=45 > > According to that bug, the fix is in 3.12.1, which is what > I'm using on client and server. > > > But I still can't seem to get it to work. Prior to doing > this, I simply had this line: > > *.* @@127.0.0.1:61514 > > > > Am I supposed to replace that line entirely with the > ActionQueue stanza, such as this one (the == is for clarity, > not actually part of my config): > > == > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 30 > $ActionResumeRetryCount -1 # infinite retries on insert failure > & @@127.0.0.1:61514 > == > > If I do, rsyslog dies immediately on restart. So I changed > it so look like this instead: > > == > *.* @@127.0.0.1:61514 > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 10 > $ActionResumeRetryCount -1 > & @@127.0.0.1:61514 > == > > This allows rsyslog to work again, but then it doesn't > actually do any resume. Just as the OP states in the bug, if > the rsyslog server is stopped, the client simply logs > messages locally. When the rsyslog server is started again, > the client once again logs locally as well as to the server. > But all the messages from the client between stop and start > on the server are not forwarded. > > I checked action.c and it does have the code-fix listed in > the forum post tied to the bug. > > I also understand that one message WILL be lost, which is a > TCP syslog protocol problem. So I sent multiple messages, > none of them came through. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 18:50:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:50:01 +0100 Subject: [rsyslog] rsyslog 2.0.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> Hi all, Rsyslog 2.0.3, a release of the stable branch, has been released today. It is a purely bug-fixing release, solving mostly bugs that occur either seldom or due to mistakes in the configuration file. It fixes some potential segfaults in those cases. Rsyslog 2.0.3 is a recommended update for all users of the rsyslog v2 branch. Download: http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml Changelog: http://www.rsyslog.com/Article189.phtml As always, feedback is appreciated. Rainer Gerhards From linuxweb at gmail.com Wed Mar 12 19:27:38 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 14:27:38 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> References: <47D81008.3090302@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Message-ID: <47D8209A.3070908@gmail.com> Rainer Gerhards wrote: > & in column one is to chain multiple actions together, but it can never > be at the start of the chain. Did you take it from some rsyslog > documentation? If so and you still know which, please let me know, I'd > like to fix that. No, I copied it from the bug report I listed. The OP must've had a regular action that wasn't listed. Anyway, I fixed it, restarted client rsyslog, but the behavior is the same. I don't need any special config on the server side, right? Well, if it helps, here's the top of my client /etc/rsyslog.conf, followed by the top of my server /etc/rsyslog.conf: == client /etc/rsyslog.conf == # Modules $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 # Standard syslog configuration (for local logging) [snip] == == server /etc/rsyslog.conf == # Modules $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so # TCP port $InputTCPServerRun 61514 # Global Directives $AllowedSender UDP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $AllowedSender TCP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $DirOwner root $DirGroup root $DirCreateMode 0755 # Templates [snip] # Standard syslog configuration [snip] == From linuxweb at gmail.com Wed Mar 12 19:32:15 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 14:32:15 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> References: <47D81008.3090302@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Message-ID: <47D821AF.9030107@gmail.com> Rainer Gerhards wrote: > $WorkDirectory /tmp/rsyslogwork Also, I should also point out that I NEVER see anything show up in my WorkDirectory. That directory above is owned by root:root and mode 0755. I turned off SELinux. Nothing EVER shows up there. johnn From milton at calnek.com Wed Mar 12 19:22:10 2008 From: milton at calnek.com (Milton Calnek) Date: Wed, 12 Mar 2008 12:22:10 -0600 Subject: [rsyslog] rsyslog 2.0.3 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> Message-ID: <47D81F52.2050701@calnek.com> The link in the web site does not work. The requested URL /rsyslog/rsyslog-2.0.23tar.gz was not found on this server. Thanks. Rainer Gerhards wrote: > Hi all, > > Rsyslog 2.0.3, a release of the stable branch, has been released today. > It is a purely bug-fixing release, solving mostly bugs that occur either > seldom or due to mistakes in the configuration file. It fixes some > potential segfaults in those cases. Rsyslog 2.0.3 is a recommended > update for all users of the rsyslog v2 branch. > > Download: > http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml > > Changelog: > http://www.rsyslog.com/Article189.phtml > > As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From rgerhards at hq.adiscon.com Wed Mar 12 19:48:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:48:45 +0100 Subject: [rsyslog] rsyslog 2.0.3 released Message-ID: <006101c88471$b85b588f$060013ac@intern.adiscon.com> Oops, as a quick fix http://download.rsyslog.com/rsyslog/rsyslog-2.0.3.tar.gz If i have not mistyped again ;) rainer ----- Urspr?ngliche Nachricht ----- Von: "Milton Calnek" An: "rsyslog-users" Gesendet: 12.03.08 19:37 Betreff: Re: [rsyslog] rsyslog 2.0.3 released The link in the web site does not work. The requested URL /rsyslog/rsyslog-2.0.23tar.gz was not found on this server. Thanks. Rainer Gerhards wrote: > Hi all, > > Rsyslog 2.0.3, a release of the stable branch, has been released today. > It is a purely bug-fixing release, solving mostly bugs that occur either > seldom or due to mistakes in the configuration file. It fixes some > potential segfaults in those cases. Rsyslog 2.0.3 is a recommended > update for all users of the rsyslog v2 branch. > > Download: > http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml > > Changelog: > http://www.rsyslog.com/Article189.phtml > > As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 19:50:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:50:55 +0100 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <006201c88472$0590e0ed$060013ac@intern.adiscon.com> That's fine, files are only created if the mem queue is filling up. By default thats after 8,000 msgs queued. And, no server needs no special config. More later, i am on a pda and can not really check out anything right now ;) rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 12.03.08 19:32 Betreff: Re: [rsyslog] actionqueue in front of tcp forward Rainer Gerhards wrote: > $WorkDirectory /tmp/rsyslogwork Also, I should also point out that I NEVER see anything show up in my WorkDirectory. That directory above is owned by root:root and mode 0755. I turned off SELinux. Nothing EVER shows up there. johnn _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 19:52:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:52:43 +0100 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <006301c88472$46360ab0$060013ac@intern.adiscon.com> Can you send me a ful debug output from the sender when it fails (maybe via mail) - if thats possible... rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 12.03.08 19:28 Betreff: Re: [rsyslog] actionqueue in front of tcp forward Rainer Gerhards wrote: > & in column one is to chain multiple actions together, but it can never > be at the start of the chain. Did you take it from some rsyslog > documentation? If so and you still know which, please let me know, I'd > like to fix that. No, I copied it from the bug report I listed. The OP must've had a regular action that wasn't listed. Anyway, I fixed it, restarted client rsyslog, but the behavior is the same. I don't need any special config on the server side, right? Well, if it helps, here's the top of my client /etc/rsyslog.conf, followed by the top of my server /etc/rsyslog.conf: == client /etc/rsyslog.conf == # Modules $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 # Standard syslog configuration (for local logging) [snip] == == server /etc/rsyslog.conf == # Modules $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so # TCP port $InputTCPServerRun 61514 # Global Directives $AllowedSender UDP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $AllowedSender TCP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $DirOwner root $DirGroup root $DirCreateMode 0755 # Templates [snip] # Standard syslog configuration [snip] == _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Wed Mar 12 20:07:34 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 15:07:34 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <006301c88472$46360ab0$060013ac@intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> Message-ID: <47D829F6.8070404@gmail.com> Rainer Gerhards wrote: > Can you send me a ful debug output from the sender when > it fails (maybe via mail) - if thats possible... I sent it via YouSendIt. Thanks for looking into this! johnn From rgerhards at hq.adiscon.com Wed Mar 12 21:40:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 21:40:22 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D829F6.8070404@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> I have received the file and had a chance to look at it. It's quite confusing. The send fails, but each retry succeeds... It looks like you use stunnel. I currently doubt that stunnel accepts each send and only reports the error after it can not connect to the remote side. So in fact we continously run into the situation that exactly *that* one message is lost. But I am still puzzled. Could you, for a test, run the same without stunnel and tell me if the problem persists or goes away? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 8:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] actionqueue in front of tcp forward > > Rainer Gerhards wrote: > > Can you send me a ful debug output from the sender when > > it fails (maybe via mail) - if thats possible... > > I sent it via YouSendIt. > Thanks for looking into this! > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 22:05:54 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 17:05:54 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> Message-ID: <47D845B2.30701@gmail.com> Rainer Gerhards wrote: > I have received the file and had a chance to look at it. It's quite > confusing. The send fails, but each retry succeeds... It looks like you > use stunnel. I currently doubt that stunnel accepts each send and only > reports the error after it can not connect to the remote side. So in > fact we continously run into the situation that exactly *that* one > message is lost. But I am still puzzled. Could you, for a test, run the > same without stunnel and tell me if the problem persists or goes away? Yes, I had the same thought about stunnel being the problem and was running some tests. It works! I now only lose the one message (known problem). But if I go over stunnel, then I lose all messages during the rsyslog server downtime. I can do without the stunnel for now. Thanks once again! johnn From rgerhards at hq.adiscon.com Wed Mar 12 22:20:15 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 22:20:15 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D845B2.30701@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 10:06 PM > To: rsyslog-users > Subject: Re: [rsyslog] actionqueue in front of tcp forward > > Rainer Gerhards wrote: > > I have received the file and had a chance to look at it. It's quite > > confusing. The send fails, but each retry succeeds... It > looks like you > > use stunnel. I currently doubt that stunnel accepts each > send and only > > reports the error after it can not connect to the remote side. So in > > fact we continously run into the situation that exactly *that* one > > message is lost. But I am still puzzled. Could you, for a > test, run the > > same without stunnel and tell me if the problem persists or > goes away? > > Yes, I had the same thought about stunnel being the problem > and was running some tests. > > It works! I now only lose the one message (known problem). > But if I go over stunnel, then I lose all messages during > the rsyslog server downtime. I have the strong feeling that it is time to do something against this plain old ack-less syslog tcp protocol... Maybe I add a half-duplex mode for starters. That's low, but quick to implement and ultra-reliable. I'll also see that I get more serious with RFC 3195 re-enabling. I've already done some basic thinking in regard to 3195 and the new syslog engine and doing it ultra-reliable will require a little bit of work. So there won't be an immediate cure - but defenitely the right route to take. How about half-duplex mode? Would that work for you? It means that each message must be acked before the next one is sent, so tcp's streaming features will almost be disabled. I'd expect a drop to at most 50% (more probable 40%) of the performance compared to what we currently run (half-duplex would obviously need to be an option...). So it would be a large performance hit. > I can do without the stunnel for now. For encryption, you could also look into the GSSAPI modules. It's contributed code, and I currently unfortunately have limited insight into it. But varmojfekoj, the contributor, has done a great job. > Thanks once again! I am very intersted in real-life experience. After all, the engine is still quite new. So I need feedback from the field to make it ultra-solid - a lab is a lab, is a lab ;) So keep the thoughts flowing. A side-note: we are rewriting phpLogCon, the web interface to syslog data. Any chance you happen to have some interest in that? ;) Rainer > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 23:10:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 18:10:22 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> Message-ID: <47D854CE.8020209@gmail.com> Rainer Gerhards wrote: > I have the strong feeling that it is time to do something against this > plain old ack-less syslog tcp protocol... Maybe I add a half-duplex mode > for starters. That's low, but quick to implement and ultra-reliable. > I'll also see that I get more serious with RFC 3195 re-enabling. I've > already done some basic thinking in regard to 3195 and the new syslog > engine and doing it ultra-reliable will require a little bit of work. So > there won't be an immediate cure - but defenitely the right route to > take. > > How about half-duplex mode? Would that work for you? It means that each > message must be acked before the next one is sent, so tcp's streaming > features will almost be disabled. I'd expect a drop to at most 50% (more > probable 40%) of the performance compared to what we currently run > (half-duplex would obviously need to be an option...). So it would be a > large performance hit. Just curious: do either of the above solutions (3195 and half-duplex) resolve the issue of losing all messages when the server goes down and you are going over stunnel? It sounds like the half-duplex does, I don't know enough about the 3195 to know. In answer to your question, I probably don't have as heavy a load as some folks out there, so it's easy for me to say, yes, half-duplex works for me, at least for the moment. >> I can do without the stunnel for now. > > For encryption, you could also look into the GSSAPI modules. It's > contributed code, and I currently unfortunately have limited insight > into it. But varmojfekoj, the contributor, has done a great job. Thanks! I'll have to look more closely at this in a staging environment down the road, as I can't afford potentially running into another stumbling block at the moment. > A side-note: we are rewriting phpLogCon, the web interface to syslog > data. Any chance you happen to have some interest in that? ;) It'll be a couple weeks before I really start looking into this, but yes, I was in fact planning to setup phpLogCon, too. johnn From rgerhards at hq.adiscon.com Thu Mar 13 08:11:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 08:11:58 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D854CE.8020209@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> <47D854CE.8020209@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> > Rainer Gerhards wrote: > > I have the strong feeling that it is time to do something against > this > > plain old ack-less syslog tcp protocol... Maybe I add a half-duplex > mode > > for starters. That's low, but quick to implement and ultra-reliable. > > I'll also see that I get more serious with RFC 3195 re-enabling. I've > > already done some basic thinking in regard to 3195 and the new syslog > > engine and doing it ultra-reliable will require a little bit of work. > So > > there won't be an immediate cure - but defenitely the right route to > > take. > > > > How about half-duplex mode? Would that work for you? It means that > each > > message must be acked before the next one is sent, so tcp's streaming > > features will almost be disabled. I'd expect a drop to at most 50% > (more > > probable 40%) of the performance compared to what we currently run > > (half-duplex would obviously need to be an option...). So it would be > a > > large performance hit. > > Just curious: do either of the above solutions (3195 and > half-duplex) resolve the issue of losing all messages when > the server goes down and you are going over stunnel? > > It sounds like the half-duplex does, I don't know enough > about the 3195 to know. > > > In answer to your question, I probably don't have as heavy a > load as some folks out there, so it's easy for me to say, > yes, half-duplex works for me, at least for the moment. I had a somewhat sleepless night, which is good news because it usually results in good ideas ;) This problem really bugged me, but I finally made up my mind. Instead of wasting time on fixing broken plain tcp syslog transport mode (e.g. by implementing half-duplex, which isn't standard anyhow), I'll do "the right thing". I thought rfc 3195 is the right thing. But it carries a lot of overhead that I really don't need. And, most importantly, any standard additions takes ages to go through the IETF (I know what I am talking about, have finally succeeded to get a better syslog rfc though it in "just" 4 (or 5?) years -- and it is still waiting to be published...). So instead of adding on any of these existing protocols, I'll do a new, lightweight but capable protocol specifically designed to solve the shortcomings we currently experience. Please welcome RELP, the "reliable event logging protocol" (name based on the quite successful selp [simple event logging protocol] effort: http://www.monitorware.com/en/workinprogress/selp.txt). Relp will evolve over time. I hope to do something useful relatively soon, and it will be extended as the project progresses. The ultimate goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog communications. I'll don't care about the outside world, so I can do the best of breed solution. For the rest of the world, rsyslog will of course continue to support plain tcp syslog and will get better support for rfc 3195. But if you wanna go hardcore on high-reliability, high-performance event logging, relp will be your choice. Technically, I'll split this off into rsyslog relp input and output plugins AND an independent librelp, which provides core protocol functionality (just in case somebody else wants to support it in the long term, so this will not be tied to rsyslog). > > >> I can do without the stunnel for now. > > > > For encryption, you could also look into the GSSAPI modules. It's > > contributed code, and I currently unfortunately have limited insight > > into it. But varmojfekoj, the contributor, has done a great job. > > Thanks! I'll have to look more closely at this in a staging > environment down the road, as I can't afford potentially > running into another stumbling block at the moment. > > > > A side-note: we are rewriting phpLogCon, the web interface to syslog > > data. Any chance you happen to have some interest in that? ;) > > It'll be a couple weeks before I really start looking into > this, but yes, I was in fact planning to setup phpLogCon, too. Excellent. Please keep an eye on its beta announcements - various stages of v2 will probably be available in a few weeks and I promise it will be much better than what is currently there. In fact, it tries to become the best-ever log web viewer. If there are some things you'd like to see in such a viewer, it would be great to hear your opinion. Rainer From maurizio.rottin at gmail.com Thu Mar 13 09:03:16 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 13 Mar 2008 09:03:16 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Wednesday, March 12, 2008 2:37 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > 2008/3/12, Rainer Gerhards : > > > > mmmmm...i'll check that better but at a glance it seems a heavy > > > > process with big logs. > > > > > > > > > Sorry, I don't really understand what you mean ;) But the problem is > > > that I probably do not really know how what you do with logger > > really > > > works. It is part of the apache config, is this right? Maybe you > > could > > > post a full config sample? > > > > > but i did... > > this is the log part in the directive > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > "combined" > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > > facility "-p local*.xxx" with tag "-t www.mysite.com" > > > > then i can use the tag to split logs by website in the remote rsyslog > > machine ;) > > > Ah, OK, I didn't realize this was actual Apache config. So your original > question > > > > just one question, do you know any way to pass the website name in > > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > > other program...i know only logger... > > > was how you could use the same apache config command for each log, so > that you don't need to have separate apache config statements for each > virtual server? > > Please bear with me if I am getting it wrong, I probably have too much > rsyslog on my mind. I am trying to help but I don't get a grip at what > the real problem is... > > Not really...i don't want to use logger, it becomes a heavy process under high traffic load....but this should be a question for the apache ML... -- mr From rgerhards at hq.adiscon.com Thu Mar 13 09:06:09 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 09:06:09 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> [sniiiiip] > Not really...i don't want to use logger, it becomes a heavy process > under high traffic load....but this should be a question for the > apache ML... OK, I don't know what makes it a heavy process... anyhow... If you come up with an explanation or what would be less resource-intense (from an apache point of view), please let me know. Maybe I can add a simple little tool to do the job. But obviously I need to know what's wrong with logger before even thinking about that ;) Rainer From Gerrard.Geldenhuis at datacash.com Thu Mar 13 10:04:11 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Thu, 13 Mar 2008 09:04:11 -0000 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com><47D854CE.8020209@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> Message-ID: > > I had a somewhat sleepless night, which is good news because it usually > results in good ideas ;) This problem really bugged me, but I finally > made up my mind. Instead of wasting time on fixing broken plain tcp > syslog transport mode (e.g. by implementing half-duplex, which isn't > standard anyhow), I'll do "the right thing". I thought rfc 3195 is the > right thing. But it carries a lot of overhead that I really don't need. > And, most importantly, any standard additions takes ages to go through > the IETF (I know what I am talking about, have finally succeeded to get > a better syslog rfc though it in "just" 4 (or 5?) years -- and it is > still waiting to be published...). > > So instead of adding on any of these existing protocols, I'll do a new, > lightweight but capable protocol specifically designed to solve the > shortcomings we currently experience. Please welcome RELP, the "reliable > event logging protocol" (name based on the quite successful selp [simple > event logging protocol] effort: > http://www.monitorware.com/en/workinprogress/selp.txt). > > Relp will evolve over time. I hope to do something useful relatively > soon, and it will be extended as the project progresses. The ultimate > goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog > communications. I'll don't care about the outside world, so I can do the > best of breed solution. For the rest of the world, rsyslog will of > course continue to support plain tcp syslog and will get better support > for rfc 3195. But if you wanna go hardcore on high-reliability, > high-performance event logging, relp will be your choice. > Maybe in another 5 years RELP will replace rfc 3195 especially. Now that rsyslog is part of official redhat release it should gain a lot of momentum, the anouncement for RHEL5.2 beta contained rsyslog. Regards From rgerhards at hq.adiscon.com Thu Mar 13 10:59:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 10:59:30 +0100 Subject: [rsyslog] RELP - was: actionqueue in front of tcp forward In-Reply-To: References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com><47D854CE.8020209@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B8E@grfint2.intern.adiscon.com> > Maybe in another 5 years RELP will replace rfc 3195 especially. Doesn't really matter ;) For the technially interested folks: I have drafted an intial spec. Still a lot is missing (of course), but it sufficient to solve the stunnel problem that we saw yesterday (and a couple of others...): http://git.adiscon.com/?p=librelp.git;a=blob;f=src/relp.c;h=a9aff8b005d2 3a8dc66aed1ffa66f41e0ba7b41c;hb=master > Now that > rsyslog is part of official redhat release it should gain a lot of > momentum, the anouncement for RHEL5.2 beta contained rsyslog. Oh, nice to hear. This is good news :) Rainer From friedl at hq.adiscon.com Thu Mar 13 12:59:00 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 13 Mar 2008 12:59:00 +0100 Subject: [rsyslog] rsyslog 3.12.2 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B96@grfint2.intern.adiscon.com> Hi all, the development branch also sees a new released. Rsyslog 3.12.2 is available immediately. It is primarily a code cleanup and bug fixing release. Feature-wise, it aims at some better recovery of aborted TCP sessions. It contains a number of important fixes, which, for example, prevent a segfault when imfile tries to read empty log file lines. Internally, loadable library modules are now automatically unloaded if they are no longer used. Rsyslog 3.12.2 is a recommended update for all v3 branch users. Download: http://www.rsyslog.com/Downloads-req-getit-lid-87.phtml Changelog: http://www.rsyslog.com/Article191.phtml Feedback is, as always, appreciated. Thank you. Florian Riedl From linuxweb at gmail.com Thu Mar 13 20:27:07 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 13 Mar 2008 15:27:07 -0400 Subject: [rsyslog] imfile with existing files, splits files on server side Message-ID: <47D9800B.40704@gmail.com> I tested this on both rsyslog 3.12.1 and this morning's 3.12.2 release. I ran a couple tests with existing log files. E.g., I have an httpd access_log which is 1.3MB and about 5,000 lines -- nowhere near as large as my real existing log, which is 136MB and over 511,000 lines. With the smaller file, as soon as I started rsyslog on the client side, it correctly started to show up on the server side in: /var/log/client1/httpd_access_log And the hostname shows up in the prefix for the message: Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip] But after a few hundred lines, it suddenly starts logging to: /var/log/httpd_access_log And the hostname no longer shows up in the prefix: Mar 13 15:12:20 tag_access_log:76.8.67.2[snip] Then, thousands of lines later, it goes back to logging to: /var/log/client1/httpd_access_log The other thing is, it never actually finishes the entire 5000 lines. It does 4000-something and then stops. My gut feeling is this is related to ActionQueue settings. So I'm going to experiment with a few different parameters. But the part that I don't get is why it's not all being logged to the same %HOSTNAME% directory? I don't know what's useful to send for diagnostics -- the original files from the client side, the split files from the server side, the client debug, the server debug. Etc. But I'd be happy to run any requested tests. Meanwhile, I'll tweak ActionQueue params. johnn From rgerhards at hq.adiscon.com Thu Mar 13 20:58:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 20:58:45 +0100 Subject: [rsyslog] imfile with existing files, splits files on server side In-Reply-To: <47D9800B.40704@gmail.com> References: <47D9800B.40704@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> Mmmhhhh... So the 5,000 line file doesn't make it completely to the server? That's strange... With the larger file, I'd assume that there are some flow control issues, but with the small one... Maybe it is related to the other problem with the hostname. Kind of sounds like it looses sync some time in between reading the files. I guess it is probably a good idea if you can send me the 5000 line file, so I can run it in my lab tomorrow. Also, please confirm that the small file actually doesn't make it to the server. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Thursday, March 13, 2008 8:27 PM > To: rsyslog-users > Subject: [rsyslog] imfile with existing files, splits files > on server side > > I tested this on both rsyslog 3.12.1 and this morning's > 3.12.2 release. > > I ran a couple tests with existing log files. E.g., I have > an httpd access_log which is 1.3MB and about 5,000 lines -- > nowhere near as large as my real existing log, which is > 136MB and over 511,000 lines. > > With the smaller file, as soon as I started rsyslog on the > client side, it correctly started to show up on the server > side in: > /var/log/client1/httpd_access_log > > And the hostname shows up in the prefix for the message: > Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip] > > > But after a few hundred lines, it suddenly starts logging to: > /var/log/httpd_access_log > > And the hostname no longer shows up in the prefix: > Mar 13 15:12:20 tag_access_log:76.8.67.2[snip] > > > Then, thousands of lines later, it goes back to logging to: > /var/log/client1/httpd_access_log > > > > The other thing is, it never actually finishes the entire > 5000 lines. It does 4000-something and then stops. > > My gut feeling is this is related to ActionQueue settings. > So I'm going to experiment with a few different parameters. > > But the part that I don't get is why it's not all being > logged to the same %HOSTNAME% directory? > > > I don't know what's useful to send for diagnostics -- the > original files from the client side, the split files from > the server side, the client debug, the server debug. Etc. > > But I'd be happy to run any requested tests. > > Meanwhile, I'll tweak ActionQueue params. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From janfrode at tanso.net Fri Mar 14 00:28:29 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 00:28:29 +0100 Subject: [rsyslog] rsyslog on RHEL5u2 ? Message-ID: Looks like rsyslogd will be available in RHEL5u2 ! Congratulations Rainer! https://www.redhat.com/archives/rhelv5-announce/2008-March/msg00000.html """ This beta release includes the following improvements: * Improved Audit and Logging + Add rsyslog logging facility New package: rsyslog-2.0.0-11.el5 Enhanced system logging and kernel message trapping daemons """ -jf From janfrode at tanso.net Fri Mar 14 00:58:55 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 00:58:55 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> Message-ID: On 2008-03-12, Johnny Tan wrote: > > This is custom. I prefer to install auditd and have the > SELinux logs separated out from /var/log/messages. Then I > use imfile to convert the audit logs to syslog in order to > send it over to the central logger. > > Yes, it's a waste of a syslog "local" facility, but I'm not > aware of any other way unless I remove auditd and have those > go back into /var/log/messages. Hmmm.. I have been wondering about how to collect the auditd logs centrally. Could you please share how you do it ? Would be great if you manage to keep the format on the receiving host.. -jf From rgerhards at hq.adiscon.com Fri Mar 14 10:49:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 10:49:45 +0100 Subject: [rsyslog] Feedback requested - length of syslog tag Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Hi all, I got the following bug report (and patch - thx) for situation where the syslog tag is larger than 32 characters. http://bugzilla.adiscon.com/show_bug.cgi?id=50 Syslog rfcs prohibit such oversize tags. However, I think seeing them in practice triggers the question if we should really follow RFC 3164 and the newer rfcs in this case. Or may it be better to have at least an option to allow tags of more than 32 characters? If you have an opinion, please voice it. Thanks, Rainer From pvrabec at redhat.com Fri Mar 14 12:15:15 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Fri, 14 Mar 2008 12:15:15 +0100 Subject: [rsyslog] HUP problem Message-ID: <200803141215.15887.pvrabec@redhat.com> Hi folks, we have some problems here :) rsyslog-3.12.1 ------------------ https://bugzilla.redhat.com/show_bug.cgi?id=437393 imklog.so + HUP generates memory leaks use: HUPs and #watch -d=cumulative cat /proc/`pgrep rsyslog`/status rsyslog-3.12.2 ------------------ is even worse, SIGSEGV occurs after every HUP :( From Gerrard.Geldenhuis at datacash.com Fri Mar 14 12:36:43 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Fri, 14 Mar 2008 11:36:43 -0000 Subject: [rsyslog] Feedback requested - length of syslog tag In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Message-ID: It's a tough one. Software that follows standards generally work well with other software and just makes life less complex. However a lot of these RFC's are very dated and have not kept track with time and 2001 when rfc3164 was published is a long time ago. If there is a flag which is not set by default and which is documented as breaking the standard it might be acceptable to allow longer tags. Users should break at own risk... Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 14 March 2008 09:50 > To: rsyslog-users > Subject: [rsyslog] Feedback requested - length of syslog tag > > Hi all, > > I got the following bug report (and patch - thx) for situation where the > syslog tag is larger than 32 characters. > > http://bugzilla.adiscon.com/show_bug.cgi?id=50 > > Syslog rfcs prohibit such oversize tags. However, I think seeing them in > practice triggers the question if we should really follow RFC 3164 and > the newer rfcs in this case. Or may it be better to have at least an > option to allow tags of more than 32 characters? > > If you have an opinion, please voice it. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Mar 14 12:57:41 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 12:57:41 +0100 Subject: [rsyslog] Feedback requested - length of syslog tag References: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Message-ID: On 2008-03-14, Rainer Gerhards wrote: > > I got the following bug report (and patch - thx) for situation where the > syslog tag is larger than 32 characters. > > http://bugzilla.adiscon.com/show_bug.cgi?id=50 > > Syslog rfcs prohibit such oversize tags. However, I think seeing them in > practice triggers the question if we should really follow RFC 3164 and > the newer rfcs in this case. Or may it be better to have at least an > option to allow tags of more than 32 characters? > > If you have an opinion, please voice it. > I think rsyslog should "be liberal in what it receives and conservative in what it send". So, maybe allow receiving messages with large tags, but don't send them out to other rfc3164-expecting servers. Sending large tags should maybe only be done in RELP.. ? -jf From linuxweb at gmail.com Fri Mar 14 15:15:26 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 14 Mar 2008 10:15:26 -0400 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> Message-ID: <47DA887E.9020601@gmail.com> Jan-Frode Myklebust wrote: > Hmmm.. I have been wondering about how to collect the auditd logs > centrally. Could you please share how you do it ? I use imfile, this is my client stanza: == # auditd audit.log $InputFileName /var/log/audit/audit.log $InputFileTag tag_audit_log: $InputFileStateFile audit_log $InputFileSeverity info $InputFileFacility local6 $InputRunFileMonitor == And on the server side: == $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" local6.* ?HostAudit == > Would be great > if you manage to keep the format on the receiving host.. The format is NOT kept due to the syslog prefix. However audit2allow can still read and process the file. If I need the original format (like for ausearch, and maybe other tools), I would need to run something like this on the server side, in the directory for the host I'm wanting to do the analysis on: sed 's/^.*tag_audit_log://' audit_log | ausearch -i Is that what you meant? johnn From rgerhards at hq.adiscon.com Fri Mar 14 15:29:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 15:29:44 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: <47DA887E.9020601@gmail.com> References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> <47DA887E.9020601@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> I guess you can keep the format if you use a template with just %msg% inside it: $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" $template auditFormat, "%msg%" local6.* ?HostAudit;auditFormat I have not tested this, just guessing. If you try it out and it does not work, I'd appreciate if you could post me a line as written by rsyslog and an original line. I'll see what I can do with these ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 14, 2008 3:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog v3 and selinux > > Jan-Frode Myklebust wrote: > > Hmmm.. I have been wondering about how to collect the auditd logs > > centrally. Could you please share how you do it ? > > I use imfile, this is my client stanza: > == > # auditd audit.log > $InputFileName /var/log/audit/audit.log > $InputFileTag tag_audit_log: > $InputFileStateFile audit_log > $InputFileSeverity info > $InputFileFacility local6 > $InputRunFileMonitor > == > > > And on the server side: > == > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > local6.* ?HostAudit > == > > > > Would be great > > if you manage to keep the format on the receiving host.. > > The format is NOT kept due to the syslog prefix. However > audit2allow can still read and process the file. > > If I need the original format (like for ausearch, and maybe > other tools), I would need to run something like this on the > server side, in the directory for the host I'm wanting to do > the analysis on: > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > > Is that what you meant? > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 14 15:33:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 15:33:43 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com><47DA887E.9020601@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BB3@grfint2.intern.adiscon.com> Ummm... I forgot two important chraceters, else you'll have no line feeds and all in one line... (see the \n after %msg$)! $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" $template auditFormat, "%msg%\n" local6.* ?HostAudit;auditFormat > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, March 14, 2008 3:30 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog v3 and selinux > > I guess you can keep the format if you use a template with just %msg% > inside it: > > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > $template auditFormat, "%msg%" > local6.* ?HostAudit;auditFormat > > I have not tested this, just guessing. If you try it out and it does > not > work, I'd appreciate if you could post me a line as written by rsyslog > and an original line. I'll see what I can do with these ;) > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > > Sent: Friday, March 14, 2008 3:15 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog v3 and selinux > > > > Jan-Frode Myklebust wrote: > > > Hmmm.. I have been wondering about how to collect the auditd logs > > > centrally. Could you please share how you do it ? > > > > I use imfile, this is my client stanza: > > == > > # auditd audit.log > > $InputFileName /var/log/audit/audit.log > > $InputFileTag tag_audit_log: > > $InputFileStateFile audit_log > > $InputFileSeverity info > > $InputFileFacility local6 > > $InputRunFileMonitor > > == > > > > > > And on the server side: > > == > > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > > local6.* ?HostAudit > > == > > > > > > > Would be great > > > if you manage to keep the format on the receiving host.. > > > > The format is NOT kept due to the syslog prefix. However > > audit2allow can still read and process the file. > > > > If I need the original format (like for ausearch, and maybe > > other tools), I would need to run something like this on the > > server side, in the directory for the host I'm wanting to do > > the analysis on: > > > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > > > > > Is that what you meant? > > johnn > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Mar 14 15:29:46 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 15:29:46 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> <47DA887E.9020601@gmail.com> Message-ID: On 2008-03-14, Johnny Tan wrote: >== > The format is NOT kept due to the syslog prefix. However > audit2allow can still read and process the file. > > If I need the original format (like for ausearch, and maybe > other tools), I would need to run something like this on the > server side, in the directory for the host I'm wanting to do > the analysis on: > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > Is that what you meant? Yes, thanks! -jf From rgerhards at hq.adiscon.com Fri Mar 14 18:09:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 18:09:35 +0100 Subject: [rsyslog] rsyslog on RHEL5u2 ? In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BBA@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, March 14, 2008 12:28 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog on RHEL5u2 ? > > Looks like rsyslogd will be available in RHEL5u2 ! Congratulations > Rainer! Thanks, much appreciated :) Let's hope to get v3 stable ready for RHEL 6 ;) Rainer > > > https://www.redhat.com/archives/rhelv5-announce/2008- > March/msg00000.html > > > """ > This beta release includes the following improvements: > > > > * Improved Audit and Logging > + Add rsyslog logging facility > > > > New package: rsyslog-2.0.0-11.el5 > Enhanced system logging and kernel message trapping daemons > > """ > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 14 18:29:40 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 18:29:40 +0100 Subject: [rsyslog] HUP problem In-Reply-To: <200803141215.15887.pvrabec@redhat.com> References: <200803141215.15887.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BBB@grfint2.intern.adiscon.com> I was busy with the other things today but had a quick look. I can repro the hup problem. Looks like it is related to the new object loader (and specifically the unloader), I guess it will be a relatively easy fix ... on Monday ;) Will look into the memleak after that. Thanks for reporting. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > Sent: Friday, March 14, 2008 12:15 PM > To: rsyslog-users > Subject: [rsyslog] HUP problem > > Hi folks, > > we have some problems here :) > > rsyslog-3.12.1 > ------------------ > https://bugzilla.redhat.com/show_bug.cgi?id=437393 > imklog.so + HUP generates memory leaks > use: > HUPs and #watch -d=cumulative cat /proc/`pgrep rsyslog`/status > > rsyslog-3.12.2 > ------------------ > is even worse, SIGSEGV occurs after every HUP :( > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 11:04:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 11:04:45 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Hi all, once again, I am in need for feedback. In order to be backwards-compatible with sysklogd, rsyslogd supported the "last message repeated n times" message compression feature. However, this feature is prone to causing user trouble. Some even think that it is a design flaw (see this discussion on the loganalysis mailing list, this posting is probably a good entry point into a lengthy thread: http://www.loganalysis.org/pipermail/loganalysis/2008-January/000547.htm l ). >From the rsyslog core engine point of view, "last message repeated n times" is quite costly in terms of code complexity and even performance. There is a -e command line switch to turn it off, which most users seem to do (and those that don't do it often seem to run into troubles). I am very tempted to DROP this feature from future builds. That would result in a great code complexity reduction (really, it takes a lot of effort...) and probably also rid us of a standard trouble spot. However, it also means that its compression features are no longer available. Question now: does anybody actually need this feature? If so, why is it good? Please provide feedback. Thanks, Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 11:15:09 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 10:15:09 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: Hi I personnally find it not very usefull to put it politely. I'd rather have every message and every time stamp for the sake of acuracy and precision then an attempt from the software to be intelligent. And if you get the benefit of a code clean up then double thumbs up from me. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 10:05 > To: rsyslog-users > Subject: [rsyslog] Feedback requested "Last message logged n times"... > > Hi all, > > once again, I am in need for feedback. In order to be > backwards-compatible with sysklogd, rsyslogd supported the "last message > repeated n times" message compression feature. However, this feature is > prone to causing user trouble. Some even think that it is a design flaw > (see this discussion on the loganalysis mailing list, this posting is > probably a good entry point into a lengthy thread: > http://www.loganalysis.org/pipermail/loganalysis/2008-January/000547.htm > l ). > > >From the rsyslog core engine point of view, "last message repeated n > times" is quite costly in terms of code complexity and even performance. > There is a -e command line switch to turn it off, which most users seem > to do (and those that don't do it often seem to run into troubles). > > I am very tempted to DROP this feature from future builds. That would > result in a great code complexity reduction (really, it takes a lot of > effort...) and probably also rid us of a standard trouble spot. However, > it also means that its compression features are no longer available. > > Question now: does anybody actually need this feature? If so, why is it > good? > > Please provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 11:23:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 11:23:37 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gerrard Geldenhuis > Sent: Tuesday, March 18, 2008 11:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n > times"... > > Hi > I personnally find it not very usefull to put it politely. I'd rather > have every message and every time stamp for the sake of acuracy and > precision then an attempt from the software to be intelligent. > > And if you get the benefit of a code clean up then double thumbs up > from > me. It's actually a *big* cleanup, because in order to support that feature, we need to have A) some background tick processing to flush the "repeated message" every few (30 or 60) seconds (it prevents rsyslogd from being tickles - think powertop, green IT and all that) B) requires rsyslogd to keep multiple message objects alive for an extended period of time, because the message object is needed for comparison when a new message comes in. Among others, this prevents some advanced flow control options which would depend on message destruction *when the message is done* and not *when the next message arrives*. C) [estimate] almost half the code of the main action calling loop, and most of the complexity, is related to "last message repeated n times" D) it prevents future end-to-end ACK capabilities (e.g. message is only acked after being written to DB) E) requires additional synchronization slowing down parallel processing (granted, just a little bit, but...) F) requires a "duplicate message capability" (some more code) G) ... and probably a few more spots ... I should have added this information to the initial message, too. These are *my* primary drivers for the idea to get rid of it ;) Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 12:12:02 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 11:12:02 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> Message-ID: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 10:24 > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n times"... > > > It's actually a *big* cleanup, because in order to support that feature, > we need to have > I have asked the question on a local lug to get some more opinions. I will post the results if any here. Regards From pvrabec at redhat.com Tue Mar 18 12:26:27 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 18 Mar 2008 12:26:27 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> Message-ID: <200803181226.27878.pvrabec@redhat.com> On Tuesday 18 March 2008 12:12:02 pm Gerrard Geldenhuis wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: 18 March 2008 10:24 > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > times"... > > > It's actually a *big* cleanup, because in order to support that > > feature, > > > we need to have > > I have asked the question on a local lug to get some more opinions. I > will post the results if any here. I did the same. :) The only arguments against removal I have received until now were: 1. DoS (but nobody explained how) 2. cleanness of logs (relative) I'm thinking about forwarding discussion to fedora-devel :) > > Regards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From raimund.sacherer at gmail.com Tue Mar 18 12:37:35 2008 From: raimund.sacherer at gmail.com (Raimund Sacherer) Date: Tue, 18 Mar 2008 12:37:35 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: <41963B0D-1D02-4C60-99E1-CF52AFF0ABCB@gmail.com> > > Question now: does anybody actually need this feature? If so, why is > it > good? > > Please provide feedback. Hi all, i think it's a biest of the past, remember those days where every byte, every bit was really valuable? i remember programing on my C64 with byte values, because they could hold 256 states and it was just using one (!, today i have to smile) byte ... Or the times i had my first 40 MB Harddrive in one of my first PC's ... i guess compressing a message which is exactly the same but only of the timestamp was virtually a lifesaver those days ... as far as i can say, from the bottom of my heart: get rid of it! disc- space is cheap, i *love* consistant logs, personally i can not think of any answer that would justify today a) log-file inconsinstency b) a known trouble-spot and c) such a future-development-stopper as this code seems to be (after reading your problems with this piece of code. again, from me: go for it best Raimund > > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 12:42:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 12:42:11 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803181226.27878.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> <200803181226.27878.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Good discussion, thanks all :) ... some more points below ... > > I have asked the question on a local lug to get some more opinions. I > > will post the results if any here. > I did the same. :) > > The only arguments against removal I have received until now were: > > 1. DoS (but nobody explained how) I think this is along the lines of making it easy to flood the log with similar messages. If that's the case, I think it is a (too) weak argument because an attacker could easily include a random pattern inside the message. HOWEVER, the current logic indeed prevents flooding the log from equal messages, e.g. if a process runs wild. > 2. cleanness of logs (relative) > > I'm thinking about forwarding discussion to fedora-devel :) Excellent. Maybe there are also alternatives to the current behavior, something in between. Right now, the core engine does this, so the reduction capability is inherent to every action (except if an action specifically forbids it, because it can not intelligently handle it - e.g. database writers). So it applies to forwarding, snmp, whatever, ... One alternative would be to remove it from the core engine and move it into the *file write* output plugin (omfile). This has its subtleties, too, so I don't like to take that approach lightly. Plus, it than means that the network may become spammed. I think one core requirement is to find people who actually *intentionally* use this feature and ask for their reasoning. That will probably be the best way to tell us if its really needed. And, of course, we need to weigh the negative effects of it against these con-points. Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 14:11:15 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 13:11:15 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: The only arguments for keeping the feature that I got on my lug was the preservartion of disk/network IO. I think to prevent DOS attacks is a valid argument but as you said can be easily circumvented by randomizing messages. To safeguard against dos attacks you could have a monitor that monitors for extra ordinary amount of traffic and then generate a snmp trap. Whether that should be a rsyslog plugin or part of other software is open to debate. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 11:42 > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n times"... > > Good discussion, thanks all :) ... some more points below ... > > > > I have asked the question on a local lug to get some more opinions. > I > > > will post the results if any here. > > I did the same. :) > > > > The only arguments against removal I have received until now were: > > > > 1. DoS (but nobody explained how) > > I think this is along the lines of making it easy to flood the log with > similar messages. If that's the case, I think it is a (too) weak > argument because an attacker could easily include a random pattern > inside the message. > > HOWEVER, the current logic indeed prevents flooding the log from equal > messages, e.g. if a process runs wild. > > > 2. cleanness of logs (relative) > > > > I'm thinking about forwarding discussion to fedora-devel :) > > Excellent. > > Maybe there are also alternatives to the current behavior, something in > between. Right now, the core engine does this, so the reduction > capability is inherent to every action (except if an action specifically > forbids it, because it can not intelligently handle it - e.g. database > writers). So it applies to forwarding, snmp, whatever, ... One > alternative would be to remove it from the core engine and move it into > the *file write* output plugin (omfile). This has its subtleties, too, > so I don't like to take that approach lightly. Plus, it than means that > the network may become spammed. I think one core requirement is to find > people who actually *intentionally* use this feature and ask for their > reasoning. That will probably be the best way to tell us if its really > needed. And, of course, we need to weigh the negative effects of it > against these con-points. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 14:22:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 14:22:30 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> > The only arguments for keeping the feature that I got on my lug was the > preservartion of disk/network IO. Did you get any "feeling" of how important this is being considered? > > I think to prevent DOS attacks is a valid argument but as you said can > be easily circumvented by randomizing messages. > > To safeguard against dos attacks you could have a monitor that monitors > for extra ordinary amount of traffic and then generate a snmp trap. > Whether that should be a rsyslog plugin or part of other software is > open to debate. This may (m-a-y it's far too early) be part of the flow control logic or an exception detector or a rate-limiting feature... Even for non-DoS cases it might be interesting to know who is sending most messages... mmmh... maybe this points into a direction on how to solve the need that is behind "last message repeated n times". Probably that need is not even fully understood... mmmhh. More thoughts are appreciated ;) Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 14:41:48 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 13:41:48 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> Message-ID: > > > The only arguments for keeping the feature that I got on my lug was > the > > preservartion of disk/network IO. > > Did you get any "feeling" of how important this is being considered? I think the general feeling is: "don't care" or not important at all but then I did not get many replies. > > Even for non-DoS cases it might be interesting to know who is sending > most messages... mmmh... maybe this points into a direction on how to > solve the need that is behind "last message repeated n times". Probably > that need is not even fully understood... mmmhh. More thoughts are > appreciated ;) When I asked about which applications has caused these type of repeat messages one culprit mentioned was the kernel. Regards From friedl at hq.adiscon.com Tue Mar 18 15:05:41 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Tue, 18 Mar 2008 15:05:41 +0100 Subject: [rsyslog] rsyslog 3.12.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDF@grfint2.intern.adiscon.com> Hi all, rsyslog 3.12.3 has been released today. It is primarily a bug-fixing release, but offers new, advanced flow control options. So far, only imfile utilizes these, but other modules will do so soon. The release contains important bug fixes, for example for a segfault on hup, improper framing for syslog/tcp communication and processing empty lines in the log file reader. Rsyslog 3.12.3 is a recommended update to all v3 branch users and users of 3.12.2 are urged to apply the update (some bugs where introduced in that release). Download: http://www.rsyslog.com/Downloads-req-getit-lid-88.phtml Changelog: http://www.rsyslog.com/Article193.phtml As always, feedback is appreciated. Florian Riedl From pvrabec at redhat.com Tue Mar 18 15:23:52 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 18 Mar 2008 15:23:52 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: <200803181523.52321.pvrabec@redhat.com> On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > The only arguments for keeping the feature that I got on my lug was the > preservartion of disk/network IO. > > I think to prevent DOS attacks is a valid argument but as you said can > be easily circumvented by randomizing messages. I'm afraid it's not true in all cases. What if you do DOS attach not directly to do rsyslog, but via other daemon. In situation when you can't send message directly to syslog, but you can make some daemon generate message for you. This message would be probably always the same content. > To safeguard against dos attacks you could have a monitor that monitors > for extra ordinary amount of traffic and then generate a snmp trap. > Whether that should be a rsyslog plugin or part of other software is > open to debate. > > Regards > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: 18 March 2008 11:42 > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > times"... > > > Good discussion, thanks all :) ... some more points below ... > > > > > > I have asked the question on a local lug to get some more > > opinions. > > > I > > > > > > will post the results if any here. > > > > > > I did the same. :) > > > > > > The only arguments against removal I have received until now were: > > > > > > 1. DoS (but nobody explained how) > > > > I think this is along the lines of making it easy to flood the log > > with > > > similar messages. If that's the case, I think it is a (too) weak > > argument because an attacker could easily include a random pattern > > inside the message. > > > > HOWEVER, the current logic indeed prevents flooding the log from equal > > messages, e.g. if a process runs wild. > > > > > 2. cleanness of logs (relative) > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > Excellent. > > > > Maybe there are also alternatives to the current behavior, something > > in > > > between. Right now, the core engine does this, so the reduction > > capability is inherent to every action (except if an action > > specifically > > > forbids it, because it can not intelligently handle it - e.g. database > > writers). So it applies to forwarding, snmp, whatever, ... One > > alternative would be to remove it from the core engine and move it > > into > > > the *file write* output plugin (omfile). This has its subtleties, too, > > so I don't like to take that approach lightly. Plus, it than means > > that > > > the network may become spammed. I think one core requirement is to > > find > > > people who actually *intentionally* use this feature and ask for their > > reasoning. That will probably be the best way to tell us if its really > > needed. And, of course, we need to weigh the negative effects of it > > against these con-points. > > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Tue Mar 18 22:12:39 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 18 Mar 2008 17:12:39 -0400 Subject: [rsyslog] imfile with existing files, splits files on server side In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> References: <47D9800B.40704@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> Message-ID: <47E03047.3090801@gmail.com> Rainer Gerhards wrote: > So the 5,000 line file doesn't make it completely to the server? That's > strange... With the larger file, I'd assume that there are some flow > control issues, but with the small one... Just wanted to follow-up... thanks to Rainer's tireless poring over my excessively long logfiles and debug outputs, this issue is finally resolved. It seems like it was a combination of tcp flow control issues (fixed with the new 3.12.3) and the server by default discarding messages above severity 4. Thanks again to Rainer for not giving up on this! johnn From rgerhards at hq.adiscon.com Thu Mar 20 11:04:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 11:04:19 +0100 Subject: [rsyslog] update on RELP Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C16@grfint2.intern.adiscon.com> Hi all, I thought I provide a quick update on RELP, the protocol rsyslog will use in the future for reliable rsyslog-to-rsyslog communication. I am currently very actively working on both the protocol and librelp, an implementation for it. The RELP "spec" is also growing. It is now available at http://www.rsyslog.com/doc-relp.html Those with some deep technical interest might want to have a peek at it. I have also setup a RELP-specific mailing list. Find it at http://lists.adiscon.net/mailman/listinfo/relp The next major feature in rsyslogd will be relp support. Until this is finished, I am focused on bug fixing. Rainer From maurizio.rottin at gmail.com Thu Mar 20 15:20:17 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:20:17 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> Message-ID: 2008/3/13, Rainer Gerhards : wow, > [sniiiiip] > > > Not really...i don't want to use logger, it becomes a heavy process > > under high traffic load....but this should be a question for the > > apache ML... > > > OK, I don't know what makes it a heavy process... anyhow... If you come > up with an explanation or what would be less resource-intense (from an > apache point of view), please let me know. Maybe I can add a simple > little tool to do the job. But obviously I need to know what's wrong > with logger before even thinking about that ;) > > ok my ideas are a little more clear now, as you said: > was how you could use the same apache config command for each log, so > that you don't need to have separate apache config statements for each > virtual server? > > Please bear with me if I am getting it wrong, I probably have too much > rsyslog on my mind. I am trying to help but I don't get a grip at what > the real problem is... > > you were right...this is what i need! just one command for ErrorLog and one for CustomLog so as i posted now i'm using: > > this is the log part in the directive > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > "combined" > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > > facility "-p local*.xxx" with tag "-t www.mysite.com" for each virtualhost, but this mean that i have 2 loggers running for every virtualhost. This is why logger becomes a heavy process. But if i use one default directive for every vhost, i don't know how to split the log per vhost in the remote machine which collects the logs... Hope now is a little bit clearer... now, new config in httpd.conf "LogLevel warn ErrorLog "|/usr/bin/logger -p local5.err" LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined CustomLog "|/usr/bin/logger -p local6.info " vcombined" and the %v gives the virtualhost name, so now i've got only two loggers. At this point how can i say to rsyslog to split the files according to the first word which is in this case the vhost name? Thanks for being patient! -- mr From rgerhards at hq.adiscon.com Thu Mar 20 15:25:09 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 15:25:09 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Can you send me a handful of the logline to play with? Probably not this week, but next... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:20 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/13, Rainer Gerhards : > > wow, > > [sniiiiip] > > > > > Not really...i don't want to use logger, it becomes a heavy process > > > under high traffic load....but this should be a question for the > > > apache ML... > > > > > > OK, I don't know what makes it a heavy process... anyhow... If you > come > > up with an explanation or what would be less resource-intense (from > an > > apache point of view), please let me know. Maybe I can add a simple > > little tool to do the job. But obviously I need to know what's wrong > > with logger before even thinking about that ;) > > > > > > ok my ideas are a little more clear now, as you said: > > was how you could use the same apache config command for each log, so > > that you don't need to have separate apache config statements for > each > > virtual server? > > > > Please bear with me if I am getting it wrong, I probably have too > much > > rsyslog on my mind. I am trying to help but I don't get a grip at > what > > the real problem is... > > > > > > you were right...this is what i need! > > just one command for ErrorLog and one for CustomLog > > so as i posted now i'm using: > > > this is the log part in the directive > > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > > "combined" > > > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger > with > > > facility "-p local*.xxx" with tag "-t www.mysite.com" > > for each virtualhost, but this mean that i have 2 loggers running for > every virtualhost. > This is why logger becomes a heavy process. > > But if i use one default directive for every vhost, i don't know how > to split the log per vhost in the remote machine which collects the > logs... > > Hope now is a little bit clearer... > > now, new config in httpd.conf > "LogLevel warn > ErrorLog "|/usr/bin/logger -p local5.err" > LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" > vcombined > CustomLog "|/usr/bin/logger -p local6.info " vcombined" > > and the %v gives the virtualhost name, so now i've got only two > loggers. > > At this point how can i say to rsyslog to split the files according to > the first word which is in this case the vhost name? > > Thanks for being patient! > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 15:46:32 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:46:32 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Message-ID: 2008/3/20, Rainer Gerhards : > Can you send me a handful of the logline to play with? Probably not this > week, but next... > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" i'm trying to use the regexp but with no success in this way: $template MsgFormat,"%msg%\n" $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ [a-z,\.]*--end%_az.log" local6.info -?ApacheRemoteCustom;MsgFormat from the documentation: "the property replacer will return the part of the property text that matches the regular expression" which should be " www.mysite.com" but i get a file named _az.log -- mr From rgerhards at hq.adiscon.com Thu Mar 20 15:49:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 15:49:21 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Let me try to avoid the regexp (its expensive and I can not debug it now ;)): so you search for the string that is at the start of the msg and delimited by the first space? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/20, Rainer Gerhards : > > Can you send me a handful of the logline to play with? Probably not > this > > week, but next... > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" > > i'm trying to use the regexp but with no success in this way: > $template MsgFormat,"%msg%\n" > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > [a-z,\.]*--end%_az.log" > local6.info -?ApacheRemoteCustom;MsgFormat > > from the documentation: "the property replacer will return the part of > the property text that matches the regular expression" which should be > " www.mysite.com" > but i get a file named _az.log > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 15:59:02 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:59:02 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Message-ID: yes! but actually there is a space at the beginning and hostname can contain the dash -, numbers, and letters. 2008/3/20, Rainer Gerhards : > Let me try to avoid the regexp (its expensive and I can not debug it now > ;)): so you search for the string that is at the start of the msg and > delimited by the first space? > > > > Rainer > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Thursday, March 20, 2008 3:47 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > 2008/3/20, Rainer Gerhards : > > > Can you send me a handful of the logline to play with? Probably not > > this > > > week, but next... > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" > > > > i'm trying to use the regexp but with no success in this way: > > $template MsgFormat,"%msg%\n" > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > [a-z,\.]*--end%_az.log" > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > from the documentation: "the property replacer will return the part of > > the property text that matches the regular expression" which should be > > " www.mysite.com" > > but i get a file named _az.log > > > > -- > > mr > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- mr From rgerhards at hq.adiscon.com Thu Mar 20 16:09:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 16:09:24 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Should work with fields (much faster). I can't try it out due to relp work, but try: %msg:F,32:2% [32 is USASCII SP, the delimiter here] But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that should work... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:59 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > yes! but actually there is a space at the beginning and hostname can > contain the dash -, numbers, and letters. > > 2008/3/20, Rainer Gerhards : > > Let me try to avoid the regexp (its expensive and I can not debug it > now > > ;)): so you search for the string that is at the start of the msg > and > > delimited by the first space? > > > > > > > > Rainer > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > Can you send me a handful of the logline to play with? Probably > not > > > this > > > > week, but next... > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > > /images/wm001.jpg HTTP/1.1" 304 - > "http://www.mysite.com/webmail.htm" > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > Gecko/20060607" > > > > > > i'm trying to use the regexp but with no success in this way: > > > $template MsgFormat,"%msg%\n" > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > [a-z,\.]*--end%_az.log" > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > from the documentation: "the property replacer will return the > part of > > > the property text that matches the regular expression" which > should be > > > " www.mysite.com" > > > but i get a file named _az.log > > > > > > -- > > > mr > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 16:22:11 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 16:22:11 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Message-ID: wow! %msg:F,32:2% works! i completely misunderstood how Fielf works...i thought the first number was the FromChar and second the ToChar... Thanks a lot!!! 2008/3/20, Rainer Gerhards : > Should work with fields (much faster). I can't try it out due to relp > work, but try: > > %msg:F,32:2% [32 is USASCII SP, the delimiter here] > > But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that > should work... > > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Thursday, March 20, 2008 3:59 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > yes! but actually there is a space at the beginning and hostname can > > contain the dash -, numbers, and letters. > > > > 2008/3/20, Rainer Gerhards : > > > Let me try to avoid the regexp (its expensive and I can not debug it > > now > > > ;)): so you search for the string that is at the start of the msg > > and > > > delimited by the first space? > > > > > > > > > > > > Rainer > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > > Can you send me a handful of the logline to play with? Probably > > not > > > > this > > > > > week, but next... > > > > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > > > /images/wm001.jpg HTTP/1.1" 304 - > > "http://www.mysite.com/webmail.htm" > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > > Gecko/20060607" > > > > > > > > i'm trying to use the regexp but with no success in this way: > > > > $template MsgFormat,"%msg%\n" > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > > [a-z,\.]*--end%_az.log" > > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > > > from the documentation: "the property replacer will return the > > part of > > > > the property text that matches the regular expression" which > > should be > > > > " www.mysite.com" > > > > but i get a file named _az.log > > > > > > > > -- > > > > mr > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > -- > > mr > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- mr From rgerhards at hq.adiscon.com Thu Mar 20 16:39:49 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 16:39:49 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1E@grfint2.intern.adiscon.com> Excellent, good to hear :) Just a side-note: when I am done with relp, I'll hopefully have time to come back to the config format. Then we'll also see much more powerful ways to extract values... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > wow! > %msg:F,32:2% works! > i completely misunderstood how Fielf works...i thought the first > number was the FromChar and second the ToChar... > Thanks a lot!!! > > 2008/3/20, Rainer Gerhards : > > Should work with fields (much faster). I can't try it out due to relp > > work, but try: > > > > %msg:F,32:2% [32 is USASCII SP, the delimiter here] > > > > But maybe %msg:F,32:1% - you need to experiment a bit. In any case, > that > > should work... > > > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > Sent: Thursday, March 20, 2008 3:59 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > yes! but actually there is a space at the beginning and hostname > can > > > contain the dash -, numbers, and letters. > > > > > > 2008/3/20, Rainer Gerhards : > > > > Let me try to avoid the regexp (its expensive and I can not > debug it > > > now > > > > ;)): so you search for the string that is at the start of the > msg > > > and > > > > delimited by the first space? > > > > > > > > > > > > > > > > Rainer > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > > > Can you send me a handful of the logline to play with? > Probably > > > not > > > > > this > > > > > > week, but next... > > > > > > > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] > "GET > > > > > /images/wm001.jpg HTTP/1.1" 304 - > > > "http://www.mysite.com/webmail.htm" > > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > > > Gecko/20060607" > > > > > > > > > > i'm trying to use the regexp but with no success in this way: > > > > > $template MsgFormat,"%msg%\n" > > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > > > [a-z,\.]*--end%_az.log" > > > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > > > > > from the documentation: "the property replacer will return > the > > > part of > > > > > the property text that matches the regular expression" which > > > should be > > > > > " www.mysite.com" > > > > > but i get a file named _az.log > > > > > > > > > > -- > > > > > mr > > > > > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > > > -- > > > mr > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 06:47:10 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 07:47:10 +0200 Subject: [rsyslog] Property-Based Filters Message-ID: <47E49D5E.5060408@pengooin.net> Hi, I'm having trouble setting up some properties-based filters. I want to see if some property is equal to "" (empty). I'm doing this: :FROMHOST, isequal, "" *.info;mail.none;authpriv.none;cron.none ?messages-per-host4 But it still matches everything it should not. Am i doing something wrong? Thanks, Radu From rgerhards at hq.adiscon.com Sat Mar 22 10:26:47 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 10:26:47 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E49D5E.5060408@pengooin.net> References: <47E49D5E.5060408@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> Hi Radu, I will look into the issue, smells like a bug. But FROMHOST can never be empty... Is it just a sample? If so, which property you are looking it (I wonder what may be empty, thus the question...). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:47 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Property-Based Filters > > Hi, > > I'm having trouble setting up some properties-based filters. I want to > see if some property is equal to "" (empty). > I'm doing this: > :FROMHOST, isequal, "" > *.info;mail.none;authpriv.none;cron.none ?messages-per- > host4 > > But it still matches everything it should not. Am i doing something > wrong? > > Thanks, > Radu > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:03:28 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:03:28 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> References: <47E49D5E.5060408@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> Message-ID: <47E4F590.8010306@pengooin.net> Hi, I'm using stable. I'm trying to create some central log machine. I made some tests .. and it seems that when i start rsyslog on the central logging machine, it creates some messages regarding rsyslog version, for those messages FROMHOST is empty. If you want i can provide you with full rsyslog.conf. Please test and reply. Thanks, Radu Gheorghiu Rainer Gerhards wrote: > Hi Radu, > > I will look into the issue, smells like a bug. But FROMHOST can never be > empty... Is it just a sample? If so, which property you are looking it > (I wonder what may be empty, thus the question...). > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:47 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] Property-Based Filters >> >> Hi, >> >> I'm having trouble setting up some properties-based filters. I want to >> see if some property is equal to "" (empty). >> I'm doing this: >> :FROMHOST, isequal, "" >> *.info;mail.none;authpriv.none;cron.none ?messages-per- >> host4 >> >> But it still matches everything it should not. Am i doing something >> wrong? >> >> Thanks, >> Radu >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 13:06:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 13:06:00 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E4F590.8010306@pengooin.net> References: <47E49D5E.5060408@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> <47E4F590.8010306@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> > Hi, > I'm using stable. I'm trying to create some central log machine. > I made some tests .. and it seems that when i start rsyslog on the > central logging machine, it creates some messages regarding rsyslog > version, for those messages FROMHOST is empty. That *is* a bug. FROMHOST should not be empty. And now that I know it'll probably won't be empty in the future ;) > If you want i can > provide > you with full rsyslog.conf. Please test and reply. So you want to filter out the rsyslog startup and shutdown messages? Please provide me a few samples of what the messages look in your log files. Thanks, Rainer > > Thanks, > Radu Gheorghiu > > Rainer Gerhards wrote: > > Hi Radu, > > > > I will look into the issue, smells like a bug. But FROMHOST can never > be > > empty... Is it just a sample? If so, which property you are looking > it > > (I wonder what may be empty, thus the question...). > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 6:47 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] Property-Based Filters > >> > >> Hi, > >> > >> I'm having trouble setting up some properties-based filters. I want > to > >> see if some property is equal to "" (empty). > >> I'm doing this: > >> :FROMHOST, isequal, "" > >> *.info;mail.none;authpriv.none;cron.none ?messages- > per- > >> host4 > >> > >> But it still matches everything it should not. Am i doing something > >> wrong? > >> > >> Thanks, > >> Radu > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:21:44 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:21:44 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> References: <47E49D5E.5060408@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> <47E4F590.8010306@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> Message-ID: <47E4F9D8.1040004@pengooin.net> Hi, Well as is said i needed a central log solution. So i did this: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" and for testing purposes (and debug): $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" And here is the rest: *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName i look in /var/log/hosts: 2008 clog-he-de messages clog-he-de is the localhost .. and 2008 + messages is created because FROMHOST is empty. proof: cat messages TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart note the "dd" after "TheHost:" . i put it there to be sure there's nothing between the two "d". Thanks, Radu Gheorghiu Rainer Gerhards wrote: >> Hi, >> I'm using stable. I'm trying to create some central log machine. >> I made some tests .. and it seems that when i start rsyslog on the >> central logging machine, it creates some messages regarding rsyslog >> version, for those messages FROMHOST is empty. >> > > That *is* a bug. FROMHOST should not be empty. And now that I know it'll > probably won't be empty in the future ;) > > >> If you want i can >> provide >> you with full rsyslog.conf. Please test and reply. >> > > So you want to filter out the rsyslog startup and shutdown messages? > Please provide me a few samples of what the messages look in your log > files. > > Thanks, > Rainer > > >> Thanks, >> Radu Gheorghiu >> >> Rainer Gerhards wrote: >> >>> Hi Radu, >>> >>> I will look into the issue, smells like a bug. But FROMHOST can >>> > never > >> be >> >>> empty... Is it just a sample? If so, which property you are looking >>> >> it >> >>> (I wonder what may be empty, thus the question...). >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 6:47 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Property-Based Filters >>>> >>>> Hi, >>>> >>>> I'm having trouble setting up some properties-based filters. I want >>>> >> to >> >>>> see if some property is equal to "" (empty). >>>> I'm doing this: >>>> :FROMHOST, isequal, "" >>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>> >> per- >> >>>> host4 >>>> >>>> But it still matches everything it should not. Am i doing something >>>> wrong? >>>> >>>> Thanks, >>>> Radu >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 13:46:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 13:46:51 +0100 Subject: [rsyslog] Property-Based Filters Message-ID: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> Ah, ok, so the problem actually is that fromhost is empty... ----- Urspr?ngliche Nachricht ----- Von: "Radu Gheorghiu" An: "rsyslog-users" Gesendet: 22.03.08 13:21 Betreff: Re: [rsyslog] Property-Based Filters Hi, Well as is said i needed a central log solution. So i did this: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" and for testing purposes (and debug): $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" And here is the rest: *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName i look in /var/log/hosts: 2008 clog-he-de messages clog-he-de is the localhost .. and 2008 + messages is created because FROMHOST is empty. proof: cat messages TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart note the "dd" after "TheHost:" . i put it there to be sure there's nothing between the two "d". Thanks, Radu Gheorghiu Rainer Gerhards wrote: >> Hi, >> I'm using stable. I'm trying to create some central log machine. >> I made some tests .. and it seems that when i start rsyslog on the >> central logging machine, it creates some messages regarding rsyslog >> version, for those messages FROMHOST is empty. >> > > That *is* a bug. FROMHOST should not be empty. And now that I know it'll > probably won't be empty in the future ;) > > >> If you want i can >> provide >> you with full rsyslog.conf. Please test and reply. >> > > So you want to filter out the rsyslog startup and shutdown messages? > Please provide me a few samples of what the messages look in your log > files. > > Thanks, > Rainer > > >> Thanks, >> Radu Gheorghiu >> >> Rainer Gerhards wrote: >> >>> Hi Radu, >>> >>> I will look into the issue, smells like a bug. But FROMHOST can >>> > never > >> be >> >>> empty... Is it just a sample? If so, which property you are looking >>> >> it >> >>> (I wonder what may be empty, thus the question...). >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 6:47 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Property-Based Filters >>>> >>>> Hi, >>>> >>>> I'm having trouble setting up some properties-based filters. I want >>>> >> to >> >>>> see if some property is equal to "" (empty). >>>> I'm doing this: >>>> :FROMHOST, isequal, "" >>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>> >> per- >> >>>> host4 >>>> >>>> But it still matches everything it should not. Am i doing something >>>> wrong? >>>> >>>> Thanks, >>>> Radu >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:54:43 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:54:43 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> Message-ID: <47E50193.4020904@pengooin.net> Yes, it is. And that wouldn't be a problem if property based filter would work fine. I tried to filter messages with FROMHOST empty and use HOSTNAME instead. It worked for localhost generated messages, but didn't work for messages coming from other host on the network. Here's sample: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" $template messages-per-host3,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host4,"/var/log/hosts/%HOSTNAME%/messages" $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" # first we check if FROMHOST is empty . if it is empty .. we use HOSTNAME :FROMHOST, isequal, "" *.info;mail.none;authpriv.none;cron.none ?messages-per-host4;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host3;MyTemplateName # we drop the messages with FROMHOST empty, and we log everything else based on FROMHOST . :FROMHOST, isequal, "" ~ *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? Well.. it looks like there are several messages that don't contain the HOSTNAME field and this breaks everything. I'm not sure these are all bugs. They may be some result of some human error of some kind (my error). Waiting for your confirmation of the above, Radu Gheorghiu Rainer Gerhards wrote: > Ah, ok, so the problem actually is that fromhost is empty... > > ----- Urspr?ngliche Nachricht ----- > Von: "Radu Gheorghiu" > An: "rsyslog-users" > Gesendet: 22.03.08 13:21 > Betreff: Re: [rsyslog] Property-Based Filters > > Hi, > > Well as is said i needed a central log solution. > So i did this: > > $template > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > > and for testing purposes (and debug): > > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: > %msg%\n" > > And here is the rest: > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host1;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host2;MyTemplateName > > i look in /var/log/hosts: > 2008 clog-he-de messages > clog-he-de is the localhost .. > and 2008 + messages is created because FROMHOST is empty. > proof: > > cat messages > TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" > x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo > udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart > > note the "dd" after "TheHost:" . i put it there to be sure there's > nothing between the two "d". > > Thanks, > Radu Gheorghiu > > Rainer Gerhards wrote: > >>> Hi, >>> I'm using stable. I'm trying to create some central log machine. >>> I made some tests .. and it seems that when i start rsyslog on the >>> central logging machine, it creates some messages regarding rsyslog >>> version, for those messages FROMHOST is empty. >>> >>> >> That *is* a bug. FROMHOST should not be empty. And now that I know it'll >> probably won't be empty in the future ;) >> >> >> >>> If you want i can >>> provide >>> you with full rsyslog.conf. Please test and reply. >>> >>> >> So you want to filter out the rsyslog startup and shutdown messages? >> Please provide me a few samples of what the messages look in your log >> files. >> >> Thanks, >> Rainer >> >> >> >>> Thanks, >>> Radu Gheorghiu >>> >>> Rainer Gerhards wrote: >>> >>> >>>> Hi Radu, >>>> >>>> I will look into the issue, smells like a bug. But FROMHOST can >>>> >>>> >> never >> >> >>> be >>> >>> >>>> empty... Is it just a sample? If so, which property you are looking >>>> >>>> >>> it >>> >>> >>>> (I wonder what may be empty, thus the question...). >>>> >>>> Rainer >>>> >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>>> Sent: Saturday, March 22, 2008 6:47 AM >>>>> To: rsyslog at lists.adiscon.com >>>>> Subject: [rsyslog] Property-Based Filters >>>>> >>>>> Hi, >>>>> >>>>> I'm having trouble setting up some properties-based filters. I want >>>>> >>>>> >>> to >>> >>> >>>>> see if some property is equal to "" (empty). >>>>> I'm doing this: >>>>> :FROMHOST, isequal, "" >>>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>>> >>>>> >>> per- >>> >>> >>>>> host4 >>>>> >>>>> But it still matches everything it should not. Am i doing something >>>>> wrong? >>>>> >>>>> Thanks, >>>>> Radu >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Sat Mar 22 14:07:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 14:07:46 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50193.4020904@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2A@grfint2.intern.adiscon.com> Ahh... That sample is most helpful. I think there is also a misunderstanding. I couldn't run a lab yet and will probably not before after easter, but (read inline below) ... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 1:55 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Yes, it is. > And that wouldn't be a problem if property based filter would work > fine. > > I tried to filter messages with FROMHOST empty and use HOSTNAME > instead. > It worked for localhost generated messages, but didn't work for > messages > coming from other host on the network. Here's sample: > > $template > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > $template > messages-per-host3,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > $template messages-per-host4,"/var/log/hosts/%HOSTNAME%/messages" > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: > %msg%\n" > > # first we check if FROMHOST is empty . if it is empty .. we use > HOSTNAME > :FROMHOST, isequal, "" Property based filters are just regular filters. So you need to specify what shall happen when the filter matches. In the above line, there is no action (sorry, looks like I overlooked that in the first message you send). So to discard these message, you'd need to do: :FROMHOST, isequal, "" ~ Filters (in v2) do NOT combine, except for the BSD-style filters. This is a feature of v3. > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host4;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host3;MyTemplateName > # we drop the messages with FROMHOST empty, and we log everything else > based on FROMHOST . > :FROMHOST, isequal, "" ~ That, of course, should work... > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host1;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host2;MyTemplateName > > Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? > Well.. it looks like there are several messages that don't contain the > HOSTNAME field and this breaks everything. > > I'm not sure these are all bugs. At may very well be. I suspect that they all have the same root cause, and that is that some message properties are not correctly being set for internally-generated messages. So one trouble spot with multiple problems resulting from it. Could you do me a favor and run rsyslogd with -d -n options interactively and send me the resulting debug log? Rainer > They may be some result of some human > error of some kind (my error). > > Waiting for your confirmation of the above, > Radu Gheorghiu > > Rainer Gerhards wrote: > > Ah, ok, so the problem actually is that fromhost is empty... > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "Radu Gheorghiu" > > An: "rsyslog-users" > > Gesendet: 22.03.08 13:21 > > Betreff: Re: [rsyslog] Property-Based Filters > > > > Hi, > > > > Well as is said i needed a central log solution. > > So i did this: > > > > $template > > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > > > > and for testing purposes (and debug): > > > > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% > TheMsg: > > %msg%\n" > > > > And here is the rest: > > *.info;mail.none;authpriv.none;cron.none > > ?messages-per-host1;MyTemplateName > > *.info;mail.none;authpriv.none;cron.none > > ?messages-per-host2;MyTemplateName > > > > i look in /var/log/hosts: > > 2008 clog-he-de messages > > clog-he-de is the localhost .. > > and 2008 + messages is created because FROMHOST is empty. > > proof: > > > > cat messages > > TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" > > x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo > > udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] > restart > > > > note the "dd" after "TheHost:" . i put it there to be sure there's > > nothing between the two "d". > > > > Thanks, > > Radu Gheorghiu > > > > Rainer Gerhards wrote: > > > >>> Hi, > >>> I'm using stable. I'm trying to create some central log machine. > >>> I made some tests .. and it seems that when i start rsyslog on the > >>> central logging machine, it creates some messages regarding rsyslog > >>> version, for those messages FROMHOST is empty. > >>> > >>> > >> That *is* a bug. FROMHOST should not be empty. And now that I know > it'll > >> probably won't be empty in the future ;) > >> > >> > >> > >>> If you want i can > >>> provide > >>> you with full rsyslog.conf. Please test and reply. > >>> > >>> > >> So you want to filter out the rsyslog startup and shutdown messages? > >> Please provide me a few samples of what the messages look in your > log > >> files. > >> > >> Thanks, > >> Rainer > >> > >> > >> > >>> Thanks, > >>> Radu Gheorghiu > >>> > >>> Rainer Gerhards wrote: > >>> > >>> > >>>> Hi Radu, > >>>> > >>>> I will look into the issue, smells like a bug. But FROMHOST can > >>>> > >>>> > >> never > >> > >> > >>> be > >>> > >>> > >>>> empty... Is it just a sample? If so, which property you are > looking > >>>> > >>>> > >>> it > >>> > >>> > >>>> (I wonder what may be empty, thus the question...). > >>>> > >>>> Rainer > >>>> > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >>>>> Sent: Saturday, March 22, 2008 6:47 AM > >>>>> To: rsyslog at lists.adiscon.com > >>>>> Subject: [rsyslog] Property-Based Filters > >>>>> > >>>>> Hi, > >>>>> > >>>>> I'm having trouble setting up some properties-based filters. I > want > >>>>> > >>>>> > >>> to > >>> > >>> > >>>>> see if some property is equal to "" (empty). > >>>>> I'm doing this: > >>>>> :FROMHOST, isequal, "" > >>>>> *.info;mail.none;authpriv.none;cron.none > ?messages- > >>>>> > >>>>> > >>> per- > >>> > >>> > >>>>> host4 > >>>>> > >>>>> But it still matches everything it should not. Am i doing > something > >>>>> wrong? > >>>>> > >>>>> Thanks, > >>>>> Radu > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> > >>>>> > >>>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 22 14:14:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 14:14:55 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50193.4020904@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> > Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? > Well.. it looks like there are several messages that don't contain the > HOSTNAME field and this breaks everything. I had a quick look at the code and I can't see why HOSTNAME should not be filled. Could you provide me a sample of such a message? That would be most interesting. I think I've also identified why FROMHOST is missing, but I'd like to evaluate it a bit more. If you can provide a sample of missing hostnames, it would very helpful to have a debug log from such a message being received and processed - but the most important thing is to see what the source of this all is... Thanks, Rainer From radu at pengooin.net Sat Mar 22 14:47:39 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 15:47:39 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> Message-ID: <47E50DFB.40505@pengooin.net> I don't think that the HOSTNAME problem is a rsyslog problem. I think that the remote host is not sending the HOSTNAME. Remote host is using the classic syslog . and it is configured like this: *.* @central-logger on the central logger .. i modify my template for debug: $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% TheMsg: %msg%\n" and in the logs: TheHost:dlastd 5 TheMsg: repeated 8 times Seems like the message had no hostname field. and rsyslog thinks "last" is the hostname. Radu From rgerhards at hq.adiscon.com Sat Mar 22 18:09:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:09:04 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50DFB.40505@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> Well... Rsyslog tries all kind of things to get hold of the real host. So if you could send me a copy of the one causing problems, I may (may ;)) be able to do something against it. HOSTNAME should always contain something usable, but as you say ... it depends ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 2:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > I don't think that the HOSTNAME problem is a rsyslog problem. > I think that the remote host is not sending the HOSTNAME. > Remote host is using the classic syslog . > and it is configured like this: > *.* @central-logger > > on the central logger .. i modify my template for debug: > $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% TheMsg: > %msg%\n" > > and in the logs: > TheHost:dlastd 5 TheMsg: repeated 8 times > > Seems like the message had no hostname field. and rsyslog thinks > "last" > is the hostname. > > Radu > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 18:13:33 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:13:33 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> Message-ID: <47E53E3D.90606@pengooin.net> Hmm i think i don't understand . Haven't you received the file that i sent you? Or did i miss something? Radu Rainer Gerhards wrote: > Well... Rsyslog tries all kind of things to get hold of the real host. > So if you could send me a copy of the one causing problems, I may (may > ;)) be able to do something against it. HOSTNAME should always contain > something usable, but as you say ... it depends ;) > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 2:48 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> I don't think that the HOSTNAME problem is a rsyslog problem. >> I think that the remote host is not sending the HOSTNAME. >> Remote host is using the classic syslog . >> and it is configured like this: >> *.* >> > @central-logger > >> on the central logger .. i modify my template for debug: >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >> > TheMsg: > >> %msg%\n" >> >> and in the logs: >> TheHost:dlastd 5 TheMsg: repeated 8 times >> >> Seems like the message had no hostname field. and rsyslog thinks >> "last" >> is the hostname. >> >> Radu >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 18:17:27 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:17:27 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E53E3D.90606@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> Replying as I can ;) I've not been able to go in-depth through the file, but I ... doh ;) I guess it's all sitting in there. Lol - sorry. As an apology, here is the FROMHOST bugfix ;) http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2. 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Hmm i think i don't understand . Haven't you received the file that i > sent you? Or did i miss something? > > Radu > > Rainer Gerhards wrote: > > Well... Rsyslog tries all kind of things to get hold of the real > host. > > So if you could send me a copy of the one causing problems, I may > (may > > ;)) be able to do something against it. HOSTNAME should always > contain > > something usable, but as you say ... it depends ;) > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 2:48 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> I don't think that the HOSTNAME problem is a rsyslog problem. > >> I think that the remote host is not sending the HOSTNAME. > >> Remote host is using the classic syslog . > >> and it is configured like this: > >> *.* > >> > > @central-logger > > > >> on the central logger .. i modify my template for debug: > >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >> > > TheMsg: > > > >> %msg%\n" > >> > >> and in the logs: > >> TheHost:dlastd 5 TheMsg: repeated 8 times > >> > >> Seems like the message had no hostname field. and rsyslog thinks > >> "last" > >> is the hostname. > >> > >> Radu > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 22 18:25:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:25:19 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E53E3D.90606@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> Ah, I've now reviewed it, but there is no message in it from a host that does not send a HOSTNAME (The last message... case). What syslogd are your running on the clients? Sysklogd? And which version? I am asking because I could check the code and see what exactly it generates. I already begin to get some bad feelings about what it sends ;) What I see is rsyslog's bug (the one just fixed) and an error message telling you that rsyslog is discarding a selector line because of no actions. That is the one where you had use the filter but without actions. Filters work only in front of actions in v2. So if you don't provide an action, nothing happens except that startup error message. Please note that the doc just talks about the property based filter. But it doesn't mean you can use it without an action. Filters only work with actions. In v3, things are already a bit different and will be much more different soon. V3 offers full expression support, so you can do Boolean operations inside filters. Also, v3 will be scriptable. HTH, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Hmm i think i don't understand . Haven't you received the file that i > sent you? Or did i miss something? > > Radu > > Rainer Gerhards wrote: > > Well... Rsyslog tries all kind of things to get hold of the real > host. > > So if you could send me a copy of the one causing problems, I may > (may > > ;)) be able to do something against it. HOSTNAME should always > contain > > something usable, but as you say ... it depends ;) > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 2:48 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> I don't think that the HOSTNAME problem is a rsyslog problem. > >> I think that the remote host is not sending the HOSTNAME. > >> Remote host is using the classic syslog . > >> and it is configured like this: > >> *.* > >> > > @central-logger > > > >> on the central logger .. i modify my template for debug: > >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >> > > TheMsg: > > > >> %msg%\n" > >> > >> and in the logs: > >> TheHost:dlastd 5 TheMsg: repeated 8 times > >> > >> Seems like the message had no hostname field. and rsyslog thinks > >> "last" > >> is the hostname. > >> > >> Radu > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 18:32:42 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:32:42 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> Message-ID: <47E542BA.4040503@pengooin.net> Ok thanks :) I only wanted to know if you received it . But if you also fixed it, it's even better :) Rainer Gerhards wrote: > Replying as I can ;) I've not been able to go in-depth through the file, > but I ... doh ;) I guess it's all sitting in there. Lol - sorry. > > As an apology, here is the FROMHOST bugfix ;) > > http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2. > 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> Hmm i think i don't understand . Haven't you received the file that i >> sent you? Or did i miss something? >> >> Radu >> >> Rainer Gerhards wrote: >> >>> Well... Rsyslog tries all kind of things to get hold of the real >>> >> host. >> >>> So if you could send me a copy of the one causing problems, I may >>> >> (may >> >>> ;)) be able to do something against it. HOSTNAME should always >>> >> contain >> >>> something usable, but as you say ... it depends ;) >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 2:48 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Property-Based Filters >>>> >>>> I don't think that the HOSTNAME problem is a rsyslog problem. >>>> I think that the remote host is not sending the HOSTNAME. >>>> Remote host is using the classic syslog . >>>> and it is configured like this: >>>> *.* >>>> >>>> >>> @central-logger >>> >>> >>>> on the central logger .. i modify my template for debug: >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >>>> >>>> >>> TheMsg: >>> >>> >>>> %msg%\n" >>>> >>>> and in the logs: >>>> TheHost:dlastd 5 TheMsg: repeated 8 times >>>> >>>> Seems like the message had no hostname field. and rsyslog thinks >>>> "last" >>>> is the hostname. >>>> >>>> Radu >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From radu at pengooin.net Sat Mar 22 18:34:09 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:34:09 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> Message-ID: <47E54311.1080502@pengooin.net> I will prepare a new file + details, in the next 30mins :) Rainer Gerhards wrote: > Ah, I've now reviewed it, but there is no message in it from a host that > does not send a HOSTNAME (The last message... case). What syslogd are > your running on the clients? Sysklogd? And which version? I am asking > because I could check the code and see what exactly it generates. I > already begin to get some bad feelings about what it sends ;) > > What I see is rsyslog's bug (the one just fixed) and an error message > telling you that rsyslog is discarding a selector line because of no > actions. That is the one where you had use the filter but without > actions. > > Filters work only in front of actions in v2. So if you don't provide an > action, nothing happens except that startup error message. Please note > that the doc just talks about the property based filter. But it doesn't > mean you can use it without an action. Filters only work with actions. > In v3, things are already a bit different and will be much more > different soon. V3 offers full expression support, so you can do > Boolean operations inside filters. Also, v3 will be scriptable. > > HTH, > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> Hmm i think i don't understand . Haven't you received the file that i >> sent you? Or did i miss something? >> >> Radu >> >> Rainer Gerhards wrote: >> >>> Well... Rsyslog tries all kind of things to get hold of the real >>> >> host. >> >>> So if you could send me a copy of the one causing problems, I may >>> >> (may >> >>> ;)) be able to do something against it. HOSTNAME should always >>> >> contain >> >>> something usable, but as you say ... it depends ;) >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 2:48 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Property-Based Filters >>>> >>>> I don't think that the HOSTNAME problem is a rsyslog problem. >>>> I think that the remote host is not sending the HOSTNAME. >>>> Remote host is using the classic syslog . >>>> and it is configured like this: >>>> *.* >>>> >>>> >>> @central-logger >>> >>> >>>> on the central logger .. i modify my template for debug: >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >>>> >>>> >>> TheMsg: >>> >>> >>>> %msg%\n" >>>> >>>> and in the logs: >>>> TheHost:dlastd 5 TheMsg: repeated 8 times >>>> >>>> Seems like the message had no hostname field. and rsyslog thinks >>>> "last" >>>> is the hostname. >>>> >>>> Radu >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 18:34:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:34:43 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E542BA.4040503@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> <47E542BA.4040503@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> No problem at all. The fix was rather trivial once I knew what I was looking for. Please let me know if it works for you. The original bug report - property based filters - however, was no bug, but a misunderstanding on how they work. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Ok thanks :) I only wanted to know if you received it . But if you also > fixed it, it's even better :) > > Rainer Gerhards wrote: > > Replying as I can ;) I've not been able to go in-depth through the > file, > > but I ... doh ;) I guess it's all sitting in there. Lol - sorry. > > > > As an apology, here is the FROMHOST bugfix ;) > > > > > http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2 > . > > 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 6:14 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> Hmm i think i don't understand . Haven't you received the file that > i > >> sent you? Or did i miss something? > >> > >> Radu > >> > >> Rainer Gerhards wrote: > >> > >>> Well... Rsyslog tries all kind of things to get hold of the real > >>> > >> host. > >> > >>> So if you could send me a copy of the one causing problems, I may > >>> > >> (may > >> > >>> ;)) be able to do something against it. HOSTNAME should always > >>> > >> contain > >> > >>> something usable, but as you say ... it depends ;) > >>> > >>> Rainer > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >>>> Sent: Saturday, March 22, 2008 2:48 PM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] Property-Based Filters > >>>> > >>>> I don't think that the HOSTNAME problem is a rsyslog problem. > >>>> I think that the remote host is not sending the HOSTNAME. > >>>> Remote host is using the classic syslog . > >>>> and it is configured like this: > >>>> *.* > >>>> > >>>> > >>> @central-logger > >>> > >>> > >>>> on the central logger .. i modify my template for debug: > >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >>>> > >>>> > >>> TheMsg: > >>> > >>> > >>>> %msg%\n" > >>>> > >>>> and in the logs: > >>>> TheHost:dlastd 5 TheMsg: repeated 8 times > >>>> > >>>> Seems like the message had no hostname field. and rsyslog thinks > >>>> "last" > >>>> is the hostname. > >>>> > >>>> Radu > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 19:41:38 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 20:41:38 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> <47E542BA.4040503@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> Message-ID: <47E552E2.3040907@pengooin.net> I just applied the patch and the problem with the empty HOSTNAME seems to be solved now. Congrats :) And thank you for your effort. Radu Rainer Gerhards wrote: > No problem at all. The fix was rather trivial once I knew what I was > looking for. Please let me know if it works for you. The original bug > report - property based filters - however, was no bug, but a > misunderstanding on how they work. > > Rainer > > From rgerhards at hq.adiscon.com Mon Mar 24 22:25:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Mar 2008 22:25:55 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803181523.52321.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> Hi all, I now have a use case where the "last message..." prevents sources from flooding syslog. But now on to a different use case: does someone actually use this feature to reduce the size of files or network traffic? Except, of course, in cases where there is a message flood due to either attact or a misbehaving source (these need to be treated differentely - I am right now after the *destination* message compression...). Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > Sent: Tuesday, March 18, 2008 3:24 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message > logged n times"... > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > The only arguments for keeping the feature that I got on my > lug was the > > preservartion of disk/network IO. > > > > I think to prevent DOS attacks is a valid argument but as > you said can > > be easily circumvented by randomizing messages. > I'm afraid it's not true in all cases. What if you do DOS > attach not directly > to do rsyslog, but via other daemon. In situation when you > can't send message > directly to syslog, but you can make some daemon generate > message for you. > This message would be probably always the same content. > > > To safeguard against dos attacks you could have a monitor > that monitors > > for extra ordinary amount of traffic and then generate a snmp trap. > > Whether that should be a rsyslog plugin or part of other software is > > open to debate. > > > > Regards > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: 18 March 2008 11:42 > > > To: rsyslog-users > > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > > > times"... > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > I have asked the question on a local lug to get some more > > > > opinions. > > > > > I > > > > > > > > will post the results if any here. > > > > > > > > I did the same. :) > > > > > > > > The only arguments against removal I have received > until now were: > > > > > > > > 1. DoS (but nobody explained how) > > > > > > I think this is along the lines of making it easy to flood the log > > > > with > > > > > similar messages. If that's the case, I think it is a (too) weak > > > argument because an attacker could easily include a random pattern > > > inside the message. > > > > > > HOWEVER, the current logic indeed prevents flooding the > log from equal > > > messages, e.g. if a process runs wild. > > > > > > > 2. cleanness of logs (relative) > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > Excellent. > > > > > > Maybe there are also alternatives to the current > behavior, something > > > > in > > > > > between. Right now, the core engine does this, so the reduction > > > capability is inherent to every action (except if an action > > > > specifically > > > > > forbids it, because it can not intelligently handle it - > e.g. database > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > alternative would be to remove it from the core engine and move it > > > > into > > > > > the *file write* output plugin (omfile). This has its > subtleties, too, > > > so I don't like to take that approach lightly. Plus, it than means > > > > that > > > > > the network may become spammed. I think one core requirement is to > > > > find > > > > > people who actually *intentionally* use this feature and > ask for their > > > reasoning. That will probably be the best way to tell us > if its really > > > needed. And, of course, we need to weigh the negative > effects of it > > > against these con-points. > > > > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From pvrabec at redhat.com Tue Mar 25 10:46:55 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 25 Mar 2008 10:46:55 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> Message-ID: <200803251046.55919.pvrabec@redhat.com> Hi folks, I talked with some people, and nobody is happy with removing this thing from rsyslog. Won't be possible to move it to separate plug-in? On Monday 24 March 2008 10:25:55 pm Rainer Gerhards wrote: > Hi all, > > I now have a use case where the "last message..." prevents sources from > flooding syslog. But now on to a different use case: does someone > actually use this feature to reduce the size of files or network > traffic? Except, of course, in cases where there is a message flood due > to either attact or a misbehaving source (these need to be treated > differentely - I am right now after the *destination* message > compression...). > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > > Sent: Tuesday, March 18, 2008 3:24 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message > > logged n times"... > > > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > > The only arguments for keeping the feature that I got on my > > > > lug was the > > > > > preservartion of disk/network IO. > > > > > > I think to prevent DOS attacks is a valid argument but as > > > > you said can > > > > > be easily circumvented by randomizing messages. > > > > I'm afraid it's not true in all cases. What if you do DOS > > attach not directly > > to do rsyslog, but via other daemon. In situation when you > > can't send message > > directly to syslog, but you can make some daemon generate > > message for you. > > This message would be probably always the same content. > > > > > To safeguard against dos attacks you could have a monitor > > > > that monitors > > > > > for extra ordinary amount of traffic and then generate a snmp trap. > > > Whether that should be a rsyslog plugin or part of other software is > > > open to debate. > > > > > > Regards > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: 18 March 2008 11:42 > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > > > > > times"... > > > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > > > I have asked the question on a local lug to get some more > > > > > > opinions. > > > > > > > I > > > > > > > > > > will post the results if any here. > > > > > > > > > > I did the same. :) > > > > > > > > > > The only arguments against removal I have received > > > > until now were: > > > > > 1. DoS (but nobody explained how) > > > > > > > > I think this is along the lines of making it easy to flood the log > > > > > > with > > > > > > > similar messages. If that's the case, I think it is a (too) weak > > > > argument because an attacker could easily include a random pattern > > > > inside the message. > > > > > > > > HOWEVER, the current logic indeed prevents flooding the > > > > log from equal > > > > > > messages, e.g. if a process runs wild. > > > > > > > > > 2. cleanness of logs (relative) > > > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > > > Excellent. > > > > > > > > Maybe there are also alternatives to the current > > > > behavior, something > > > > > in > > > > > > > between. Right now, the core engine does this, so the reduction > > > > capability is inherent to every action (except if an action > > > > > > specifically > > > > > > > forbids it, because it can not intelligently handle it - > > > > e.g. database > > > > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > > alternative would be to remove it from the core engine and move it > > > > > > into > > > > > > > the *file write* output plugin (omfile). This has its > > > > subtleties, too, > > > > > > so I don't like to take that approach lightly. Plus, it than means > > > > > > that > > > > > > > the network may become spammed. I think one core requirement is to > > > > > > find > > > > > > > people who actually *intentionally* use this feature and > > > > ask for their > > > > > > reasoning. That will probably be the best way to tell us > > > > if its really > > > > > > needed. And, of course, we need to weigh the negative > > > > effects of it > > > > > > against these con-points. > > > > > > > > Rainer > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 25 10:52:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 10:52:29 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803251046.55919.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> <200803251046.55919.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C3D@grfint2.intern.adiscon.com> Pluging... well, that depends on the use cases. I am trying to identify the actual use cases, so far I found the one with flooding/attack. The problem is that the current logic can not be moved to a plugin because it is right at the most inner workings of the engine. If we can identify the actual use cases, things will probably be much easier to work with. For example, I currently think that I can build an optional interface that enables flood-controlling inputs and reducing messages if some condition is given there. Note, however, that this is different from current logic, where the flood control only applies to some outputs, because the output decides if it can work with the repeated lines. The bottom line is that I think there are some valid use cases, we just don't know about them. The current implementation/idea is soooo bad because we don't know what we really need to do. So identifying the actual use cases is the priority and then doing a good design that covers them. I'll probably follow advise from the loganalysis mailing list and change the default to not do the "last message..." processing. Then, see who complains and if nobody does drop the feature. It would also be extremely useful to know *why* people are unhappy (uses cases ;)), instead of just knowing they are... Can you shed some light on that? Thanks for all your help, Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Tuesday, March 25, 2008 10:47 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] Feedback requested "Last message logged n > times"... > > Hi folks, > > I talked with some people, and nobody is happy with removing this thing > from > rsyslog. Won't be possible to move it to separate plug-in? > > On Monday 24 March 2008 10:25:55 pm Rainer Gerhards wrote: > > Hi all, > > > > I now have a use case where the "last message..." prevents sources > from > > flooding syslog. But now on to a different use case: does someone > > actually use this feature to reduce the size of files or network > > traffic? Except, of course, in cases where there is a message flood > due > > to either attact or a misbehaving source (these need to be treated > > differentely - I am right now after the *destination* message > > compression...). > > > > Thanks, > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter > Vrabec > > > Sent: Tuesday, March 18, 2008 3:24 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] Feedback requested "Last message > > > logged n times"... > > > > > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > > > The only arguments for keeping the feature that I got on my > > > > > > lug was the > > > > > > > preservartion of disk/network IO. > > > > > > > > I think to prevent DOS attacks is a valid argument but as > > > > > > you said can > > > > > > > be easily circumvented by randomizing messages. > > > > > > I'm afraid it's not true in all cases. What if you do DOS > > > attach not directly > > > to do rsyslog, but via other daemon. In situation when you > > > can't send message > > > directly to syslog, but you can make some daemon generate > > > message for you. > > > This message would be probably always the same content. > > > > > > > To safeguard against dos attacks you could have a monitor > > > > > > that monitors > > > > > > > for extra ordinary amount of traffic and then generate a snmp > trap. > > > > Whether that should be a rsyslog plugin or part of other software > is > > > > open to debate. > > > > > > > > Regards > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: 18 March 2008 11:42 > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] Feedback requested "Last message logged > n > > > > > > > > times"... > > > > > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > > > > > I have asked the question on a local lug to get some more > > > > > > > > opinions. > > > > > > > > > I > > > > > > > > > > > > will post the results if any here. > > > > > > > > > > > > I did the same. :) > > > > > > > > > > > > The only arguments against removal I have received > > > > > > until now were: > > > > > > 1. DoS (but nobody explained how) > > > > > > > > > > I think this is along the lines of making it easy to flood the > log > > > > > > > > with > > > > > > > > > similar messages. If that's the case, I think it is a (too) > weak > > > > > argument because an attacker could easily include a random > pattern > > > > > inside the message. > > > > > > > > > > HOWEVER, the current logic indeed prevents flooding the > > > > > > log from equal > > > > > > > > messages, e.g. if a process runs wild. > > > > > > > > > > > 2. cleanness of logs (relative) > > > > > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > > > > > Excellent. > > > > > > > > > > Maybe there are also alternatives to the current > > > > > > behavior, something > > > > > > > in > > > > > > > > > between. Right now, the core engine does this, so the reduction > > > > > capability is inherent to every action (except if an action > > > > > > > > specifically > > > > > > > > > forbids it, because it can not intelligently handle it - > > > > > > e.g. database > > > > > > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > > > alternative would be to remove it from the core engine and move > it > > > > > > > > into > > > > > > > > > the *file write* output plugin (omfile). This has its > > > > > > subtleties, too, > > > > > > > > so I don't like to take that approach lightly. Plus, it than > means > > > > > > > > that > > > > > > > > > the network may become spammed. I think one core requirement is > to > > > > > > > > find > > > > > > > > > people who actually *intentionally* use this feature and > > > > > > ask for their > > > > > > > > reasoning. That will probably be the best way to tell us > > > > > > if its really > > > > > > > > needed. And, of course, we need to weigh the negative > > > > > > effects of it > > > > > > > > against these con-points. > > > > > > > > > > Rainer > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > From friedl at hq.adiscon.com Tue Mar 25 11:31:39 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Tue, 25 Mar 2008 11:31:39 +0100 Subject: [rsyslog] rsyslog 3.12.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C41@grfint2.intern.adiscon.com> Hi all, rsyslog 3.12.4 has been released today. Thanks to a changed default, it now offers much faster log file writing. Also, advanced flow control has now been added to all inputs, giving an even more reliable system. There are some other small enhancements (see change log). There are also a number of bug fixes, the most important one being a fix for a massive memory leak in imklog. It lost several hundred kilobytes on each hup. Rsyslog 3.12.4 is a recommended update for all v3 branch users. Download: http://www.rsyslog.com/Downloads-req-getit-lid-89.phtml Changelog: http://www.rsyslog.com/Article195.phtml As always, feedback is appreciated. Florian Riedl From rgerhards at hq.adiscon.com Tue Mar 25 16:33:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 16:33:55 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> Hi all, a question on the forum (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a bit asking around finally made me think about the possibility of a v3 stable version, something that dangled on my mind for a while now... While it is just three month since v2 stable, we have gone a long way and during that time the new queue engine has much matured. As "development" state is most often a show-stopper for putting code into production, I think about releasing a 3.0.0 stable (based on the most recent 3.12.x version WITHOUT RELP support). >From a support perspective, I'd like to continue support v2 stable and obviously v3 stable and development concurrently. Development would continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug fixes. Some time from now, 3.. would become 3.1.0 stable. At this point, I would drop support/fixing for 3.0. and ask users with problems in the branch to update to 3.1.. Does this sound reasonable? The big problem is that I must be careful not to create too many version which I must support, but at the same time allow new, matured, features to go into broader production. And, of course, 3.0.0 stable would probably be a bit less stable than 2.0.4, but only if you use features that are not present in v2. All in all, the queue engine has received more than a fire drill and is ready for production - though probably not fully trouble free. But we won't see that until it gets more used. Feedback is appreciated. Thanks, Rainer From r.bhatia at ipax.at Tue Mar 25 17:04:10 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Tue, 25 Mar 2008 17:04:10 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> Message-ID: <47E9227A.5010903@ipax.at> Rainer Gerhards wrote: > Hi all, > > a question on the forum > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a bit > asking around finally made me think about the possibility of a v3 stable > version, something that dangled on my mind for a while now... While it > is just three month since v2 stable, we have gone a long way and during > that time the new queue engine has much matured. As "development" state > is most often a show-stopper for putting code into production, I think > about releasing a 3.0.0 stable (based on the most recent 3.12.x version > WITHOUT RELP support). > >>From a support perspective, I'd like to continue support v2 stable and > obviously v3 stable and development concurrently. Development would > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > fixes. Some time from now, 3.. would become 3.1.0 > stable. At this point, I would drop support/fixing for 3.0. > and ask users with problems in the branch to update to 3.1.. i guess this will cause serious headake because people tend to think like this: 3.0.x is less mature/was released before 3.1.x is less mature/was released before 3.12.x is less mature/was released before 3.13.x i guess there should be a deliberate versioning scheme. examples are: - versioning similar to the linux kernel - release 3.12.x as 3.12.x-stable and continue development without this tag. - something else but i guess its no good to jump from 3.12.x back to 3.0.x and from a future revision such as 3.40.x back to 3.1.x cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Tue Mar 25 17:06:52 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 17:06:52 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <47E9227A.5010903@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> Hi Raoul, OK, good feedback on the version number scheme - it has been sitting this (confusing) way all time since the early days of rsyslog... Probably a sufficiently good cleanup would be to go to 4.x and the follow a reasonable scheme from then on... But besides that: how do you feel about the facts other then the version numbering? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Tuesday, March 25, 2008 5:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.0.0 stable? - feedback, please > > Rainer Gerhards wrote: > > Hi all, > > > > a question on the forum > > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a > bit > > asking around finally made me think about the possibility of a v3 > stable > > version, something that dangled on my mind for a while now... While > it > > is just three month since v2 stable, we have gone a long way and > during > > that time the new queue engine has much matured. As "development" > state > > is most often a show-stopper for putting code into production, I > think > > about releasing a 3.0.0 stable (based on the most recent 3.12.x > version > > WITHOUT RELP support). > > > >>From a support perspective, I'd like to continue support v2 stable > and > > obviously v3 stable and development concurrently. Development would > > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > > fixes. Some time from now, 3.. would become 3.1.0 > > stable. At this point, I would drop support/fixing for 3.0. > > and ask users with problems in the branch to update to > 3.1.. > > i guess this will cause serious headake because people tend to think > like this: > > 3.0.x is less mature/was released before > 3.1.x is less mature/was released before > 3.12.x is less mature/was released before > 3.13.x > > i guess there should be a deliberate versioning scheme. examples are: > > - versioning similar to the linux kernel > - release 3.12.x as 3.12.x-stable and continue development without this > tag. > - something else > > but i guess its no good to jump from 3.12.x back to 3.0.x and from a > future revision such as 3.40.x back to 3.1.x > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Tue Mar 25 18:05:04 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Tue, 25 Mar 2008 18:05:04 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> Message-ID: <47E930C0.6020903@ipax.at> hi, Rainer Gerhards wrote: > OK, good feedback on the version number scheme - it has been sitting > this (confusing) way all time since the early days of rsyslog... > Probably a sufficiently good cleanup would be to go to 4.x and the > follow a reasonable scheme from then on... > > But besides that: how do you feel about the facts other then the version > numbering? sorry, i cannot quote on this as i am not using v3 as of now. this might change in the next couple of weeks but as of now we're still on version v1.x, as this version is not giving much trouble. cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Tue Mar 25 18:16:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 18:16:05 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <47E9227A.5010903@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> Hi all, I have thought about the version numbering. Obviously, the old scheme (http://www.rsyslog.com/doc-version_naming.html ) is confusing and should be dropped. Rsyslog currently follows a three-number version scheme: major.minor.patchlevel I can follow an odd/even scheme on major for unstable/stable branches. However, with major features being continuously introduced may bring us to rsyslog 10.x.x by the end of the year. While I have no hard concerns, it seems to be unusual for open source to increase the major version that fast. On the other hand, I need minor.patchlevel to handle the smaller fixes and new major feature. I increase minor for each new feature (like queue engine, expression-based filters, relp, to name a few actual ones) and increase patchlevel for smaller changes. The three number scheme works well, but, again, keeps the major number very quickly growing given the current fast pace of development. An alternative would be to use politicalmajor.major.minor.patchlevel Where politicalmajor would only be incremented every now and then and actually without much technical reasoning for doing so - only then when we "feel" it is time for a new major release. It would probably turn out to be incremented around once a year, just to keep the corporate folks happy and the major version number reasonable low. Question now: which scheme should I follow? Is there any other one? If so, why should I follow that other scheme? I would appreciate quick feedback, as I'd like to settle this issue before releasing RELP support, which currently looks like some time (very) early April. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Tuesday, March 25, 2008 5:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.0.0 stable? - feedback, please > > Rainer Gerhards wrote: > > Hi all, > > > > a question on the forum > > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a > bit > > asking around finally made me think about the possibility of a v3 > stable > > version, something that dangled on my mind for a while now... While > it > > is just three month since v2 stable, we have gone a long way and > during > > that time the new queue engine has much matured. As "development" > state > > is most often a show-stopper for putting code into production, I > think > > about releasing a 3.0.0 stable (based on the most recent 3.12.x > version > > WITHOUT RELP support). > > > >>From a support perspective, I'd like to continue support v2 stable > and > > obviously v3 stable and development concurrently. Development would > > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > > fixes. Some time from now, 3.. would become 3.1.0 > > stable. At this point, I would drop support/fixing for 3.0. > > and ask users with problems in the branch to update to > 3.1.. > > i guess this will cause serious headake because people tend to think > like this: > > 3.0.x is less mature/was released before > 3.1.x is less mature/was released before > 3.12.x is less mature/was released before > 3.13.x > > i guess there should be a deliberate versioning scheme. examples are: > > - versioning similar to the linux kernel > - release 3.12.x as 3.12.x-stable and continue development without this > tag. > - something else > > but i guess its no good to jump from 3.12.x back to 3.0.x and from a > future revision such as 3.40.x back to 3.1.x > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From pvrabec at redhat.com Wed Mar 26 11:47:21 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Wed, 26 Mar 2008 11:47:21 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> Message-ID: <200803261147.21726.pvrabec@redhat.com> Hi, politicalmajor.major.minor.patchlevel seems good to me. politicalmajor - something great happened, like v2 and v3 major - odd, even - developer/stable release minor - new features patchlevel - patchlevel so current 3.12.4 becomes 3.0.12.4 development starts with 3.1.0.0 and continues to 3.1.0.1, 3.1.0.2, 3.1.1.0, 3.1.2.0, ... 3.1.6.10 than we decide to release stable 3.2.6.0, 3.2.6.1, 3.2.6.2 or 3.2.0.0, 3.2.0.1, 3.2.0.2? development 3.3.0.0, ... 3.3.x.y There is one problem 3.12.4 > 3.0.12.4 == problem with upgrade solutions: 1) 3.12.4 -> 3.14.0.0 2) 3.12.4 -> 3.0.12.4 and I'll start new epoch of rsyslog package (Epoch: 2 in specfile, don't know how is it in other distros) On Tuesday 25 March 2008 06:16:05 pm Rainer Gerhards wrote: > Hi all, > > I have thought about the version numbering. Obviously, the old scheme > (http://www.rsyslog.com/doc-version_naming.html ) is confusing and > should be dropped. > > Rsyslog currently follows a three-number version scheme: > > major.minor.patchlevel > > I can follow an odd/even scheme on major for unstable/stable branches. > However, with major features being continuously introduced may bring us > to rsyslog 10.x.x by the end of the year. While I have no hard concerns, > it seems to be unusual for open source to increase the major version > that fast. On the other hand, I need minor.patchlevel to handle the > smaller fixes and new major feature. I increase minor for each new > feature (like queue engine, expression-based filters, relp, to name a > few actual ones) and increase patchlevel for smaller changes. The three > number scheme works well, but, again, keeps the major number very > quickly growing given the current fast pace of development. > > An alternative would be to use > > politicalmajor.major.minor.patchlevel > > Where politicalmajor would only be incremented every now and then and > actually without much technical reasoning for doing so - only then when > we "feel" it is time for a new major release. It would probably turn out > to be incremented around once a year, just to keep the corporate folks > happy and the major version number reasonable low. > > Question now: which scheme should I follow? Is there any other one? If > so, why should I follow that other scheme? > I would appreciate quick feedback, as I'd like to settle this issue > before releasing RELP support, which currently looks like some time > (very) early April. > > Thanks, > Rainer From linuxweb at gmail.com Thu Mar 27 02:28:46 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 26 Mar 2008 21:28:46 -0400 Subject: [rsyslog] central logserver log rotation Message-ID: <47EAF84E.4010909@gmail.com> Thanks to Rainer, I've got my shiny new central log server up & running. I'd like to rotate the logs on there. If I've got a separate directory for each host, though (i.e., /var/log/rsyslog/%HOSTNAME%), how can I do this? Does something like this work with logrotate: /var/log/rsyslog/*/* { weekly notifempty missingok } What are people doing for logrotation on their CENTRAL logging server? johnn From rgerhards at hq.adiscon.com Thu Mar 27 10:00:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 10:00:46 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <200803261147.21726.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> Hi Peter, thanks for the feedback. So it looks like we go for the 4-number scheme. If I look at the migration case, I am tempted to do a one-time move to 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as interim devel release) to prevent any confusion about version numbers. Changing from a 3-number to a 4-number scheme within the same major release sounds confusing in itself to me. Any concerns with such an approach? Anyone? Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Wednesday, March 26, 2008 11:47 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] rsyslog version numbering - was: RE: rsyslog > 3.0.0 stable? - feedback, please > > Hi, > > politicalmajor.major.minor.patchlevel seems good to me. > > politicalmajor - something great happened, like v2 and v3 > > major - odd, even - developer/stable release > > minor - new features > > patchlevel - patchlevel > > so current 3.12.4 becomes 3.0.12.4 > > development starts with 3.1.0.0 and continues to > > 3.1.0.1, 3.1.0.2, 3.1.1.0, 3.1.2.0, ... 3.1.6.10 > > than we decide to release stable > > 3.2.6.0, 3.2.6.1, 3.2.6.2 > > or > > 3.2.0.0, 3.2.0.1, 3.2.0.2? > > development 3.3.0.0, ... 3.3.x.y > > There is one problem 3.12.4 > 3.0.12.4 == problem with upgrade > > solutions: > > 1) 3.12.4 -> 3.14.0.0 > > 2) 3.12.4 -> 3.0.12.4 and I'll start new epoch of rsyslog package > > (Epoch: 2 in specfile, don't know how is it in other distros) > > On Tuesday 25 March 2008 06:16:05 pm Rainer Gerhards wrote: > > > Hi all, > > > > > > I have thought about the version numbering. Obviously, the old scheme > > > (http://www.rsyslog.com/doc-version_naming.html ) is confusing and > > > should be dropped. > > > > > > Rsyslog currently follows a three-number version scheme: > > > > > > major.minor.patchlevel > > > > > > I can follow an odd/even scheme on major for unstable/stable > branches. > > > However, with major features being continuously introduced may bring > us > > > to rsyslog 10.x.x by the end of the year. While I have no hard > concerns, > > > it seems to be unusual for open source to increase the major version > > > that fast. On the other hand, I need minor.patchlevel to handle the > > > smaller fixes and new major feature. I increase minor for each new > > > feature (like queue engine, expression-based filters, relp, to name a > > > few actual ones) and increase patchlevel for smaller changes. The > three > > > number scheme works well, but, again, keeps the major number very > > > quickly growing given the current fast pace of development. > > > > > > An alternative would be to use > > > > > > politicalmajor.major.minor.patchlevel > > > > > > Where politicalmajor would only be incremented every now and then and > > > actually without much technical reasoning for doing so - only then > when > > > we "feel" it is time for a new major release. It would probably turn > out > > > to be incremented around once a year, just to keep the corporate > folks > > > happy and the major version number reasonable low. > > > > > > Question now: which scheme should I follow? Is there any other one? > If > > > so, why should I follow that other scheme? > > > I would appreciate quick feedback, as I'd like to settle this issue > > > before releasing RELP support, which currently looks like some time > > > (very) early April. > > > > > > Thanks, > > > Rainer From rgerhards at hq.adiscon.com Thu Mar 27 10:13:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 10:13:25 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: <47EAF84E.4010909@gmail.com> References: <47EAF84E.4010909@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Hi Johnn, the typical solution seems to be to include a date-specific part in the filename: /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% And clean up the directories every now and then... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Thursday, March 27, 2008 2:29 AM > To: rsyslog-users > Subject: [rsyslog] central logserver log rotation > > Thanks to Rainer, I've got my shiny new central log server > up & running. I'd like to rotate the logs on there. > > If I've got a separate directory for each host, though > (i.e., /var/log/rsyslog/%HOSTNAME%), how can I do this? > > Does something like this work with logrotate: > > /var/log/rsyslog/*/* { > weekly > notifempty > missingok > } > > > What are people doing for logrotation on their CENTRAL > logging server? > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Thu Mar 27 15:39:11 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 27 Mar 2008 10:39:11 -0400 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> Message-ID: <47EBB18F.1040803@gmail.com> Rainer Gerhards wrote: > Hi Peter, > > thanks for the feedback. So it looks like we go for the 4-number scheme. > > If I look at the migration case, I am tempted to do a one-time move to > 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as > interim devel release) to prevent any confusion about version numbers. > Changing from a 3-number to a 4-number scheme within the same major > release sounds confusing in itself to me. I support the suggestion above. Trying to number within v3 is going to wreak havoc on attempts to update properly. p.s. To be perfectly honest, I didn't think a move to 4-number was even necessary, but I think you'd know better than anyone else what is in the pipeline for rsyslog, and if those are major enough to require "major" number to be changed, then, yeah, it wouldn't make sense to see 10.x.x within a year or two. johnn From rgerhards at hq.adiscon.com Thu Mar 27 16:39:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 16:39:45 +0100 Subject: [rsyslog] timestamp format - why use the old one? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C74@grfint2.intern.adiscon.com> Hi all, I got an interesting question from another discussion list. As you (hopefully) are aware, rsyslog supports high precision timestamps. However, the default format does not output them in order to avoid confusing log parsers. That, in turn, leads to few people actually ever using it. So in the end run, rsyslog's high-precision timestamps have never been used much in the past 3+ years. One suggestion to solve the log parser problem was that we should begin to use the high-precision stamp by default and work with converters/filters when data is fed to log analysers. What is your opinion on this topic? Is it time to change the default output format for timestamps? Rainer From friedl at hq.adiscon.com Thu Mar 27 17:32:56 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 27 Mar 2008 17:32:56 +0100 Subject: [rsyslog] rsyslog 2.0.4 stable released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C78@grfint2.intern.adiscon.com> Hi all, rsyslog 2.0.4, a stable release version, has been released today. It contains some bug fixes, the most important one is that FROMHOST was not properly set on internal messages. There are also two new properties ($HHOUR and $QHOUR) which can be used in dynamic log file generation. Rsyslog 2.0.4 is a recommended update for all users of the stable v2 branch. Changelog: http://www.rsyslog.com/Article197.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-90.phtml Florian Riedl From janfrode at tanso.net Fri Mar 28 08:51:22 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 28 Mar 2008 08:51:22 +0100 Subject: [rsyslog] central logserver log rotation References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: On 2008-03-27, Rainer Gerhards wrote: > the typical solution seems to be to include a date-specific part in the > filename: > > /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% > > And clean up the directories every now and then... /etc/rsyslog.conf: ##################################################### # Log everything to a per host daily logfile # ##################################################### $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log" *.* -?DailyPerHostLogs /etc/cron.hourly/syslog-bzip2: find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec bzip2 '{}' \; -jf From rgerhards at hq.adiscon.com Fri Mar 28 10:17:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 10:17:44 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: References: <47EAF84E.4010909@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C82@grfint2.intern.adiscon.com> Hi JF, thanks, I've made a wiki page out of it: http://wiki.rsyslog.com/index.php/DailyLogRotation Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, March 28, 2008 8:51 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] central logserver log rotation > > On 2008-03-27, Rainer Gerhards wrote: > > the typical solution seems to be to include a date-specific part in > the > > filename: > > > > /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% > > > > And clean up the directories every now and then... > > /etc/rsyslog.conf: > > ##################################################### > # Log everything to a per host daily logfile # > ##################################################### > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/me > ssages.log" > *.* -?DailyPerHostLogs > > /etc/cron.hourly/syslog-bzip2: > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec > bzip2 '{}' \; > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 28 12:58:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 12:58:37 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47EBB18F.1040803@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> > > If I look at the migration case, I am tempted to do a one-time move > to > > 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as > > interim devel release) to prevent any confusion about version > numbers. > > Changing from a 3-number to a 4-number scheme within the same major > > release sounds confusing in itself to me. > > I support the suggestion above. Trying to number within v3 > is going to wreak havoc on attempts to update properly. > > p.s. To be perfectly honest, I didn't think a move to > 4-number was even necessary, I thought quite a while over your message. I have to admit that I personally would prefer a three-number scheme. Maybe there is a solution without going overboard on the major version number increments. First of all, things become stable because bugs are fixed - and no new ones added. "No new ones added" means every stable version is a dead-end when it comes to features. So, 2.0.4 may become 2.0.9 or even 2.0.9999, but we won't see a 2.1.x as that would imply a sufficiently large new feature. This is also a quite hard limit as it means the stable version will always starve for new features. But that's how it works today. In devel, the minor is updated whenever something cool enough is added to justify that. Which means typically every 2 to 6 weeks on the current schedule. So 3.10.0 has become 3.10.2, later 3.11.0 and we are now (soon;)) at 3.12.5. The interesting thing is what happens next: 3.12.5 is quite stable now. There are some feature introduced in 3.10.0 being very stable, some from 3.11.0 which are the same and there as some new ones in 3.12.0 introduced which are not as stable as others (because of limited exposure to practice). Anyhow, I'll soon convert that whole version to 4.0.0 (maybe 4.0.0.0) stable. Then I'll start a new devel branch and the stable build again begins to mature but also become feature-less compared to the devel - which has various stages of maturity some time later. I now got a new school of thinking: let's stick with the 3-digit scheme: major.minor.patchlevel Major will be incremented whenever there is sufficiently large change (typically some inner workings must be changed quite a bit). Stabel and Devel will coexist within the same major release. For minor, we have even - stable and odd - devel. Patchlevel again is smaller magnitude changes within the minor (or evolution of a new feature not yet announced to be usable). But that doesn't solve my need to sometimes have two or three devel versions with new features in a row. To solve that, I suggest that we simply skip the stable releases. For example: 4.0.0 has been released, 4.1.0 is the new stable. I add relp support in 4.1.0. Then I do some changes and end up at 4.1.4. Then, I decide to do the new config file format. That'll be 4.3.0. Note that there now is a stable version missing: we have 4.0.0, 4.1.4 and 4.3.0. I work some more on devel. Then comes a new feature (let's guess TLS support...). So we finally have 4.0.0, 4.1.4, 4.3.x and 4.5.x. After a while, I am at 4.5.5 and also got enough bug fixes to think that the feature introduced in 4.3.x is now ready for prime time. Then, I'll release 4.4.0, based on the 4.3.x codebase (and missing the new 4.5.x feature). With the release of 4.4.0, bugfixing for 4.0.x will end, as we now have a new stable version of the v4 branch (but fixes would still be applied to stable v2). So we end up with 4.0.x (deadend), 4.1.x (dead), NO 4.2.x, 4.3.x (dead), 4.4.x (current stable v4) and 4.5.x (current stable v5). The key thing is that I left out 4.2.x. If that seems to be acceptable, we do not need the 4-number scheme. I know it's pretty wild with all these version numbers... I hope I still could get the idea over. Feedback is highly appreciated. If there is none and no objection by next week Tuesday, I'll go for that scheme. I intend to release 4.0.0(.0) next Wednesday. Today's release of 3.12.5 will set the stage for it. > but I think you'd know better > than anyone else what is in the pipeline for rsyslog, and if > those are major enough to require "major" number to be > changed, then, yeah, it wouldn't make sense to see 10.x.x > within a year or two. Just some thoughts of the pipeline. No guarantees on the order, but the order is current thinking: * relp support * relp with TLS support (followed by relp/GSSAPI) * config file format (scriptable, RainerScript) * email notification output * native TCP w. TLS/syslog-transport-tls internet draft * cryptographically signed messages * support for other message types but syslog * tamperproof (compliance-compliant ;)) system that forces rsyslogd to work with a centralized configuration and provides the ability to detect unauthorized client configuration changes (where they can't be prevented) ... and probably much more ;) Rainer From r.bhatia at ipax.at Fri Mar 28 14:26:14 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 28 Mar 2008 14:26:14 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> Message-ID: <47ECF1F6.8030909@ipax.at> hi, what about the following major.minor.patchlevel with 3.x.y as an exception, used as: STABLE RELEASES =============== 1) release 3.x.y (e.g. 3.12.0) as stable. we might continue providing patches, fixes, etc. in a fashion like: - 3.12.0 (= stable) - 3.13.0 (= major fix) - 3.14.0 (= major fix) - 3.15.0 (= small fix) ... 2) continue development with 4.0.x. release 4.0.14 as stable with: - 4.0.14 (= stable) - 4.0.15 (= fix) - 4.0.16 (= fix) - 4.0.17 (= fix) ... 3) continue development with 4.1.x, release e.g. 4.1.25 as stable with: - 4.1.25 (= stable) - 4.1.26 (= fix) - 4.1.27 (= fix) - 4.1.28 (= fix) ... that's for the handling of stable releases. ad 1) as a more consistent use of the major.minor.patchlevel, we could use: - 3.12.0 - 3.12.1 - 3.12.2 ... - 3.12.1255 DEVELOPMENT SUGGESTION A ======================== inside the development tree, one could do: 4.1.0 = new dev release 4.1.1-rc1 = major new features - exerimental - only for hardcore testers 4.1.1-rc2 = getting more stable 4.1.1-rc3 = getting more stable 4.1.1 = let more ppl test this release with its new features 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not be continued/patched/etc. DEVELOPMENT SUGGESTION B ======================== do not release that many devel releases but put out nightlies with a git "short" tag (i think i've seen this on hg). [1] this would mean: 4.1.0 = new dev tree is started. 4.1.1-d12080bf3d99 snapshot/interim release 2008-03-28 big new features! 4.1.1-c377eb7b2a22 snapshot/interim release 2008-03-29 fixing 4.1.1-b7b2a2515d3a snapshot/interim release 2008-03-30 fixing 4.1.1 - now we're confident enough to make a release for testing. 4.1.2-10e7d626f05c - big new features snapshot/interim release ... 4.1.2 - we're confident - public testers wanted ... 4.1.25 - now we're stable - declare stable, no more features. ;) of course, such interim releases must not happen on a daily basis. everyone should be free to do a recent checkout and those interim releases could be limited to rainer feeling confident enough ;) so, what about this suggestion? cheers, raoul [1] how to shorten the git revisions. long: 4.1.1-d120806a4549ae6c377eb7b2a25172251fbf3d99 4.1.1-d12080(6a4549ae6c377eb7b2a25172251f)bf3d99 4.1.1-d12080...bf3d99 4.1.1-d12080.bf3d99 4.1.1-d12080bf3d99 -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Mar 28 14:56:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 14:56:51 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ECF1F6.8030909@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> > what about the following major.minor.patchlevel with 3.x.y as an > exception, used as: > > STABLE RELEASES > =============== > 1) release 3.x.y (e.g. 3.12.0) as stable. > we might continue providing patches, fixes, etc. in a fashion like: > - 3.12.0 (= stable) > - 3.13.0 (= major fix) > - 3.14.0 (= major fix) > - 3.15.0 (= small fix) > ... > > 2) continue development with 4.0.x. release 4.0.14 as stable with: > - 4.0.14 (= stable) > - 4.0.15 (= fix) > - 4.0.16 (= fix) > - 4.0.17 (= fix) > ... > > 3) continue development with 4.1.x, release e.g. 4.1.25 as stable with: > - 4.1.25 (= stable) > - 4.1.26 (= fix) > - 4.1.27 (= fix) > - 4.1.28 (= fix) > ... > > that's for the handling of stable releases. So in essence, at one patchlevel a major.minor becomes stable and a new (major.minor) is begun - that would work for me and sound quite logical... A key point still would be that there could be a 4.1.24 unstable, while I begin to work on 4.2.0. After all, things are unstable at first ;) So we have two unstable in a row. After a while (we may be at 4.2.6) we can release 4.1.25 (stable) as it has matured enough. Any more comments on that? > ad 1) as a more consistent use of the major.minor.patchlevel, we could > use: > - 3.12.0 > - 3.12.1 > - 3.12.2 > ... > - 3.12.1255 > > > > DEVELOPMENT SUGGESTION A > ======================== > inside the development tree, one could do: > 4.1.0 = new dev release > 4.1.1-rc1 = major new features - exerimental - only for hardcore > testers > 4.1.1-rc2 = getting more stable > 4.1.1-rc3 = getting more stable > 4.1.1 = let more ppl test this release with its new features > > 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not be > continued/patched/etc. I see where you are coming from. This is actually a three and a half version numbering scheme - but also makes sense to me. Looks like a good solution. I like this much more than suggestion B, I have to admit ;) -- the main reason is that I tend to commit very frequently, but a nightly release will probably bad to follow up on. Overall, these suggestions look easier to follow than mine. So if nobody objects, I'll implement them by mid next week. BUT... Feedback is still most welcome! Rainer > > > > DEVELOPMENT SUGGESTION B > ======================== > do not release that many devel releases but put out nightlies with > a git "short" tag (i think i've seen this on hg). [1] > > this would mean: > 4.1.0 = new dev tree is started. > 4.1.1-d12080bf3d99 snapshot/interim release 2008-03-28 big new > features! > 4.1.1-c377eb7b2a22 snapshot/interim release 2008-03-29 fixing > 4.1.1-b7b2a2515d3a snapshot/interim release 2008-03-30 fixing > 4.1.1 - now we're confident enough to make a release for testing. > > 4.1.2-10e7d626f05c - big new features snapshot/interim release > ... > 4.1.2 - we're confident - public testers wanted > ... > 4.1.25 - now we're stable - declare stable, no more features. ;) > > of course, such interim releases must not happen on a daily basis. > everyone should be free to do a recent checkout and those interim > releases could be limited to rainer feeling confident enough ;) > > > > so, what about this suggestion? > > cheers, > raoul > > [1] how to shorten the git revisions. > long: 4.1.1-d120806a4549ae6c377eb7b2a25172251fbf3d99 > 4.1.1-d12080(6a4549ae6c377eb7b2a25172251f)bf3d99 > 4.1.1-d12080...bf3d99 > 4.1.1-d12080.bf3d99 > 4.1.1-d12080bf3d99 > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Fri Mar 28 16:03:05 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 11:03:05 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> Message-ID: <47ED08A9.40500@gmail.com> Rainer Gerhards wrote: >> inside the development tree, one could do: >> 4.1.0 = new dev release >> 4.1.1-rc1 = major new features - exerimental - only for hardcore >> testers >> 4.1.1-rc2 = getting more stable >> 4.1.1-rc3 = getting more stable >> 4.1.1 = let more ppl test this release with its new features >> >> 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not > be >> continued/patched/etc. > > I see where you are coming from. This is actually a three and a half > version numbering scheme - but also makes sense to me. Looks like a good > solution. I like this much more than suggestion B, I have to admit ;) -- > the main reason is that I tend to commit very frequently, but a nightly > release will probably bad to follow up on. I like this idea too (or the general direction of it, anyway -- I'm sure once you start down it, you will likely tweak it to fit actual needs). It lets you keep the 3-number scheme, while still providing for interim releases that don't overload the version number. And a few other projects seem to use this scheme too (consistency is good). > Overall, these suggestions look easier to follow than mine. So if nobody > objects, I'll implement them by mid next week. BUT... Feedback is still > most welcome! So, starting next week, there will be 3 branches? * 2.0.x -- stable, this is the one that systems like RHEL will track for awhile * 3.12.x -- what is this called? this is the "new" stable, for those that need the new 3.x features, but also want a relatively stabilized version, once the numbering is sufficiently high * 4.0.x -- devel, this is where the new major features start going in I think this works. Actually, taking a look at mysql just for comparison sake, they appear to have something similar: 5.0.x (for super-stable, like RHEL), 5.1.x (for those who need full NDB support, but mostly stable once it comes out of RC), and 6.0.x (alpha). johnn From linuxweb at gmail.com Fri Mar 28 16:06:48 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 11:06:48 -0400 Subject: [rsyslog] central logserver log rotation In-Reply-To: References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: <47ED0988.2060705@gmail.com> Jan-Frode Myklebust wrote: > /etc/rsyslog.conf: > > ##################################################### > # Log everything to a per host daily logfile # > ##################################################### > $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log" > *.* -?DailyPerHostLogs > > /etc/cron.hourly/syslog-bzip2: > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec bzip2 '{}' \; Thanks for the example. And ... I was just thinking of doing some gzip/bzip on my morning commute today, so your cron example comes just in time too :). johnn From rgerhards at hq.adiscon.com Fri Mar 28 16:25:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 16:25:12 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: <47ED0988.2060705@gmail.com> References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> <47ED0988.2060705@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C8C@grfint2.intern.adiscon.com> In the long term (think tank!) it may be possible do some automagic with dynafiles. Let's say specify a timeout after which, when no longer accessed, some (external) rotation logic is invoked. The good thing, probably, would be that rsyslog knows best when it can and need to do so. It could even invoke it somewhat quicker if it runs out of disk space ;) Again, nothing to be implemented soon, but if you have an opinion on it, I'd like to hear about it. Side-Note: we are getting close to an initial release of the web interface. It will not only work with databases, but also with text files. I have this, too, on my mind when I think about rotation... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 28, 2008 4:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] central logserver log rotation > > Jan-Frode Myklebust wrote: > > /etc/rsyslog.conf: > > > > ##################################################### > > # Log everything to a per host daily logfile # > > ##################################################### > > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/me > ssages.log" > > *.* -?DailyPerHostLogs > > > > /etc/cron.hourly/syslog-bzip2: > > > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec > bzip2 '{}' \; > > Thanks for the example. And ... I was just thinking of doing > some gzip/bzip on my morning commute today, so your cron > example comes just in time too :). > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Fri Mar 28 16:48:16 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 28 Mar 2008 16:48:16 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> Message-ID: <47ED1340.8030300@ipax.at> Rainer Gerhards wrote: > A key point still would be that there could be a 4.1.24 unstable, while > I begin to work on 4.2.0. After all, things are unstable at first ;) So > we have two unstable in a row. After a while (we may be at 4.2.6) we can > release 4.1.25 (stable) as it has matured enough. i see no problem with this. there even does not need to be a 4.1.x which is declared as stable. > I see where you are coming from. This is actually a three and a half > version numbering scheme - but also makes sense to me. Looks like a good > solution. I like this much more than suggestion B, I have to admit ;) -- > the main reason is that I tend to commit very frequently, but a nightly > release will probably bad to follow up on. there need not be a nightly release. after ordering things in my mind, the main difference is: do we need to map -rcX to git snapshot 45b6c18f1d13be2ff89ee62024ba... or do we allready have this information somewhere accessible. besides that, we could skip the "rc" string and simply use 4.0.12-1 - like a build number which is used by some linux distributions. another nice aspect would be to integrate the git hash into the initial startup message or in some status output. see: > # cat /proc/drbd > version: 8.0.8 (api:86/proto:86) > GIT-hash: bd3e2c922f95c4fa0dca57a4f8c24bf8b249cc02 build by root at webcluster01, 2008-01-29 18:36:06 > Mar 23 06:25:27 webcluster02 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="2525"][x-configInfo udpReception="No" gitHash="bd3e2c922f95c4fa0dca57a4f8c24bf8b249cc02" > udpPort="514" tcpReception="No" tcpPort="0"] > Overall, these suggestions look easier to follow than mine. So if nobody > objects, I'll implement them by mid next week. BUT... Feedback is still > most welcome! maybe it would be good to make a final statement before implementing and releasing it. so we do not have to revert/skip/create the version 5 and 6 and discard 3/4 as a "versioning experiment" ;) cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From linuxweb at gmail.com Fri Mar 28 17:00:37 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 12:00:37 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1340.8030300@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> Message-ID: <47ED1625.9030305@gmail.com> Raoul Bhatia [IPAX] wrote: > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > like a build number which is used by some linux distributions. Actually, Red Hat-based distros (at least) use that "-1" for internal changes. Let's say there is a version: 4.0.12rc3 I (or Red Hat) decide to build an RPM based on that, so my version is: 4.0.12rc3-1 Then, Rainer releases a 4.0.12rc4. It has too many new features in it, so I decide not to move to it. HOWEVER, it also has one crucial bugfix that I need. So I take that patch and backport it into my version. Now, my new *local* version is: 4.0.12rc3-2 So, I do like the idea of attaching rc* (because that makes it clear the changes are from the developer), but not a dash-number (-1, etc.) to the version, if at all possible, because those tend to be localized/internal version number changes. johnn From rgerhards at hq.adiscon.com Fri Mar 28 17:58:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 17:58:58 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1625.9030305@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C91@grfint2.intern.adiscon.com> And this sounds very reasonable, too. Also -rc seems to alert some folks and makes clear what they are doing ;) Please keep the feedback flowing, very good discussion. Just a quick explanation why I would like to settle this out of the sudden quickly. It's a somewhat political issue. While I have no distro-preferrence (rsyslog should run great on as many platforms as possible), I can get the new versioning scheme into Fedora 9 if I do it by mid next week. That is a very nice incentive ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 28, 2008 5:01 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Raoul Bhatia [IPAX] wrote: > > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > > like a build number which is used by some linux distributions. > > Actually, Red Hat-based distros (at least) use that "-1" for > internal changes. > > Let's say there is a version: 4.0.12rc3 > > I (or Red Hat) decide to build an RPM based on that, so my > version is: > 4.0.12rc3-1 > > > Then, Rainer releases a 4.0.12rc4. It has too many new > features in it, so I decide not to move to it. HOWEVER, it > also has one crucial bugfix that I need. So I take that > patch and backport it into my version. Now, my new *local* > version is: > 4.0.12rc3-2 > > So, I do like the idea of attaching rc* (because that makes > it clear the changes are from the developer), but not a > dash-number (-1, etc.) to the version, if at all possible, > because those tend to be localized/internal version number > changes. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 28 18:45:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 18:45:58 +0100 Subject: [rsyslog] rsyslog 3.12.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C93@grfint2.intern.adiscon.com> Hi All, rsyslog 3.12.5 has just been released. It is mostly a bug-fixing and cleanup release. Some changed defaults make it easier to use advanced features. For example, high-precision timestamps are now enabled by default when writing log files and "last message repeated n times" processing, a typical trouble spot, is turned off by default. The documentation has been improved and the v2-backward compatibility layer has been completed. This version is a candidate to become the next stable release. Changelog: http://www.rsyslog.com/Article199.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-91.phtml I suggest to (re-) read ./doc/v3compatibility.html, which also talks about the new defaults. Also available online at: http://www.rsyslog.com/doc-v3compatibility.html Feedback on this release is appreciated. I hope it will be useful. Rainer From rgerhards at hq.adiscon.com Sat Mar 29 10:44:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 29 Mar 2008 10:44:26 +0100 Subject: [rsyslog] debug setting in 3.12.5 active Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C96@grfint2.intern.adiscon.com> Hi all, I just noticed that a debug setting is active in 3.12.5. This disables the actual dlclose. It should not have too bad side effects. I will fix this in 3.12.6, with a few other cleanups, which I hopefully release later today. So you may want to hold updating if you have not done so already. Sorry for the hassle, Rainer From mbiebl at gmail.com Sat Mar 29 12:47:44 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 12:47:44 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1625.9030305@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: 2008/3/28, Johnny Tan : > Raoul Bhatia [IPAX] wrote: > > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > > like a build number which is used by some linux distributions. > > > Actually, Red Hat-based distros (at least) use that "-1" for > internal changes. Yeah, please don't use "-1" as version suffix, this is usually reserved for distros. -rc? would be ok though. Rainer, could you please summarise, what your current idea for the version numbering is. With all the different proposals I somehow lost track about the latest preferred one. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sat Mar 29 14:56:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 29 Mar 2008 14:56:30 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com><200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> I needed some time to digest all this myself ;) I've now done that and put it all into a HTML page. That doesn't imply it is the scheme we will finally use, but if so, it saves me some time. I have also described the development process a bit. I think this is vital for understanding why I need certain release numbers. The info is here (with the now irrelevant text at the end of the doc): http://www.rsyslog.com/doc-version_naming.html I am still a bit undecided if we really need to go v4 or can start this (or whatever else) in the v3 branch... Feedback appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Saturday, March 29, 2008 12:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/28, Johnny Tan : > > Raoul Bhatia [IPAX] wrote: > > > besides that, we could skip the "rc" string and simply use 4.0.12- > 1 - > > > like a build number which is used by some linux distributions. > > > > > > Actually, Red Hat-based distros (at least) use that "-1" for > > internal changes. > > Yeah, please don't use "-1" as version suffix, this is usually > reserved for distros. > -rc? would be ok though. > > Rainer, could you please summarise, what your current idea for the > version numbering is. > With all the different proposals I somehow lost track about the latest > preferred one. > > Cheers, > Michael > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Sat Mar 29 15:13:03 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 15:13:03 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: 2008/3/29, Rainer Gerhards : > I needed some time to digest all this myself ;) I've now done that and > put it all into a HTML page. That doesn't imply it is the scheme we will > finally use, but if so, it saves me some time. I have also described the > development process a bit. I think this is vital for understanding why I > need certain release numbers. The info is here (with the now irrelevant > text at the end of the doc): > > http://www.rsyslog.com/doc-version_naming.html > > I am still a bit undecided if we really need to go v4 or can start this > (or whatever else) in the v3 branch... > > Feedback appreciated. What I'm missing from the document, is a distinction between stable and unstable releases (or are all releases with -rc? unstable and without stable?) Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Sat Mar 29 15:16:18 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 15:16:18 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: 2008/3/29, Michael Biebl : > 2008/3/29, Rainer Gerhards : > > > I needed some time to digest all this myself ;) I've now done that and > > put it all into a HTML page. That doesn't imply it is the scheme we will > > finally use, but if so, it saves me some time. I have also described the > > development process a bit. I think this is vital for understanding why I > > need certain release numbers. The info is here (with the now irrelevant > > text at the end of the doc): > > > > http://www.rsyslog.com/doc-version_naming.html > > > > I am still a bit undecided if we really need to go v4 or can start this > > (or whatever else) in the v3 branch... > > > > Feedback appreciated. > > > What I'm missing from the document, is a distinction between stable > and unstable releases > (or are all releases with -rc? unstable and without stable?) I.e., are -mf releases to be considered stable or not? Would the current 3.12.5 be an rc release in the new versioning scheme? Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sun Mar 30 11:01:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 30 Mar 2008 11:01:22 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> [Mmmhhh. I thought I had sent a reply from the pda. Well, please don't wonder if it shows up some time later ;) So here is the full response:] Any version with a devstate designation (-mf, -rc) is unstable. Every version without it is stable (this will be far less versions as we reach stable only every now and then). So what would the current version be named? Mhh... It's not an easy one-to-one mapping. Because the process changes a bit. Today, I add new major and minor features at the same time while stabilizing old features. This is also the primary reason why we so far have very infrequently new features in the stable release: whenever I stabilize an "older" feature, I introduce new bugs while working on the new ones. That'll change a bit with the new scheme, as I then will freeze code once the focus feature has been implemented. When I start with the next focus feature, patches will be applied to both the "older" code as well as the one I am working on. As such, features become stable during the process. It obviously is some more work to do for me, as I now need to apply patches to multiple code paths (to keep this reasonable, I limited the stable releases to only the latest minor version). The plus is that we get much more feature-rich stable releases, so I think the additional development time is well spent. And for the ultra-conservative folks, there is always the old, feature-less previous version stable (v2 in this case). After having said this, the current release would probably have two names: 3.12.0-rc3 --> the stabilized module loader release 3.13.0-mf2 --> the new relp-enabled release which has not yet real relp support I am also now of the thought that we do not necessarily need to move to 4.0.0 to cover this new scheme. I'd say I can simply release a 3.13.0 stable next week (based on the current 3.12.5) and then continue to work on relp in 3.14.0-rc0 (there will probably be no -mf for that release as relp support is already quite far). Question now: how can I just declare the release to be stable even though it contains the RELP code? Answer: I am lucky ;) the relp code *inside* rsyslog is just two slim inout/output plugins. No core changes. The actual relp code comes via librelp, which is external. Thanks to this coincidence, I did not really introduce any new features since the last feature focus and so the rest of rsyslog could mature. At no other point in the v3 tree I would have been able to do this. That is yet another reason that I'd like to settle the issue by mid next week - the initial version of relp is almost done and I am eager to move on. Again, feedback appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Saturday, March 29, 2008 3:16 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/29, Michael Biebl : > > 2008/3/29, Rainer Gerhards : > > > > > I needed some time to digest all this myself ;) I've now done that > and > > > put it all into a HTML page. That doesn't imply it is the scheme > we will > > > finally use, but if so, it saves me some time. I have also > described the > > > development process a bit. I think this is vital for > understanding why I > > > need certain release numbers. The info is here (with the now > irrelevant > > > text at the end of the doc): > > > > > > http://www.rsyslog.com/doc-version_naming.html > > > > > > I am still a bit undecided if we really need to go v4 or can > start this > > > (or whatever else) in the v3 branch... > > > > > > Feedback appreciated. > > > > > > What I'm missing from the document, is a distinction between stable > > and unstable releases > > (or are all releases with -rc? unstable and without stable?) > > I.e., are -mf releases to be considered stable or not? > Would the current 3.12.5 be an rc release in the new versioning scheme? > > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 11:20:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 11:20:58 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> I'll do a new release according to the new versioning scheme (3.13.0-rc1) later today if nobody objects. It's still not finalized, but doing so makes everybody aware and is also a test for me (plus, we could move to 3.13.0 [stable] on April, 2nd). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Sunday, March 30, 2008 11:01 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > [Mmmhhh. I thought I had sent a reply from the pda. Well, please don't > wonder if it shows up some time later ;) So here is the full response:] > > Any version with a devstate designation (-mf, -rc) is unstable. Every > version without it is stable (this will be far less versions as we > reach > stable only every now and then). > > So what would the current version be named? Mhh... It's not an easy > one-to-one mapping. Because the process changes a bit. Today, I add new > major and minor features at the same time while stabilizing old > features. This is also the primary reason why we so far have very > infrequently new features in the stable release: whenever I stabilize > an > "older" feature, I introduce new bugs while working on the new ones. > > That'll change a bit with the new scheme, as I then will freeze code > once the focus feature has been implemented. When I start with the next > focus feature, patches will be applied to both the "older" code as well > as the one I am working on. As such, features become stable during the > process. It obviously is some more work to do for me, as I now need to > apply patches to multiple code paths (to keep this reasonable, I > limited > the stable releases to only the latest minor version). The plus is that > we get much more feature-rich stable releases, so I think the > additional > development time is well spent. And for the ultra-conservative folks, > there is always the old, feature-less previous version stable (v2 in > this case). > > After having said this, the current release would probably have two > names: > > 3.12.0-rc3 --> the stabilized module loader release > 3.13.0-mf2 --> the new relp-enabled release which has not yet real relp > support > > I am also now of the thought that we do not necessarily need to move to > 4.0.0 to cover this new scheme. I'd say I can simply release a 3.13.0 > stable next week (based on the current 3.12.5) and then continue to > work > on relp in 3.14.0-rc0 (there will probably be no -mf for that release > as > relp support is already quite far). > > Question now: how can I just declare the release to be stable even > though it contains the RELP code? Answer: I am lucky ;) the relp code > *inside* rsyslog is just two slim inout/output plugins. No core > changes. > The actual relp code comes via librelp, which is external. Thanks to > this coincidence, I did not really introduce any new features since the > last feature focus and so the rest of rsyslog could mature. At no other > point in the v3 tree I would have been able to do this. That is yet > another reason that I'd like to settle the issue by mid next week - the > initial version of relp is almost done and I am eager to move on. > > Again, feedback appreciated. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > > Sent: Saturday, March 29, 2008 3:16 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog version numbering > > > > 2008/3/29, Michael Biebl : > > > 2008/3/29, Rainer Gerhards : > > > > > > > I needed some time to digest all this myself ;) I've now done > that > > and > > > > put it all into a HTML page. That doesn't imply it is the > scheme > > we will > > > > finally use, but if so, it saves me some time. I have also > > described the > > > > development process a bit. I think this is vital for > > understanding why I > > > > need certain release numbers. The info is here (with the now > > irrelevant > > > > text at the end of the doc): > > > > > > > > http://www.rsyslog.com/doc-version_naming.html > > > > > > > > I am still a bit undecided if we really need to go v4 or can > > start this > > > > (or whatever else) in the v3 branch... > > > > > > > > Feedback appreciated. > > > > > > > > > What I'm missing from the document, is a distinction between stable > > > and unstable releases > > > (or are all releases with -rc? unstable and without stable?) > > > > I.e., are -mf releases to be considered stable or not? > > Would the current 3.12.5 be an rc release in the new versioning > scheme? > > > > Michael > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 12:48:00 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 12:48:00 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > I'll do a new release according to the new versioning scheme > (3.13.0-rc1) later today if nobody objects. It's still not finalized, > but doing so makes everybody aware and is also a test for me (plus, we > could move to 3.13.0 [stable] on April, 2nd). > Ok, a few comments: I thinks the distinction between mf? (minor features) and rc? (major features) is a bit artifical and counterintuitive imho. For me, rc means "release candidate", meaning that we are close to a stable release (contrary to your proposal, where rc means major new feature, possibly destabilizing a lot). Also, if I read your proposal correctly, as soon as there was an rc1 release, there can be no more mf? releases (even if the added features would be minor) I'd rather use something like -pre1, -pre2 ... (no matter if you have a major or minor feature) and if we are nearing a stable release switch to -rc Or you could do it like the kernel, and only use -rc?. Another proposal for the version scheme (slightly adopted from KDE or Gnome): As an example, your currently stable release is 3.1.x, eg. 3.1.4. Now you want to work against the next minor release, which would be 3.2 So you start with 3.1.80 (80 meaning alpha/beta quality) and increment the last digit for every new release, eg. 3.1.81 (if you reach 89, you start numbering 89.1..) When you enter RC stage, you switch to 3.1.90 and increment for each new release. The final release being 3.2. If you are working towards a next major release, you would start with 3.80 -> 3.90 -> 4.0 Cheers, Michael [1] If you run out -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 12:57:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 12:57:56 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Quick comment... Others please also comment... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 12:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > I'll do a new release according to the new versioning scheme > > (3.13.0-rc1) later today if nobody objects. It's still not > finalized, > > but doing so makes everybody aware and is also a test for me (plus, > we > > could move to 3.13.0 [stable] on April, 2nd). > > > > Ok, a few comments: > I thinks the distinction between mf? (minor features) and rc? (major > features) is a bit artifical and counterintuitive imho. > For me, rc means "release candidate", meaning that we are close to a > stable release (contrary to your proposal, where rc means major new > feature, possibly destabilizing a lot). Also, if I read your proposal > correctly, as soon as there was an rc1 release, there can be no more > mf? releases (even if the added features would be minor) > I'd rather use something like -pre1, -pre2 ... (no matter if you have > a major or minor feature) and if we are nearing a stable release > switch to -rc It would be fine with me to call all -pre1, -pre2 etc... > Or you could do it like the kernel, and only use -rc?. To me, it is quite illogical to call something a -rc, when it not even contains the feature that it shall be a candidate for ;) That was the primary driver for -mf. > > Another proposal for the version scheme (slightly adopted from KDE or > Gnome): > As an example, your currently stable release is > 3.1.x, eg. 3.1.4. > Now you want to work against the next minor release, which would be 3.2 > So you start with > 3.1.80 (80 meaning alpha/beta quality) and increment the last digit > for every new release, eg. 3.1.81 (if you reach 89, you start > numbering 89.1..) > When you enter RC stage, you switch to > 3.1.90 and increment for each new release. The final release being 3.2. Well, that is somewhat along the lines of what I currently do. Remember, I start at 3.10.x for deve builds, then (in the original scheme), I'd go to 3.0, continue with e.g. 3.12.3, which becomes 3.1. The problem were the bouncing numbers. But maybe using patchlevel for that solves the issues. Let's use an actual sample. 3.13.0 is the current stable. The next major feature is RELP, which has been completed, but is not yet stable. I am now working on plain tcp TLS. So would this be the release numbers: 3.13.5 stable 3.13.87 relp 3.14.82 relp/tcp Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.13.88 relp 3.14.83 rep/tcp Once relp is stable, we will have 3.13.6 deprecated 3.14.0 stable 3.14.83 rep/tcp unstable So actually the 3.14.0 release happens possibly much later than 3.14.80? And in that scheme we never have a development designation but the patch level contains it (being >= 80 implies being unstable, while being less implies being stable)? >From what I see, I think that would work for me. Rainer > > If you are working towards a next major release, you would start with > 3.80 -> 3.90 -> 4.0 > > Cheers, > Michael > > [1] If you run out > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 13:02:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:02:56 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Question ;) > If you are working towards a next major release, you would start with > 3.80 -> 3.90 -> 4.0 That also means once we have reached 3.80, we can/will no longer apply the new features to the 3.x stable tree, but rather only the 4.x stable tree? I am asking because I just realized we would otherwise be back to the up and down bouncing version numbers - this happens to be exactly the scheme I used so far... Rainer From mbiebl at gmail.com Mon Mar 31 13:10:07 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 13:10:07 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Question ;) > > > > If you are working towards a next major release, you would start with > > 3.80 -> 3.90 -> 4.0 > > > That also means once we have reached 3.80, we can/will no longer apply > the new features to the 3.x stable tree, but rather only the 4.x stable > tree? I am asking because I just realized we would otherwise be back to > the up and down bouncing version numbers - this happens to be exactly > the scheme I used so far... I don't see the problem here. Say you have a current 3.2.1 stable release. Then you start working for the next major release 4. After 4.0 has been released, you want to add a new feature to the old stable branch, so you release 3.3.0. That's something completely different than renaming 3.12.x to 3.1 when it becomes stable. Michaell -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 13:12:55 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 13:12:55 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Quick comment... Others please also comment... > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > > > Sent: Monday, March 31, 2008 12:48 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog version numbering > > > > > 2008/3/31, Rainer Gerhards : > > > I'll do a new release according to the new versioning scheme > > > (3.13.0-rc1) later today if nobody objects. It's still not > > finalized, > > > but doing so makes everybody aware and is also a test for me (plus, > > we > > > could move to 3.13.0 [stable] on April, 2nd). > > > > > > > Ok, a few comments: > > I thinks the distinction between mf? (minor features) and rc? (major > > features) is a bit artifical and counterintuitive imho. > > For me, rc means "release candidate", meaning that we are close to a > > stable release (contrary to your proposal, where rc means major new > > feature, possibly destabilizing a lot). Also, if I read your proposal > > correctly, as soon as there was an rc1 release, there can be no more > > mf? releases (even if the added features would be minor) > > I'd rather use something like -pre1, -pre2 ... (no matter if you have > > a major or minor feature) and if we are nearing a stable release > > switch to -rc > > > It would be fine with me to call all -pre1, -pre2 etc... > > > > Or you could do it like the kernel, and only use -rc?. > > > To me, it is quite illogical to call something a -rc, when it not even > contains the feature that it shall be a candidate for ;) That was the > primary driver for -mf. The point is, that the don't try to draw a line where they start calling something alpha, beta then rc. The simply use one, rc, and increment that. > > > > > Another proposal for the version scheme (slightly adopted from KDE or > > Gnome): > > As an example, your currently stable release is > > 3.1.x, eg. 3.1.4. > > Now you want to work against the next minor release, which would be > 3.2 > > So you start with > > 3.1.80 (80 meaning alpha/beta quality) and increment the last digit > > for every new release, eg. 3.1.81 (if you reach 89, you start > > numbering 89.1..) > > When you enter RC stage, you switch to > > 3.1.90 and increment for each new release. The final release being > 3.2. > > > Well, that is somewhat along the lines of what I currently do. Remember, > I start at 3.10.x for deve builds, then (in the original scheme), I'd go > to 3.0, continue with e.g. 3.12.3, which becomes 3.1. The problem were The problem is, that for package management systems (and for people) 3.12.x > 3.1.x > the bouncing numbers. But maybe using patchlevel for that solves the > issues. > > Let's use an actual sample. > > 3.13.0 is the current stable. The next major feature is RELP, which has > been completed, but is not yet stable. I am now working on plain tcp > TLS. So would this be the release numbers: > > 3.13.5 stable > 3.13.87 relp > 3.14.82 relp/tcp > > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.13.88 relp > 3.14.83 rep/tcp > > Once relp is stable, we will have > > 3.13.6 deprecated > 3.14.0 stable > 3.14.83 rep/tcp unstable > > So actually the 3.14.0 release happens possibly much later than 3.14.80? This only happens, if you have two unstable branches at the same time. Do you expect that to happen that often? Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From Gerrard.Geldenhuis at datacash.com Mon Mar 31 13:28:26 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Mon, 31 Mar 2008 12:28:26 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: I have not really followed the thread but thought I would give a quick opinion. My requirements would be: Easy to follow numbering. I should be able to see at a glance what the newer version is. There should not be confusing things like 3.12.3 which becomes 3.1 as Michael highligted. Other than that I am not really bothered. Stable and unstable numbering is not realy relevant in my opinion anymore. I would rather look at the website or an rss feed to ascertain which version is the latest stable of a specific product. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: 31 March 2008 12:13 > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > Quick comment... Others please also comment... > > From rgerhards at hq.adiscon.com Mon Mar 31 13:39:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:39:57 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> > > So actually the 3.14.0 release happens possibly much later than > 3.14.80? > > This only happens, if you have two unstable branches at the same time. > Do you expect that to happen that often? I'd say that's the usual case given the current pace of development. I could even envision three unstables at one time... Rainer From rgerhards at hq.adiscon.com Mon Mar 31 13:51:39 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:51:39 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 1:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > Question ;) > > > > > > > If you are working towards a next major release, you would start > with > > > 3.80 -> 3.90 -> 4.0 > > > > > > That also means once we have reached 3.80, we can/will no longer > apply > > the new features to the 3.x stable tree, but rather only the 4.x > stable > > tree? I am asking because I just realized we would otherwise be back > to > > the up and down bouncing version numbers - this happens to be > exactly > > the scheme I used so far... > > I don't see the problem here. > Say you have a current 3.2.1 stable release. > Then you start working for the next major release 4. > After 4.0 has been released, you want to add a new feature to the old > stable branch, so you release 3.3.0. > That's something completely different than renaming 3.12.x to 3.1 when > it becomes stable. I was thinking about the .80+ case. Back to the sample, this time just assuming that relp is scheduled for the next major release. 3.13.0 is the current stable. 3.13.5 stable 3.80.5 relp 3.81.2 relp/tcp Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.80.6 relp 3.81.3 rep/tcp Once relp is stable, we will have... well, what will we have? RELP is stable, but the v4 goal is not yet reached? I thought it means it'll become 3.14.0: 3.13.6 deprecated 3.14.0 stable relp 3.81.3 rep/tcp unstable But now we are back to a case wher 3.81.6 (higher version number) is the same a 3.14.0. And now let's consider: 3.13.6 deprecated 3.14.0 stable relp (based on 3.80.6) 3.81.3 relp/tcp unstable 3.82.0 relp w/ tls unstable Now tcp becomes stable: 3.13.6 deprecated 3.14.0 stable relp (based on 3.80.6) 3.15.0 relp/tcp stable (based on 3.81.3) 3.82.0 relp w/ tls unstable But now we have the situation that 3.15 is more capable than 3.80. This is what I meant. It's exactly the scheme rsyslog used so far, except that I started unstable builds at .10 and not .80. If things working towards a new release shall start with .80, we need to either hold development on the stable (just as I did so far) or we need to have more capable builds with fewer IDs. Maybe this whole stable/unstable thing is not really worth the trouble and I should instead provide the feature stability matrix instead - where for each feature is shown how mature it is. Actually, I'd think that would probably make up for better information. Comments please. Rainer From rgerhards at hq.adiscon.com Mon Mar 31 13:59:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:59:23 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAC@grfint2.intern.adiscon.com> > > > So actually the 3.14.0 release happens possibly much later than > > 3.14.80? > > > > This only happens, if you have two unstable branches at the same > time. > > Do you expect that to happen that often? > > I'd say that's the usual case given the current pace of development. I > could even envision three unstables at one time... This was maybe a bit brief. Let me add the reasoning, it may not be obvious... What makes a stable a stable? Bug fixes? Well, kind of. But actually it is *TIME*. It takes some time to find the "initial bug set". So the most bug reports come in in the 2 to 3 weeks after release. Depending on the feature complexity, it may take even longer (just think about the queue engine and its worker-thread system). So in essence, to get to a stable, I need to have the version just sit around and wait for bugs and their fixes. This will start whenever I have implemented a feature initially. So to continue, I'll immediately fork and work on the next feature set - else I would be idling for at least a couple of weeks. And, depending on how things go, I may even finish that one feature (minor release) and go to the next one before even the version before it matures (especially if my confidence in that code for some reason is reduced or the code is quite complex). During that course, it may happen that a minor feature will never become stable, but only the next one will (if that's quicker stabilized than the previous one). Rainer From r.bhatia at ipax.at Mon Mar 31 15:18:32 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 15:18:32 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> Message-ID: <47F0E4A8.50207@ipax.at> Rainer Gerhards wrote: i clearly object this numbering scheme as it gives much trouble to both ppl and package maintainers. moreover, i would see the 3.x.y branch as a branch depending on some "old" kind of version numbering and start with a fresh version scheme from 4.x.y onwards. coming back to your example, what would you think of: > 3.13.0 is the current stable. > 3.13.5 stable 3.13.5-relp5 (3.80.5 relp) 3.13.5-relp+tcp2 (3.81.2 relp/tcp) > Now let's assume I add a bugfix for the core engine. Would that bring us > to > 3.13.6 stable 3.13.6-relp6 (3.80.6 relp) 3.13.5-relp+tcp3 (3.81.3 rep/tcp) > Once relp is stable, we will have... well, what will we have? RELP is > stable, but the v4 goal is not yet reached? I thought it means it'll > become 3.14.0: 3.13.6 deprecated 3.14.0 stable relp 3.14.0-tcp3 (3.81.3 rep/tcp unstable) 3.13.6 deprecated 3.14.0 stable relp 3.14.3-tcp3 (3.81.3 relp/tcp unstable) 3.14.0-tls (3.82.0 relp w/ tls unstable) > Now tcp becomes stable: 3.13.6 deprecated 3.14.0 stable relp 3.15.0 relp/tcp stable 3.15.0-tls (3.82.0 relp w/ tls unstable) this means we're basically working with patches which can be applied on top of the current stable release. in git you can of course maintain them as branches and merge as to your liking. cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Mon Mar 31 15:36:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 15:36:31 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F0E4A8.50207@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> This has some subtleties for me, too (well, maybe its not really subtleties but simply frightening me ;)). Let me try to get this back to a simple path. How about a simple major.minor.patchlevel-dev So basically the "Sunday scheme" without -mf/-rc but instead just -dev. I guess the scheme was good for all, but the -mf/-rc was deemed confusing. In the sample, it would look like: 3.13.5 stable 3.14.0-dev6 (relp) 3.15.0-dev3 (relp/tcp) Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.14.0-dev7 (relp) 3.15.0-dev4 (relp/tcp) Once relp is stable, we have 3.13.6 deprecated 3.14.0 stable relp 3.15.0-dev4 (relp/tcp) TLS begun: 3.13.6 deprecated 3.14.0 stable relp 3.15.0-dev4 (relp/tcp) 3.16.0-dev0 (tls) Now tcp becomes stable: 3.13.6 deprecated 3.14.0 deprecated 3.15.0 stable (relp/tcp) 3.16.0-dev0 (tls) Bottom line: versions with -dev are unstable, all others are stable. Does this sound acceptable? Rainer From rgerhards at hq.adiscon.com Mon Mar 31 16:25:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 16:25:54 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com><47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB2@grfint2.intern.adiscon.com> FYI: I'll release as 3.13.0-dev0 soon to check out if that scheme brings some problems to me in regard of the systems where I enter the version number. Again, nothing is decided yet. But I think it is better to do that experiment in the v3 tree than to potentially lose a v4 version number ;) (and, as a side note, my v4 release goal is not yet reached, so I'd prefer to stay at v3 if that's possible). Please let us know if you don't like that 3.13.0-dev0 scheme so that we can move to something different before I need to release the next stable. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, March 31, 2008 3:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > This has some subtleties for me, too (well, maybe its not really > subtleties but simply frightening me ;)). Let me try to get this back > to > a simple path. How about a simple > > major.minor.patchlevel-dev > > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) > > > Now let's assume I add a bugfix for the core engine. Would that bring > us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) > > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) > > Now tcp becomes stable: > > 3.13.6 deprecated > 3.14.0 deprecated > 3.15.0 stable (relp/tcp) > 3.16.0-dev0 (tls) > > Bottom line: versions with -dev are unstable, all others are > stable. > > Does this sound acceptable? > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From pvrabec at redhat.com Mon Mar 31 16:51:12 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Mon, 31 Mar 2008 16:51:12 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <200803311651.12753.pvrabec@redhat.com> Hi, > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) > > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) There is a great disadvatage of this scheme. 3.14.0-dev7 > 3.14.0, BUT 3.14.0 is newer. From linuxweb at gmail.com Mon Mar 31 16:53:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 10:53:22 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F0FAE2.3070405@gmail.com> Rainer Gerhards wrote: > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed Since many folks are already familiar with alpha, beta, and rc, I think it'd be good to stick with those as much as possible. > confusing. In the sample, it would look like: > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) 3.13.5 stable 3.14.0-beta6 (relp) 3.15.0-alpha3 (relp/tcp) > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.0-beta7 (relp) 3.15.0-alpha4 (relp/tcp) > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) 3.13.6 deprecated 3.14.0-rc1 (?) relp becoming stable 3.15.0-alpha4 (relp/tcp) > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) 3.13.6 deprecated 3.14.0 stable relp 3.15.0-beta1 (relp/tcp) (3.15 now becomes beta) 3.16.0-alpha1 (tls) (the new alpha/dev branch) In a nutshell: alpha - very (b)leading edge dev release branch beta - dev branch that has been out for awhile but still being worked on rc - the beta/dev branch finally getting ready to be declared stable, pretty much bug-fix-only at this point stable deprecated From r.bhatia at ipax.at Mon Mar 31 16:58:02 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 16:58:02 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F0FBFA.3020802@ipax.at> Rainer Gerhards wrote: > This has some subtleties for me, too (well, maybe its not really > subtleties but simply frightening me ;)). Let me try to get this back to > a simple path. How about a simple > > major.minor.patchlevel-dev > > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) what i'm not getting is, that you a) want to maintain some work-in-progress-patches as seperate trees. what happens if you want to implement feature-x (ssl/tls) first (which would then be 3.17.0) and this becomes stable faster than relp/other stuff? then we have 3.13.5 3.14.0-devX 3.15.0-devY 3.16.0 stable uhm ... ? what happens if a patch will be unstable for ages as it was some kind of paid-for-feature but time ran out? or if a feature becomes obsolete because of another feature? b) why do you want to stick to some in-between-version - especially with the v3.x scheme which i still see as progressing as 3.14.0, 3.15.0, 3.16.0 as in the old-style so nobody is confused. therefore i call 3.14.0 3.15.0 etc. in-betwee-versions, if 3.16 is under development) mhm, now an idea begins to take hold of me ;) do you want to totally switch to the new scheme now? or do you want to keep some kind of legacy numbering for v3? maybe my problem is, that i somehow see the versioning this way: v3.pl or rather v3.subminor.pl and v4.minor.pl or rather v4.minor.subminor.pl (such as we can see with kernel v2.6.24.4) so 3.15.0 is translated to 3.0.15 in my mind. so the next step would be 3.0.16 which would become 3.16.x in the "old" release style. perhaps you can enlighten me/us? :) and maybe, if you want to switch to the final version scheme, we should jump over a couple of releases and start with 3.20.0 to make things clear? cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Mon Mar 31 17:01:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:01:35 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <200803311651.12753.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> Peter, Ah, so you compare the strings... makes sense. But also makes clear we have a problem with any such scheme... ... that means it applies to Johnn's suggestion, too :( Next try, back to odd/even. So any odd version number is unstable, any even one is stable. Once something gets stable, it is incremented to the even version. -rc and similar things may (may!) be added to alert users. Again into the sample case: 3.14.0 stable 3.15.6 unstable (relp) 3.17.2 unstable (relp/tcp) Now let's assume I add a bugfix for the core engine. Would that bring us to 3.14.1 stable 3.15.7 unstable (relp) 3.17.3 unstable (relp/tcp) Once relp is stable, we have 3.14.1 deprecated 3.16.0 stable (relp) 3.17.3 unstable (relp/tcp) TLS begins: 3.14.1 deprecated 3.16.0 stable (relp) 3.17.3 unstable (relp/tcp) 3.19.0 unstable (tls) Now tcp becomes stable: 3.14.1 deprecated 3.16.0 deprecated 3.18.0 stable (relp/tcp) 3.19.0 unstable (tls) In all cases stable > devel, at least as far as the some feature set is concerned. There may be gaps in the minor version numbers when versions are not yet stable. Does that work? Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Monday, March 31, 2008 4:51 PM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] rsyslog version numbering > > Hi, > > > 3.13.6 stable > > 3.14.0-dev7 (relp) > > 3.15.0-dev4 (relp/tcp) > > > > Once relp is stable, we have > > > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > There is a great disadvatage of this scheme. > 3.14.0-dev7 > 3.14.0, BUT 3.14.0 is newer. From friedl at hq.adiscon.com Mon Mar 31 17:08:57 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 31 Mar 2008 17:08:57 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Hi all, I have just release rsyslog 3.13.0-dev0, which will hopefully become the next stable release by mid-week. It fixes two bugs, one is a potential segfault in the syslog/plain tcp receiver. The other one is removal of some debug instrumentation that accidently made it into 3.12.5. There is also a new ./configure option (--enable/disable-rsyslogd) which permits to build just specific plugins without the main binary. This has been added to support source-based packaging systems (like in FreeBSD). This is also a test for one potential new versioning scheme. 3.13.0-dev0 is recommended for all v3 branch users. Feedback on it is highly appreciated. Changelog: http://www.rsyslog.com/Article201.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-92.phtml Florian Riedl From linuxweb at gmail.com Mon Mar 31 17:21:19 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 11:21:19 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> Message-ID: <47F1016F.6010200@gmail.com> Rainer Gerhards wrote: > Peter, > > Ah, so you compare the strings... makes sense. But also makes clear we > have a problem with any such scheme... > > ... that means it applies to Johnn's suggestion, too :( No, I made a mistake of keeping the numbering when my main point was more about, if you are going to use appendages, to stick with the well-known ones (pre, alpha, beta, rc, stable, patch). It should work with *any* numbering scheme (yours, Michael's, the even/odd, etc.). The value-added benefit is it gives people an instant knowledge of where the branch stands. My company doesn't mind using rc's in production if we absolutely need the features it provides. But using an alpha would be much harder to justify. Granted, in the end, these are somewhat all arbitrary (even declaring something stable). But I think, like you said, most devs (or the communities that surround a project) have a general sense that the passage of time and what actually gets added (bug-fix-only, etc.) to a branch help determine these labels. That's generally good enough for most people who end up using the software. johnn From rgerhards at hq.adiscon.com Mon Mar 31 17:25:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:25:57 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F1016F.6010200@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> <47F1016F.6010200@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB7@grfint2.intern.adiscon.com> I couldn't agree more. I have a maturity matrix on my mind. I guess I'll do a sample of it. Maybe that helps find a solution. The core thing is that with a truly modular design like we have in rsyslog nothing is stable or unstable per se. Some components are extremely stable, while others are bleeding edge. As long as you don't use the bleeding edge ones (and they did not touch core features) the last ironed-out alpha devel version is rock solid. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Monday, March 31, 2008 5:21 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Rainer Gerhards wrote: > > Peter, > > > > Ah, so you compare the strings... makes sense. But also makes clear > we > > have a problem with any such scheme... > > > > ... that means it applies to Johnn's suggestion, too :( > > No, I made a mistake of keeping the numbering when my main > point was more about, if you are going to use appendages, to > stick with the well-known ones (pre, alpha, beta, rc, > stable, patch). > > It should work with *any* numbering scheme (yours, > Michael's, the even/odd, etc.). > > > The value-added benefit is it gives people an instant > knowledge of where the branch stands. > > My company doesn't mind using rc's in production if we > absolutely need the features it provides. But using an alpha > would be much harder to justify. > > Granted, in the end, these are somewhat all arbitrary (even > declaring something stable). But I think, like you said, > most devs (or the communities that surround a project) have > a general sense that the passage of time and what actually > gets added (bug-fix-only, etc.) to a branch help determine > these labels. That's generally good enough for most people > who end up using the software. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 17:32:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:32:06 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F0FBFA.3020802@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F0FBFA.3020802@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB8@grfint2.intern.adiscon.com> Well... First thing is that I have urgent need to release -- there are a couple of things in the queue. If I don't release them this week, I'll probably don't have a chance to get to a stable any time soon. So I will release, even at the price that the version number scheme may be less than optimum this time. But now on to the real meat (and thanks for sticking around on this topic! ;))... > Rainer Gerhards wrote: > > This has some subtleties for me, too (well, maybe its not really > > subtleties but simply frightening me ;)). Let me try to get this back > to > > a simple path. How about a simple > > > > major.minor.patchlevel-dev > > > > So basically the "Sunday scheme" without -mf/-rc but instead just - > dev. > > I guess the scheme was good for all, but the -mf/-rc was deemed > > confusing. In the sample, it would look like: > > > > 3.13.5 stable > > 3.14.0-dev6 (relp) > > 3.15.0-dev3 (relp/tcp) > > what i'm not getting is, that you > a) want to maintain some work-in-progress-patches as seperate trees. > what happens if you want to implement feature-x (ssl/tls) first > (which would then be 3.17.0) and this becomes stable faster than > relp/other stuff? > > then we have > 3.13.5 > 3.14.0-devX > 3.15.0-devY > 3.16.0 stable > > uhm ... ? [I am answering this in the context of the version numbers above. After Peter's note on the strcmp() of version numbers, I think that we need to use something like odd/even where stable is compares always greater than unstable.] In this case, there will never be a 3.14 and 3.15 stable. This scenario may happen. In my current scheme, we'd probably never seen a 3.11 stable (queue engine) but a 3.12, because the queue engine needed much more time to mature. > what happens if a patch will be unstable for ages as it was some kind > of > paid-for-feature but time ran out? or if a feature becomes obsolete > because of another feature? Maybe the problem is that I am NOT branching off for new features. I branch off stable releases. I fear that branching of for new features will be much more time consuming at the current pace of development and has high potential for conflict. Maybe I am wrong ;) To the cases above: the paid-for-feature would never be released. That case is not fully thought out as I did not yet find someone who want's to pay. My perception may change when this happens ;) If a feature that is not yet stable is obsoleted, that version will never become stable (gap in release history). > > b) why do you want to stick to some in-between-version - especially > with > the v3.x scheme > > which i still see as progressing as 3.14.0, 3.15.0, > 3.16.0 as in the old-style so nobody is confused. therefore i call > 3.14.0 3.15.0 etc. in-betwee-versions, if 3.16 is under development) I have a major release focus for v3: full modularization. Not reached yet ;) Of course, that can be sacrificed. But as long as we experiment (and we seem to do), I prefer to do that in v3 instead of risking to lose v4, too (when we go to something else) ;) > > mhm, now an idea begins to take hold of me ;) > > do you want to totally switch to the new scheme now? or do you want to > keep some kind of legacy numbering for v3? > > maybe my problem is, that i somehow see the versioning this way: > > v3.pl or rather v3.subminor.pl and > v4.minor.pl or rather v4.minor.subminor.pl (such as we can see with > kernel v2.6.24.4) > > so 3.15.0 is translated to 3.0.15 in my mind. so the next step would be > 3.0.16 which would become 3.16.x in the "old" release style. > > perhaps you can enlighten me/us? :) Well, *if* I'd taken proper stable branchpoints, we would now probably be at 3.3 or 3.4 speaking stable-wise (3.0 intial release and input plugins, 3.1 queue (probably never delivered), 3.2 expression support, ...). So far, rsyslog did a stable release once every 12 to 24 month. From there on, everything was experimental. We can keep with that scheme (less work for me), but the drawback is that it takes an immense amount of time before those looking for stable can find new features. IMHO this hasn't worked out well. > > and maybe, if you want to switch to the final version scheme, we should > jump over a couple of releases and start with 3.20.0 to make things > clear? I agree, maybe even v4 - but then the version scheme itself must have reached stable state ;) Rainer > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Mon Mar 31 17:44:43 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 11:44:43 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F106EB.6090504@gmail.com> Rainer Gerhards wrote: > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) Ok, let me take one last swipe at this, with numbering AND labels: 3.13.5 stable 3.14.6-beta 3.15.3-alpha > > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.7-beta 3.15.4-alpha > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.8-rc 3.15.5-alpha > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) 3.13.6 deprecated 3.14.9 stable 3.15.5-beta 3.16.0-alpha > Now tcp becomes stable: > > 3.13.6 deprecated > 3.14.0 deprecated > 3.15.0 stable (relp/tcp) > 3.16.0-dev0 (tls) Well, this is kind of a big jump, but assuming it goes through all the proper alpha/beta/rc phases: 3.13.6 deprecated 3.14.9 deprecated 3.15.6 stable 3.16.1-alpha So, you increment the patchlevels, as you've been doing, but you use -alpha, -beta, -rc (with no numbers) to designate the newness/readiness of the branch. From rgerhards at hq.adiscon.com Mon Mar 31 18:06:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 18:06:34 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F106EB.6090504@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F106EB.6090504@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> Please have a look at: http://www.rsyslog.com/doc-maturity.html It's not complete yet, because it requires some work. If it is done parallel to the releases, it's not much work. I personally would find this much more useful than a single binary switch for all of the modules (which never is really right ;)). That, of course, doesn't solve the version numbering question ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Monday, March 31, 2008 5:45 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Rainer Gerhards wrote: > > confusing. In the sample, it would look like: > > > > 3.13.5 stable > > 3.14.0-dev6 (relp) > > 3.15.0-dev3 (relp/tcp) > > Ok, let me take one last swipe at this, with numbering AND > labels: > > 3.13.5 stable > 3.14.6-beta > 3.15.3-alpha > > > > > > Now let's assume I add a bugfix for the core engine. Would that bring > us > > to > > > > 3.13.6 stable > > 3.14.0-dev7 (relp) > > 3.15.0-dev4 (relp/tcp) > > 3.13.6 stable > 3.14.7-beta > 3.15.4-alpha > > > > Once relp is stable, we have > > > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > 3.13.6 stable > 3.14.8-rc > 3.15.5-alpha > > > > TLS begun: > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > 3.16.0-dev0 (tls) > > 3.13.6 deprecated > 3.14.9 stable > 3.15.5-beta > 3.16.0-alpha > > > > Now tcp becomes stable: > > > > 3.13.6 deprecated > > 3.14.0 deprecated > > 3.15.0 stable (relp/tcp) > > 3.16.0-dev0 (tls) > > Well, this is kind of a big jump, but assuming it goes > through all the proper alpha/beta/rc phases: > > 3.13.6 deprecated > 3.14.9 deprecated > 3.15.6 stable > 3.16.1-alpha > > So, you increment the patchlevels, as you've been doing, but > you use -alpha, -beta, -rc (with no numbers) to designate > the newness/readiness of the branch. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 19:15:59 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 19:15:59 +0200 Subject: [rsyslog] openssl vs rsyslog Message-ID: Hi Rainer, I noticed that you added an openssl configure check. That probably means, that you plan to add native SSL support via openssl. Please note that OpenSSL and the GPL are incompatible [1], such packages are usually rejected from Debian. You have two options: - Either add a special exemption for OpenSSL - Use another SSL implementation like GnuTLS [2] or NSS [3] Cheers, Michael [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html [2] http://www.gnu.org/software/gnutls/ [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 19:52:37 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 19:52:37 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Florian Riedl : > Hi all, > > I have just release rsyslog 3.13.0-dev0, which will hopefully become the > next stable release by mid-week. > > It fixes two bugs, one is a potential segfault in the syslog/plain tcp > receiver. The other one is removal of some debug instrumentation that > accidently made it into 3.12.5. There is also a new ./configure option > (--enable/disable-rsyslogd) which permits to build just specific plugins Hm, the default of enable_rsyslogd should imho be "yes" > without the main binary. This has been added to support source-based > packaging systems (like in FreeBSD). This is also a test for one > potential new versioning scheme. I don't see the use case yet. Why would someone want to build the plugins without the rsyslogd binary? Does it actually work if you mix different versions of plugins and rsyslogd? If someone is interested in only a specific plugin, why not use "make -C plugins/foo"? Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 20:04:18 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 20:04:18 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Michael Biebl : > 2008/3/31, Florian Riedl : > > > Hi all, > > > > I have just release rsyslog 3.13.0-dev0, which will hopefully become the > > next stable release by mid-week. > > > > It fixes two bugs, one is a potential segfault in the syslog/plain tcp > > receiver. The other one is removal of some debug instrumentation that > > accidently made it into 3.12.5. There is also a new ./configure option > > (--enable/disable-rsyslogd) which permits to build just specific plugins > > > Hm, the default of enable_rsyslogd should imho be "yes" > > > > without the main binary. This has been added to support source-based > > packaging systems (like in FreeBSD). This is also a test for one > > potential new versioning scheme. > > > I don't see the use case yet. Why would someone want to build the > plugins without the rsyslogd binary? Does it actually work if you mix > different versions of plugins and rsyslogd? > If someone is interested in only a specific plugin, why not use "make > -C plugins/foo"? Forgot to add: If you intend to build rsyslogd and rfc3195d conditionally, the corresponding man pages should also be installed conditionally. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 20:14:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 20:14:01 +0200 Subject: [rsyslog] openssl vs rsyslog Message-ID: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) Can you point me to the problem description? What would i need to exempt? Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... Any advise is deeply appreciated. Thanks, rainer ----- Urspr?ngliche Nachricht ----- Von: "Michael Biebl" An: "rsyslog-users" Gesendet: 31.03.08 19:16 Betreff: [rsyslog] openssl vs rsyslog Hi Rainer, I noticed that you added an openssl configure check. That probably means, that you plan to add native SSL support via openssl. Please note that OpenSSL and the GPL are incompatible [1], such packages are usually rejected from Debian. You have two options: - Either add a special exemption for OpenSSL - Use another SSL implementation like GnuTLS [2] or NSS [3] Cheers, Michael [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html [2] http://www.gnu.org/software/gnutls/ [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 20:34:40 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 20:34:40 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) > > Can you point me to the problem description? What would i need to exempt? Google for "openssl gpl" and you should find relevant information. E.g. [1] tries to summarize the problem. It also has a proposal for such a OpensSSL exemption. You should add this addendum to COPYING and probably also ship the original OpenSSL COPYING file (e.g. as COPYING.OpenSSL) > > Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... A prominent example of a project using GnuTLS is e.g. samba (if it's highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. A prominent example of a project using NSS is mozilla/firefox/thunderbird. I think the more widely used library is GnuTLS. I really don't have that much experience with either of these libraries, so it's hard to give a recommendation. Cheers, Michael > [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html > [2] http://www.gnu.org/software/gnutls/ > [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 20:47:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 20:47:03 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 8:35 PM > To: rsyslog-users > Subject: Re: [rsyslog] openssl vs rsyslog > > 2008/3/31, Rainer Gerhards : > > Oh, excellent feedback. And I thought that openssl is the > workhorse of os tls - wrong preception obviously ;) > > > > Can you point me to the problem description? What would i > need to exempt? > > Google for "openssl gpl" and you should find relevant information. > E.g. [1] tries to summarize the problem. > It also has a proposal for such a OpensSSL exemption. You should add > this addendum to COPYING and probably also ship the original OpenSSL > COPYING file (e.g. as COPYING.OpenSSL) Yes - sorry. I overlooked the link on the PDA. Will mow read through it. But if openssl means trouble, I'll prefer to stay away from it unless there is *very* good reason. You wake-up call was *right* in time. I planned to do some major hacking tomorrow - nothing lost so far :) [and, in case you wonder, yes, initial RELP is done ;)]. > > Is any of the alternates suitable for production use, > especially in a highly threaded environment? GNU, from the > name, sounds appealing... > > > A prominent example of a project using GnuTLS is e.g. samba (if it's > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > A prominent example of a project using NSS is > mozilla/firefox/thunderbird. > > I think the more widely used library is GnuTLS. I really don't have > that much experience with either of these libraries, so it's hard to > give a recommendation. At least it doesn't sound too bad ;) Anyone with experience please comment. Thanks again, Rainer > > Cheers, > Michael > > > > [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html > > [2] http://www.gnu.org/software/gnutls/ > > [3] http://www.mozilla.org/projects/security/pki/nss/ > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From milton at calnek.com Mon Mar 31 21:00:11 2008 From: milton at calnek.com (Milton Calnek) Date: Mon, 31 Mar 2008 13:00:11 -0600 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> Message-ID: <47F134BB.9090705@calnek.com> Don't know if this helps, or if you got the same from COPYING.Openssl. http://www.openssl.org/support/faq.html#LEGAL1 Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl >> Sent: Monday, March 31, 2008 8:35 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] openssl vs rsyslog >> >> 2008/3/31, Rainer Gerhards : >>> Oh, excellent feedback. And I thought that openssl is the >> workhorse of os tls - wrong preception obviously ;) >>> Can you point me to the problem description? What would i >> need to exempt? >> >> Google for "openssl gpl" and you should find relevant information. >> E.g. [1] tries to summarize the problem. >> It also has a proposal for such a OpensSSL exemption. You should add >> this addendum to COPYING and probably also ship the original OpenSSL >> COPYING file (e.g. as COPYING.OpenSSL) > > Yes - sorry. I overlooked the link on the PDA. Will mow read through it. > But if openssl means trouble, I'll prefer to stay away from it unless > there is *very* good reason. You wake-up call was *right* in time. I > planned to do some major hacking tomorrow - nothing lost so far :) [and, > in case you wonder, yes, initial RELP is done ;)]. > >>> Is any of the alternates suitable for production use, >> especially in a highly threaded environment? GNU, from the >> name, sounds appealing... >> >> >> A prominent example of a project using GnuTLS is e.g. samba (if it's >> highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. >> A prominent example of a project using NSS is >> mozilla/firefox/thunderbird. >> >> I think the more widely used library is GnuTLS. I really don't have >> that much experience with either of these libraries, so it's hard to >> give a recommendation. > > At least it doesn't sound too bad ;) > > Anyone with experience please comment. > > Thanks again, > Rainer >> Cheers, >> Michael >> >> >>> [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html >>> [2] http://www.gnu.org/software/gnutls/ >>> [3] http://www.mozilla.org/projects/security/pki/nss/ >> >> -- >> Why is it that all of the instruments seeking intelligent life in the >> universe are pointed away from Earth? >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From skvidal at fedoraproject.org Mon Mar 31 20:38:19 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 14:38:19 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: <1206988699.9240.38.camel@cutter> On Mon, 2008-03-31 at 20:34 +0200, Michael Biebl wrote: > 2008/3/31, Rainer Gerhards : > > Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) > > > > Can you point me to the problem description? What would i need to exempt? > > Google for "openssl gpl" and you should find relevant information. > E.g. [1] tries to summarize the problem. > It also has a proposal for such a OpensSSL exemption. You should add > this addendum to COPYING and probably also ship the original OpenSSL > COPYING file (e.g. as COPYING.OpenSSL) > > > > > Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... > > > A prominent example of a project using GnuTLS is e.g. samba (if it's > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > A prominent example of a project using NSS is mozilla/firefox/thunderbird. > > I think the more widely used library is GnuTLS. I really don't have > that much experience with either of these libraries, so it's hard to > give a recommendation. > the library that a lot of folks inside red hat and fedora are driving people to is the mozilla nss library. It handles all the bits openssl does and w/o the licensing problems. -sv From rgerhards at hq.adiscon.com Mon Mar 31 21:10:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:10:04 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206988699.9240.38.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> Hi sv, > the library that a lot of folks inside red hat and fedora are driving > people to is the mozilla nss library. It handles all the bits openssl > does and w/o the licensing problems. Sounds promising. I just found http://www.gnu.org/software/gnutls/comparison.html And that makes GNU TLS quite appealing (no wonder given the source ;)). Do you happen to have a link which tells why use nss? That would be excellent (but I am of course searching myself). Thanks, Rainer From r.bhatia at ipax.at Mon Mar 31 21:16:16 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 21:16:16 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F106EB.6090504@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> Message-ID: <47F13880.3020506@ipax.at> Rainer Gerhards wrote: > Please have a look at: > > http://www.rsyslog.com/doc-maturity.html > > It's not complete yet, because it requires some work. If it is done > parallel to the releases, it's not much work. I personally would find > this much more useful than a single binary switch for all of the modules > (which never is really right ;)). That, of course, doesn't solve the > version numbering question ;) as a short input if you want to provide a feature matrix: http://linux-vserver.org/Feature_Matrix cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From mbiebl at gmail.com Mon Mar 31 21:18:33 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 21:18:33 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206988699.9240.38.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: 2008/3/31, seth vidal : > > A prominent example of a project using GnuTLS is e.g. samba (if it's > > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > > A prominent example of a project using NSS is mozilla/firefox/thunderbird. > > > > I think the more widely used library is GnuTLS. I really don't have > > that much experience with either of these libraries, so it's hard to > > give a recommendation. > > > > > the library that a lot of folks inside red hat and fedora are driving > people to is the mozilla nss library. It handles all the bits openssl > does and w/o the licensing problems. Interesting to know. Do you know any technical advantages of NSS over GnuTLS (stability, features, nicer API, etc)? At least on Debian I had the impression that GnuTLS was the preferred solution. E.g. samba, openldap and exim4 are originally using openssl but contain a patch to use GnuTLS in Debian. One reason might be, that libgnutls is quite a bit smaller: gnutls: installed size (on i386): 920 kb [1] libnss3: installed size (on i386): 2472 kb, also requires the netscape portable runtime library libnpr (588 kb) => ~3Mb [2] Which imho is a plus for GnuTLS. [1] http://packages.debian.org/sid/libgnutls26 [2] http://packages.debian.org/sid/libnss3-1d -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From skvidal at fedoraproject.org Mon Mar 31 21:21:26 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 15:21:26 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> Message-ID: <1206991286.9240.43.camel@cutter> On Mon, 2008-03-31 at 21:10 +0200, Rainer Gerhards wrote: > Hi sv, > > > the library that a lot of folks inside red hat and fedora are driving > > people to is the mozilla nss library. It handles all the bits openssl > > does and w/o the licensing problems. > > Sounds promising. I just found > > http://www.gnu.org/software/gnutls/comparison.html > > > And that makes GNU TLS quite appealing (no wonder given the source ;)). > Do you happen to have a link which tells why use nss? That would be > excellent (but I am of course searching myself). > nss is fips certified: http://www.mozilla.org/projects/security/pki/nss/fips/ which makes it easier to use in large gov't agencies. gnutls is not certified. nss is also where the fedora/red hat crypto consolidation is going.... there is also an api-helper library for transitioning from openssl to nss. if at all possible, go with nss. -sv From skvidal at fedoraproject.org Mon Mar 31 21:22:19 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 15:22:19 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: <1206991339.9240.45.camel@cutter> On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > Interesting to know. Do you know any technical advantages of NSS over > GnuTLS (stability, features, nicer API, etc)? openssl transition library fips certification. http://www.mozilla.org/projects/security/pki/nss/fips/ the second one being the most important, I think. -sv From mbiebl at gmail.com Mon Mar 31 21:26:51 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 21:26:51 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206991339.9240.45.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> <1206991339.9240.45.camel@cutter> Message-ID: 2008/3/31, seth vidal : > On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > > Interesting to know. Do you know any technical advantages of NSS over > > GnuTLS (stability, features, nicer API, etc)? > > > openssl transition library > fips certification. > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > the second one being the most important, I think. Yeah, the openssl transition library doesn't concern us. Why is the fips certification important for rsyslog? Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 21:26:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:26:59 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206991339.9240.45.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com><1206988699.9240.38.camel@cutter> <1206991339.9240.45.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC7@grfint2.intern.adiscon.com> > > Interesting to know. Do you know any technical advantages > of NSS over > > GnuTLS (stability, features, nicer API, etc)? > > openssl transition library > fips certification. > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > the second one being the most important, I think. > > -sv FIPS is a strong selling point, I have to admit. Especially as rsyslog is heading towards the compliance space and trying to be as tamper-proof as possible. On a quick lock, GNUTLS looks a bit nicer to me (the NSPL doesn't sound really good whereas the GNUTLS zlib implementations solves something I intended to do). But FIPS... Rainer From rgerhards at hq.adiscon.com Mon Mar 31 21:29:52 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:29:52 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com><1206988699.9240.38.camel@cutter><1206991339.9240.45.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC8@grfint2.intern.adiscon.com> Whatever I do - I'll try to create a small driver shim so that hopefully different libraries could be used together with rsyslog. It depends on the APIs, but in general that should not be too hard (but practice often tells the difference). On FIPS I made I comment, where I think the mails crossed. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 9:27 PM > To: rsyslog-users > Subject: Re: [rsyslog] openssl vs rsyslog > > 2008/3/31, seth vidal : > > On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > > > Interesting to know. Do you know any technical > advantages of NSS over > > > GnuTLS (stability, features, nicer API, etc)? > > > > > > openssl transition library > > fips certification. > > > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > > > the second one being the most important, I think. > > Yeah, the openssl transition library doesn't concern us. > > Why is the fips certification important for rsyslog? > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From leitstelle at bnet.li Mon Mar 3 15:07:23 2008 From: leitstelle at bnet.li (leitstelle at bnet.li) Date: Mon, 03 Mar 2008 15:07:23 +0100 Subject: [rsyslog] Selector and MySQL Message-ID: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> Hi all, i tried to write data into a DB with a selector in rsyslog.conf: ... !dhcpd $template test, "...",sql :msg, contains, "dinfo" >127.0.0.1,db,user,pw;test #eof one of the messages, which rsyslog is like: 2008-03-03 14:14.20 dhcpd : dinfo:, myipadress and mac So from my point of view the selector should write this message to the DB. BUT it doesn`t. If i replace the selector e.g. with *.info it works (but then all message are written to the DB) Thanks for your support Andre From rgerhards at hq.adiscon.com Mon Mar 3 15:17:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 3 Mar 2008 15:17:21 +0100 Subject: [rsyslog] Selector and MySQL In-Reply-To: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> References: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AC0@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of leitstelle at bnet.li > Sent: Monday, March 03, 2008 3:07 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Selector and MySQL > > Hi all, > > i tried to write data into a DB with a selector > > in rsyslog.conf: > ... > !dhcpd > $template test, "...",sql > :msg, contains, "dinfo" >127.0.0.1,db,user,pw;test > #eof > > one of the messages, which rsyslog is like: > 2008-03-03 14:14.20 dhcpd : dinfo:, myipadress and mac Umm... That message format looks strange. I guess the parser gets confused by it. Can you please run rsyslogd in interactively in debug mode (-d -n) and post the resulting log while such a message is received. Thanks, Rainer > > So from my point of view the selector should write this message to the > DB. > BUT it doesn`t. > > If i replace the selector e.g. with *.info it works (but then all > message are written to the DB) > > Thanks for your support > Andre > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 3 16:27:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 3 Mar 2008 16:27:30 +0100 Subject: [rsyslog] Solaris Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> Hi all, is there anybody on this list with (open) Solaris? I installed a system and tried to compile rsyslog on it, but I can't get the environment ready :( I somebody is in the know, advise would deeply be appreciated (if not, that's a good indication I still do not need to care about Solaris, which also solves my issue ;)). Thanks, Rainer From Mark.Capelle at pcmc.com Tue Mar 4 15:43:19 2008 From: Mark.Capelle at pcmc.com (Capelle, Mark (PCMC-GB)) Date: Tue, 4 Mar 2008 08:43:19 -0600 Subject: [rsyslog] Recording IP address of sender in logs Message-ID: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> I have been using syslogd 1.4.1 for quite some time to gather syslogs from multiple hosts which I then have a script which separates out the logs based on the source host. I just rebuilt my syslog server and took the opportunity to use rsyslogd 2.0.2 instead of the syslogd, but I noticed that my syslogs no longer record the address of the source host which will break my scripts. Is there any way to change this behavior? Examples - Syslogd: Mar 4 08:05:47 10.10.1.1 Mar 04 2008 08:05:47: %ASA-4-106023: Deny tcp src inside:x.x.x.x/3713 dst outside:x.x.x.x/1021 by access-group "inside_access_out" [0x0, 0x0] ^^^^^^^^^ Rsyslogd: Mar 4 08:41:17 Mar 04 2008 08:41:25: %ASA-4-106023: Deny tcp src inside:x.x.x.x/2125 dst outside:x.x.x.x/9018 by access-group "inside_access_out" [0x0, 0x0] Regards, Mark From rgerhards at hq.adiscon.com Tue Mar 4 16:58:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Mar 2008 16:58:23 +0100 Subject: [rsyslog] Recording IP address of sender in logs In-Reply-To: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> References: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AD4@grfint2.intern.adiscon.com> Mark, it looks like message format received is somewhat invalid. Rsyslog tries to parse according to rfc 3164 but also applies legacy legacy parsing if it finds a way to do so. If you could provide me a wireshark capture of the message, I could see if I can modify the parser. HOWEVER, that shouldn't limit to do you what you want. Nothing is fixed in rsyslog and you can modify the format via a template. It is just the default template that gives you the result below. Please see the rsyslog doc on how to define other template. If you have a hard time, let me know and I'll figure it out for you, but I am currently very busy with new work, so I can't promise on the timeline. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Capelle, Mark (PCMC-GB) > Sent: Tuesday, March 04, 2008 3:43 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Recording IP address of sender in logs > > I have been using syslogd 1.4.1 for quite some time to gather syslogs > from multiple hosts which I then have a script which separates out the > logs based on the source host. I just rebuilt my syslog server and > took > the opportunity to use rsyslogd 2.0.2 instead of the syslogd, but I > noticed that my syslogs no longer record the address of the source host > which will break my scripts. Is there any way to change this behavior? > > > > Examples - > > > > Syslogd: > > Mar 4 08:05:47 10.10.1.1 Mar 04 2008 08:05:47: > %ASA-4-106023: Deny tcp src inside:x.x.x.x/3713 dst > outside:x.x.x.x/1021 > by access-group "inside_access_out" [0x0, 0x0] > > ^^^^^^^^^ > > > > Rsyslogd: > > Mar 4 08:41:17 Mar 04 2008 08:41:25: %ASA-4-106023: Deny > tcp src inside:x.x.x.x/2125 dst outside:x.x.x.x/9018 by access-group > "inside_access_out" [0x0, 0x0] > > > > > > Regards, > > Mark > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Thu Mar 6 01:49:46 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 05 Mar 2008 19:49:46 -0500 Subject: [rsyslog] local/remote logging Message-ID: <47CF3FAA.6050302@gmail.com> I want to have logs written locally (the way they usually are) as well as sent to a central logger. The two situations that I want local logs for are: 1) in case the central logger is not available for any reason, I don't lose messages 2) if I need to troubleshoot something, I can just look at the local logs Obviously, I will rotate the local logs and have older ones discarded. Whereas the central logger will archive them for longer periods. Can I accomplish this with rsyslog? AIUI, I can, and I have to move to at least v3.11.x -- is that correct? Assuming yes, I already got the remote logging working. What do I do to get the simultaneous local logging to work too? I can't find any wiki or faq on this. If I get it working, I'd be happy to post it on the site. johnn From mbiebl at gmail.com Thu Mar 6 02:24:12 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Thu, 6 Mar 2008 02:24:12 +0100 Subject: [rsyslog] Solaris In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> Message-ID: 2008/3/3, Rainer Gerhards : > Hi all, > > is there anybody on this list with (open) Solaris? I installed a system > and tried to compile rsyslog on it, but I can't get the environment > ready :( > > I somebody is in the know, advise would deeply be appreciated (if not, > that's a good indication I still do not need to care about Solaris, > which also solves my issue ;)). http://blogs.sun.com/AngeloJ/entry/virtualbox_and_opensolaris_indiana ? -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Wed Mar 5 19:16:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 05 Mar 2008 19:16:18 +0100 Subject: [rsyslog] local/remote logging In-Reply-To: <47CF3FAA.6050302@gmail.com> References: <47CF3FAA.6050302@gmail.com> Message-ID: <1204740978.567.17.camel@localhost.localdomain> Hi Johnn, On Wed, 2008-03-05 at 19:49 -0500, Johnny Tan wrote: > I want to have logs written locally (the way they usually > are) as well as sent to a central logger. > > The two situations that I want local logs for are: > > 1) in case the central logger is not available for any > reason, I don't lose messages You don't need to lose message when the remote host is unavailable. Use queued mode. This is currently documented for the database, but works for any other action as well: http://www.rsyslog.com/doc-rsyslog_high_database_rate.html > > 2) if I need to troubleshoot something, I can just look at > the local logs > > Obviously, I will rotate the local logs and have older ones > discarded. Whereas the central logger will archive them for > longer periods. > > > Can I accomplish this with rsyslog? AIUI, I can, and I have > to move to at least v3.11.x -- is that correct? Indeed. But don't use anything earlier than 3.12.0 - if you do, you'll run into a couple of bugs. > Assuming yes, I already got the remote logging working. What > do I do to get the simultaneous local logging to work too? Rsyslog carries out whatever you configure. So just add a rule for local logging, e.g. *.* -/var/log/local-logfile That's it ;) > I can't find any wiki or faq on this. If I get it working, > I'd be happy to post it on the site. That would be much appreciated! HTH Rainer From rgerhards at hq.adiscon.com Thu Mar 6 12:24:16 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 12:24:16 +0100 Subject: [rsyslog] rsyslog on HP UX Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B02@grfint2.intern.adiscon.com> Hi all, thanks to HP's testdrive, I got rsyslog to compile and basically run on HP UX. My testing abilities on the testdrive are limited (non-root access), but I was successfully able to receive a UDP syslog message and store it to a file. So the engine and the plugins seem to run quite OK. If anybody has HP UX and is interested in trying rsyslog out, please let me know. Obviously, you'll receive great support if you try ;) The HP UX "support" will be part of 3.12.1, which I hope to release soon (today or tomorrow). During the compile, there is a set of warnings that I do not fully understand. If somebody can shed some light on them, I'd deeply appreciate that: net.c: In function `AddAllowedSender': net.c:203: warning: cast increases required alignment of target type net.c:212: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:290: warning: cast increases required alignment of target type net.c:294: warning: cast increases required alignment of target type net.c:295: warning: cast increases required alignment of target type net.c:296: warning: cast increases required alignment of target type net.c: In function `MaskCmp': net.c:491: warning: cast increases required alignment of target type net.c:492: warning: cast increases required alignment of target type net.c:502: warning: cast increases required alignment of target type net.c:503: warning: cast increases required alignment of target type net.c:512: warning: cast increases required alignment of target type net.c:513: warning: cast increases required alignment of target type net.c:513: warning: cast increases required alignment of target type net.c:516: warning: cast increases required alignment of target type net.c:517: warning: cast increases required alignment of target type net.c: In function `debugListenInfo': net.c:744: warning: cast increases required alignment of target type net.c:749: warning: cast increases required alignment of target type No suffix list. They seem to be related to casting IPv4/IPv6 structures. Feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Thu Mar 6 16:39:10 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 16:39:10 +0100 Subject: [rsyslog] rsyslog 3.12.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B0E@grfint2.intern.adiscon.com> Hi all, Rsyslog 3.12.1 is available immediately. There has been a log going on under the hood, code has been cleaned up and become more modular (you'll see more of that in the upcoming releases) and a library plugin-interface has been added (which so far is not really visible to the end user). We also worked a bit on portability and rsyslogd now compiles under HP UX. We are looking for testers, so if you have HP UX, you are invited to try it out. Feature-wise, a new syslog mib as been added, runtime debugging support enhanced and the ability to discard timestamps received via unix sockets has now been added (and is the default). This is necessary to keep time information consistent even if local programs log with wrong time information (what they do in practice). The gssapi code has greatly been changed and uses the new dynamic plugin library system. Consequently, some trouble in this regard cannot be ruled out ;). Finally, there is an important bugfix: failed actions were not properly retried, causing message loss in a destination-failure scenario. This is now corrected. Download: http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml Change Log: http://www.rsyslog.com/Article187.phtml As always, feedback is appreciated. Rainer Gerhards From stephen.carville at gmail.com Thu Mar 6 18:54:06 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Thu, 6 Mar 2008 09:54:06 -0800 Subject: [rsyslog] Log watch software Message-ID: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> I have a cenltralized repository usng rsyslogm and syslog to mirror /var/log/messages, /var/log/secure ,and information messages from cfengine. In the near future I hope to get auditd reporting to a central server. My immedate taks is to add some log analysis software on teh central server. I've started modifiying LogWatch to work with MySQL -- thats pretty straightforward -- but I'm curious what other solutions there may be out there. FOSS is preferred but a I'm not against a reasonably priced commercial product. So far everything Google has returned are commercial products for Windows sytems. -- Stephen Carville From rgerhards at hq.adiscon.com Thu Mar 6 18:55:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 18:55:51 +0100 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> I am not so involved with logwatch. Let me ask feature-wise: what capabilities do you need to do the job? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: Thursday, March 06, 2008 6:54 PM > To: rsyslog-users > Subject: [rsyslog] Log watch software > > I have a cenltralized repository usng rsyslogm and syslog to mirror > /var/log/messages, /var/log/secure ,and information messages from > cfengine. In the near future I hope to get auditd reporting to a > central server. My immedate taks is to add some log analysis software > on teh central server. I've started modifiying LogWatch to work with > MySQL -- thats pretty straightforward -- but I'm curious what other > solutions there may be out there. FOSS is preferred but a I'm not > against a reasonably priced commercial product. So far everything > Google has returned are commercial products for Windows sytems. > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From Gerrard.Geldenhuis at datacash.com Thu Mar 6 18:57:35 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Thu, 6 Mar 2008 17:57:35 -0000 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> Message-ID: Hi Stephen, Did you mean to ask a question... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: 06 March 2008 17:54 > To: rsyslog-users > Subject: [rsyslog] Log watch software > > I have a cenltralized repository usng rsyslogm and syslog to mirror > /var/log/messages, /var/log/secure ,and information messages from > cfengine. In the near future I hope to get auditd reporting to a > central server. My immedate taks is to add some log analysis software > on teh central server. I've started modifiying LogWatch to work with > MySQL -- thats pretty straightforward -- but I'm curious what other > solutions there may be out there. FOSS is preferred but a I'm not > against a reasonably priced commercial product. So far everything > Google has returned are commercial products for Windows sytems. > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From stephen.carville at gmail.com Thu Mar 6 20:44:11 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Thu, 6 Mar 2008 11:44:11 -0800 Subject: [rsyslog] Log watch software In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> Message-ID: <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> On Thu, Mar 6, 2008 at 9:55 AM, Rainer Gerhards wrote: > I am not so involved with logwatch. Let me ask feature-wise: what > capabilities do you need to do the job? About 99% of what's in messages or secure is trivia. JoeBob logged in, ran a sudo command and logged off. An authenticated mount request was received from ip.add.re.ss. That sort of thing. What I'm looking for is a parser that can pick out the (hopefully) rare messages that indicates a problem like a disk drive is reporting errors. I can modify big brother and logwatch to do this but I am curious if anyone has a favorite package I haven't heard of yet. > Rainer > > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > > Sent: Thursday, March 06, 2008 6:54 PM > > To: rsyslog-users > > Subject: [rsyslog] Log watch software > > > > I have a cenltralized repository usng rsyslogm and syslog to mirror > > /var/log/messages, /var/log/secure ,and information messages from > > cfengine. In the near future I hope to get auditd reporting to a > > central server. My immedate taks is to add some log analysis software > > on teh central server. I've started modifiying LogWatch to work with > > MySQL -- thats pretty straightforward -- but I'm curious what other > > solutions there may be out there. FOSS is preferred but a I'm not > > against a reasonably priced commercial product. So far everything > > Google has returned are commercial products for Windows sytems. > > > > -- > > Stephen Carville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Stephen Carville From rgerhards at hq.adiscon.com Thu Mar 6 21:27:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 21:27:13 +0100 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> Hi, I am seeing where you come from. That's the million-dollar question ;) I suggest you also post to the loganalysis list, that's probably a better place than over here: http://www.loganalysis.org/mailman/listinfo/loganalysis Let me hijack this thread to share an idea. Rsyslog has a lot of infrastructure in place. Once I am finished with the essentials (which will of course be in a few month...), I'd like to put that infrastructure to better use than just drive the simple outputs we currently have. One thing I have on my mind is an output plugin which stores (hashes) of all message within a timeframe (e.g. last 7 days). Then, when a new message comes in, it compares it to all previous messages and emits a special message itself if the message occured less than "n" times in the past. I think this goes into the direction of what you are looking for. But would it generally be considered to be a useful idea? Even though we are months away from an implementation, feedack would be very valuable to me as it helps me shape my mid- to long-term direction. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Stephen Carville > Sent: Thursday, March 06, 2008 8:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] Log watch software > > On Thu, Mar 6, 2008 at 9:55 AM, Rainer Gerhards > wrote: > > I am not so involved with logwatch. Let me ask feature-wise: what > > capabilities do you need to do the job? > > About 99% of what's in messages or secure is trivia. JoeBob logged > in, ran a sudo command and logged off. An authenticated mount request > was received from ip.add.re.ss. That sort of thing. What I'm looking > for is a parser that can pick out the (hopefully) rare messages that > indicates a problem like a disk drive is reporting errors. > > I can modify big brother and logwatch to do this but I am curious if > anyone has a favorite package I haven't heard of yet. > > > Rainer > > > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > > > Sent: Thursday, March 06, 2008 6:54 PM > > > To: rsyslog-users > > > Subject: [rsyslog] Log watch software > > > > > > I have a cenltralized repository usng rsyslogm and > syslog to mirror > > > /var/log/messages, /var/log/secure ,and information messages from > > > cfengine. In the near future I hope to get auditd reporting to a > > > central server. My immedate taks is to add some log > analysis software > > > on teh central server. I've started modifiying LogWatch > to work with > > > MySQL -- thats pretty straightforward -- but I'm curious > what other > > > solutions there may be out there. FOSS is preferred but a I'm not > > > against a reasonably priced commercial product. So far > everything > > > Google has returned are commercial products for Windows sytems. > > > > > > -- > > > Stephen Carville > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Thu Mar 6 21:47:47 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 06 Mar 2008 15:47:47 -0500 Subject: [rsyslog] local/remote logging In-Reply-To: <1204740978.567.17.camel@localhost.localdomain> References: <47CF3FAA.6050302@gmail.com> <1204740978.567.17.camel@localhost.localdomain> Message-ID: <47D05873.60300@gmail.com> Rainer Gerhards wrote: > Hi Johnn, > > On Wed, 2008-03-05 at 19:49 -0500, Johnny Tan wrote: >> I want to have logs written locally (the way they usually >> are) as well as sent to a central logger. >> >> The two situations that I want local logs for are: >> >> 1) in case the central logger is not available for any >> reason, I don't lose messages > > You don't need to lose message when the remote host is unavailable. Use > queued mode. This is currently documented for the database, but works > for any other action as well: Hmm... and this is doable even with 2.x, right? So I could have it queued in memory buffer long enough to bring it back up or bring up another one with same ip to accept the messages? >> Assuming yes, I already got the remote logging working. What >> do I do to get the simultaneous local logging to work too? > > Rsyslog carries out whatever you configure. So just add a rule for local > logging, e.g. > > *.* -/var/log/local-logfile Awesome! Thanks. I'll post on the wiki once I get it working. johnn From rgerhards at hq.adiscon.com Thu Mar 6 21:53:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 21:53:59 +0100 Subject: [rsyslog] local/remote logging In-Reply-To: <47D05873.60300@gmail.com> References: <47CF3FAA.6050302@gmail.com><1204740978.567.17.camel@localhost.localdomain> <47D05873.60300@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B18@grfint2.intern.adiscon.com> > >> 1) in case the central logger is not available for any > >> reason, I don't lose messages > > > > You don't need to lose message when the remote host is > unavailable. Use > > queued mode. This is currently documented for the database, > but works > > for any other action as well: > > Hmm... and this is doable even with 2.x, right? Sorry, Dude, no such queues in v2 :-( > So I could have it queued in memory buffer long enough to > bring it back up or bring up another one with same ip to > accept the messages? It's *not* a main memory buffer. If you configure it to use disk *and* you have enough disk space, it can queue for ages. It's pretty advanced, the full doc on queues is here: http://www.rsyslog.com/doc-queues.html The disk, of course, is only touched if needed. If the destination server is only offline for a brief period, messages may be buffered in memory only. But if the volume is too large or you stop rsyslogd for a while, it is written disk. So if you have enough disk space and your destination dies on Friday and you are too lazy to come in over the weekend, rsyslogd will send all queued elements after the restart on Monday. The same, of course, works if you are on vacation ;) Rainer From Gerrard.Geldenhuis at datacash.com Fri Mar 7 10:34:17 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Fri, 7 Mar 2008 09:34:17 -0000 Subject: [rsyslog] Log watch software In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com><2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> Message-ID: > Let me hijack this thread to share an idea. Rsyslog has a lot of > infrastructure in place. Once I am finished with the essentials (which > will of course be in a few month...), I'd like to put that > infrastructure to better use than just drive the simple outputs we > currently have. One thing I have on my mind is an output plugin which > stores (hashes) of all message within a timeframe (e.g. last 7 days). > Then, when a new message comes in, it compares it to all previous > messages and emits a special message itself if the message occured less > than "n" times in the past. I think this goes into the direction of what > you are looking for. > > But would it generally be considered to be a useful idea? Even though we > are months away from an implementation, feedack would be very valuable > to me as it helps me shape my mid- to long-term direction. > > Rainer Just thinking out loud... it would be very cool if one could build in some AI in such a plugin. You could then spend time "training" the plugin and buying "trained" AI's to regonize certain patterns in the logs etc. Regards From linuxweb at gmail.com Sat Mar 8 00:22:11 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 07 Mar 2008 18:22:11 -0500 Subject: [rsyslog] wiki with various application logging examples Message-ID: <47D1CE23.2010005@gmail.com> Just wondering if there is a wiki with examples of how to log to syslog/rsyslog for various apps. For my needs, I was specifically looking for apache, mysql, tomcat, puppet, etc. For apache, I already figured out error_log can just point to syslog, but access_log, you do: CustomLog "| /usr/bin/logger -p local0.info" combined Do I do something similar for any application where I can specify the logfile destination? johnn From rgerhards at hq.adiscon.com Sat Mar 8 13:54:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 8 Mar 2008 13:54:19 +0100 Subject: [rsyslog] wiki with various application logging examples Message-ID: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> I think nobody has added descriptions so far, but maybe that changes now ;) On the file logs - I suggest you look into imfule, that sound much cleaner and reliable to me than the logger trick. rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 08.03.08 00:22 Betreff: [rsyslog] wiki with various application logging examples Just wondering if there is a wiki with examples of how to log to syslog/rsyslog for various apps. For my needs, I was specifically looking for apache, mysql, tomcat, puppet, etc. For apache, I already figured out error_log can just point to syslog, but access_log, you do: CustomLog "| /usr/bin/logger -p local0.info" combined Do I do something similar for any application where I can specify the logfile destination? johnn _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 8 17:04:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 8 Mar 2008 17:04:18 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Just to facilitate things: this is what I am thinking about: http://www.rsyslog.com/doc-imfile.html It can monitor all application log files natively. Did I understand your need right? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Rainer Gerhards > Sent: Saturday, March 08, 2008 1:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > I think nobody has added descriptions so far, but maybe that > changes now ;) > > On the file logs - I suggest you look into imfule, that sound > much cleaner and reliable to me than the logger trick. > > rainer > > ----- Urspr?ngliche Nachricht ----- > Von: "Johnny Tan" > An: "rsyslog-users" > Gesendet: 08.03.08 00:22 > Betreff: [rsyslog] wiki with various application logging examples > > Just wondering if there is a wiki with examples of how to > log to syslog/rsyslog for various apps. > > For my needs, I was specifically looking for apache, mysql, > tomcat, puppet, etc. > > For apache, I already figured out error_log can just point > to syslog, but access_log, you do: > CustomLog "| /usr/bin/logger -p local0.info" combined > > Do I do something similar for any application where I can > specify the logfile destination? > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Sun Mar 9 01:14:47 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Sat, 08 Mar 2008 19:14:47 -0500 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: <47D32BF7.5030108@gmail.com> Rainer Gerhards wrote: > Just to facilitate things: this is what I am thinking > about: > > http://www.rsyslog.com/doc-imfile.html > > It can monitor all application log files natively. Did I > understand your need right? Yes, that's exactly what I was looking for. Thanks, really really awesome! One quick error -- in your Sample, you use: $InputFilePollingInterval 10 It should be $InputFilePollInterval (no -ing on Poll) Thanks again! johnn From janfrode at tanso.net Sun Mar 9 12:56:15 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Mar 2008 12:56:15 +0100 Subject: [rsyslog] wiki with various application logging examples References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: On 2008-03-08, Rainer Gerhards wrote: > Just to facilitate things: this is what I am thinking about: > > http://www.rsyslog.com/doc-imfile.html > > It can monitor all application log files natively. Did I understand your need right? > Seems like "imfile" could be trivially implemented trough: while sleep $InputFilePollInterval ; do logtail $InputFileName $InputFileStateFile | \ logger -t $InputFileTag -p $InputFileFacility.$InputFileSeverity done http://debian.stro.at/bzr-test/logtail/ Or for realtime monitoring: tail --follow=name --retry $InputFileName | \ logger -t $InputFileTag -p $InputFileFacility.$InputFileSeverity IMHO rsyslogd might be trying to do too much internally. When will it start reading mail ? That might not sound too far fetched.. :-) "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." -- jwz -jf From mbiebl at gmail.com Sun Mar 9 15:02:04 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sun, 9 Mar 2008 15:02:04 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: 2008/3/9, Jan-Frode Myklebust : > IMHO rsyslogd might be trying to do too much internally. When will it start > reading mail ? That might not sound too far fetched.. :-) > > "Every program attempts to expand until it can read mail. Those programs > which cannot so expand are replaced by ones which can." -- jwz To some extent, I share your concerns. I very much believe in the Unix philosophy of small tools, which do one job, and do that right and can be chained/combined easily. rsyslog should keeps it's focus and stay what it's primary focus is, a syslog daemon. Taking your email example: Having the functionality to send out email notifications on certain events is certainly useful. But we should not build the email sending functionality into rsyslog itself, but simply hand out (pipe) the data to external tools (mail/sendmail). The point is, what we can expect to be available on a given system or not ("logtail" probably isn't, "mail" very likely is). Also, as it's implemented as a loadable module, it doesn't clutter the main rsyslog code and so keeps the code base maintainable. But again ,I think your concern is valid and we should evaluate carefully what to implement directly into rsyslog and where it makes more sense to utilise external tools. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sun Mar 9 18:38:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 9 Mar 2008 18:38:00 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Hi all, and especially JF and Michael, Good discussion, please keep the thoughts flowing :) Let me start by tell you I am guilty of a much broader view of what a syslogd can do than most people probably have. To me, a syslogd processes *system events* and that's much more than just those pieces sent via syslog. I know that I should probably stopped the rsyslog project at some point in time and fork an eventlogd from it. But, why would this have made any sense? To me, the core point is maintainability of code and the freedom for people to use only (and exactly) what they need (and want!) to use. After all, there are also lots of folks how say that database support has nothing to do with a syslogd and thus should not be part of if (and that was the birthplace of rsyslog... ;)). So for me the natural thing was to make rsyslog pluggable AND keep only in the actual core what is ultimately needed. I got very serious about this last summer. There are a couple of posts from last August in my blog and I'd like to highlight this one: http://rgerhards.blogspot.com/2007/08/on-importance-of-plug-ins-for-rsys log.html We have now reached a point where we have a quite well implemented plugin interface and rsyslogd has become much more modular. Still, there are things that people need to run even if they never need them (like the syslog-protocol message parser and a couple of "standard" outputs that come with sysklogd). I am continuously working to remove these dependencies. For example, on Friday I moved regex support out to a separate module. So you no longer need to load regexpes if you do not need them. Now to the plugins. What is a plugin? Good question. A plugin is *a project in itself*. Just for convenience, it is part of the main tarball (we had a couple of real headaches when we tried to separate the initial plugins and came to the conclusion that it is by far more convenient to deliver them as part of the main tarball). Plugins need not be written by the rsyslog team. Maybe somebody has already written an email reception plugin and just not told us - may very well be (though I doubt ;)). Now let me take on the imfile example. The key point - at least IMHO - is that there is no single line in rsyslog core's code that has been added in support of imfile. And, more importantly, if imfile would go away, not a single line of code could be removed. So the imfile plugin (project) does neither add complexity nor code nor other overhead to rsyslog core. So can it be evil? Imfile was added in response of a user request. When I heared it, I thought it would be useful to have such a plugin. Did I know about the logger approach? Yes - and I have read some many folks complain over this or that aspect of a logger-approach that I knew I didn't want it. Maybe these were all false claims, maybe people simply did not understand to do it right. I don't know and I honestly admit that I don't care. The presence of many people feeling bugged about that approach made me not like it (if you look for actual samples, I suggest going through the loganalysis mailing list archive). Did I know about logtail? No. Had I written imfile if I did? Not sure: from the small shell script you provided, it looks like there is a problem if a) script is in sleep period b) data is appended to text file c) text file is rotated d) new lines are written to text file e) script awake for new polling loop On a quick look, it looks like the data written in b) will never make it to the syslogd. Imfile handles that. But even if logtail (or the script) handles it - as I said, I did not know about it. So I took roughly a day of my time to create an initial imfile. If logtail handles the situation described above, would be available on the platforms I was asked for and I knew about it, I'd probably refrained from implementing imfile. But, again, no harm happened. If you don't like imfile, simply don't use it. It adds nothing to the rsyslog core. You can still use the logtail approach. IMHO this is what we want: freedom of choice. And rsyslog provides this freedom. On the mail output case (though I need to be a bit brief as dinner is approaching ;)): I actually intend to add an email output plugin. It's on the list for long a time. I personally see some benefit in the ability to talk to a tightly coupled, in-memory, very performant accessible way to send mail message (just like with the SNMP traps). On the other hand, you can see that I didn't implement it for a long time - simply because I did not consider it important enough given what else needs to be done. But I now need to speak up, because I have actually begun to seriously look into implementing it before this discussion begun ;) Before I do, I will make sure I understand that other options and if there is some benefit in doing so. But if I implement it for one reason or the other, nothing bad happens to rsyslog: if you don't like it, you don't need to use it. It doesn't add any overhead to the core. And now please feedback. Am I overlooking something real evil? What is the bad thing in offering exotic functionality that only few will ever need? Thanks again for the discussion, I am *really* interested in getting more feedback on this topic. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Sunday, March 09, 2008 3:02 PM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > 2008/3/9, Jan-Frode Myklebust : > > IMHO rsyslogd might be trying to do too much internally. When will > it start > > reading mail ? That might not sound too far fetched.. :-) > > > > "Every program attempts to expand until it can read mail. > Those programs > > which cannot so expand are replaced by ones which can." -- > jwz > > To some extent, I share your concerns. > I very much believe in the Unix philosophy of small tools, which do > one job, and do that right and can be chained/combined easily. > > rsyslog should keeps it's focus and stay what it's primary focus is, a > syslog daemon. > Taking your email example: > Having the functionality to send out email notifications on certain > events is certainly useful. But we should not build the email sending > functionality into rsyslog itself, but simply hand out (pipe) the data > to external tools (mail/sendmail). > > The point is, what we can expect to be available on a given system or > not ("logtail" probably isn't, "mail" very likely is). Also, as it's > implemented as a loadable module, it doesn't clutter the main rsyslog > code and so keeps the code base maintainable. > > But again ,I think your concern is valid and we should evaluate > carefully what to implement directly into rsyslog and where it makes > more sense to utilise external tools. > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Sun Mar 9 23:05:10 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Mar 2008 23:05:10 +0100 Subject: [rsyslog] wiki with various application logging examples References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Message-ID: On 2008-03-09, Rainer Gerhards wrote: > > Now let me take on the imfile example. The key point - at least IMHO - > is that there is no single line in rsyslog core's code that has been > added in support of imfile. And, more importantly, if imfile would go > away, not a single line of code could be removed. So the imfile plugin > (project) does neither add complexity nor code nor other overhead to > rsyslog core. So can it be evil? It's evil that you're forcing me to upgrade to rsyslog v3.x to take advantage of it ;-) That's what triggered my previous post.. I want to be tracking non-syslog logfiles, and if imfile/plugins where more in the unix philosophy of small tools that chain easily.. I might have been able to pick it from v3.x and use it on stable systems. > from the small shell script you provided, it looks like there is a > problem if > > a) script is in sleep period > b) data is appended to text file > c) text file is rotated > d) new lines are written to text file > e) script awake for new polling loop > > On a quick look, it looks like the data written in b) will never make it > to the syslogd. Imfile handles that. Yes, you're right. Cool that imfile handles it. > On the mail output case (though I need to be a bit brief as dinner is > approaching ;)): I actually intend to add an email output plugin. The quote was about being able to *read* mail: "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." and I was thinking it not too far fetched since "splunk" can do it (it can download email messages via IMAP, index them and create alerts on suspicious content). It might seem like you want to take rsyslog in that direction, i.e. your complete eventlogd&alertSystem fork of rsyslogd that can read any input (syslog, other-logfile, email, snmptraps) and analyze and alert on the data. That's not what *I* want from a syslog server. I just want it to reliably collect and store the logs in an organized manner. Then I'll use other tools to read and analyze them. -jf From rgerhards at hq.adiscon.com Mon Mar 10 08:52:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Mar 2008 08:52:46 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B3B@grfint2.intern.adiscon.com> Hi JF, thanks for the note. Of course, it triggers a couple of responses (see below). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Sunday, March 09, 2008 11:05 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] wiki with various application logging examples > > On 2008-03-09, Rainer Gerhards wrote: > > > > Now let me take on the imfile example. The key point - at least IMHO > - > > is that there is no single line in rsyslog core's code that has been > > added in support of imfile. And, more importantly, if imfile would go > > away, not a single line of code could be removed. So the imfile > plugin > > (project) does neither add complexity nor code nor other overhead to > > rsyslog core. So can it be evil? > > It's evil that you're forcing me to upgrade to rsyslog v3.x to take > advantage of it ;-) That's what triggered my previous post.. I want > to be tracking non-syslog logfiles, and if imfile/plugins where more > in the unix philosophy of small tools that chain easily.. I might have > been able to pick it from v3.x and use it on stable systems. I've see the smily but let me do a couple of comments. First of all, the "philosophy of small tools that chain easily" implies that they chain via a pipe. This is great for a lot of applications, but it has its drawbacks. A plain pipe is a simplex, relatively loosely coupled IPC method. So if one part of the pipe dies, other parts will learn about it eventually, but not at the same instant and, most importantly, they do not exactly know what was processed and what not. It's pretty much the same thing as plain TCP transport, which pretends reliability but still has a few windows of exposure where message loss may occur (see http://www.monitorware.com/en/workinprogress/selp.txt section 2.4 and, yes, rsyslog is victim to this as long as we don't have a full RFC 3195 implementation). So relying on plain pipes is not exactly my premier communications method if I would like to implement a reliable syslogd (and "reliable" is the "r" in rsyslog). One can circumvent the problem by defining an app-level protocol on top of the pipe, working with app-level acks. HOWEVER, with that you would run into the pretty same situation, namely that you could not use a new version together with an old engine. One can circumvent that, too, but only as far as the old engine has implemented such methodology. And v1/2 do not have this (due to time constraints, rsyslog is still only about half finished...). There is some other reason that makes me avoid pipes. If I do an app-level ack, I need to do at least 4 system calls to pass a single message: 1. input write to pipe 2. engine read msg from pipe 3. engine post processing stat (ACK) to pipe 4. input read ACK from pipe If you do a little bit of math, you'll see with how many user/kernel space transitions you end up, plus how many cycles are needed to run the necessary housekeeping code inside the kernel and libraries. I don't sum them up now, but I am pretty sure that I can completely process the message in less time than is need just for the IPC in that case. That probably is fine if you look at a low-end workstation syslogd, but you don't want to have this overhead if you aim for an enterprise solutions capable of handling massive data input. Also, if everything goes to the system log socket, there is little you can multithread. But we need to be able to multithread as much as possible. If I write a new engine (as I currently do), I would like to see it well working for at least the next 10 years. If I think about hardware trends during that period, it is clear that a single core will become not much faster than it is today. But the number of cores will greatly increase. In order to utilize that, an application must be able to run on as many threads as possible - all with reasonable overhead, of course. So my conclusion is that rsyslog must be able to run massively multithreaded for the high-end use case. This also prohibits using primary interfaces which cannot easily be multithreaded. Finally there is the issue of flow control. Rsyslog *does* flow control, and will do more advanced flow control in the future. Especially with world-dominating UDP syslog it is vitally important to do flow control, because UDP cannot be flow controlled. Sound strange? Well... If we can not flow control UDP syslog, we need to apply intelligent and adoptable flow control that flow controls sources who can (tcp syslog, rfc 3195 and of course file data!) so that buffer space is kept for those precious UDP message which are lost if we can not buffer them at the right instant. If you think this through, you'll see that this requires different level and methods of flow control, depending on the source [so far, I see three levels: can not (UDP), can somewhat (TCP, local sockets), can easily (log files and other sources that generate data themselves)]. Having a native interface greatly reduces code complexity and thus program reliability when it comes at implementing these features. Also, it would require specialized plugins in any case, you couldn't do it with a simple "pipe me in approach" (well, ... some things yes, but at a complexity cost). Also, while I too believe in the Unix approach of small tools, I also think it is important that the average user is able to configure it. Rsyslog aims not only at being enterprise-class but at the some time aims at being easy to use for the novice. Novices don't understand complex scripting to get the job done. I think relying too much on complex glue doesn't help getting the job done. To come back to your orginal post, rsyslogd *should* of course support the Unix way of piping. It looks like there is a small plugin missing to read natively from a pipe. However, I never got a request to implement it. I guess most people use the logger trick to accomplish that task. I know syslog-ng can natively read from pipes, but have not yet considered this important enough given the lack of requests from the community. If you like such a plugin, it's probably a good idea to speak up now ;) > > > > from the small shell script you provided, it looks like there is a > > problem if > > > > a) script is in sleep period > > b) data is appended to text file > > c) text file is rotated > > d) new lines are written to text file > > e) script awake for new polling loop > > > > On a quick look, it looks like the data written in b) will never make > it > > to the syslogd. Imfile handles that. > > Yes, you're right. Cool that imfile handles it. > > > > On the mail output case (though I need to be a bit brief as dinner is > > approaching ;)): I actually intend to add an email output plugin. > > The quote was about being able to *read* mail: I was replying here to Michael Biebel. It looks I was a bit too much in hurry to point this out. Sorry... > > "Every program attempts to expand until it can read mail. > Those programs which cannot so expand are replaced by ones > which can." > > and I was thinking it not too far fetched since "splunk" can do it (it > can > download email messages via IMAP, index them and create alerts on > suspicious > content). In short: receiving email is very low on my agenda. Keep in mind that I already have architecture and at least partly written such a beast on Windows: http://www.monitorware.com/en/Product/product_comparision.php The email question never was in much demand. But if demand comes up, it for sure is not a big thing to add it... (and, of course, you already can do it today with a bit of scripting, the right mailbox rules and logger -- but that isn't appealing to most folks and is one reason I tend to write plugins ;)). > It might seem like you want to take rsyslog in that > direction, i.e. > your complete eventlogd&alertSystem fork of rsyslogd that can read any > input > (syslog, other-logfile, email, snmptraps) and analyze and alert on the > data. > > That's not what *I* want from a syslog server. I just want it to > reliably > collect and store the logs in an organized manner. Then I'll use other > tools to read and analyze them. ... and this is of course perfectly fine with me, too. However, if you look at the core engine needs, you'll see that the "do it all" eventprocessor and the "plain simple syslogd" have exactly the same needs - at least if you would like to extend the syslogd to be enterprise class. Take the queue engine in v3. Its complex. Actually horribly complex. I didn't like to include that complex beast, but it was the only clean solution to the need of being massively concurrent AND being able to queue data while a destination is down. Any other alternative IMHO would have been dirty and hackish - and in the long term much less maintainable. So I just did the right thing (hopefully), even though it was a really big effort and even though it probably will need a few more firedrills before it is really ready for prime time in all scenarios. Another alternative would have been to use one of the big enterprise class message queueing projects. But that would have created a dependency for such a system on each desktop - ouch... I hope I made the right compromise. Currently, the full queue engine is part of the core. If that turns out to be a problem, I can outsource that to a plugin, but that isn't currently very appealing to me. After all, it's "just" some memory overhead - if you don't need the advanced features, no code is executed to do that. The queue uses an internal driver model and simply configuration means simple code. Thus, the v3 queue engine is as reliable as v2, except if you use all those bells and whistles where I am sure currently a few bugs wait to be detected (even though the situation has much improved recently and will improve with each new feedback I receive). Another good example is the config file: of course, there is no need to have a scriptable configuration for a simple syslogd. But while thinking about the (necessary) expression support and a lot of user requests for a better to read config file format, I came to the conclusion that creating a scriptable format is actually the right route to take: http://rgerhards.blogspot.com/2008/02/introducing-rainerscript-and-some. html Anything else (IMHO) would again be less clean, less maintainable and, in this case, would even take longer to implement. So one might think it is evil to include a virtual machine inside a syslogd, but to me it is actually the least effort to implement things. Of course, you can rightly argue that all of this is over the board if you just want to have a plain local logger that takes messages from the local log socket, maybe UDP syslog and store it to local files. You are probably right. But in this case, you can still continue to use sysklogd. After all, rsyslog was spawned from it to create and advanced syslogd. So it comes at no surprise that I am adding features which may not be required for the simple use cases. ;) Let me conclude with two core points: - rsyslog core is as slim as possible, plugins are separate projects that extend the core; This means nobody is forced to run more code than actually required for his job - both a simple, but fairly enhanced syslogd as well as a full network Event processor share the same root engine needs Thus, rsyslog implements this core engine and I occasionally add a plugin here and there to take advantage of the core. Right now, rsyslog core is far from being finished, as are the plugins. At this time, I am working on getting the core right and doing the most requested plugins. When I am done with that, I'll look at the *real* advanced plugins for all kinds of things that users have interest in. I don't see any need to fork of a separate core project for that. In fact, I think it would be counter-productive as I would need to maintain two code bases and the newly forked project would always be able to do what rsyslogd does. So what would be the motivation to maintain another, feature-less project...? Anyhow, I may be totally wrong. Feedback on this topic is still highly appreciated (be it brief or elaborate ;)). I am probably abusing this thread to also tell you a bit about design decisions I have not yet communicated (sorry for that, so much to do, so few time...;)). Rainer > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 10 08:54:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Mar 2008 08:54:42 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <47D32BF7.5030108@gmail.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <47D32BF7.5030108@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> Hi Johnn, I'd really appreciate if you could any actual samples (even brief) to the wiki at http://wiki.rsyslog.com They would definitely be of great benefit for other users, too. As you have probably seen in my other mails, I am pretty busy with developing right now so I hope that users contribute use cases. Thanks, Raiener > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Sunday, March 09, 2008 1:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > Rainer Gerhards wrote: > > Just to facilitate things: this is what I am thinking > > about: > > > > http://www.rsyslog.com/doc-imfile.html > > > > It can monitor all application log files natively. Did I > > understand your need right? > > Yes, that's exactly what I was looking for. Thanks, really > really awesome! > > One quick error -- in your Sample, you use: > $InputFilePollingInterval 10 > > It should be $InputFilePollInterval (no -ing on Poll) > > Thanks again! > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Mon Mar 10 17:01:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 10 Mar 2008 12:01:22 -0400 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <47D32BF7.5030108@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> Message-ID: <47D55B52.7010905@gmail.com> Rainer Gerhards wrote: > Hi Johnn, > > I'd really appreciate if you could any actual samples (even brief) to > the wiki at > > http://wiki.rsyslog.com > > They would definitely be of great benefit for other users, too. As you > have probably seen in my other mails, I am pretty busy with developing > right now so I hope that users contribute use cases. Yes, absolutely! I'm a rsyslog convert :). Since we're going live this week, I've got a ton of things on my plate. But I fully intend to put up my samples (however basic they are) on the wiki soon after launch! Thanks for a great program! johnn From maurizio.rottin at gmail.com Tue Mar 11 13:01:57 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Tue, 11 Mar 2008 13:01:57 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log Message-ID: Hi all, i set up an apache webserver wich logs with directives: ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined" then syslog forwards them with local5.err @10.10.10.89 local6.info @10.10.10.89 and it works good! now on 10.10.10.89 i set up an rsyslog server with directives $template ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" local5.err -?ApacheRemoteErr $template ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Custom.log" local6.info -?ApacheRemoteCustom the result is that i get these files in /var/log/rsyslog/Apache/ ./www.mysite.com:/Custom.log with customLog ./error_www.mysite.com:/Err.log with ErroLog ./message/Err.log with lines like Mar 11 12:32:52 last message repeated 9 times Mar 11 12:32:52 last message repeated 5 times (Note that this is very annoying!!!) Now this a simple way i have found to divide logs per VHost. I don't know if there is a better way, but the real question is different: In the Custom or Error logs i have lines like: Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part do i have to use a double template setup? one for the file and one for the content? the template in the file seems to be this: $template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" but i need just $template TheOneINeed,%msg%\n" i want only the %msg%\n in my file Thanks in advance -- mr From linuxweb at gmail.com Tue Mar 11 19:08:06 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 14:08:06 -0400 Subject: [rsyslog] rsyslog v3 and selinux Message-ID: <47D6CA86.6060501@gmail.com> I took Fedora 8's rsyslog v2.0.2 SRPM and rebuilt it for v3.12.1 I'm using the same init script. When I used that init script (i.e., "/etc/init.d/rsyslog start") on v2.0.2, the only SELinux problem I had was the domain transition, as documented here: http://tanso.net/rsyslog/ However, when I rebuilt the RPM for v3.12.1, and used the same init script to run it, I get many SELinux errors. Eventually, I worked out all the things I had to allow for rsyslog v3.12.1 to run properly. It's enclosed below as a semanage module. Just curious if there was some change to rsyslog between versions 2 and 3 which would make rsyslog, even when running properly in domain syslogd_exec_t, to cause so many SELinux denials, including not being able to do TCP bind?? Thanks for any clues, johnn p.s. Here's the module that finally worked. Will document on wiki when all is done. ========== module rsyslog 1.0; require { class dir search; class file { getattr read write }; class filesystem remount; class tcp_socket { create accept read setopt bind name_bind node_bind listen }; type boot_t; type auditd_log_t; type var_log_t; type syslogd_t; type syslogd_port_t; type port_t; type mount_t; type system_map_t; type inaddr_any_node_t; type unspec_node_t; role system_r; }; allow syslogd_t boot_t:dir search; allow syslogd_t auditd_log_t:dir search; allow syslogd_t auditd_log_t:file { getattr read }; allow syslogd_t self:tcp_socket { create accept read setopt bind listen }; allow syslogd_t syslogd_port_t:tcp_socket name_bind; allow syslogd_t port_t:tcp_socket name_bind; allow syslogd_t system_map_t:file { read getattr }; allow syslogd_t inaddr_any_node_t:tcp_socket node_bind; allow syslogd_t unspec_node_t:tcp_socket node_bind; ========== From rgerhards at hq.adiscon.com Tue Mar 11 20:05:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Mar 2008 20:05:13 +0100 Subject: [rsyslog] rsyslog v3 and selinux Message-ID: <005c01c883aa$d9cecd2e$060013ac@intern.adiscon.com> I have absolutely no idea about selinux ... But: may it be that the fact that now plugins - separate libraries - are loaded by rsyslogd to do the job? rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 11.03.08 19:08 Betreff: [rsyslog] rsyslog v3 and selinux I took Fedora 8's rsyslog v2.0.2 SRPM and rebuilt it for v3.12.1 I'm using the same init script. When I used that init script (i.e., "/etc/init.d/rsyslog start") on v2.0.2, the only SELinux problem I had was the domain transition, as documented here: http://tanso.net/rsyslog/ However, when I rebuilt the RPM for v3.12.1, and used the same init script to run it, I get many SELinux errors. Eventually, I worked out all the things I had to allow for rsyslog v3.12.1 to run properly. It's enclosed below as a semanage module. Just curious if there was some change to rsyslog between versions 2 and 3 which would make rsyslog, even when running properly in domain syslogd_exec_t, to cause so many SELinux denials, including not being able to do TCP bind?? Thanks for any clues, johnn p.s. Here's the module that finally worked. Will document on wiki when all is done. ========== module rsyslog 1.0; require { class dir search; class file { getattr read write }; class filesystem remount; class tcp_socket { create accept read setopt bind name_bind node_bind listen }; type boot_t; type auditd_log_t; type var_log_t; type syslogd_t; type syslogd_port_t; type port_t; type mount_t; type system_map_t; type inaddr_any_node_t; type unspec_node_t; role system_r; }; allow syslogd_t boot_t:dir search; allow syslogd_t auditd_log_t:dir search; allow syslogd_t auditd_log_t:file { getattr read }; allow syslogd_t self:tcp_socket { create accept read setopt bind listen }; allow syslogd_t syslogd_port_t:tcp_socket name_bind; allow syslogd_t port_t:tcp_socket name_bind; allow syslogd_t system_map_t:file { read getattr }; allow syslogd_t inaddr_any_node_t:tcp_socket node_bind; allow syslogd_t unspec_node_t:tcp_socket node_bind; ========== _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 11 21:45:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Mar 2008 21:45:00 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Maurizio Rottin > Sent: Tuesday, March 11, 2008 1:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog with apache and per vhost log > > Hi all, > i set up an apache webserver wich logs with directives: > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com > " "combined" > > then syslog forwards them with > local5.err @10.10.10.89 > local6.info @10.10.10.89 > and it works good! > > now on 10.10.10.89 i set up an rsyslog server with directives > $template > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" > local5.err -?ApacheRemoteErr > $template > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C > ustom.log" > local6.info -?ApacheRemoteCustom > > the result is that i get these files in /var/log/rsyslog/Apache/ > ./www.mysite.com:/Custom.log > with customLog > > ./error_www.mysite.com:/Err.log > with ErroLog > > ./message/Err.log > with lines like > Mar 11 12:32:52 last message repeated 9 times > Mar 11 12:32:52 last message repeated 5 times > (Note that this is very annoying!!!) Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd compatibility, -e stand for "log *e*very message") > > > Now this a simple way i have found to divide logs per VHost. > I don't know if there is a better way, but the real question > is different: > In the Custom or Error logs i have lines like: > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.7.13) Gecko/20060607" > > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part > do i have to use a double template setup? one for the file and one for > the content? > This is caused by the default template, which is what most peiople expect in their syslog files. But, as you write, the soluton is quite easy, just use another template ;) > the template in the file seems to be this: > $template TraditionalFormat,%timegenerated% %HOSTNAME% > %syslogtag%%msg%\n" > but i need just > $template TheOneINeed,%msg%\n" > i want only the %msg%\n in my file > So, use local6.info -?ApacheRemoteCustom;TheOneINeed HTH, feel free to ask any follow-up questions. :) Rainer > > Thanks in advance > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 00:42:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 19:42:22 -0400 Subject: [rsyslog] segmentation fault Message-ID: <47D718DE.90603@gmail.com> Ahh, so close yet so far. I've setup a central rsyslog server and am setting up my first client to point to it. I am going over stunnel. I set it up pretty much like it is in the secure rsyslog article. There are some existing logs on the client box. I load imfile.so module to convert those logs to syslog and send them over TCP/stunnel to my central logger. Everytime I try to start it, it dies within seconds. When I turn on debug mode, I notice the client gets a segmentation fault. I have pasted the tail end of the debug output here: http://pastebin.com/m9a83946 Is it just the sheer number of messages that is causing it to segfault, or is something else going on? johnn From linuxweb at gmail.com Wed Mar 12 03:09:00 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 22:09:00 -0400 Subject: [rsyslog] segmentation fault In-Reply-To: <47D718DE.90603@gmail.com> References: <47D718DE.90603@gmail.com> Message-ID: <47D73B3C.1000509@gmail.com> After MUCH trial & error, I've found the problem. If there is a blank line in one of the log files being read by imfile, then rsyslog dies. I filed a bug: http://bugzilla.adiscon.com/show_bug.cgi?id=49 johnn From rgerhards at hq.adiscon.com Wed Mar 12 08:00:40 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 08:00:40 +0100 Subject: [rsyslog] segmentation fault In-Reply-To: <47D73B3C.1000509@gmail.com> References: <47D718DE.90603@gmail.com> <47D73B3C.1000509@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B64@grfint2.intern.adiscon.com> Thanks for finding out. Looks like we got a bug-fixing day today... Will update the bug tracker as things progress. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 3:09 AM > To: rsyslog-users > Subject: Re: [rsyslog] segmentation fault > > After MUCH trial & error, I've found the problem. If there > is a blank line in one of the log files being read by > imfile, then rsyslog dies. > > I filed a bug: > http://bugzilla.adiscon.com/show_bug.cgi?id=49 > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Wed Mar 12 10:47:31 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 10:47:31 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> Message-ID: 2008/3/11, Rainer Gerhards : > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > > Maurizio Rottin > > Sent: Tuesday, March 11, 2008 1:02 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] rsyslog with apache and per vhost log > > > > Hi all, > > i set up an apache webserver wich logs with directives: > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com > > " "combined" > > > > then syslog forwards them with > > local5.err @10.10.10.89 > > local6.info @10.10.10.89 > > and it works good! > > > > now on 10.10.10.89 i set up an rsyslog server with directives > > $template > > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" > > local5.err -?ApacheRemoteErr > > $template > > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C > > ustom.log" > > local6.info -?ApacheRemoteCustom > > > > the result is that i get these files in /var/log/rsyslog/Apache/ > > ./www.mysite.com:/Custom.log > > with customLog > > > > ./error_www.mysite.com:/Err.log > > with ErroLog > > > > ./message/Err.log > > with lines like > > Mar 11 12:32:52 last message repeated 9 times > > Mar 11 12:32:52 last message repeated 5 times > > (Note that this is very annoying!!!) > > > Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd > compatibility, -e stand for "log *e*very message") And i red about the -e options a lot of times...thank! > > > > > > > Now this a simple way i have found to divide logs per VHost. > > I don't know if there is a better way, but the real question > > is different: > > In the Custom or Error logs i have lines like: > > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - > > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 > > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; > > en-US; rv:1.7.13) Gecko/20060607" > > > > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part > > do i have to use a double template setup? one for the file and one for > > the content? > > > > This is caused by the default template, which is what most peiople > expect in their syslog files. But, as you write, the soluton is quite > easy, just use another template ;) > > > > the template in the file seems to be this: > > $template TraditionalFormat,%timegenerated% %HOSTNAME% > > %syslogtag%%msg%\n" > > but i need just > > $template TheOneINeed,%msg%\n" > > i want only the %msg%\n in my file > > > > > So, use > > local6.info -?ApacheRemoteCustom;TheOneINeed works great man! Thanks! > HTH, feel free to ask any follow-up questions. :) > just one question, do you know any way to pass the website name in "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an other program...i know only logger... -- mr From rgerhards at hq.adiscon.com Wed Mar 12 10:50:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 10:50:06 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> [happy snip] > just one question, do you know any way to pass the website name in > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > other program...i know only logger... Well, you could use the imfile plugin to read the log files directly: http://www.rsyslog.com/doc-imfile.html Then, I'd set the site name as tag and so you can process it from the tag value. HTH Rainer From maurizio.rottin at gmail.com Wed Mar 12 11:24:08 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 11:24:08 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > [happy snip] > > > just one question, do you know any way to pass the website name in > > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > > other program...i know only logger... > > > Well, you could use the imfile plugin to read the log files directly: > > http://www.rsyslog.com/doc-imfile.html > > Then, I'd set the site name as tag and so you can process it from the > tag value. > > HTH > > Rainer > mmmmm...i'll check that better but at a glance it seems a heavy process with big logs. -- mr From rgerhards at hq.adiscon.com Wed Mar 12 11:25:53 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 11:25:53 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> > mmmmm...i'll check that better but at a glance it seems a heavy > process with big logs. Sorry, I don't really understand what you mean ;) But the problem is that I probably do not really know how what you do with logger really works. It is part of the apache config, is this right? Maybe you could post a full config sample? Rainer From janfrode at tanso.net Wed Mar 12 12:39:04 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 12 Mar 2008 12:39:04 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> Message-ID: On 2008-03-11, Johnny Tan wrote: > > However, when I rebuilt the RPM for v3.12.1, and used the > same init script to run it, I get many SELinux errors. > Eventually, I worked out all the things I had to allow for > rsyslog v3.12.1 to run properly. It's enclosed below as a > semanage module. These seems very strange: > allow syslogd_t boot_t:dir search; > allow syslogd_t system_map_t:file { read getattr }; Why does it want to read /boot/System.map* ? > allow syslogd_t auditd_log_t:dir search; > allow syslogd_t auditd_log_t:file { getattr read }; And why read /var/log/audit/* ? -jf From rgerhards at hq.adiscon.com Wed Mar 12 13:48:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 13:48:42 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6D@grfint2.intern.adiscon.com> That's part of the klog(d) code I inherited from sysklogd. I've never questioned if there is another way to do it. It seems to use these to obtain kernel symbols. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Wednesday, March 12, 2008 12:39 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog v3 and selinux > > On 2008-03-11, Johnny Tan wrote: > > > > However, when I rebuilt the RPM for v3.12.1, and used the > > same init script to run it, I get many SELinux errors. > > Eventually, I worked out all the things I had to allow for > > rsyslog v3.12.1 to run properly. It's enclosed below as a > > semanage module. > > These seems very strange: > > > allow syslogd_t boot_t:dir search; > > allow syslogd_t system_map_t:file { read getattr }; > > Why does it want to read /boot/System.map* ? > > > allow syslogd_t auditd_log_t:dir search; > > allow syslogd_t auditd_log_t:file { getattr read }; > > And why read /var/log/audit/* ? > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Wed Mar 12 14:37:25 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 14:37:25 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > > mmmmm...i'll check that better but at a glance it seems a heavy > > process with big logs. > > > Sorry, I don't really understand what you mean ;) But the problem is > that I probably do not really know how what you do with logger really > works. It is part of the apache config, is this right? Maybe you could > post a full config sample? > but i did... this is the log part in the directive ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined" in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with facility "-p local*.xxx" with tag "-t www.mysite.com" then i can use the tag to split logs by website in the remote rsyslog machine ;) -- mr From rgerhards at hq.adiscon.com Wed Mar 12 14:44:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 14:44:31 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Wednesday, March 12, 2008 2:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/12, Rainer Gerhards : > > > mmmmm...i'll check that better but at a glance it seems a heavy > > > process with big logs. > > > > > > Sorry, I don't really understand what you mean ;) But the problem is > > that I probably do not really know how what you do with logger > really > > works. It is part of the apache config, is this right? Maybe you > could > > post a full config sample? > > > but i did... > this is the log part in the directive > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > "combined" > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > facility "-p local*.xxx" with tag "-t www.mysite.com" > > then i can use the tag to split logs by website in the remote rsyslog > machine ;) Ah, OK, I didn't realize this was actual Apache config. So your original question > just one question, do you know any way to pass the website name in > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > other program...i know only logger... was how you could use the same apache config command for each log, so that you don't need to have separate apache config statements for each virtual server? Please bear with me if I am getting it wrong, I probably have too much rsyslog on my mind. I am trying to help but I don't get a grip at what the real problem is... Rainer From linuxweb at gmail.com Wed Mar 12 15:08:17 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 10:08:17 -0400 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> Message-ID: <47D7E3D1.9010309@gmail.com> Jan-Frode Myklebust wrote: >> allow syslogd_t auditd_log_t:dir search; >> allow syslogd_t auditd_log_t:file { getattr read }; > > And why read /var/log/audit/* ? This is custom. I prefer to install auditd and have the SELinux logs separated out from /var/log/messages. Then I use imfile to convert the audit logs to syslog in order to send it over to the central logger. Yes, it's a waste of a syslog "local" facility, but I'm not aware of any other way unless I remove auditd and have those go back into /var/log/messages. johnn From linuxweb at gmail.com Wed Mar 12 18:16:56 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 13:16:56 -0400 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <47D81008.3090302@gmail.com> I wanted to setup my rsyslog clients such that when the rsyslog server is down and then comes back up, the clients will resume forwarding messages to it. I'm running into this same bug as another user: http://bugzilla.adiscon.com/show_bug.cgi?id=45 According to that bug, the fix is in 3.12.1, which is what I'm using on client and server. But I still can't seem to get it to work. Prior to doing this, I simply had this line: *.* @@127.0.0.1:61514 Am I supposed to replace that line entirely with the ActionQueue stanza, such as this one (the == is for clarity, not actually part of my config): == $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 30 $ActionResumeRetryCount -1 # infinite retries on insert failure & @@127.0.0.1:61514 == If I do, rsyslog dies immediately on restart. So I changed it so look like this instead: == *.* @@127.0.0.1:61514 $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 & @@127.0.0.1:61514 == This allows rsyslog to work again, but then it doesn't actually do any resume. Just as the OP states in the bug, if the rsyslog server is stopped, the client simply logs messages locally. When the rsyslog server is started again, the client once again logs locally as well as to the server. But all the messages from the client between stop and start on the server are not forwarded. I checked action.c and it does have the code-fix listed in the forum post tied to the bug. I also understand that one message WILL be lost, which is a TCP syslog protocol problem. So I sent multiple messages, none of them came through. johnn From rgerhards at hq.adiscon.com Wed Mar 12 18:22:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:22:12 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D81008.3090302@gmail.com> References: <47D81008.3090302@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B81@grfint2.intern.adiscon.com> I need to check for the restart problem (currently working on something else). Please also follow this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=48 This is the "one message lost" problem, and I have created at least some cure for it today (to be verified). Will check your bug report when I verify it. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 6:17 PM > To: rsyslog-users > Subject: [rsyslog] actionqueue in front of tcp forward > > I wanted to setup my rsyslog clients such that when the > rsyslog server is down and then comes back up, the clients > will resume forwarding messages to it. > > I'm running into this same bug as another user: > http://bugzilla.adiscon.com/show_bug.cgi?id=45 > > According to that bug, the fix is in 3.12.1, which is what > I'm using on client and server. > > > But I still can't seem to get it to work. Prior to doing > this, I simply had this line: > > *.* @@127.0.0.1:61514 > > > > Am I supposed to replace that line entirely with the > ActionQueue stanza, such as this one (the == is for clarity, > not actually part of my config): > > == > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 30 > $ActionResumeRetryCount -1 # infinite retries on insert failure > & @@127.0.0.1:61514 > == > > If I do, rsyslog dies immediately on restart. So I changed > it so look like this instead: > > == > *.* @@127.0.0.1:61514 > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 10 > $ActionResumeRetryCount -1 > & @@127.0.0.1:61514 > == > > This allows rsyslog to work again, but then it doesn't > actually do any resume. Just as the OP states in the bug, if > the rsyslog server is stopped, the client simply logs > messages locally. When the rsyslog server is started again, > the client once again logs locally as well as to the server. > But all the messages from the client between stop and start > on the server are not forwarded. > > I checked action.c and it does have the code-fix listed in > the forum post tied to the bug. > > I also understand that one message WILL be lost, which is a > TCP syslog protocol problem. So I sent multiple messages, > none of them came through. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 18:32:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:32:35 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D81008.3090302@gmail.com> References: <47D81008.3090302@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Argh, some config parsing seems to be wrong. There is a problem with the config file. The right config format is this: == $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 == & in column one is to chain multiple actions together, but it can never be at the start of the chain. Did you take it from some rsyslog documentation? If so and you still know which, please let me know, I'd like to fix that. If I think correctly, the queued action is now chained to the regular action, and probably never executed because the regular action fails ... or something along these lines. Please try the config above, I think it will work (at least it does in my lab). And of course I'll look into the config parser and co ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 6:17 PM > To: rsyslog-users > Subject: [rsyslog] actionqueue in front of tcp forward > > I wanted to setup my rsyslog clients such that when the > rsyslog server is down and then comes back up, the clients > will resume forwarding messages to it. > > I'm running into this same bug as another user: > http://bugzilla.adiscon.com/show_bug.cgi?id=45 > > According to that bug, the fix is in 3.12.1, which is what > I'm using on client and server. > > > But I still can't seem to get it to work. Prior to doing > this, I simply had this line: > > *.* @@127.0.0.1:61514 > > > > Am I supposed to replace that line entirely with the > ActionQueue stanza, such as this one (the == is for clarity, > not actually part of my config): > > == > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 30 > $ActionResumeRetryCount -1 # infinite retries on insert failure > & @@127.0.0.1:61514 > == > > If I do, rsyslog dies immediately on restart. So I changed > it so look like this instead: > > == > *.* @@127.0.0.1:61514 > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 10 > $ActionResumeRetryCount -1 > & @@127.0.0.1:61514 > == > > This allows rsyslog to work again, but then it doesn't > actually do any resume. Just as the OP states in the bug, if > the rsyslog server is stopped, the client simply logs > messages locally. When the rsyslog server is started again, > the client once again logs locally as well as to the server. > But all the messages from the client between stop and start > on the server are not forwarded. > > I checked action.c and it does have the code-fix listed in > the forum post tied to the bug. > > I also understand that one message WILL be lost, which is a > TCP syslog protocol problem. So I sent multiple messages, > none of them came through. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 18:50:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:50:01 +0100 Subject: [rsyslog] rsyslog 2.0.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> Hi all, Rsyslog 2.0.3, a release of the stable branch, has been released today. It is a purely bug-fixing release, solving mostly bugs that occur either seldom or due to mistakes in the configuration file. It fixes some potential segfaults in those cases. Rsyslog 2.0.3 is a recommended update for all users of the rsyslog v2 branch. Download: http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml Changelog: http://www.rsyslog.com/Article189.phtml As always, feedback is appreciated. Rainer Gerhards From linuxweb at gmail.com Wed Mar 12 19:27:38 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 14:27:38 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> References: <47D81008.3090302@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Message-ID: <47D8209A.3070908@gmail.com> Rainer Gerhards wrote: > & in column one is to chain multiple actions together, but it can never > be at the start of the chain. Did you take it from some rsyslog > documentation? If so and you still know which, please let me know, I'd > like to fix that. No, I copied it from the bug report I listed. The OP must've had a regular action that wasn't listed. Anyway, I fixed it, restarted client rsyslog, but the behavior is the same. I don't need any special config on the server side, right? Well, if it helps, here's the top of my client /etc/rsyslog.conf, followed by the top of my server /etc/rsyslog.conf: == client /etc/rsyslog.conf == # Modules $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 # Standard syslog configuration (for local logging) [snip] == == server /etc/rsyslog.conf == # Modules $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so # TCP port $InputTCPServerRun 61514 # Global Directives $AllowedSender UDP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $AllowedSender TCP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $DirOwner root $DirGroup root $DirCreateMode 0755 # Templates [snip] # Standard syslog configuration [snip] == From linuxweb at gmail.com Wed Mar 12 19:32:15 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 14:32:15 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> References: <47D81008.3090302@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Message-ID: <47D821AF.9030107@gmail.com> Rainer Gerhards wrote: > $WorkDirectory /tmp/rsyslogwork Also, I should also point out that I NEVER see anything show up in my WorkDirectory. That directory above is owned by root:root and mode 0755. I turned off SELinux. Nothing EVER shows up there. johnn From milton at calnek.com Wed Mar 12 19:22:10 2008 From: milton at calnek.com (Milton Calnek) Date: Wed, 12 Mar 2008 12:22:10 -0600 Subject: [rsyslog] rsyslog 2.0.3 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> Message-ID: <47D81F52.2050701@calnek.com> The link in the web site does not work. The requested URL /rsyslog/rsyslog-2.0.23tar.gz was not found on this server. Thanks. Rainer Gerhards wrote: > Hi all, > > Rsyslog 2.0.3, a release of the stable branch, has been released today. > It is a purely bug-fixing release, solving mostly bugs that occur either > seldom or due to mistakes in the configuration file. It fixes some > potential segfaults in those cases. Rsyslog 2.0.3 is a recommended > update for all users of the rsyslog v2 branch. > > Download: > http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml > > Changelog: > http://www.rsyslog.com/Article189.phtml > > As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From rgerhards at hq.adiscon.com Wed Mar 12 19:48:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:48:45 +0100 Subject: [rsyslog] rsyslog 2.0.3 released Message-ID: <006101c88471$b85b588f$060013ac@intern.adiscon.com> Oops, as a quick fix http://download.rsyslog.com/rsyslog/rsyslog-2.0.3.tar.gz If i have not mistyped again ;) rainer ----- Urspr?ngliche Nachricht ----- Von: "Milton Calnek" An: "rsyslog-users" Gesendet: 12.03.08 19:37 Betreff: Re: [rsyslog] rsyslog 2.0.3 released The link in the web site does not work. The requested URL /rsyslog/rsyslog-2.0.23tar.gz was not found on this server. Thanks. Rainer Gerhards wrote: > Hi all, > > Rsyslog 2.0.3, a release of the stable branch, has been released today. > It is a purely bug-fixing release, solving mostly bugs that occur either > seldom or due to mistakes in the configuration file. It fixes some > potential segfaults in those cases. Rsyslog 2.0.3 is a recommended > update for all users of the rsyslog v2 branch. > > Download: > http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml > > Changelog: > http://www.rsyslog.com/Article189.phtml > > As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 19:50:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:50:55 +0100 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <006201c88472$0590e0ed$060013ac@intern.adiscon.com> That's fine, files are only created if the mem queue is filling up. By default thats after 8,000 msgs queued. And, no server needs no special config. More later, i am on a pda and can not really check out anything right now ;) rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 12.03.08 19:32 Betreff: Re: [rsyslog] actionqueue in front of tcp forward Rainer Gerhards wrote: > $WorkDirectory /tmp/rsyslogwork Also, I should also point out that I NEVER see anything show up in my WorkDirectory. That directory above is owned by root:root and mode 0755. I turned off SELinux. Nothing EVER shows up there. johnn _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 19:52:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:52:43 +0100 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <006301c88472$46360ab0$060013ac@intern.adiscon.com> Can you send me a ful debug output from the sender when it fails (maybe via mail) - if thats possible... rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 12.03.08 19:28 Betreff: Re: [rsyslog] actionqueue in front of tcp forward Rainer Gerhards wrote: > & in column one is to chain multiple actions together, but it can never > be at the start of the chain. Did you take it from some rsyslog > documentation? If so and you still know which, please let me know, I'd > like to fix that. No, I copied it from the bug report I listed. The OP must've had a regular action that wasn't listed. Anyway, I fixed it, restarted client rsyslog, but the behavior is the same. I don't need any special config on the server side, right? Well, if it helps, here's the top of my client /etc/rsyslog.conf, followed by the top of my server /etc/rsyslog.conf: == client /etc/rsyslog.conf == # Modules $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 # Standard syslog configuration (for local logging) [snip] == == server /etc/rsyslog.conf == # Modules $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so # TCP port $InputTCPServerRun 61514 # Global Directives $AllowedSender UDP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $AllowedSender TCP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $DirOwner root $DirGroup root $DirCreateMode 0755 # Templates [snip] # Standard syslog configuration [snip] == _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Wed Mar 12 20:07:34 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 15:07:34 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <006301c88472$46360ab0$060013ac@intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> Message-ID: <47D829F6.8070404@gmail.com> Rainer Gerhards wrote: > Can you send me a ful debug output from the sender when > it fails (maybe via mail) - if thats possible... I sent it via YouSendIt. Thanks for looking into this! johnn From rgerhards at hq.adiscon.com Wed Mar 12 21:40:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 21:40:22 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D829F6.8070404@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> I have received the file and had a chance to look at it. It's quite confusing. The send fails, but each retry succeeds... It looks like you use stunnel. I currently doubt that stunnel accepts each send and only reports the error after it can not connect to the remote side. So in fact we continously run into the situation that exactly *that* one message is lost. But I am still puzzled. Could you, for a test, run the same without stunnel and tell me if the problem persists or goes away? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 8:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] actionqueue in front of tcp forward > > Rainer Gerhards wrote: > > Can you send me a ful debug output from the sender when > > it fails (maybe via mail) - if thats possible... > > I sent it via YouSendIt. > Thanks for looking into this! > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 22:05:54 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 17:05:54 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> Message-ID: <47D845B2.30701@gmail.com> Rainer Gerhards wrote: > I have received the file and had a chance to look at it. It's quite > confusing. The send fails, but each retry succeeds... It looks like you > use stunnel. I currently doubt that stunnel accepts each send and only > reports the error after it can not connect to the remote side. So in > fact we continously run into the situation that exactly *that* one > message is lost. But I am still puzzled. Could you, for a test, run the > same without stunnel and tell me if the problem persists or goes away? Yes, I had the same thought about stunnel being the problem and was running some tests. It works! I now only lose the one message (known problem). But if I go over stunnel, then I lose all messages during the rsyslog server downtime. I can do without the stunnel for now. Thanks once again! johnn From rgerhards at hq.adiscon.com Wed Mar 12 22:20:15 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 22:20:15 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D845B2.30701@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 10:06 PM > To: rsyslog-users > Subject: Re: [rsyslog] actionqueue in front of tcp forward > > Rainer Gerhards wrote: > > I have received the file and had a chance to look at it. It's quite > > confusing. The send fails, but each retry succeeds... It > looks like you > > use stunnel. I currently doubt that stunnel accepts each > send and only > > reports the error after it can not connect to the remote side. So in > > fact we continously run into the situation that exactly *that* one > > message is lost. But I am still puzzled. Could you, for a > test, run the > > same without stunnel and tell me if the problem persists or > goes away? > > Yes, I had the same thought about stunnel being the problem > and was running some tests. > > It works! I now only lose the one message (known problem). > But if I go over stunnel, then I lose all messages during > the rsyslog server downtime. I have the strong feeling that it is time to do something against this plain old ack-less syslog tcp protocol... Maybe I add a half-duplex mode for starters. That's low, but quick to implement and ultra-reliable. I'll also see that I get more serious with RFC 3195 re-enabling. I've already done some basic thinking in regard to 3195 and the new syslog engine and doing it ultra-reliable will require a little bit of work. So there won't be an immediate cure - but defenitely the right route to take. How about half-duplex mode? Would that work for you? It means that each message must be acked before the next one is sent, so tcp's streaming features will almost be disabled. I'd expect a drop to at most 50% (more probable 40%) of the performance compared to what we currently run (half-duplex would obviously need to be an option...). So it would be a large performance hit. > I can do without the stunnel for now. For encryption, you could also look into the GSSAPI modules. It's contributed code, and I currently unfortunately have limited insight into it. But varmojfekoj, the contributor, has done a great job. > Thanks once again! I am very intersted in real-life experience. After all, the engine is still quite new. So I need feedback from the field to make it ultra-solid - a lab is a lab, is a lab ;) So keep the thoughts flowing. A side-note: we are rewriting phpLogCon, the web interface to syslog data. Any chance you happen to have some interest in that? ;) Rainer > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 23:10:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 18:10:22 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> Message-ID: <47D854CE.8020209@gmail.com> Rainer Gerhards wrote: > I have the strong feeling that it is time to do something against this > plain old ack-less syslog tcp protocol... Maybe I add a half-duplex mode > for starters. That's low, but quick to implement and ultra-reliable. > I'll also see that I get more serious with RFC 3195 re-enabling. I've > already done some basic thinking in regard to 3195 and the new syslog > engine and doing it ultra-reliable will require a little bit of work. So > there won't be an immediate cure - but defenitely the right route to > take. > > How about half-duplex mode? Would that work for you? It means that each > message must be acked before the next one is sent, so tcp's streaming > features will almost be disabled. I'd expect a drop to at most 50% (more > probable 40%) of the performance compared to what we currently run > (half-duplex would obviously need to be an option...). So it would be a > large performance hit. Just curious: do either of the above solutions (3195 and half-duplex) resolve the issue of losing all messages when the server goes down and you are going over stunnel? It sounds like the half-duplex does, I don't know enough about the 3195 to know. In answer to your question, I probably don't have as heavy a load as some folks out there, so it's easy for me to say, yes, half-duplex works for me, at least for the moment. >> I can do without the stunnel for now. > > For encryption, you could also look into the GSSAPI modules. It's > contributed code, and I currently unfortunately have limited insight > into it. But varmojfekoj, the contributor, has done a great job. Thanks! I'll have to look more closely at this in a staging environment down the road, as I can't afford potentially running into another stumbling block at the moment. > A side-note: we are rewriting phpLogCon, the web interface to syslog > data. Any chance you happen to have some interest in that? ;) It'll be a couple weeks before I really start looking into this, but yes, I was in fact planning to setup phpLogCon, too. johnn From rgerhards at hq.adiscon.com Thu Mar 13 08:11:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 08:11:58 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D854CE.8020209@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> <47D854CE.8020209@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> > Rainer Gerhards wrote: > > I have the strong feeling that it is time to do something against > this > > plain old ack-less syslog tcp protocol... Maybe I add a half-duplex > mode > > for starters. That's low, but quick to implement and ultra-reliable. > > I'll also see that I get more serious with RFC 3195 re-enabling. I've > > already done some basic thinking in regard to 3195 and the new syslog > > engine and doing it ultra-reliable will require a little bit of work. > So > > there won't be an immediate cure - but defenitely the right route to > > take. > > > > How about half-duplex mode? Would that work for you? It means that > each > > message must be acked before the next one is sent, so tcp's streaming > > features will almost be disabled. I'd expect a drop to at most 50% > (more > > probable 40%) of the performance compared to what we currently run > > (half-duplex would obviously need to be an option...). So it would be > a > > large performance hit. > > Just curious: do either of the above solutions (3195 and > half-duplex) resolve the issue of losing all messages when > the server goes down and you are going over stunnel? > > It sounds like the half-duplex does, I don't know enough > about the 3195 to know. > > > In answer to your question, I probably don't have as heavy a > load as some folks out there, so it's easy for me to say, > yes, half-duplex works for me, at least for the moment. I had a somewhat sleepless night, which is good news because it usually results in good ideas ;) This problem really bugged me, but I finally made up my mind. Instead of wasting time on fixing broken plain tcp syslog transport mode (e.g. by implementing half-duplex, which isn't standard anyhow), I'll do "the right thing". I thought rfc 3195 is the right thing. But it carries a lot of overhead that I really don't need. And, most importantly, any standard additions takes ages to go through the IETF (I know what I am talking about, have finally succeeded to get a better syslog rfc though it in "just" 4 (or 5?) years -- and it is still waiting to be published...). So instead of adding on any of these existing protocols, I'll do a new, lightweight but capable protocol specifically designed to solve the shortcomings we currently experience. Please welcome RELP, the "reliable event logging protocol" (name based on the quite successful selp [simple event logging protocol] effort: http://www.monitorware.com/en/workinprogress/selp.txt). Relp will evolve over time. I hope to do something useful relatively soon, and it will be extended as the project progresses. The ultimate goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog communications. I'll don't care about the outside world, so I can do the best of breed solution. For the rest of the world, rsyslog will of course continue to support plain tcp syslog and will get better support for rfc 3195. But if you wanna go hardcore on high-reliability, high-performance event logging, relp will be your choice. Technically, I'll split this off into rsyslog relp input and output plugins AND an independent librelp, which provides core protocol functionality (just in case somebody else wants to support it in the long term, so this will not be tied to rsyslog). > > >> I can do without the stunnel for now. > > > > For encryption, you could also look into the GSSAPI modules. It's > > contributed code, and I currently unfortunately have limited insight > > into it. But varmojfekoj, the contributor, has done a great job. > > Thanks! I'll have to look more closely at this in a staging > environment down the road, as I can't afford potentially > running into another stumbling block at the moment. > > > > A side-note: we are rewriting phpLogCon, the web interface to syslog > > data. Any chance you happen to have some interest in that? ;) > > It'll be a couple weeks before I really start looking into > this, but yes, I was in fact planning to setup phpLogCon, too. Excellent. Please keep an eye on its beta announcements - various stages of v2 will probably be available in a few weeks and I promise it will be much better than what is currently there. In fact, it tries to become the best-ever log web viewer. If there are some things you'd like to see in such a viewer, it would be great to hear your opinion. Rainer From maurizio.rottin at gmail.com Thu Mar 13 09:03:16 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 13 Mar 2008 09:03:16 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Wednesday, March 12, 2008 2:37 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > 2008/3/12, Rainer Gerhards : > > > > mmmmm...i'll check that better but at a glance it seems a heavy > > > > process with big logs. > > > > > > > > > Sorry, I don't really understand what you mean ;) But the problem is > > > that I probably do not really know how what you do with logger > > really > > > works. It is part of the apache config, is this right? Maybe you > > could > > > post a full config sample? > > > > > but i did... > > this is the log part in the directive > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > "combined" > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > > facility "-p local*.xxx" with tag "-t www.mysite.com" > > > > then i can use the tag to split logs by website in the remote rsyslog > > machine ;) > > > Ah, OK, I didn't realize this was actual Apache config. So your original > question > > > > just one question, do you know any way to pass the website name in > > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > > other program...i know only logger... > > > was how you could use the same apache config command for each log, so > that you don't need to have separate apache config statements for each > virtual server? > > Please bear with me if I am getting it wrong, I probably have too much > rsyslog on my mind. I am trying to help but I don't get a grip at what > the real problem is... > > Not really...i don't want to use logger, it becomes a heavy process under high traffic load....but this should be a question for the apache ML... -- mr From rgerhards at hq.adiscon.com Thu Mar 13 09:06:09 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 09:06:09 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> [sniiiiip] > Not really...i don't want to use logger, it becomes a heavy process > under high traffic load....but this should be a question for the > apache ML... OK, I don't know what makes it a heavy process... anyhow... If you come up with an explanation or what would be less resource-intense (from an apache point of view), please let me know. Maybe I can add a simple little tool to do the job. But obviously I need to know what's wrong with logger before even thinking about that ;) Rainer From Gerrard.Geldenhuis at datacash.com Thu Mar 13 10:04:11 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Thu, 13 Mar 2008 09:04:11 -0000 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com><47D854CE.8020209@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> Message-ID: > > I had a somewhat sleepless night, which is good news because it usually > results in good ideas ;) This problem really bugged me, but I finally > made up my mind. Instead of wasting time on fixing broken plain tcp > syslog transport mode (e.g. by implementing half-duplex, which isn't > standard anyhow), I'll do "the right thing". I thought rfc 3195 is the > right thing. But it carries a lot of overhead that I really don't need. > And, most importantly, any standard additions takes ages to go through > the IETF (I know what I am talking about, have finally succeeded to get > a better syslog rfc though it in "just" 4 (or 5?) years -- and it is > still waiting to be published...). > > So instead of adding on any of these existing protocols, I'll do a new, > lightweight but capable protocol specifically designed to solve the > shortcomings we currently experience. Please welcome RELP, the "reliable > event logging protocol" (name based on the quite successful selp [simple > event logging protocol] effort: > http://www.monitorware.com/en/workinprogress/selp.txt). > > Relp will evolve over time. I hope to do something useful relatively > soon, and it will be extended as the project progresses. The ultimate > goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog > communications. I'll don't care about the outside world, so I can do the > best of breed solution. For the rest of the world, rsyslog will of > course continue to support plain tcp syslog and will get better support > for rfc 3195. But if you wanna go hardcore on high-reliability, > high-performance event logging, relp will be your choice. > Maybe in another 5 years RELP will replace rfc 3195 especially. Now that rsyslog is part of official redhat release it should gain a lot of momentum, the anouncement for RHEL5.2 beta contained rsyslog. Regards From rgerhards at hq.adiscon.com Thu Mar 13 10:59:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 10:59:30 +0100 Subject: [rsyslog] RELP - was: actionqueue in front of tcp forward In-Reply-To: References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com><47D854CE.8020209@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B8E@grfint2.intern.adiscon.com> > Maybe in another 5 years RELP will replace rfc 3195 especially. Doesn't really matter ;) For the technially interested folks: I have drafted an intial spec. Still a lot is missing (of course), but it sufficient to solve the stunnel problem that we saw yesterday (and a couple of others...): http://git.adiscon.com/?p=librelp.git;a=blob;f=src/relp.c;h=a9aff8b005d2 3a8dc66aed1ffa66f41e0ba7b41c;hb=master > Now that > rsyslog is part of official redhat release it should gain a lot of > momentum, the anouncement for RHEL5.2 beta contained rsyslog. Oh, nice to hear. This is good news :) Rainer From friedl at hq.adiscon.com Thu Mar 13 12:59:00 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 13 Mar 2008 12:59:00 +0100 Subject: [rsyslog] rsyslog 3.12.2 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B96@grfint2.intern.adiscon.com> Hi all, the development branch also sees a new released. Rsyslog 3.12.2 is available immediately. It is primarily a code cleanup and bug fixing release. Feature-wise, it aims at some better recovery of aborted TCP sessions. It contains a number of important fixes, which, for example, prevent a segfault when imfile tries to read empty log file lines. Internally, loadable library modules are now automatically unloaded if they are no longer used. Rsyslog 3.12.2 is a recommended update for all v3 branch users. Download: http://www.rsyslog.com/Downloads-req-getit-lid-87.phtml Changelog: http://www.rsyslog.com/Article191.phtml Feedback is, as always, appreciated. Thank you. Florian Riedl From linuxweb at gmail.com Thu Mar 13 20:27:07 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 13 Mar 2008 15:27:07 -0400 Subject: [rsyslog] imfile with existing files, splits files on server side Message-ID: <47D9800B.40704@gmail.com> I tested this on both rsyslog 3.12.1 and this morning's 3.12.2 release. I ran a couple tests with existing log files. E.g., I have an httpd access_log which is 1.3MB and about 5,000 lines -- nowhere near as large as my real existing log, which is 136MB and over 511,000 lines. With the smaller file, as soon as I started rsyslog on the client side, it correctly started to show up on the server side in: /var/log/client1/httpd_access_log And the hostname shows up in the prefix for the message: Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip] But after a few hundred lines, it suddenly starts logging to: /var/log/httpd_access_log And the hostname no longer shows up in the prefix: Mar 13 15:12:20 tag_access_log:76.8.67.2[snip] Then, thousands of lines later, it goes back to logging to: /var/log/client1/httpd_access_log The other thing is, it never actually finishes the entire 5000 lines. It does 4000-something and then stops. My gut feeling is this is related to ActionQueue settings. So I'm going to experiment with a few different parameters. But the part that I don't get is why it's not all being logged to the same %HOSTNAME% directory? I don't know what's useful to send for diagnostics -- the original files from the client side, the split files from the server side, the client debug, the server debug. Etc. But I'd be happy to run any requested tests. Meanwhile, I'll tweak ActionQueue params. johnn From rgerhards at hq.adiscon.com Thu Mar 13 20:58:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 20:58:45 +0100 Subject: [rsyslog] imfile with existing files, splits files on server side In-Reply-To: <47D9800B.40704@gmail.com> References: <47D9800B.40704@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> Mmmhhhh... So the 5,000 line file doesn't make it completely to the server? That's strange... With the larger file, I'd assume that there are some flow control issues, but with the small one... Maybe it is related to the other problem with the hostname. Kind of sounds like it looses sync some time in between reading the files. I guess it is probably a good idea if you can send me the 5000 line file, so I can run it in my lab tomorrow. Also, please confirm that the small file actually doesn't make it to the server. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Thursday, March 13, 2008 8:27 PM > To: rsyslog-users > Subject: [rsyslog] imfile with existing files, splits files > on server side > > I tested this on both rsyslog 3.12.1 and this morning's > 3.12.2 release. > > I ran a couple tests with existing log files. E.g., I have > an httpd access_log which is 1.3MB and about 5,000 lines -- > nowhere near as large as my real existing log, which is > 136MB and over 511,000 lines. > > With the smaller file, as soon as I started rsyslog on the > client side, it correctly started to show up on the server > side in: > /var/log/client1/httpd_access_log > > And the hostname shows up in the prefix for the message: > Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip] > > > But after a few hundred lines, it suddenly starts logging to: > /var/log/httpd_access_log > > And the hostname no longer shows up in the prefix: > Mar 13 15:12:20 tag_access_log:76.8.67.2[snip] > > > Then, thousands of lines later, it goes back to logging to: > /var/log/client1/httpd_access_log > > > > The other thing is, it never actually finishes the entire > 5000 lines. It does 4000-something and then stops. > > My gut feeling is this is related to ActionQueue settings. > So I'm going to experiment with a few different parameters. > > But the part that I don't get is why it's not all being > logged to the same %HOSTNAME% directory? > > > I don't know what's useful to send for diagnostics -- the > original files from the client side, the split files from > the server side, the client debug, the server debug. Etc. > > But I'd be happy to run any requested tests. > > Meanwhile, I'll tweak ActionQueue params. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From janfrode at tanso.net Fri Mar 14 00:28:29 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 00:28:29 +0100 Subject: [rsyslog] rsyslog on RHEL5u2 ? Message-ID: Looks like rsyslogd will be available in RHEL5u2 ! Congratulations Rainer! https://www.redhat.com/archives/rhelv5-announce/2008-March/msg00000.html """ This beta release includes the following improvements: * Improved Audit and Logging + Add rsyslog logging facility New package: rsyslog-2.0.0-11.el5 Enhanced system logging and kernel message trapping daemons """ -jf From janfrode at tanso.net Fri Mar 14 00:58:55 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 00:58:55 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> Message-ID: On 2008-03-12, Johnny Tan wrote: > > This is custom. I prefer to install auditd and have the > SELinux logs separated out from /var/log/messages. Then I > use imfile to convert the audit logs to syslog in order to > send it over to the central logger. > > Yes, it's a waste of a syslog "local" facility, but I'm not > aware of any other way unless I remove auditd and have those > go back into /var/log/messages. Hmmm.. I have been wondering about how to collect the auditd logs centrally. Could you please share how you do it ? Would be great if you manage to keep the format on the receiving host.. -jf From rgerhards at hq.adiscon.com Fri Mar 14 10:49:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 10:49:45 +0100 Subject: [rsyslog] Feedback requested - length of syslog tag Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Hi all, I got the following bug report (and patch - thx) for situation where the syslog tag is larger than 32 characters. http://bugzilla.adiscon.com/show_bug.cgi?id=50 Syslog rfcs prohibit such oversize tags. However, I think seeing them in practice triggers the question if we should really follow RFC 3164 and the newer rfcs in this case. Or may it be better to have at least an option to allow tags of more than 32 characters? If you have an opinion, please voice it. Thanks, Rainer From pvrabec at redhat.com Fri Mar 14 12:15:15 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Fri, 14 Mar 2008 12:15:15 +0100 Subject: [rsyslog] HUP problem Message-ID: <200803141215.15887.pvrabec@redhat.com> Hi folks, we have some problems here :) rsyslog-3.12.1 ------------------ https://bugzilla.redhat.com/show_bug.cgi?id=437393 imklog.so + HUP generates memory leaks use: HUPs and #watch -d=cumulative cat /proc/`pgrep rsyslog`/status rsyslog-3.12.2 ------------------ is even worse, SIGSEGV occurs after every HUP :( From Gerrard.Geldenhuis at datacash.com Fri Mar 14 12:36:43 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Fri, 14 Mar 2008 11:36:43 -0000 Subject: [rsyslog] Feedback requested - length of syslog tag In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Message-ID: It's a tough one. Software that follows standards generally work well with other software and just makes life less complex. However a lot of these RFC's are very dated and have not kept track with time and 2001 when rfc3164 was published is a long time ago. If there is a flag which is not set by default and which is documented as breaking the standard it might be acceptable to allow longer tags. Users should break at own risk... Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 14 March 2008 09:50 > To: rsyslog-users > Subject: [rsyslog] Feedback requested - length of syslog tag > > Hi all, > > I got the following bug report (and patch - thx) for situation where the > syslog tag is larger than 32 characters. > > http://bugzilla.adiscon.com/show_bug.cgi?id=50 > > Syslog rfcs prohibit such oversize tags. However, I think seeing them in > practice triggers the question if we should really follow RFC 3164 and > the newer rfcs in this case. Or may it be better to have at least an > option to allow tags of more than 32 characters? > > If you have an opinion, please voice it. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Mar 14 12:57:41 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 12:57:41 +0100 Subject: [rsyslog] Feedback requested - length of syslog tag References: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Message-ID: On 2008-03-14, Rainer Gerhards wrote: > > I got the following bug report (and patch - thx) for situation where the > syslog tag is larger than 32 characters. > > http://bugzilla.adiscon.com/show_bug.cgi?id=50 > > Syslog rfcs prohibit such oversize tags. However, I think seeing them in > practice triggers the question if we should really follow RFC 3164 and > the newer rfcs in this case. Or may it be better to have at least an > option to allow tags of more than 32 characters? > > If you have an opinion, please voice it. > I think rsyslog should "be liberal in what it receives and conservative in what it send". So, maybe allow receiving messages with large tags, but don't send them out to other rfc3164-expecting servers. Sending large tags should maybe only be done in RELP.. ? -jf From linuxweb at gmail.com Fri Mar 14 15:15:26 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 14 Mar 2008 10:15:26 -0400 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> Message-ID: <47DA887E.9020601@gmail.com> Jan-Frode Myklebust wrote: > Hmmm.. I have been wondering about how to collect the auditd logs > centrally. Could you please share how you do it ? I use imfile, this is my client stanza: == # auditd audit.log $InputFileName /var/log/audit/audit.log $InputFileTag tag_audit_log: $InputFileStateFile audit_log $InputFileSeverity info $InputFileFacility local6 $InputRunFileMonitor == And on the server side: == $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" local6.* ?HostAudit == > Would be great > if you manage to keep the format on the receiving host.. The format is NOT kept due to the syslog prefix. However audit2allow can still read and process the file. If I need the original format (like for ausearch, and maybe other tools), I would need to run something like this on the server side, in the directory for the host I'm wanting to do the analysis on: sed 's/^.*tag_audit_log://' audit_log | ausearch -i Is that what you meant? johnn From rgerhards at hq.adiscon.com Fri Mar 14 15:29:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 15:29:44 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: <47DA887E.9020601@gmail.com> References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> <47DA887E.9020601@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> I guess you can keep the format if you use a template with just %msg% inside it: $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" $template auditFormat, "%msg%" local6.* ?HostAudit;auditFormat I have not tested this, just guessing. If you try it out and it does not work, I'd appreciate if you could post me a line as written by rsyslog and an original line. I'll see what I can do with these ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 14, 2008 3:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog v3 and selinux > > Jan-Frode Myklebust wrote: > > Hmmm.. I have been wondering about how to collect the auditd logs > > centrally. Could you please share how you do it ? > > I use imfile, this is my client stanza: > == > # auditd audit.log > $InputFileName /var/log/audit/audit.log > $InputFileTag tag_audit_log: > $InputFileStateFile audit_log > $InputFileSeverity info > $InputFileFacility local6 > $InputRunFileMonitor > == > > > And on the server side: > == > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > local6.* ?HostAudit > == > > > > Would be great > > if you manage to keep the format on the receiving host.. > > The format is NOT kept due to the syslog prefix. However > audit2allow can still read and process the file. > > If I need the original format (like for ausearch, and maybe > other tools), I would need to run something like this on the > server side, in the directory for the host I'm wanting to do > the analysis on: > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > > Is that what you meant? > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 14 15:33:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 15:33:43 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com><47DA887E.9020601@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BB3@grfint2.intern.adiscon.com> Ummm... I forgot two important chraceters, else you'll have no line feeds and all in one line... (see the \n after %msg$)! $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" $template auditFormat, "%msg%\n" local6.* ?HostAudit;auditFormat > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, March 14, 2008 3:30 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog v3 and selinux > > I guess you can keep the format if you use a template with just %msg% > inside it: > > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > $template auditFormat, "%msg%" > local6.* ?HostAudit;auditFormat > > I have not tested this, just guessing. If you try it out and it does > not > work, I'd appreciate if you could post me a line as written by rsyslog > and an original line. I'll see what I can do with these ;) > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > > Sent: Friday, March 14, 2008 3:15 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog v3 and selinux > > > > Jan-Frode Myklebust wrote: > > > Hmmm.. I have been wondering about how to collect the auditd logs > > > centrally. Could you please share how you do it ? > > > > I use imfile, this is my client stanza: > > == > > # auditd audit.log > > $InputFileName /var/log/audit/audit.log > > $InputFileTag tag_audit_log: > > $InputFileStateFile audit_log > > $InputFileSeverity info > > $InputFileFacility local6 > > $InputRunFileMonitor > > == > > > > > > And on the server side: > > == > > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > > local6.* ?HostAudit > > == > > > > > > > Would be great > > > if you manage to keep the format on the receiving host.. > > > > The format is NOT kept due to the syslog prefix. However > > audit2allow can still read and process the file. > > > > If I need the original format (like for ausearch, and maybe > > other tools), I would need to run something like this on the > > server side, in the directory for the host I'm wanting to do > > the analysis on: > > > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > > > > > Is that what you meant? > > johnn > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Mar 14 15:29:46 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 15:29:46 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> <47DA887E.9020601@gmail.com> Message-ID: On 2008-03-14, Johnny Tan wrote: >== > The format is NOT kept due to the syslog prefix. However > audit2allow can still read and process the file. > > If I need the original format (like for ausearch, and maybe > other tools), I would need to run something like this on the > server side, in the directory for the host I'm wanting to do > the analysis on: > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > Is that what you meant? Yes, thanks! -jf From rgerhards at hq.adiscon.com Fri Mar 14 18:09:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 18:09:35 +0100 Subject: [rsyslog] rsyslog on RHEL5u2 ? In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BBA@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, March 14, 2008 12:28 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog on RHEL5u2 ? > > Looks like rsyslogd will be available in RHEL5u2 ! Congratulations > Rainer! Thanks, much appreciated :) Let's hope to get v3 stable ready for RHEL 6 ;) Rainer > > > https://www.redhat.com/archives/rhelv5-announce/2008- > March/msg00000.html > > > """ > This beta release includes the following improvements: > > > > * Improved Audit and Logging > + Add rsyslog logging facility > > > > New package: rsyslog-2.0.0-11.el5 > Enhanced system logging and kernel message trapping daemons > > """ > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 14 18:29:40 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 18:29:40 +0100 Subject: [rsyslog] HUP problem In-Reply-To: <200803141215.15887.pvrabec@redhat.com> References: <200803141215.15887.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BBB@grfint2.intern.adiscon.com> I was busy with the other things today but had a quick look. I can repro the hup problem. Looks like it is related to the new object loader (and specifically the unloader), I guess it will be a relatively easy fix ... on Monday ;) Will look into the memleak after that. Thanks for reporting. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > Sent: Friday, March 14, 2008 12:15 PM > To: rsyslog-users > Subject: [rsyslog] HUP problem > > Hi folks, > > we have some problems here :) > > rsyslog-3.12.1 > ------------------ > https://bugzilla.redhat.com/show_bug.cgi?id=437393 > imklog.so + HUP generates memory leaks > use: > HUPs and #watch -d=cumulative cat /proc/`pgrep rsyslog`/status > > rsyslog-3.12.2 > ------------------ > is even worse, SIGSEGV occurs after every HUP :( > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 11:04:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 11:04:45 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Hi all, once again, I am in need for feedback. In order to be backwards-compatible with sysklogd, rsyslogd supported the "last message repeated n times" message compression feature. However, this feature is prone to causing user trouble. Some even think that it is a design flaw (see this discussion on the loganalysis mailing list, this posting is probably a good entry point into a lengthy thread: http://www.loganalysis.org/pipermail/loganalysis/2008-January/000547.htm l ). >From the rsyslog core engine point of view, "last message repeated n times" is quite costly in terms of code complexity and even performance. There is a -e command line switch to turn it off, which most users seem to do (and those that don't do it often seem to run into troubles). I am very tempted to DROP this feature from future builds. That would result in a great code complexity reduction (really, it takes a lot of effort...) and probably also rid us of a standard trouble spot. However, it also means that its compression features are no longer available. Question now: does anybody actually need this feature? If so, why is it good? Please provide feedback. Thanks, Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 11:15:09 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 10:15:09 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: Hi I personnally find it not very usefull to put it politely. I'd rather have every message and every time stamp for the sake of acuracy and precision then an attempt from the software to be intelligent. And if you get the benefit of a code clean up then double thumbs up from me. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 10:05 > To: rsyslog-users > Subject: [rsyslog] Feedback requested "Last message logged n times"... > > Hi all, > > once again, I am in need for feedback. In order to be > backwards-compatible with sysklogd, rsyslogd supported the "last message > repeated n times" message compression feature. However, this feature is > prone to causing user trouble. Some even think that it is a design flaw > (see this discussion on the loganalysis mailing list, this posting is > probably a good entry point into a lengthy thread: > http://www.loganalysis.org/pipermail/loganalysis/2008-January/000547.htm > l ). > > >From the rsyslog core engine point of view, "last message repeated n > times" is quite costly in terms of code complexity and even performance. > There is a -e command line switch to turn it off, which most users seem > to do (and those that don't do it often seem to run into troubles). > > I am very tempted to DROP this feature from future builds. That would > result in a great code complexity reduction (really, it takes a lot of > effort...) and probably also rid us of a standard trouble spot. However, > it also means that its compression features are no longer available. > > Question now: does anybody actually need this feature? If so, why is it > good? > > Please provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 11:23:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 11:23:37 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gerrard Geldenhuis > Sent: Tuesday, March 18, 2008 11:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n > times"... > > Hi > I personnally find it not very usefull to put it politely. I'd rather > have every message and every time stamp for the sake of acuracy and > precision then an attempt from the software to be intelligent. > > And if you get the benefit of a code clean up then double thumbs up > from > me. It's actually a *big* cleanup, because in order to support that feature, we need to have A) some background tick processing to flush the "repeated message" every few (30 or 60) seconds (it prevents rsyslogd from being tickles - think powertop, green IT and all that) B) requires rsyslogd to keep multiple message objects alive for an extended period of time, because the message object is needed for comparison when a new message comes in. Among others, this prevents some advanced flow control options which would depend on message destruction *when the message is done* and not *when the next message arrives*. C) [estimate] almost half the code of the main action calling loop, and most of the complexity, is related to "last message repeated n times" D) it prevents future end-to-end ACK capabilities (e.g. message is only acked after being written to DB) E) requires additional synchronization slowing down parallel processing (granted, just a little bit, but...) F) requires a "duplicate message capability" (some more code) G) ... and probably a few more spots ... I should have added this information to the initial message, too. These are *my* primary drivers for the idea to get rid of it ;) Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 12:12:02 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 11:12:02 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> Message-ID: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 10:24 > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n times"... > > > It's actually a *big* cleanup, because in order to support that feature, > we need to have > I have asked the question on a local lug to get some more opinions. I will post the results if any here. Regards From pvrabec at redhat.com Tue Mar 18 12:26:27 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 18 Mar 2008 12:26:27 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> Message-ID: <200803181226.27878.pvrabec@redhat.com> On Tuesday 18 March 2008 12:12:02 pm Gerrard Geldenhuis wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: 18 March 2008 10:24 > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > times"... > > > It's actually a *big* cleanup, because in order to support that > > feature, > > > we need to have > > I have asked the question on a local lug to get some more opinions. I > will post the results if any here. I did the same. :) The only arguments against removal I have received until now were: 1. DoS (but nobody explained how) 2. cleanness of logs (relative) I'm thinking about forwarding discussion to fedora-devel :) > > Regards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From raimund.sacherer at gmail.com Tue Mar 18 12:37:35 2008 From: raimund.sacherer at gmail.com (Raimund Sacherer) Date: Tue, 18 Mar 2008 12:37:35 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: <41963B0D-1D02-4C60-99E1-CF52AFF0ABCB@gmail.com> > > Question now: does anybody actually need this feature? If so, why is > it > good? > > Please provide feedback. Hi all, i think it's a biest of the past, remember those days where every byte, every bit was really valuable? i remember programing on my C64 with byte values, because they could hold 256 states and it was just using one (!, today i have to smile) byte ... Or the times i had my first 40 MB Harddrive in one of my first PC's ... i guess compressing a message which is exactly the same but only of the timestamp was virtually a lifesaver those days ... as far as i can say, from the bottom of my heart: get rid of it! disc- space is cheap, i *love* consistant logs, personally i can not think of any answer that would justify today a) log-file inconsinstency b) a known trouble-spot and c) such a future-development-stopper as this code seems to be (after reading your problems with this piece of code. again, from me: go for it best Raimund > > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 12:42:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 12:42:11 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803181226.27878.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> <200803181226.27878.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Good discussion, thanks all :) ... some more points below ... > > I have asked the question on a local lug to get some more opinions. I > > will post the results if any here. > I did the same. :) > > The only arguments against removal I have received until now were: > > 1. DoS (but nobody explained how) I think this is along the lines of making it easy to flood the log with similar messages. If that's the case, I think it is a (too) weak argument because an attacker could easily include a random pattern inside the message. HOWEVER, the current logic indeed prevents flooding the log from equal messages, e.g. if a process runs wild. > 2. cleanness of logs (relative) > > I'm thinking about forwarding discussion to fedora-devel :) Excellent. Maybe there are also alternatives to the current behavior, something in between. Right now, the core engine does this, so the reduction capability is inherent to every action (except if an action specifically forbids it, because it can not intelligently handle it - e.g. database writers). So it applies to forwarding, snmp, whatever, ... One alternative would be to remove it from the core engine and move it into the *file write* output plugin (omfile). This has its subtleties, too, so I don't like to take that approach lightly. Plus, it than means that the network may become spammed. I think one core requirement is to find people who actually *intentionally* use this feature and ask for their reasoning. That will probably be the best way to tell us if its really needed. And, of course, we need to weigh the negative effects of it against these con-points. Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 14:11:15 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 13:11:15 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: The only arguments for keeping the feature that I got on my lug was the preservartion of disk/network IO. I think to prevent DOS attacks is a valid argument but as you said can be easily circumvented by randomizing messages. To safeguard against dos attacks you could have a monitor that monitors for extra ordinary amount of traffic and then generate a snmp trap. Whether that should be a rsyslog plugin or part of other software is open to debate. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 11:42 > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n times"... > > Good discussion, thanks all :) ... some more points below ... > > > > I have asked the question on a local lug to get some more opinions. > I > > > will post the results if any here. > > I did the same. :) > > > > The only arguments against removal I have received until now were: > > > > 1. DoS (but nobody explained how) > > I think this is along the lines of making it easy to flood the log with > similar messages. If that's the case, I think it is a (too) weak > argument because an attacker could easily include a random pattern > inside the message. > > HOWEVER, the current logic indeed prevents flooding the log from equal > messages, e.g. if a process runs wild. > > > 2. cleanness of logs (relative) > > > > I'm thinking about forwarding discussion to fedora-devel :) > > Excellent. > > Maybe there are also alternatives to the current behavior, something in > between. Right now, the core engine does this, so the reduction > capability is inherent to every action (except if an action specifically > forbids it, because it can not intelligently handle it - e.g. database > writers). So it applies to forwarding, snmp, whatever, ... One > alternative would be to remove it from the core engine and move it into > the *file write* output plugin (omfile). This has its subtleties, too, > so I don't like to take that approach lightly. Plus, it than means that > the network may become spammed. I think one core requirement is to find > people who actually *intentionally* use this feature and ask for their > reasoning. That will probably be the best way to tell us if its really > needed. And, of course, we need to weigh the negative effects of it > against these con-points. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 14:22:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 14:22:30 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> > The only arguments for keeping the feature that I got on my lug was the > preservartion of disk/network IO. Did you get any "feeling" of how important this is being considered? > > I think to prevent DOS attacks is a valid argument but as you said can > be easily circumvented by randomizing messages. > > To safeguard against dos attacks you could have a monitor that monitors > for extra ordinary amount of traffic and then generate a snmp trap. > Whether that should be a rsyslog plugin or part of other software is > open to debate. This may (m-a-y it's far too early) be part of the flow control logic or an exception detector or a rate-limiting feature... Even for non-DoS cases it might be interesting to know who is sending most messages... mmmh... maybe this points into a direction on how to solve the need that is behind "last message repeated n times". Probably that need is not even fully understood... mmmhh. More thoughts are appreciated ;) Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 14:41:48 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 13:41:48 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> Message-ID: > > > The only arguments for keeping the feature that I got on my lug was > the > > preservartion of disk/network IO. > > Did you get any "feeling" of how important this is being considered? I think the general feeling is: "don't care" or not important at all but then I did not get many replies. > > Even for non-DoS cases it might be interesting to know who is sending > most messages... mmmh... maybe this points into a direction on how to > solve the need that is behind "last message repeated n times". Probably > that need is not even fully understood... mmmhh. More thoughts are > appreciated ;) When I asked about which applications has caused these type of repeat messages one culprit mentioned was the kernel. Regards From friedl at hq.adiscon.com Tue Mar 18 15:05:41 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Tue, 18 Mar 2008 15:05:41 +0100 Subject: [rsyslog] rsyslog 3.12.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDF@grfint2.intern.adiscon.com> Hi all, rsyslog 3.12.3 has been released today. It is primarily a bug-fixing release, but offers new, advanced flow control options. So far, only imfile utilizes these, but other modules will do so soon. The release contains important bug fixes, for example for a segfault on hup, improper framing for syslog/tcp communication and processing empty lines in the log file reader. Rsyslog 3.12.3 is a recommended update to all v3 branch users and users of 3.12.2 are urged to apply the update (some bugs where introduced in that release). Download: http://www.rsyslog.com/Downloads-req-getit-lid-88.phtml Changelog: http://www.rsyslog.com/Article193.phtml As always, feedback is appreciated. Florian Riedl From pvrabec at redhat.com Tue Mar 18 15:23:52 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 18 Mar 2008 15:23:52 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: <200803181523.52321.pvrabec@redhat.com> On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > The only arguments for keeping the feature that I got on my lug was the > preservartion of disk/network IO. > > I think to prevent DOS attacks is a valid argument but as you said can > be easily circumvented by randomizing messages. I'm afraid it's not true in all cases. What if you do DOS attach not directly to do rsyslog, but via other daemon. In situation when you can't send message directly to syslog, but you can make some daemon generate message for you. This message would be probably always the same content. > To safeguard against dos attacks you could have a monitor that monitors > for extra ordinary amount of traffic and then generate a snmp trap. > Whether that should be a rsyslog plugin or part of other software is > open to debate. > > Regards > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: 18 March 2008 11:42 > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > times"... > > > Good discussion, thanks all :) ... some more points below ... > > > > > > I have asked the question on a local lug to get some more > > opinions. > > > I > > > > > > will post the results if any here. > > > > > > I did the same. :) > > > > > > The only arguments against removal I have received until now were: > > > > > > 1. DoS (but nobody explained how) > > > > I think this is along the lines of making it easy to flood the log > > with > > > similar messages. If that's the case, I think it is a (too) weak > > argument because an attacker could easily include a random pattern > > inside the message. > > > > HOWEVER, the current logic indeed prevents flooding the log from equal > > messages, e.g. if a process runs wild. > > > > > 2. cleanness of logs (relative) > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > Excellent. > > > > Maybe there are also alternatives to the current behavior, something > > in > > > between. Right now, the core engine does this, so the reduction > > capability is inherent to every action (except if an action > > specifically > > > forbids it, because it can not intelligently handle it - e.g. database > > writers). So it applies to forwarding, snmp, whatever, ... One > > alternative would be to remove it from the core engine and move it > > into > > > the *file write* output plugin (omfile). This has its subtleties, too, > > so I don't like to take that approach lightly. Plus, it than means > > that > > > the network may become spammed. I think one core requirement is to > > find > > > people who actually *intentionally* use this feature and ask for their > > reasoning. That will probably be the best way to tell us if its really > > needed. And, of course, we need to weigh the negative effects of it > > against these con-points. > > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Tue Mar 18 22:12:39 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 18 Mar 2008 17:12:39 -0400 Subject: [rsyslog] imfile with existing files, splits files on server side In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> References: <47D9800B.40704@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> Message-ID: <47E03047.3090801@gmail.com> Rainer Gerhards wrote: > So the 5,000 line file doesn't make it completely to the server? That's > strange... With the larger file, I'd assume that there are some flow > control issues, but with the small one... Just wanted to follow-up... thanks to Rainer's tireless poring over my excessively long logfiles and debug outputs, this issue is finally resolved. It seems like it was a combination of tcp flow control issues (fixed with the new 3.12.3) and the server by default discarding messages above severity 4. Thanks again to Rainer for not giving up on this! johnn From rgerhards at hq.adiscon.com Thu Mar 20 11:04:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 11:04:19 +0100 Subject: [rsyslog] update on RELP Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C16@grfint2.intern.adiscon.com> Hi all, I thought I provide a quick update on RELP, the protocol rsyslog will use in the future for reliable rsyslog-to-rsyslog communication. I am currently very actively working on both the protocol and librelp, an implementation for it. The RELP "spec" is also growing. It is now available at http://www.rsyslog.com/doc-relp.html Those with some deep technical interest might want to have a peek at it. I have also setup a RELP-specific mailing list. Find it at http://lists.adiscon.net/mailman/listinfo/relp The next major feature in rsyslogd will be relp support. Until this is finished, I am focused on bug fixing. Rainer From maurizio.rottin at gmail.com Thu Mar 20 15:20:17 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:20:17 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> Message-ID: 2008/3/13, Rainer Gerhards : wow, > [sniiiiip] > > > Not really...i don't want to use logger, it becomes a heavy process > > under high traffic load....but this should be a question for the > > apache ML... > > > OK, I don't know what makes it a heavy process... anyhow... If you come > up with an explanation or what would be less resource-intense (from an > apache point of view), please let me know. Maybe I can add a simple > little tool to do the job. But obviously I need to know what's wrong > with logger before even thinking about that ;) > > ok my ideas are a little more clear now, as you said: > was how you could use the same apache config command for each log, so > that you don't need to have separate apache config statements for each > virtual server? > > Please bear with me if I am getting it wrong, I probably have too much > rsyslog on my mind. I am trying to help but I don't get a grip at what > the real problem is... > > you were right...this is what i need! just one command for ErrorLog and one for CustomLog so as i posted now i'm using: > > this is the log part in the directive > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > "combined" > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > > facility "-p local*.xxx" with tag "-t www.mysite.com" for each virtualhost, but this mean that i have 2 loggers running for every virtualhost. This is why logger becomes a heavy process. But if i use one default directive for every vhost, i don't know how to split the log per vhost in the remote machine which collects the logs... Hope now is a little bit clearer... now, new config in httpd.conf "LogLevel warn ErrorLog "|/usr/bin/logger -p local5.err" LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined CustomLog "|/usr/bin/logger -p local6.info " vcombined" and the %v gives the virtualhost name, so now i've got only two loggers. At this point how can i say to rsyslog to split the files according to the first word which is in this case the vhost name? Thanks for being patient! -- mr From rgerhards at hq.adiscon.com Thu Mar 20 15:25:09 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 15:25:09 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Can you send me a handful of the logline to play with? Probably not this week, but next... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:20 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/13, Rainer Gerhards : > > wow, > > [sniiiiip] > > > > > Not really...i don't want to use logger, it becomes a heavy process > > > under high traffic load....but this should be a question for the > > > apache ML... > > > > > > OK, I don't know what makes it a heavy process... anyhow... If you > come > > up with an explanation or what would be less resource-intense (from > an > > apache point of view), please let me know. Maybe I can add a simple > > little tool to do the job. But obviously I need to know what's wrong > > with logger before even thinking about that ;) > > > > > > ok my ideas are a little more clear now, as you said: > > was how you could use the same apache config command for each log, so > > that you don't need to have separate apache config statements for > each > > virtual server? > > > > Please bear with me if I am getting it wrong, I probably have too > much > > rsyslog on my mind. I am trying to help but I don't get a grip at > what > > the real problem is... > > > > > > you were right...this is what i need! > > just one command for ErrorLog and one for CustomLog > > so as i posted now i'm using: > > > this is the log part in the directive > > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > > "combined" > > > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger > with > > > facility "-p local*.xxx" with tag "-t www.mysite.com" > > for each virtualhost, but this mean that i have 2 loggers running for > every virtualhost. > This is why logger becomes a heavy process. > > But if i use one default directive for every vhost, i don't know how > to split the log per vhost in the remote machine which collects the > logs... > > Hope now is a little bit clearer... > > now, new config in httpd.conf > "LogLevel warn > ErrorLog "|/usr/bin/logger -p local5.err" > LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" > vcombined > CustomLog "|/usr/bin/logger -p local6.info " vcombined" > > and the %v gives the virtualhost name, so now i've got only two > loggers. > > At this point how can i say to rsyslog to split the files according to > the first word which is in this case the vhost name? > > Thanks for being patient! > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 15:46:32 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:46:32 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Message-ID: 2008/3/20, Rainer Gerhards : > Can you send me a handful of the logline to play with? Probably not this > week, but next... > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" i'm trying to use the regexp but with no success in this way: $template MsgFormat,"%msg%\n" $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ [a-z,\.]*--end%_az.log" local6.info -?ApacheRemoteCustom;MsgFormat from the documentation: "the property replacer will return the part of the property text that matches the regular expression" which should be " www.mysite.com" but i get a file named _az.log -- mr From rgerhards at hq.adiscon.com Thu Mar 20 15:49:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 15:49:21 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Let me try to avoid the regexp (its expensive and I can not debug it now ;)): so you search for the string that is at the start of the msg and delimited by the first space? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/20, Rainer Gerhards : > > Can you send me a handful of the logline to play with? Probably not > this > > week, but next... > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" > > i'm trying to use the regexp but with no success in this way: > $template MsgFormat,"%msg%\n" > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > [a-z,\.]*--end%_az.log" > local6.info -?ApacheRemoteCustom;MsgFormat > > from the documentation: "the property replacer will return the part of > the property text that matches the regular expression" which should be > " www.mysite.com" > but i get a file named _az.log > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 15:59:02 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:59:02 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Message-ID: yes! but actually there is a space at the beginning and hostname can contain the dash -, numbers, and letters. 2008/3/20, Rainer Gerhards : > Let me try to avoid the regexp (its expensive and I can not debug it now > ;)): so you search for the string that is at the start of the msg and > delimited by the first space? > > > > Rainer > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Thursday, March 20, 2008 3:47 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > 2008/3/20, Rainer Gerhards : > > > Can you send me a handful of the logline to play with? Probably not > > this > > > week, but next... > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" > > > > i'm trying to use the regexp but with no success in this way: > > $template MsgFormat,"%msg%\n" > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > [a-z,\.]*--end%_az.log" > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > from the documentation: "the property replacer will return the part of > > the property text that matches the regular expression" which should be > > " www.mysite.com" > > but i get a file named _az.log > > > > -- > > mr > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- mr From rgerhards at hq.adiscon.com Thu Mar 20 16:09:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 16:09:24 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Should work with fields (much faster). I can't try it out due to relp work, but try: %msg:F,32:2% [32 is USASCII SP, the delimiter here] But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that should work... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:59 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > yes! but actually there is a space at the beginning and hostname can > contain the dash -, numbers, and letters. > > 2008/3/20, Rainer Gerhards : > > Let me try to avoid the regexp (its expensive and I can not debug it > now > > ;)): so you search for the string that is at the start of the msg > and > > delimited by the first space? > > > > > > > > Rainer > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > Can you send me a handful of the logline to play with? Probably > not > > > this > > > > week, but next... > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > > /images/wm001.jpg HTTP/1.1" 304 - > "http://www.mysite.com/webmail.htm" > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > Gecko/20060607" > > > > > > i'm trying to use the regexp but with no success in this way: > > > $template MsgFormat,"%msg%\n" > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > [a-z,\.]*--end%_az.log" > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > from the documentation: "the property replacer will return the > part of > > > the property text that matches the regular expression" which > should be > > > " www.mysite.com" > > > but i get a file named _az.log > > > > > > -- > > > mr > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 16:22:11 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 16:22:11 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Message-ID: wow! %msg:F,32:2% works! i completely misunderstood how Fielf works...i thought the first number was the FromChar and second the ToChar... Thanks a lot!!! 2008/3/20, Rainer Gerhards : > Should work with fields (much faster). I can't try it out due to relp > work, but try: > > %msg:F,32:2% [32 is USASCII SP, the delimiter here] > > But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that > should work... > > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Thursday, March 20, 2008 3:59 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > yes! but actually there is a space at the beginning and hostname can > > contain the dash -, numbers, and letters. > > > > 2008/3/20, Rainer Gerhards : > > > Let me try to avoid the regexp (its expensive and I can not debug it > > now > > > ;)): so you search for the string that is at the start of the msg > > and > > > delimited by the first space? > > > > > > > > > > > > Rainer > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > > Can you send me a handful of the logline to play with? Probably > > not > > > > this > > > > > week, but next... > > > > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > > > /images/wm001.jpg HTTP/1.1" 304 - > > "http://www.mysite.com/webmail.htm" > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > > Gecko/20060607" > > > > > > > > i'm trying to use the regexp but with no success in this way: > > > > $template MsgFormat,"%msg%\n" > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > > [a-z,\.]*--end%_az.log" > > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > > > from the documentation: "the property replacer will return the > > part of > > > > the property text that matches the regular expression" which > > should be > > > > " www.mysite.com" > > > > but i get a file named _az.log > > > > > > > > -- > > > > mr > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > -- > > mr > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- mr From rgerhards at hq.adiscon.com Thu Mar 20 16:39:49 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 16:39:49 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1E@grfint2.intern.adiscon.com> Excellent, good to hear :) Just a side-note: when I am done with relp, I'll hopefully have time to come back to the config format. Then we'll also see much more powerful ways to extract values... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > wow! > %msg:F,32:2% works! > i completely misunderstood how Fielf works...i thought the first > number was the FromChar and second the ToChar... > Thanks a lot!!! > > 2008/3/20, Rainer Gerhards : > > Should work with fields (much faster). I can't try it out due to relp > > work, but try: > > > > %msg:F,32:2% [32 is USASCII SP, the delimiter here] > > > > But maybe %msg:F,32:1% - you need to experiment a bit. In any case, > that > > should work... > > > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > Sent: Thursday, March 20, 2008 3:59 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > yes! but actually there is a space at the beginning and hostname > can > > > contain the dash -, numbers, and letters. > > > > > > 2008/3/20, Rainer Gerhards : > > > > Let me try to avoid the regexp (its expensive and I can not > debug it > > > now > > > > ;)): so you search for the string that is at the start of the > msg > > > and > > > > delimited by the first space? > > > > > > > > > > > > > > > > Rainer > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > > > Can you send me a handful of the logline to play with? > Probably > > > not > > > > > this > > > > > > week, but next... > > > > > > > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] > "GET > > > > > /images/wm001.jpg HTTP/1.1" 304 - > > > "http://www.mysite.com/webmail.htm" > > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > > > Gecko/20060607" > > > > > > > > > > i'm trying to use the regexp but with no success in this way: > > > > > $template MsgFormat,"%msg%\n" > > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > > > [a-z,\.]*--end%_az.log" > > > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > > > > > from the documentation: "the property replacer will return > the > > > part of > > > > > the property text that matches the regular expression" which > > > should be > > > > > " www.mysite.com" > > > > > but i get a file named _az.log > > > > > > > > > > -- > > > > > mr > > > > > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > > > -- > > > mr > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 06:47:10 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 07:47:10 +0200 Subject: [rsyslog] Property-Based Filters Message-ID: <47E49D5E.5060408@pengooin.net> Hi, I'm having trouble setting up some properties-based filters. I want to see if some property is equal to "" (empty). I'm doing this: :FROMHOST, isequal, "" *.info;mail.none;authpriv.none;cron.none ?messages-per-host4 But it still matches everything it should not. Am i doing something wrong? Thanks, Radu From rgerhards at hq.adiscon.com Sat Mar 22 10:26:47 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 10:26:47 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E49D5E.5060408@pengooin.net> References: <47E49D5E.5060408@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> Hi Radu, I will look into the issue, smells like a bug. But FROMHOST can never be empty... Is it just a sample? If so, which property you are looking it (I wonder what may be empty, thus the question...). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:47 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Property-Based Filters > > Hi, > > I'm having trouble setting up some properties-based filters. I want to > see if some property is equal to "" (empty). > I'm doing this: > :FROMHOST, isequal, "" > *.info;mail.none;authpriv.none;cron.none ?messages-per- > host4 > > But it still matches everything it should not. Am i doing something > wrong? > > Thanks, > Radu > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:03:28 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:03:28 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> References: <47E49D5E.5060408@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> Message-ID: <47E4F590.8010306@pengooin.net> Hi, I'm using stable. I'm trying to create some central log machine. I made some tests .. and it seems that when i start rsyslog on the central logging machine, it creates some messages regarding rsyslog version, for those messages FROMHOST is empty. If you want i can provide you with full rsyslog.conf. Please test and reply. Thanks, Radu Gheorghiu Rainer Gerhards wrote: > Hi Radu, > > I will look into the issue, smells like a bug. But FROMHOST can never be > empty... Is it just a sample? If so, which property you are looking it > (I wonder what may be empty, thus the question...). > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:47 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] Property-Based Filters >> >> Hi, >> >> I'm having trouble setting up some properties-based filters. I want to >> see if some property is equal to "" (empty). >> I'm doing this: >> :FROMHOST, isequal, "" >> *.info;mail.none;authpriv.none;cron.none ?messages-per- >> host4 >> >> But it still matches everything it should not. Am i doing something >> wrong? >> >> Thanks, >> Radu >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 13:06:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 13:06:00 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E4F590.8010306@pengooin.net> References: <47E49D5E.5060408@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> <47E4F590.8010306@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> > Hi, > I'm using stable. I'm trying to create some central log machine. > I made some tests .. and it seems that when i start rsyslog on the > central logging machine, it creates some messages regarding rsyslog > version, for those messages FROMHOST is empty. That *is* a bug. FROMHOST should not be empty. And now that I know it'll probably won't be empty in the future ;) > If you want i can > provide > you with full rsyslog.conf. Please test and reply. So you want to filter out the rsyslog startup and shutdown messages? Please provide me a few samples of what the messages look in your log files. Thanks, Rainer > > Thanks, > Radu Gheorghiu > > Rainer Gerhards wrote: > > Hi Radu, > > > > I will look into the issue, smells like a bug. But FROMHOST can never > be > > empty... Is it just a sample? If so, which property you are looking > it > > (I wonder what may be empty, thus the question...). > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 6:47 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] Property-Based Filters > >> > >> Hi, > >> > >> I'm having trouble setting up some properties-based filters. I want > to > >> see if some property is equal to "" (empty). > >> I'm doing this: > >> :FROMHOST, isequal, "" > >> *.info;mail.none;authpriv.none;cron.none ?messages- > per- > >> host4 > >> > >> But it still matches everything it should not. Am i doing something > >> wrong? > >> > >> Thanks, > >> Radu > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:21:44 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:21:44 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> References: <47E49D5E.5060408@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> <47E4F590.8010306@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> Message-ID: <47E4F9D8.1040004@pengooin.net> Hi, Well as is said i needed a central log solution. So i did this: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" and for testing purposes (and debug): $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" And here is the rest: *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName i look in /var/log/hosts: 2008 clog-he-de messages clog-he-de is the localhost .. and 2008 + messages is created because FROMHOST is empty. proof: cat messages TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart note the "dd" after "TheHost:" . i put it there to be sure there's nothing between the two "d". Thanks, Radu Gheorghiu Rainer Gerhards wrote: >> Hi, >> I'm using stable. I'm trying to create some central log machine. >> I made some tests .. and it seems that when i start rsyslog on the >> central logging machine, it creates some messages regarding rsyslog >> version, for those messages FROMHOST is empty. >> > > That *is* a bug. FROMHOST should not be empty. And now that I know it'll > probably won't be empty in the future ;) > > >> If you want i can >> provide >> you with full rsyslog.conf. Please test and reply. >> > > So you want to filter out the rsyslog startup and shutdown messages? > Please provide me a few samples of what the messages look in your log > files. > > Thanks, > Rainer > > >> Thanks, >> Radu Gheorghiu >> >> Rainer Gerhards wrote: >> >>> Hi Radu, >>> >>> I will look into the issue, smells like a bug. But FROMHOST can >>> > never > >> be >> >>> empty... Is it just a sample? If so, which property you are looking >>> >> it >> >>> (I wonder what may be empty, thus the question...). >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 6:47 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Property-Based Filters >>>> >>>> Hi, >>>> >>>> I'm having trouble setting up some properties-based filters. I want >>>> >> to >> >>>> see if some property is equal to "" (empty). >>>> I'm doing this: >>>> :FROMHOST, isequal, "" >>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>> >> per- >> >>>> host4 >>>> >>>> But it still matches everything it should not. Am i doing something >>>> wrong? >>>> >>>> Thanks, >>>> Radu >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 13:46:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 13:46:51 +0100 Subject: [rsyslog] Property-Based Filters Message-ID: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> Ah, ok, so the problem actually is that fromhost is empty... ----- Urspr?ngliche Nachricht ----- Von: "Radu Gheorghiu" An: "rsyslog-users" Gesendet: 22.03.08 13:21 Betreff: Re: [rsyslog] Property-Based Filters Hi, Well as is said i needed a central log solution. So i did this: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" and for testing purposes (and debug): $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" And here is the rest: *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName i look in /var/log/hosts: 2008 clog-he-de messages clog-he-de is the localhost .. and 2008 + messages is created because FROMHOST is empty. proof: cat messages TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart note the "dd" after "TheHost:" . i put it there to be sure there's nothing between the two "d". Thanks, Radu Gheorghiu Rainer Gerhards wrote: >> Hi, >> I'm using stable. I'm trying to create some central log machine. >> I made some tests .. and it seems that when i start rsyslog on the >> central logging machine, it creates some messages regarding rsyslog >> version, for those messages FROMHOST is empty. >> > > That *is* a bug. FROMHOST should not be empty. And now that I know it'll > probably won't be empty in the future ;) > > >> If you want i can >> provide >> you with full rsyslog.conf. Please test and reply. >> > > So you want to filter out the rsyslog startup and shutdown messages? > Please provide me a few samples of what the messages look in your log > files. > > Thanks, > Rainer > > >> Thanks, >> Radu Gheorghiu >> >> Rainer Gerhards wrote: >> >>> Hi Radu, >>> >>> I will look into the issue, smells like a bug. But FROMHOST can >>> > never > >> be >> >>> empty... Is it just a sample? If so, which property you are looking >>> >> it >> >>> (I wonder what may be empty, thus the question...). >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 6:47 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Property-Based Filters >>>> >>>> Hi, >>>> >>>> I'm having trouble setting up some properties-based filters. I want >>>> >> to >> >>>> see if some property is equal to "" (empty). >>>> I'm doing this: >>>> :FROMHOST, isequal, "" >>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>> >> per- >> >>>> host4 >>>> >>>> But it still matches everything it should not. Am i doing something >>>> wrong? >>>> >>>> Thanks, >>>> Radu >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:54:43 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:54:43 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> Message-ID: <47E50193.4020904@pengooin.net> Yes, it is. And that wouldn't be a problem if property based filter would work fine. I tried to filter messages with FROMHOST empty and use HOSTNAME instead. It worked for localhost generated messages, but didn't work for messages coming from other host on the network. Here's sample: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" $template messages-per-host3,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host4,"/var/log/hosts/%HOSTNAME%/messages" $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" # first we check if FROMHOST is empty . if it is empty .. we use HOSTNAME :FROMHOST, isequal, "" *.info;mail.none;authpriv.none;cron.none ?messages-per-host4;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host3;MyTemplateName # we drop the messages with FROMHOST empty, and we log everything else based on FROMHOST . :FROMHOST, isequal, "" ~ *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? Well.. it looks like there are several messages that don't contain the HOSTNAME field and this breaks everything. I'm not sure these are all bugs. They may be some result of some human error of some kind (my error). Waiting for your confirmation of the above, Radu Gheorghiu Rainer Gerhards wrote: > Ah, ok, so the problem actually is that fromhost is empty... > > ----- Urspr?ngliche Nachricht ----- > Von: "Radu Gheorghiu" > An: "rsyslog-users" > Gesendet: 22.03.08 13:21 > Betreff: Re: [rsyslog] Property-Based Filters > > Hi, > > Well as is said i needed a central log solution. > So i did this: > > $template > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > > and for testing purposes (and debug): > > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: > %msg%\n" > > And here is the rest: > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host1;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host2;MyTemplateName > > i look in /var/log/hosts: > 2008 clog-he-de messages > clog-he-de is the localhost .. > and 2008 + messages is created because FROMHOST is empty. > proof: > > cat messages > TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" > x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo > udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart > > note the "dd" after "TheHost:" . i put it there to be sure there's > nothing between the two "d". > > Thanks, > Radu Gheorghiu > > Rainer Gerhards wrote: > >>> Hi, >>> I'm using stable. I'm trying to create some central log machine. >>> I made some tests .. and it seems that when i start rsyslog on the >>> central logging machine, it creates some messages regarding rsyslog >>> version, for those messages FROMHOST is empty. >>> >>> >> That *is* a bug. FROMHOST should not be empty. And now that I know it'll >> probably won't be empty in the future ;) >> >> >> >>> If you want i can >>> provide >>> you with full rsyslog.conf. Please test and reply. >>> >>> >> So you want to filter out the rsyslog startup and shutdown messages? >> Please provide me a few samples of what the messages look in your log >> files. >> >> Thanks, >> Rainer >> >> >> >>> Thanks, >>> Radu Gheorghiu >>> >>> Rainer Gerhards wrote: >>> >>> >>>> Hi Radu, >>>> >>>> I will look into the issue, smells like a bug. But FROMHOST can >>>> >>>> >> never >> >> >>> be >>> >>> >>>> empty... Is it just a sample? If so, which property you are looking >>>> >>>> >>> it >>> >>> >>>> (I wonder what may be empty, thus the question...). >>>> >>>> Rainer >>>> >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>>> Sent: Saturday, March 22, 2008 6:47 AM >>>>> To: rsyslog at lists.adiscon.com >>>>> Subject: [rsyslog] Property-Based Filters >>>>> >>>>> Hi, >>>>> >>>>> I'm having trouble setting up some properties-based filters. I want >>>>> >>>>> >>> to >>> >>> >>>>> see if some property is equal to "" (empty). >>>>> I'm doing this: >>>>> :FROMHOST, isequal, "" >>>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>>> >>>>> >>> per- >>> >>> >>>>> host4 >>>>> >>>>> But it still matches everything it should not. Am i doing something >>>>> wrong? >>>>> >>>>> Thanks, >>>>> Radu >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Sat Mar 22 14:07:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 14:07:46 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50193.4020904@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2A@grfint2.intern.adiscon.com> Ahh... That sample is most helpful. I think there is also a misunderstanding. I couldn't run a lab yet and will probably not before after easter, but (read inline below) ... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 1:55 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Yes, it is. > And that wouldn't be a problem if property based filter would work > fine. > > I tried to filter messages with FROMHOST empty and use HOSTNAME > instead. > It worked for localhost generated messages, but didn't work for > messages > coming from other host on the network. Here's sample: > > $template > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > $template > messages-per-host3,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > $template messages-per-host4,"/var/log/hosts/%HOSTNAME%/messages" > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: > %msg%\n" > > # first we check if FROMHOST is empty . if it is empty .. we use > HOSTNAME > :FROMHOST, isequal, "" Property based filters are just regular filters. So you need to specify what shall happen when the filter matches. In the above line, there is no action (sorry, looks like I overlooked that in the first message you send). So to discard these message, you'd need to do: :FROMHOST, isequal, "" ~ Filters (in v2) do NOT combine, except for the BSD-style filters. This is a feature of v3. > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host4;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host3;MyTemplateName > # we drop the messages with FROMHOST empty, and we log everything else > based on FROMHOST . > :FROMHOST, isequal, "" ~ That, of course, should work... > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host1;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host2;MyTemplateName > > Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? > Well.. it looks like there are several messages that don't contain the > HOSTNAME field and this breaks everything. > > I'm not sure these are all bugs. At may very well be. I suspect that they all have the same root cause, and that is that some message properties are not correctly being set for internally-generated messages. So one trouble spot with multiple problems resulting from it. Could you do me a favor and run rsyslogd with -d -n options interactively and send me the resulting debug log? Rainer > They may be some result of some human > error of some kind (my error). > > Waiting for your confirmation of the above, > Radu Gheorghiu > > Rainer Gerhards wrote: > > Ah, ok, so the problem actually is that fromhost is empty... > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "Radu Gheorghiu" > > An: "rsyslog-users" > > Gesendet: 22.03.08 13:21 > > Betreff: Re: [rsyslog] Property-Based Filters > > > > Hi, > > > > Well as is said i needed a central log solution. > > So i did this: > > > > $template > > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > > > > and for testing purposes (and debug): > > > > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% > TheMsg: > > %msg%\n" > > > > And here is the rest: > > *.info;mail.none;authpriv.none;cron.none > > ?messages-per-host1;MyTemplateName > > *.info;mail.none;authpriv.none;cron.none > > ?messages-per-host2;MyTemplateName > > > > i look in /var/log/hosts: > > 2008 clog-he-de messages > > clog-he-de is the localhost .. > > and 2008 + messages is created because FROMHOST is empty. > > proof: > > > > cat messages > > TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" > > x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo > > udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] > restart > > > > note the "dd" after "TheHost:" . i put it there to be sure there's > > nothing between the two "d". > > > > Thanks, > > Radu Gheorghiu > > > > Rainer Gerhards wrote: > > > >>> Hi, > >>> I'm using stable. I'm trying to create some central log machine. > >>> I made some tests .. and it seems that when i start rsyslog on the > >>> central logging machine, it creates some messages regarding rsyslog > >>> version, for those messages FROMHOST is empty. > >>> > >>> > >> That *is* a bug. FROMHOST should not be empty. And now that I know > it'll > >> probably won't be empty in the future ;) > >> > >> > >> > >>> If you want i can > >>> provide > >>> you with full rsyslog.conf. Please test and reply. > >>> > >>> > >> So you want to filter out the rsyslog startup and shutdown messages? > >> Please provide me a few samples of what the messages look in your > log > >> files. > >> > >> Thanks, > >> Rainer > >> > >> > >> > >>> Thanks, > >>> Radu Gheorghiu > >>> > >>> Rainer Gerhards wrote: > >>> > >>> > >>>> Hi Radu, > >>>> > >>>> I will look into the issue, smells like a bug. But FROMHOST can > >>>> > >>>> > >> never > >> > >> > >>> be > >>> > >>> > >>>> empty... Is it just a sample? If so, which property you are > looking > >>>> > >>>> > >>> it > >>> > >>> > >>>> (I wonder what may be empty, thus the question...). > >>>> > >>>> Rainer > >>>> > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >>>>> Sent: Saturday, March 22, 2008 6:47 AM > >>>>> To: rsyslog at lists.adiscon.com > >>>>> Subject: [rsyslog] Property-Based Filters > >>>>> > >>>>> Hi, > >>>>> > >>>>> I'm having trouble setting up some properties-based filters. I > want > >>>>> > >>>>> > >>> to > >>> > >>> > >>>>> see if some property is equal to "" (empty). > >>>>> I'm doing this: > >>>>> :FROMHOST, isequal, "" > >>>>> *.info;mail.none;authpriv.none;cron.none > ?messages- > >>>>> > >>>>> > >>> per- > >>> > >>> > >>>>> host4 > >>>>> > >>>>> But it still matches everything it should not. Am i doing > something > >>>>> wrong? > >>>>> > >>>>> Thanks, > >>>>> Radu > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> > >>>>> > >>>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 22 14:14:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 14:14:55 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50193.4020904@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> > Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? > Well.. it looks like there are several messages that don't contain the > HOSTNAME field and this breaks everything. I had a quick look at the code and I can't see why HOSTNAME should not be filled. Could you provide me a sample of such a message? That would be most interesting. I think I've also identified why FROMHOST is missing, but I'd like to evaluate it a bit more. If you can provide a sample of missing hostnames, it would very helpful to have a debug log from such a message being received and processed - but the most important thing is to see what the source of this all is... Thanks, Rainer From radu at pengooin.net Sat Mar 22 14:47:39 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 15:47:39 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> Message-ID: <47E50DFB.40505@pengooin.net> I don't think that the HOSTNAME problem is a rsyslog problem. I think that the remote host is not sending the HOSTNAME. Remote host is using the classic syslog . and it is configured like this: *.* @central-logger on the central logger .. i modify my template for debug: $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% TheMsg: %msg%\n" and in the logs: TheHost:dlastd 5 TheMsg: repeated 8 times Seems like the message had no hostname field. and rsyslog thinks "last" is the hostname. Radu From rgerhards at hq.adiscon.com Sat Mar 22 18:09:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:09:04 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50DFB.40505@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> Well... Rsyslog tries all kind of things to get hold of the real host. So if you could send me a copy of the one causing problems, I may (may ;)) be able to do something against it. HOSTNAME should always contain something usable, but as you say ... it depends ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 2:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > I don't think that the HOSTNAME problem is a rsyslog problem. > I think that the remote host is not sending the HOSTNAME. > Remote host is using the classic syslog . > and it is configured like this: > *.* @central-logger > > on the central logger .. i modify my template for debug: > $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% TheMsg: > %msg%\n" > > and in the logs: > TheHost:dlastd 5 TheMsg: repeated 8 times > > Seems like the message had no hostname field. and rsyslog thinks > "last" > is the hostname. > > Radu > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 18:13:33 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:13:33 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> Message-ID: <47E53E3D.90606@pengooin.net> Hmm i think i don't understand . Haven't you received the file that i sent you? Or did i miss something? Radu Rainer Gerhards wrote: > Well... Rsyslog tries all kind of things to get hold of the real host. > So if you could send me a copy of the one causing problems, I may (may > ;)) be able to do something against it. HOSTNAME should always contain > something usable, but as you say ... it depends ;) > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 2:48 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> I don't think that the HOSTNAME problem is a rsyslog problem. >> I think that the remote host is not sending the HOSTNAME. >> Remote host is using the classic syslog . >> and it is configured like this: >> *.* >> > @central-logger > >> on the central logger .. i modify my template for debug: >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >> > TheMsg: > >> %msg%\n" >> >> and in the logs: >> TheHost:dlastd 5 TheMsg: repeated 8 times >> >> Seems like the message had no hostname field. and rsyslog thinks >> "last" >> is the hostname. >> >> Radu >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 18:17:27 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:17:27 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E53E3D.90606@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> Replying as I can ;) I've not been able to go in-depth through the file, but I ... doh ;) I guess it's all sitting in there. Lol - sorry. As an apology, here is the FROMHOST bugfix ;) http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2. 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Hmm i think i don't understand . Haven't you received the file that i > sent you? Or did i miss something? > > Radu > > Rainer Gerhards wrote: > > Well... Rsyslog tries all kind of things to get hold of the real > host. > > So if you could send me a copy of the one causing problems, I may > (may > > ;)) be able to do something against it. HOSTNAME should always > contain > > something usable, but as you say ... it depends ;) > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 2:48 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> I don't think that the HOSTNAME problem is a rsyslog problem. > >> I think that the remote host is not sending the HOSTNAME. > >> Remote host is using the classic syslog . > >> and it is configured like this: > >> *.* > >> > > @central-logger > > > >> on the central logger .. i modify my template for debug: > >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >> > > TheMsg: > > > >> %msg%\n" > >> > >> and in the logs: > >> TheHost:dlastd 5 TheMsg: repeated 8 times > >> > >> Seems like the message had no hostname field. and rsyslog thinks > >> "last" > >> is the hostname. > >> > >> Radu > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 22 18:25:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:25:19 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E53E3D.90606@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> Ah, I've now reviewed it, but there is no message in it from a host that does not send a HOSTNAME (The last message... case). What syslogd are your running on the clients? Sysklogd? And which version? I am asking because I could check the code and see what exactly it generates. I already begin to get some bad feelings about what it sends ;) What I see is rsyslog's bug (the one just fixed) and an error message telling you that rsyslog is discarding a selector line because of no actions. That is the one where you had use the filter but without actions. Filters work only in front of actions in v2. So if you don't provide an action, nothing happens except that startup error message. Please note that the doc just talks about the property based filter. But it doesn't mean you can use it without an action. Filters only work with actions. In v3, things are already a bit different and will be much more different soon. V3 offers full expression support, so you can do Boolean operations inside filters. Also, v3 will be scriptable. HTH, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Hmm i think i don't understand . Haven't you received the file that i > sent you? Or did i miss something? > > Radu > > Rainer Gerhards wrote: > > Well... Rsyslog tries all kind of things to get hold of the real > host. > > So if you could send me a copy of the one causing problems, I may > (may > > ;)) be able to do something against it. HOSTNAME should always > contain > > something usable, but as you say ... it depends ;) > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 2:48 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> I don't think that the HOSTNAME problem is a rsyslog problem. > >> I think that the remote host is not sending the HOSTNAME. > >> Remote host is using the classic syslog . > >> and it is configured like this: > >> *.* > >> > > @central-logger > > > >> on the central logger .. i modify my template for debug: > >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >> > > TheMsg: > > > >> %msg%\n" > >> > >> and in the logs: > >> TheHost:dlastd 5 TheMsg: repeated 8 times > >> > >> Seems like the message had no hostname field. and rsyslog thinks > >> "last" > >> is the hostname. > >> > >> Radu > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 18:32:42 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:32:42 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> Message-ID: <47E542BA.4040503@pengooin.net> Ok thanks :) I only wanted to know if you received it . But if you also fixed it, it's even better :) Rainer Gerhards wrote: > Replying as I can ;) I've not been able to go in-depth through the file, > but I ... doh ;) I guess it's all sitting in there. Lol - sorry. > > As an apology, here is the FROMHOST bugfix ;) > > http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2. > 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> Hmm i think i don't understand . Haven't you received the file that i >> sent you? Or did i miss something? >> >> Radu >> >> Rainer Gerhards wrote: >> >>> Well... Rsyslog tries all kind of things to get hold of the real >>> >> host. >> >>> So if you could send me a copy of the one causing problems, I may >>> >> (may >> >>> ;)) be able to do something against it. HOSTNAME should always >>> >> contain >> >>> something usable, but as you say ... it depends ;) >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 2:48 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Property-Based Filters >>>> >>>> I don't think that the HOSTNAME problem is a rsyslog problem. >>>> I think that the remote host is not sending the HOSTNAME. >>>> Remote host is using the classic syslog . >>>> and it is configured like this: >>>> *.* >>>> >>>> >>> @central-logger >>> >>> >>>> on the central logger .. i modify my template for debug: >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >>>> >>>> >>> TheMsg: >>> >>> >>>> %msg%\n" >>>> >>>> and in the logs: >>>> TheHost:dlastd 5 TheMsg: repeated 8 times >>>> >>>> Seems like the message had no hostname field. and rsyslog thinks >>>> "last" >>>> is the hostname. >>>> >>>> Radu >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From radu at pengooin.net Sat Mar 22 18:34:09 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:34:09 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> Message-ID: <47E54311.1080502@pengooin.net> I will prepare a new file + details, in the next 30mins :) Rainer Gerhards wrote: > Ah, I've now reviewed it, but there is no message in it from a host that > does not send a HOSTNAME (The last message... case). What syslogd are > your running on the clients? Sysklogd? And which version? I am asking > because I could check the code and see what exactly it generates. I > already begin to get some bad feelings about what it sends ;) > > What I see is rsyslog's bug (the one just fixed) and an error message > telling you that rsyslog is discarding a selector line because of no > actions. That is the one where you had use the filter but without > actions. > > Filters work only in front of actions in v2. So if you don't provide an > action, nothing happens except that startup error message. Please note > that the doc just talks about the property based filter. But it doesn't > mean you can use it without an action. Filters only work with actions. > In v3, things are already a bit different and will be much more > different soon. V3 offers full expression support, so you can do > Boolean operations inside filters. Also, v3 will be scriptable. > > HTH, > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> Hmm i think i don't understand . Haven't you received the file that i >> sent you? Or did i miss something? >> >> Radu >> >> Rainer Gerhards wrote: >> >>> Well... Rsyslog tries all kind of things to get hold of the real >>> >> host. >> >>> So if you could send me a copy of the one causing problems, I may >>> >> (may >> >>> ;)) be able to do something against it. HOSTNAME should always >>> >> contain >> >>> something usable, but as you say ... it depends ;) >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 2:48 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Property-Based Filters >>>> >>>> I don't think that the HOSTNAME problem is a rsyslog problem. >>>> I think that the remote host is not sending the HOSTNAME. >>>> Remote host is using the classic syslog . >>>> and it is configured like this: >>>> *.* >>>> >>>> >>> @central-logger >>> >>> >>>> on the central logger .. i modify my template for debug: >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >>>> >>>> >>> TheMsg: >>> >>> >>>> %msg%\n" >>>> >>>> and in the logs: >>>> TheHost:dlastd 5 TheMsg: repeated 8 times >>>> >>>> Seems like the message had no hostname field. and rsyslog thinks >>>> "last" >>>> is the hostname. >>>> >>>> Radu >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 18:34:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:34:43 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E542BA.4040503@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> <47E542BA.4040503@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> No problem at all. The fix was rather trivial once I knew what I was looking for. Please let me know if it works for you. The original bug report - property based filters - however, was no bug, but a misunderstanding on how they work. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Ok thanks :) I only wanted to know if you received it . But if you also > fixed it, it's even better :) > > Rainer Gerhards wrote: > > Replying as I can ;) I've not been able to go in-depth through the > file, > > but I ... doh ;) I guess it's all sitting in there. Lol - sorry. > > > > As an apology, here is the FROMHOST bugfix ;) > > > > > http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2 > . > > 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 6:14 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> Hmm i think i don't understand . Haven't you received the file that > i > >> sent you? Or did i miss something? > >> > >> Radu > >> > >> Rainer Gerhards wrote: > >> > >>> Well... Rsyslog tries all kind of things to get hold of the real > >>> > >> host. > >> > >>> So if you could send me a copy of the one causing problems, I may > >>> > >> (may > >> > >>> ;)) be able to do something against it. HOSTNAME should always > >>> > >> contain > >> > >>> something usable, but as you say ... it depends ;) > >>> > >>> Rainer > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >>>> Sent: Saturday, March 22, 2008 2:48 PM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] Property-Based Filters > >>>> > >>>> I don't think that the HOSTNAME problem is a rsyslog problem. > >>>> I think that the remote host is not sending the HOSTNAME. > >>>> Remote host is using the classic syslog . > >>>> and it is configured like this: > >>>> *.* > >>>> > >>>> > >>> @central-logger > >>> > >>> > >>>> on the central logger .. i modify my template for debug: > >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >>>> > >>>> > >>> TheMsg: > >>> > >>> > >>>> %msg%\n" > >>>> > >>>> and in the logs: > >>>> TheHost:dlastd 5 TheMsg: repeated 8 times > >>>> > >>>> Seems like the message had no hostname field. and rsyslog thinks > >>>> "last" > >>>> is the hostname. > >>>> > >>>> Radu > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 19:41:38 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 20:41:38 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> <47E542BA.4040503@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> Message-ID: <47E552E2.3040907@pengooin.net> I just applied the patch and the problem with the empty HOSTNAME seems to be solved now. Congrats :) And thank you for your effort. Radu Rainer Gerhards wrote: > No problem at all. The fix was rather trivial once I knew what I was > looking for. Please let me know if it works for you. The original bug > report - property based filters - however, was no bug, but a > misunderstanding on how they work. > > Rainer > > From rgerhards at hq.adiscon.com Mon Mar 24 22:25:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Mar 2008 22:25:55 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803181523.52321.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> Hi all, I now have a use case where the "last message..." prevents sources from flooding syslog. But now on to a different use case: does someone actually use this feature to reduce the size of files or network traffic? Except, of course, in cases where there is a message flood due to either attact or a misbehaving source (these need to be treated differentely - I am right now after the *destination* message compression...). Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > Sent: Tuesday, March 18, 2008 3:24 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message > logged n times"... > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > The only arguments for keeping the feature that I got on my > lug was the > > preservartion of disk/network IO. > > > > I think to prevent DOS attacks is a valid argument but as > you said can > > be easily circumvented by randomizing messages. > I'm afraid it's not true in all cases. What if you do DOS > attach not directly > to do rsyslog, but via other daemon. In situation when you > can't send message > directly to syslog, but you can make some daemon generate > message for you. > This message would be probably always the same content. > > > To safeguard against dos attacks you could have a monitor > that monitors > > for extra ordinary amount of traffic and then generate a snmp trap. > > Whether that should be a rsyslog plugin or part of other software is > > open to debate. > > > > Regards > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: 18 March 2008 11:42 > > > To: rsyslog-users > > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > > > times"... > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > I have asked the question on a local lug to get some more > > > > opinions. > > > > > I > > > > > > > > will post the results if any here. > > > > > > > > I did the same. :) > > > > > > > > The only arguments against removal I have received > until now were: > > > > > > > > 1. DoS (but nobody explained how) > > > > > > I think this is along the lines of making it easy to flood the log > > > > with > > > > > similar messages. If that's the case, I think it is a (too) weak > > > argument because an attacker could easily include a random pattern > > > inside the message. > > > > > > HOWEVER, the current logic indeed prevents flooding the > log from equal > > > messages, e.g. if a process runs wild. > > > > > > > 2. cleanness of logs (relative) > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > Excellent. > > > > > > Maybe there are also alternatives to the current > behavior, something > > > > in > > > > > between. Right now, the core engine does this, so the reduction > > > capability is inherent to every action (except if an action > > > > specifically > > > > > forbids it, because it can not intelligently handle it - > e.g. database > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > alternative would be to remove it from the core engine and move it > > > > into > > > > > the *file write* output plugin (omfile). This has its > subtleties, too, > > > so I don't like to take that approach lightly. Plus, it than means > > > > that > > > > > the network may become spammed. I think one core requirement is to > > > > find > > > > > people who actually *intentionally* use this feature and > ask for their > > > reasoning. That will probably be the best way to tell us > if its really > > > needed. And, of course, we need to weigh the negative > effects of it > > > against these con-points. > > > > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From pvrabec at redhat.com Tue Mar 25 10:46:55 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 25 Mar 2008 10:46:55 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> Message-ID: <200803251046.55919.pvrabec@redhat.com> Hi folks, I talked with some people, and nobody is happy with removing this thing from rsyslog. Won't be possible to move it to separate plug-in? On Monday 24 March 2008 10:25:55 pm Rainer Gerhards wrote: > Hi all, > > I now have a use case where the "last message..." prevents sources from > flooding syslog. But now on to a different use case: does someone > actually use this feature to reduce the size of files or network > traffic? Except, of course, in cases where there is a message flood due > to either attact or a misbehaving source (these need to be treated > differentely - I am right now after the *destination* message > compression...). > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > > Sent: Tuesday, March 18, 2008 3:24 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message > > logged n times"... > > > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > > The only arguments for keeping the feature that I got on my > > > > lug was the > > > > > preservartion of disk/network IO. > > > > > > I think to prevent DOS attacks is a valid argument but as > > > > you said can > > > > > be easily circumvented by randomizing messages. > > > > I'm afraid it's not true in all cases. What if you do DOS > > attach not directly > > to do rsyslog, but via other daemon. In situation when you > > can't send message > > directly to syslog, but you can make some daemon generate > > message for you. > > This message would be probably always the same content. > > > > > To safeguard against dos attacks you could have a monitor > > > > that monitors > > > > > for extra ordinary amount of traffic and then generate a snmp trap. > > > Whether that should be a rsyslog plugin or part of other software is > > > open to debate. > > > > > > Regards > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: 18 March 2008 11:42 > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > > > > > times"... > > > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > > > I have asked the question on a local lug to get some more > > > > > > opinions. > > > > > > > I > > > > > > > > > > will post the results if any here. > > > > > > > > > > I did the same. :) > > > > > > > > > > The only arguments against removal I have received > > > > until now were: > > > > > 1. DoS (but nobody explained how) > > > > > > > > I think this is along the lines of making it easy to flood the log > > > > > > with > > > > > > > similar messages. If that's the case, I think it is a (too) weak > > > > argument because an attacker could easily include a random pattern > > > > inside the message. > > > > > > > > HOWEVER, the current logic indeed prevents flooding the > > > > log from equal > > > > > > messages, e.g. if a process runs wild. > > > > > > > > > 2. cleanness of logs (relative) > > > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > > > Excellent. > > > > > > > > Maybe there are also alternatives to the current > > > > behavior, something > > > > > in > > > > > > > between. Right now, the core engine does this, so the reduction > > > > capability is inherent to every action (except if an action > > > > > > specifically > > > > > > > forbids it, because it can not intelligently handle it - > > > > e.g. database > > > > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > > alternative would be to remove it from the core engine and move it > > > > > > into > > > > > > > the *file write* output plugin (omfile). This has its > > > > subtleties, too, > > > > > > so I don't like to take that approach lightly. Plus, it than means > > > > > > that > > > > > > > the network may become spammed. I think one core requirement is to > > > > > > find > > > > > > > people who actually *intentionally* use this feature and > > > > ask for their > > > > > > reasoning. That will probably be the best way to tell us > > > > if its really > > > > > > needed. And, of course, we need to weigh the negative > > > > effects of it > > > > > > against these con-points. > > > > > > > > Rainer > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 25 10:52:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 10:52:29 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803251046.55919.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> <200803251046.55919.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C3D@grfint2.intern.adiscon.com> Pluging... well, that depends on the use cases. I am trying to identify the actual use cases, so far I found the one with flooding/attack. The problem is that the current logic can not be moved to a plugin because it is right at the most inner workings of the engine. If we can identify the actual use cases, things will probably be much easier to work with. For example, I currently think that I can build an optional interface that enables flood-controlling inputs and reducing messages if some condition is given there. Note, however, that this is different from current logic, where the flood control only applies to some outputs, because the output decides if it can work with the repeated lines. The bottom line is that I think there are some valid use cases, we just don't know about them. The current implementation/idea is soooo bad because we don't know what we really need to do. So identifying the actual use cases is the priority and then doing a good design that covers them. I'll probably follow advise from the loganalysis mailing list and change the default to not do the "last message..." processing. Then, see who complains and if nobody does drop the feature. It would also be extremely useful to know *why* people are unhappy (uses cases ;)), instead of just knowing they are... Can you shed some light on that? Thanks for all your help, Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Tuesday, March 25, 2008 10:47 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] Feedback requested "Last message logged n > times"... > > Hi folks, > > I talked with some people, and nobody is happy with removing this thing > from > rsyslog. Won't be possible to move it to separate plug-in? > > On Monday 24 March 2008 10:25:55 pm Rainer Gerhards wrote: > > Hi all, > > > > I now have a use case where the "last message..." prevents sources > from > > flooding syslog. But now on to a different use case: does someone > > actually use this feature to reduce the size of files or network > > traffic? Except, of course, in cases where there is a message flood > due > > to either attact or a misbehaving source (these need to be treated > > differentely - I am right now after the *destination* message > > compression...). > > > > Thanks, > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter > Vrabec > > > Sent: Tuesday, March 18, 2008 3:24 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] Feedback requested "Last message > > > logged n times"... > > > > > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > > > The only arguments for keeping the feature that I got on my > > > > > > lug was the > > > > > > > preservartion of disk/network IO. > > > > > > > > I think to prevent DOS attacks is a valid argument but as > > > > > > you said can > > > > > > > be easily circumvented by randomizing messages. > > > > > > I'm afraid it's not true in all cases. What if you do DOS > > > attach not directly > > > to do rsyslog, but via other daemon. In situation when you > > > can't send message > > > directly to syslog, but you can make some daemon generate > > > message for you. > > > This message would be probably always the same content. > > > > > > > To safeguard against dos attacks you could have a monitor > > > > > > that monitors > > > > > > > for extra ordinary amount of traffic and then generate a snmp > trap. > > > > Whether that should be a rsyslog plugin or part of other software > is > > > > open to debate. > > > > > > > > Regards > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: 18 March 2008 11:42 > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] Feedback requested "Last message logged > n > > > > > > > > times"... > > > > > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > > > > > I have asked the question on a local lug to get some more > > > > > > > > opinions. > > > > > > > > > I > > > > > > > > > > > > will post the results if any here. > > > > > > > > > > > > I did the same. :) > > > > > > > > > > > > The only arguments against removal I have received > > > > > > until now were: > > > > > > 1. DoS (but nobody explained how) > > > > > > > > > > I think this is along the lines of making it easy to flood the > log > > > > > > > > with > > > > > > > > > similar messages. If that's the case, I think it is a (too) > weak > > > > > argument because an attacker could easily include a random > pattern > > > > > inside the message. > > > > > > > > > > HOWEVER, the current logic indeed prevents flooding the > > > > > > log from equal > > > > > > > > messages, e.g. if a process runs wild. > > > > > > > > > > > 2. cleanness of logs (relative) > > > > > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > > > > > Excellent. > > > > > > > > > > Maybe there are also alternatives to the current > > > > > > behavior, something > > > > > > > in > > > > > > > > > between. Right now, the core engine does this, so the reduction > > > > > capability is inherent to every action (except if an action > > > > > > > > specifically > > > > > > > > > forbids it, because it can not intelligently handle it - > > > > > > e.g. database > > > > > > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > > > alternative would be to remove it from the core engine and move > it > > > > > > > > into > > > > > > > > > the *file write* output plugin (omfile). This has its > > > > > > subtleties, too, > > > > > > > > so I don't like to take that approach lightly. Plus, it than > means > > > > > > > > that > > > > > > > > > the network may become spammed. I think one core requirement is > to > > > > > > > > find > > > > > > > > > people who actually *intentionally* use this feature and > > > > > > ask for their > > > > > > > > reasoning. That will probably be the best way to tell us > > > > > > if its really > > > > > > > > needed. And, of course, we need to weigh the negative > > > > > > effects of it > > > > > > > > against these con-points. > > > > > > > > > > Rainer > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > From friedl at hq.adiscon.com Tue Mar 25 11:31:39 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Tue, 25 Mar 2008 11:31:39 +0100 Subject: [rsyslog] rsyslog 3.12.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C41@grfint2.intern.adiscon.com> Hi all, rsyslog 3.12.4 has been released today. Thanks to a changed default, it now offers much faster log file writing. Also, advanced flow control has now been added to all inputs, giving an even more reliable system. There are some other small enhancements (see change log). There are also a number of bug fixes, the most important one being a fix for a massive memory leak in imklog. It lost several hundred kilobytes on each hup. Rsyslog 3.12.4 is a recommended update for all v3 branch users. Download: http://www.rsyslog.com/Downloads-req-getit-lid-89.phtml Changelog: http://www.rsyslog.com/Article195.phtml As always, feedback is appreciated. Florian Riedl From rgerhards at hq.adiscon.com Tue Mar 25 16:33:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 16:33:55 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> Hi all, a question on the forum (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a bit asking around finally made me think about the possibility of a v3 stable version, something that dangled on my mind for a while now... While it is just three month since v2 stable, we have gone a long way and during that time the new queue engine has much matured. As "development" state is most often a show-stopper for putting code into production, I think about releasing a 3.0.0 stable (based on the most recent 3.12.x version WITHOUT RELP support). >From a support perspective, I'd like to continue support v2 stable and obviously v3 stable and development concurrently. Development would continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug fixes. Some time from now, 3.. would become 3.1.0 stable. At this point, I would drop support/fixing for 3.0. and ask users with problems in the branch to update to 3.1.. Does this sound reasonable? The big problem is that I must be careful not to create too many version which I must support, but at the same time allow new, matured, features to go into broader production. And, of course, 3.0.0 stable would probably be a bit less stable than 2.0.4, but only if you use features that are not present in v2. All in all, the queue engine has received more than a fire drill and is ready for production - though probably not fully trouble free. But we won't see that until it gets more used. Feedback is appreciated. Thanks, Rainer From r.bhatia at ipax.at Tue Mar 25 17:04:10 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Tue, 25 Mar 2008 17:04:10 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> Message-ID: <47E9227A.5010903@ipax.at> Rainer Gerhards wrote: > Hi all, > > a question on the forum > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a bit > asking around finally made me think about the possibility of a v3 stable > version, something that dangled on my mind for a while now... While it > is just three month since v2 stable, we have gone a long way and during > that time the new queue engine has much matured. As "development" state > is most often a show-stopper for putting code into production, I think > about releasing a 3.0.0 stable (based on the most recent 3.12.x version > WITHOUT RELP support). > >>From a support perspective, I'd like to continue support v2 stable and > obviously v3 stable and development concurrently. Development would > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > fixes. Some time from now, 3.. would become 3.1.0 > stable. At this point, I would drop support/fixing for 3.0. > and ask users with problems in the branch to update to 3.1.. i guess this will cause serious headake because people tend to think like this: 3.0.x is less mature/was released before 3.1.x is less mature/was released before 3.12.x is less mature/was released before 3.13.x i guess there should be a deliberate versioning scheme. examples are: - versioning similar to the linux kernel - release 3.12.x as 3.12.x-stable and continue development without this tag. - something else but i guess its no good to jump from 3.12.x back to 3.0.x and from a future revision such as 3.40.x back to 3.1.x cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Tue Mar 25 17:06:52 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 17:06:52 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <47E9227A.5010903@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> Hi Raoul, OK, good feedback on the version number scheme - it has been sitting this (confusing) way all time since the early days of rsyslog... Probably a sufficiently good cleanup would be to go to 4.x and the follow a reasonable scheme from then on... But besides that: how do you feel about the facts other then the version numbering? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Tuesday, March 25, 2008 5:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.0.0 stable? - feedback, please > > Rainer Gerhards wrote: > > Hi all, > > > > a question on the forum > > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a > bit > > asking around finally made me think about the possibility of a v3 > stable > > version, something that dangled on my mind for a while now... While > it > > is just three month since v2 stable, we have gone a long way and > during > > that time the new queue engine has much matured. As "development" > state > > is most often a show-stopper for putting code into production, I > think > > about releasing a 3.0.0 stable (based on the most recent 3.12.x > version > > WITHOUT RELP support). > > > >>From a support perspective, I'd like to continue support v2 stable > and > > obviously v3 stable and development concurrently. Development would > > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > > fixes. Some time from now, 3.. would become 3.1.0 > > stable. At this point, I would drop support/fixing for 3.0. > > and ask users with problems in the branch to update to > 3.1.. > > i guess this will cause serious headake because people tend to think > like this: > > 3.0.x is less mature/was released before > 3.1.x is less mature/was released before > 3.12.x is less mature/was released before > 3.13.x > > i guess there should be a deliberate versioning scheme. examples are: > > - versioning similar to the linux kernel > - release 3.12.x as 3.12.x-stable and continue development without this > tag. > - something else > > but i guess its no good to jump from 3.12.x back to 3.0.x and from a > future revision such as 3.40.x back to 3.1.x > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Tue Mar 25 18:05:04 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Tue, 25 Mar 2008 18:05:04 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> Message-ID: <47E930C0.6020903@ipax.at> hi, Rainer Gerhards wrote: > OK, good feedback on the version number scheme - it has been sitting > this (confusing) way all time since the early days of rsyslog... > Probably a sufficiently good cleanup would be to go to 4.x and the > follow a reasonable scheme from then on... > > But besides that: how do you feel about the facts other then the version > numbering? sorry, i cannot quote on this as i am not using v3 as of now. this might change in the next couple of weeks but as of now we're still on version v1.x, as this version is not giving much trouble. cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Tue Mar 25 18:16:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 18:16:05 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <47E9227A.5010903@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> Hi all, I have thought about the version numbering. Obviously, the old scheme (http://www.rsyslog.com/doc-version_naming.html ) is confusing and should be dropped. Rsyslog currently follows a three-number version scheme: major.minor.patchlevel I can follow an odd/even scheme on major for unstable/stable branches. However, with major features being continuously introduced may bring us to rsyslog 10.x.x by the end of the year. While I have no hard concerns, it seems to be unusual for open source to increase the major version that fast. On the other hand, I need minor.patchlevel to handle the smaller fixes and new major feature. I increase minor for each new feature (like queue engine, expression-based filters, relp, to name a few actual ones) and increase patchlevel for smaller changes. The three number scheme works well, but, again, keeps the major number very quickly growing given the current fast pace of development. An alternative would be to use politicalmajor.major.minor.patchlevel Where politicalmajor would only be incremented every now and then and actually without much technical reasoning for doing so - only then when we "feel" it is time for a new major release. It would probably turn out to be incremented around once a year, just to keep the corporate folks happy and the major version number reasonable low. Question now: which scheme should I follow? Is there any other one? If so, why should I follow that other scheme? I would appreciate quick feedback, as I'd like to settle this issue before releasing RELP support, which currently looks like some time (very) early April. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Tuesday, March 25, 2008 5:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.0.0 stable? - feedback, please > > Rainer Gerhards wrote: > > Hi all, > > > > a question on the forum > > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a > bit > > asking around finally made me think about the possibility of a v3 > stable > > version, something that dangled on my mind for a while now... While > it > > is just three month since v2 stable, we have gone a long way and > during > > that time the new queue engine has much matured. As "development" > state > > is most often a show-stopper for putting code into production, I > think > > about releasing a 3.0.0 stable (based on the most recent 3.12.x > version > > WITHOUT RELP support). > > > >>From a support perspective, I'd like to continue support v2 stable > and > > obviously v3 stable and development concurrently. Development would > > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > > fixes. Some time from now, 3.. would become 3.1.0 > > stable. At this point, I would drop support/fixing for 3.0. > > and ask users with problems in the branch to update to > 3.1.. > > i guess this will cause serious headake because people tend to think > like this: > > 3.0.x is less mature/was released before > 3.1.x is less mature/was released before > 3.12.x is less mature/was released before > 3.13.x > > i guess there should be a deliberate versioning scheme. examples are: > > - versioning similar to the linux kernel > - release 3.12.x as 3.12.x-stable and continue development without this > tag. > - something else > > but i guess its no good to jump from 3.12.x back to 3.0.x and from a > future revision such as 3.40.x back to 3.1.x > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From pvrabec at redhat.com Wed Mar 26 11:47:21 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Wed, 26 Mar 2008 11:47:21 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> Message-ID: <200803261147.21726.pvrabec@redhat.com> Hi, politicalmajor.major.minor.patchlevel seems good to me. politicalmajor - something great happened, like v2 and v3 major - odd, even - developer/stable release minor - new features patchlevel - patchlevel so current 3.12.4 becomes 3.0.12.4 development starts with 3.1.0.0 and continues to 3.1.0.1, 3.1.0.2, 3.1.1.0, 3.1.2.0, ... 3.1.6.10 than we decide to release stable 3.2.6.0, 3.2.6.1, 3.2.6.2 or 3.2.0.0, 3.2.0.1, 3.2.0.2? development 3.3.0.0, ... 3.3.x.y There is one problem 3.12.4 > 3.0.12.4 == problem with upgrade solutions: 1) 3.12.4 -> 3.14.0.0 2) 3.12.4 -> 3.0.12.4 and I'll start new epoch of rsyslog package (Epoch: 2 in specfile, don't know how is it in other distros) On Tuesday 25 March 2008 06:16:05 pm Rainer Gerhards wrote: > Hi all, > > I have thought about the version numbering. Obviously, the old scheme > (http://www.rsyslog.com/doc-version_naming.html ) is confusing and > should be dropped. > > Rsyslog currently follows a three-number version scheme: > > major.minor.patchlevel > > I can follow an odd/even scheme on major for unstable/stable branches. > However, with major features being continuously introduced may bring us > to rsyslog 10.x.x by the end of the year. While I have no hard concerns, > it seems to be unusual for open source to increase the major version > that fast. On the other hand, I need minor.patchlevel to handle the > smaller fixes and new major feature. I increase minor for each new > feature (like queue engine, expression-based filters, relp, to name a > few actual ones) and increase patchlevel for smaller changes. The three > number scheme works well, but, again, keeps the major number very > quickly growing given the current fast pace of development. > > An alternative would be to use > > politicalmajor.major.minor.patchlevel > > Where politicalmajor would only be incremented every now and then and > actually without much technical reasoning for doing so - only then when > we "feel" it is time for a new major release. It would probably turn out > to be incremented around once a year, just to keep the corporate folks > happy and the major version number reasonable low. > > Question now: which scheme should I follow? Is there any other one? If > so, why should I follow that other scheme? > I would appreciate quick feedback, as I'd like to settle this issue > before releasing RELP support, which currently looks like some time > (very) early April. > > Thanks, > Rainer From linuxweb at gmail.com Thu Mar 27 02:28:46 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 26 Mar 2008 21:28:46 -0400 Subject: [rsyslog] central logserver log rotation Message-ID: <47EAF84E.4010909@gmail.com> Thanks to Rainer, I've got my shiny new central log server up & running. I'd like to rotate the logs on there. If I've got a separate directory for each host, though (i.e., /var/log/rsyslog/%HOSTNAME%), how can I do this? Does something like this work with logrotate: /var/log/rsyslog/*/* { weekly notifempty missingok } What are people doing for logrotation on their CENTRAL logging server? johnn From rgerhards at hq.adiscon.com Thu Mar 27 10:00:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 10:00:46 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <200803261147.21726.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> Hi Peter, thanks for the feedback. So it looks like we go for the 4-number scheme. If I look at the migration case, I am tempted to do a one-time move to 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as interim devel release) to prevent any confusion about version numbers. Changing from a 3-number to a 4-number scheme within the same major release sounds confusing in itself to me. Any concerns with such an approach? Anyone? Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Wednesday, March 26, 2008 11:47 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] rsyslog version numbering - was: RE: rsyslog > 3.0.0 stable? - feedback, please > > Hi, > > politicalmajor.major.minor.patchlevel seems good to me. > > politicalmajor - something great happened, like v2 and v3 > > major - odd, even - developer/stable release > > minor - new features > > patchlevel - patchlevel > > so current 3.12.4 becomes 3.0.12.4 > > development starts with 3.1.0.0 and continues to > > 3.1.0.1, 3.1.0.2, 3.1.1.0, 3.1.2.0, ... 3.1.6.10 > > than we decide to release stable > > 3.2.6.0, 3.2.6.1, 3.2.6.2 > > or > > 3.2.0.0, 3.2.0.1, 3.2.0.2? > > development 3.3.0.0, ... 3.3.x.y > > There is one problem 3.12.4 > 3.0.12.4 == problem with upgrade > > solutions: > > 1) 3.12.4 -> 3.14.0.0 > > 2) 3.12.4 -> 3.0.12.4 and I'll start new epoch of rsyslog package > > (Epoch: 2 in specfile, don't know how is it in other distros) > > On Tuesday 25 March 2008 06:16:05 pm Rainer Gerhards wrote: > > > Hi all, > > > > > > I have thought about the version numbering. Obviously, the old scheme > > > (http://www.rsyslog.com/doc-version_naming.html ) is confusing and > > > should be dropped. > > > > > > Rsyslog currently follows a three-number version scheme: > > > > > > major.minor.patchlevel > > > > > > I can follow an odd/even scheme on major for unstable/stable > branches. > > > However, with major features being continuously introduced may bring > us > > > to rsyslog 10.x.x by the end of the year. While I have no hard > concerns, > > > it seems to be unusual for open source to increase the major version > > > that fast. On the other hand, I need minor.patchlevel to handle the > > > smaller fixes and new major feature. I increase minor for each new > > > feature (like queue engine, expression-based filters, relp, to name a > > > few actual ones) and increase patchlevel for smaller changes. The > three > > > number scheme works well, but, again, keeps the major number very > > > quickly growing given the current fast pace of development. > > > > > > An alternative would be to use > > > > > > politicalmajor.major.minor.patchlevel > > > > > > Where politicalmajor would only be incremented every now and then and > > > actually without much technical reasoning for doing so - only then > when > > > we "feel" it is time for a new major release. It would probably turn > out > > > to be incremented around once a year, just to keep the corporate > folks > > > happy and the major version number reasonable low. > > > > > > Question now: which scheme should I follow? Is there any other one? > If > > > so, why should I follow that other scheme? > > > I would appreciate quick feedback, as I'd like to settle this issue > > > before releasing RELP support, which currently looks like some time > > > (very) early April. > > > > > > Thanks, > > > Rainer From rgerhards at hq.adiscon.com Thu Mar 27 10:13:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 10:13:25 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: <47EAF84E.4010909@gmail.com> References: <47EAF84E.4010909@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Hi Johnn, the typical solution seems to be to include a date-specific part in the filename: /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% And clean up the directories every now and then... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Thursday, March 27, 2008 2:29 AM > To: rsyslog-users > Subject: [rsyslog] central logserver log rotation > > Thanks to Rainer, I've got my shiny new central log server > up & running. I'd like to rotate the logs on there. > > If I've got a separate directory for each host, though > (i.e., /var/log/rsyslog/%HOSTNAME%), how can I do this? > > Does something like this work with logrotate: > > /var/log/rsyslog/*/* { > weekly > notifempty > missingok > } > > > What are people doing for logrotation on their CENTRAL > logging server? > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Thu Mar 27 15:39:11 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 27 Mar 2008 10:39:11 -0400 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> Message-ID: <47EBB18F.1040803@gmail.com> Rainer Gerhards wrote: > Hi Peter, > > thanks for the feedback. So it looks like we go for the 4-number scheme. > > If I look at the migration case, I am tempted to do a one-time move to > 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as > interim devel release) to prevent any confusion about version numbers. > Changing from a 3-number to a 4-number scheme within the same major > release sounds confusing in itself to me. I support the suggestion above. Trying to number within v3 is going to wreak havoc on attempts to update properly. p.s. To be perfectly honest, I didn't think a move to 4-number was even necessary, but I think you'd know better than anyone else what is in the pipeline for rsyslog, and if those are major enough to require "major" number to be changed, then, yeah, it wouldn't make sense to see 10.x.x within a year or two. johnn From rgerhards at hq.adiscon.com Thu Mar 27 16:39:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 16:39:45 +0100 Subject: [rsyslog] timestamp format - why use the old one? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C74@grfint2.intern.adiscon.com> Hi all, I got an interesting question from another discussion list. As you (hopefully) are aware, rsyslog supports high precision timestamps. However, the default format does not output them in order to avoid confusing log parsers. That, in turn, leads to few people actually ever using it. So in the end run, rsyslog's high-precision timestamps have never been used much in the past 3+ years. One suggestion to solve the log parser problem was that we should begin to use the high-precision stamp by default and work with converters/filters when data is fed to log analysers. What is your opinion on this topic? Is it time to change the default output format for timestamps? Rainer From friedl at hq.adiscon.com Thu Mar 27 17:32:56 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 27 Mar 2008 17:32:56 +0100 Subject: [rsyslog] rsyslog 2.0.4 stable released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C78@grfint2.intern.adiscon.com> Hi all, rsyslog 2.0.4, a stable release version, has been released today. It contains some bug fixes, the most important one is that FROMHOST was not properly set on internal messages. There are also two new properties ($HHOUR and $QHOUR) which can be used in dynamic log file generation. Rsyslog 2.0.4 is a recommended update for all users of the stable v2 branch. Changelog: http://www.rsyslog.com/Article197.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-90.phtml Florian Riedl From janfrode at tanso.net Fri Mar 28 08:51:22 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 28 Mar 2008 08:51:22 +0100 Subject: [rsyslog] central logserver log rotation References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: On 2008-03-27, Rainer Gerhards wrote: > the typical solution seems to be to include a date-specific part in the > filename: > > /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% > > And clean up the directories every now and then... /etc/rsyslog.conf: ##################################################### # Log everything to a per host daily logfile # ##################################################### $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log" *.* -?DailyPerHostLogs /etc/cron.hourly/syslog-bzip2: find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec bzip2 '{}' \; -jf From rgerhards at hq.adiscon.com Fri Mar 28 10:17:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 10:17:44 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: References: <47EAF84E.4010909@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C82@grfint2.intern.adiscon.com> Hi JF, thanks, I've made a wiki page out of it: http://wiki.rsyslog.com/index.php/DailyLogRotation Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, March 28, 2008 8:51 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] central logserver log rotation > > On 2008-03-27, Rainer Gerhards wrote: > > the typical solution seems to be to include a date-specific part in > the > > filename: > > > > /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% > > > > And clean up the directories every now and then... > > /etc/rsyslog.conf: > > ##################################################### > # Log everything to a per host daily logfile # > ##################################################### > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/me > ssages.log" > *.* -?DailyPerHostLogs > > /etc/cron.hourly/syslog-bzip2: > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec > bzip2 '{}' \; > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 28 12:58:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 12:58:37 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47EBB18F.1040803@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> > > If I look at the migration case, I am tempted to do a one-time move > to > > 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as > > interim devel release) to prevent any confusion about version > numbers. > > Changing from a 3-number to a 4-number scheme within the same major > > release sounds confusing in itself to me. > > I support the suggestion above. Trying to number within v3 > is going to wreak havoc on attempts to update properly. > > p.s. To be perfectly honest, I didn't think a move to > 4-number was even necessary, I thought quite a while over your message. I have to admit that I personally would prefer a three-number scheme. Maybe there is a solution without going overboard on the major version number increments. First of all, things become stable because bugs are fixed - and no new ones added. "No new ones added" means every stable version is a dead-end when it comes to features. So, 2.0.4 may become 2.0.9 or even 2.0.9999, but we won't see a 2.1.x as that would imply a sufficiently large new feature. This is also a quite hard limit as it means the stable version will always starve for new features. But that's how it works today. In devel, the minor is updated whenever something cool enough is added to justify that. Which means typically every 2 to 6 weeks on the current schedule. So 3.10.0 has become 3.10.2, later 3.11.0 and we are now (soon;)) at 3.12.5. The interesting thing is what happens next: 3.12.5 is quite stable now. There are some feature introduced in 3.10.0 being very stable, some from 3.11.0 which are the same and there as some new ones in 3.12.0 introduced which are not as stable as others (because of limited exposure to practice). Anyhow, I'll soon convert that whole version to 4.0.0 (maybe 4.0.0.0) stable. Then I'll start a new devel branch and the stable build again begins to mature but also become feature-less compared to the devel - which has various stages of maturity some time later. I now got a new school of thinking: let's stick with the 3-digit scheme: major.minor.patchlevel Major will be incremented whenever there is sufficiently large change (typically some inner workings must be changed quite a bit). Stabel and Devel will coexist within the same major release. For minor, we have even - stable and odd - devel. Patchlevel again is smaller magnitude changes within the minor (or evolution of a new feature not yet announced to be usable). But that doesn't solve my need to sometimes have two or three devel versions with new features in a row. To solve that, I suggest that we simply skip the stable releases. For example: 4.0.0 has been released, 4.1.0 is the new stable. I add relp support in 4.1.0. Then I do some changes and end up at 4.1.4. Then, I decide to do the new config file format. That'll be 4.3.0. Note that there now is a stable version missing: we have 4.0.0, 4.1.4 and 4.3.0. I work some more on devel. Then comes a new feature (let's guess TLS support...). So we finally have 4.0.0, 4.1.4, 4.3.x and 4.5.x. After a while, I am at 4.5.5 and also got enough bug fixes to think that the feature introduced in 4.3.x is now ready for prime time. Then, I'll release 4.4.0, based on the 4.3.x codebase (and missing the new 4.5.x feature). With the release of 4.4.0, bugfixing for 4.0.x will end, as we now have a new stable version of the v4 branch (but fixes would still be applied to stable v2). So we end up with 4.0.x (deadend), 4.1.x (dead), NO 4.2.x, 4.3.x (dead), 4.4.x (current stable v4) and 4.5.x (current stable v5). The key thing is that I left out 4.2.x. If that seems to be acceptable, we do not need the 4-number scheme. I know it's pretty wild with all these version numbers... I hope I still could get the idea over. Feedback is highly appreciated. If there is none and no objection by next week Tuesday, I'll go for that scheme. I intend to release 4.0.0(.0) next Wednesday. Today's release of 3.12.5 will set the stage for it. > but I think you'd know better > than anyone else what is in the pipeline for rsyslog, and if > those are major enough to require "major" number to be > changed, then, yeah, it wouldn't make sense to see 10.x.x > within a year or two. Just some thoughts of the pipeline. No guarantees on the order, but the order is current thinking: * relp support * relp with TLS support (followed by relp/GSSAPI) * config file format (scriptable, RainerScript) * email notification output * native TCP w. TLS/syslog-transport-tls internet draft * cryptographically signed messages * support for other message types but syslog * tamperproof (compliance-compliant ;)) system that forces rsyslogd to work with a centralized configuration and provides the ability to detect unauthorized client configuration changes (where they can't be prevented) ... and probably much more ;) Rainer From r.bhatia at ipax.at Fri Mar 28 14:26:14 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 28 Mar 2008 14:26:14 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> Message-ID: <47ECF1F6.8030909@ipax.at> hi, what about the following major.minor.patchlevel with 3.x.y as an exception, used as: STABLE RELEASES =============== 1) release 3.x.y (e.g. 3.12.0) as stable. we might continue providing patches, fixes, etc. in a fashion like: - 3.12.0 (= stable) - 3.13.0 (= major fix) - 3.14.0 (= major fix) - 3.15.0 (= small fix) ... 2) continue development with 4.0.x. release 4.0.14 as stable with: - 4.0.14 (= stable) - 4.0.15 (= fix) - 4.0.16 (= fix) - 4.0.17 (= fix) ... 3) continue development with 4.1.x, release e.g. 4.1.25 as stable with: - 4.1.25 (= stable) - 4.1.26 (= fix) - 4.1.27 (= fix) - 4.1.28 (= fix) ... that's for the handling of stable releases. ad 1) as a more consistent use of the major.minor.patchlevel, we could use: - 3.12.0 - 3.12.1 - 3.12.2 ... - 3.12.1255 DEVELOPMENT SUGGESTION A ======================== inside the development tree, one could do: 4.1.0 = new dev release 4.1.1-rc1 = major new features - exerimental - only for hardcore testers 4.1.1-rc2 = getting more stable 4.1.1-rc3 = getting more stable 4.1.1 = let more ppl test this release with its new features 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not be continued/patched/etc. DEVELOPMENT SUGGESTION B ======================== do not release that many devel releases but put out nightlies with a git "short" tag (i think i've seen this on hg). [1] this would mean: 4.1.0 = new dev tree is started. 4.1.1-d12080bf3d99 snapshot/interim release 2008-03-28 big new features! 4.1.1-c377eb7b2a22 snapshot/interim release 2008-03-29 fixing 4.1.1-b7b2a2515d3a snapshot/interim release 2008-03-30 fixing 4.1.1 - now we're confident enough to make a release for testing. 4.1.2-10e7d626f05c - big new features snapshot/interim release ... 4.1.2 - we're confident - public testers wanted ... 4.1.25 - now we're stable - declare stable, no more features. ;) of course, such interim releases must not happen on a daily basis. everyone should be free to do a recent checkout and those interim releases could be limited to rainer feeling confident enough ;) so, what about this suggestion? cheers, raoul [1] how to shorten the git revisions. long: 4.1.1-d120806a4549ae6c377eb7b2a25172251fbf3d99 4.1.1-d12080(6a4549ae6c377eb7b2a25172251f)bf3d99 4.1.1-d12080...bf3d99 4.1.1-d12080.bf3d99 4.1.1-d12080bf3d99 -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Mar 28 14:56:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 14:56:51 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ECF1F6.8030909@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> > what about the following major.minor.patchlevel with 3.x.y as an > exception, used as: > > STABLE RELEASES > =============== > 1) release 3.x.y (e.g. 3.12.0) as stable. > we might continue providing patches, fixes, etc. in a fashion like: > - 3.12.0 (= stable) > - 3.13.0 (= major fix) > - 3.14.0 (= major fix) > - 3.15.0 (= small fix) > ... > > 2) continue development with 4.0.x. release 4.0.14 as stable with: > - 4.0.14 (= stable) > - 4.0.15 (= fix) > - 4.0.16 (= fix) > - 4.0.17 (= fix) > ... > > 3) continue development with 4.1.x, release e.g. 4.1.25 as stable with: > - 4.1.25 (= stable) > - 4.1.26 (= fix) > - 4.1.27 (= fix) > - 4.1.28 (= fix) > ... > > that's for the handling of stable releases. So in essence, at one patchlevel a major.minor becomes stable and a new (major.minor) is begun - that would work for me and sound quite logical... A key point still would be that there could be a 4.1.24 unstable, while I begin to work on 4.2.0. After all, things are unstable at first ;) So we have two unstable in a row. After a while (we may be at 4.2.6) we can release 4.1.25 (stable) as it has matured enough. Any more comments on that? > ad 1) as a more consistent use of the major.minor.patchlevel, we could > use: > - 3.12.0 > - 3.12.1 > - 3.12.2 > ... > - 3.12.1255 > > > > DEVELOPMENT SUGGESTION A > ======================== > inside the development tree, one could do: > 4.1.0 = new dev release > 4.1.1-rc1 = major new features - exerimental - only for hardcore > testers > 4.1.1-rc2 = getting more stable > 4.1.1-rc3 = getting more stable > 4.1.1 = let more ppl test this release with its new features > > 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not be > continued/patched/etc. I see where you are coming from. This is actually a three and a half version numbering scheme - but also makes sense to me. Looks like a good solution. I like this much more than suggestion B, I have to admit ;) -- the main reason is that I tend to commit very frequently, but a nightly release will probably bad to follow up on. Overall, these suggestions look easier to follow than mine. So if nobody objects, I'll implement them by mid next week. BUT... Feedback is still most welcome! Rainer > > > > DEVELOPMENT SUGGESTION B > ======================== > do not release that many devel releases but put out nightlies with > a git "short" tag (i think i've seen this on hg). [1] > > this would mean: > 4.1.0 = new dev tree is started. > 4.1.1-d12080bf3d99 snapshot/interim release 2008-03-28 big new > features! > 4.1.1-c377eb7b2a22 snapshot/interim release 2008-03-29 fixing > 4.1.1-b7b2a2515d3a snapshot/interim release 2008-03-30 fixing > 4.1.1 - now we're confident enough to make a release for testing. > > 4.1.2-10e7d626f05c - big new features snapshot/interim release > ... > 4.1.2 - we're confident - public testers wanted > ... > 4.1.25 - now we're stable - declare stable, no more features. ;) > > of course, such interim releases must not happen on a daily basis. > everyone should be free to do a recent checkout and those interim > releases could be limited to rainer feeling confident enough ;) > > > > so, what about this suggestion? > > cheers, > raoul > > [1] how to shorten the git revisions. > long: 4.1.1-d120806a4549ae6c377eb7b2a25172251fbf3d99 > 4.1.1-d12080(6a4549ae6c377eb7b2a25172251f)bf3d99 > 4.1.1-d12080...bf3d99 > 4.1.1-d12080.bf3d99 > 4.1.1-d12080bf3d99 > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Fri Mar 28 16:03:05 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 11:03:05 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> Message-ID: <47ED08A9.40500@gmail.com> Rainer Gerhards wrote: >> inside the development tree, one could do: >> 4.1.0 = new dev release >> 4.1.1-rc1 = major new features - exerimental - only for hardcore >> testers >> 4.1.1-rc2 = getting more stable >> 4.1.1-rc3 = getting more stable >> 4.1.1 = let more ppl test this release with its new features >> >> 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not > be >> continued/patched/etc. > > I see where you are coming from. This is actually a three and a half > version numbering scheme - but also makes sense to me. Looks like a good > solution. I like this much more than suggestion B, I have to admit ;) -- > the main reason is that I tend to commit very frequently, but a nightly > release will probably bad to follow up on. I like this idea too (or the general direction of it, anyway -- I'm sure once you start down it, you will likely tweak it to fit actual needs). It lets you keep the 3-number scheme, while still providing for interim releases that don't overload the version number. And a few other projects seem to use this scheme too (consistency is good). > Overall, these suggestions look easier to follow than mine. So if nobody > objects, I'll implement them by mid next week. BUT... Feedback is still > most welcome! So, starting next week, there will be 3 branches? * 2.0.x -- stable, this is the one that systems like RHEL will track for awhile * 3.12.x -- what is this called? this is the "new" stable, for those that need the new 3.x features, but also want a relatively stabilized version, once the numbering is sufficiently high * 4.0.x -- devel, this is where the new major features start going in I think this works. Actually, taking a look at mysql just for comparison sake, they appear to have something similar: 5.0.x (for super-stable, like RHEL), 5.1.x (for those who need full NDB support, but mostly stable once it comes out of RC), and 6.0.x (alpha). johnn From linuxweb at gmail.com Fri Mar 28 16:06:48 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 11:06:48 -0400 Subject: [rsyslog] central logserver log rotation In-Reply-To: References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: <47ED0988.2060705@gmail.com> Jan-Frode Myklebust wrote: > /etc/rsyslog.conf: > > ##################################################### > # Log everything to a per host daily logfile # > ##################################################### > $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log" > *.* -?DailyPerHostLogs > > /etc/cron.hourly/syslog-bzip2: > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec bzip2 '{}' \; Thanks for the example. And ... I was just thinking of doing some gzip/bzip on my morning commute today, so your cron example comes just in time too :). johnn From rgerhards at hq.adiscon.com Fri Mar 28 16:25:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 16:25:12 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: <47ED0988.2060705@gmail.com> References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> <47ED0988.2060705@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C8C@grfint2.intern.adiscon.com> In the long term (think tank!) it may be possible do some automagic with dynafiles. Let's say specify a timeout after which, when no longer accessed, some (external) rotation logic is invoked. The good thing, probably, would be that rsyslog knows best when it can and need to do so. It could even invoke it somewhat quicker if it runs out of disk space ;) Again, nothing to be implemented soon, but if you have an opinion on it, I'd like to hear about it. Side-Note: we are getting close to an initial release of the web interface. It will not only work with databases, but also with text files. I have this, too, on my mind when I think about rotation... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 28, 2008 4:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] central logserver log rotation > > Jan-Frode Myklebust wrote: > > /etc/rsyslog.conf: > > > > ##################################################### > > # Log everything to a per host daily logfile # > > ##################################################### > > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/me > ssages.log" > > *.* -?DailyPerHostLogs > > > > /etc/cron.hourly/syslog-bzip2: > > > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec > bzip2 '{}' \; > > Thanks for the example. And ... I was just thinking of doing > some gzip/bzip on my morning commute today, so your cron > example comes just in time too :). > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Fri Mar 28 16:48:16 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 28 Mar 2008 16:48:16 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> Message-ID: <47ED1340.8030300@ipax.at> Rainer Gerhards wrote: > A key point still would be that there could be a 4.1.24 unstable, while > I begin to work on 4.2.0. After all, things are unstable at first ;) So > we have two unstable in a row. After a while (we may be at 4.2.6) we can > release 4.1.25 (stable) as it has matured enough. i see no problem with this. there even does not need to be a 4.1.x which is declared as stable. > I see where you are coming from. This is actually a three and a half > version numbering scheme - but also makes sense to me. Looks like a good > solution. I like this much more than suggestion B, I have to admit ;) -- > the main reason is that I tend to commit very frequently, but a nightly > release will probably bad to follow up on. there need not be a nightly release. after ordering things in my mind, the main difference is: do we need to map -rcX to git snapshot 45b6c18f1d13be2ff89ee62024ba... or do we allready have this information somewhere accessible. besides that, we could skip the "rc" string and simply use 4.0.12-1 - like a build number which is used by some linux distributions. another nice aspect would be to integrate the git hash into the initial startup message or in some status output. see: > # cat /proc/drbd > version: 8.0.8 (api:86/proto:86) > GIT-hash: bd3e2c922f95c4fa0dca57a4f8c24bf8b249cc02 build by root at webcluster01, 2008-01-29 18:36:06 > Mar 23 06:25:27 webcluster02 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="2525"][x-configInfo udpReception="No" gitHash="bd3e2c922f95c4fa0dca57a4f8c24bf8b249cc02" > udpPort="514" tcpReception="No" tcpPort="0"] > Overall, these suggestions look easier to follow than mine. So if nobody > objects, I'll implement them by mid next week. BUT... Feedback is still > most welcome! maybe it would be good to make a final statement before implementing and releasing it. so we do not have to revert/skip/create the version 5 and 6 and discard 3/4 as a "versioning experiment" ;) cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From linuxweb at gmail.com Fri Mar 28 17:00:37 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 12:00:37 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1340.8030300@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> Message-ID: <47ED1625.9030305@gmail.com> Raoul Bhatia [IPAX] wrote: > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > like a build number which is used by some linux distributions. Actually, Red Hat-based distros (at least) use that "-1" for internal changes. Let's say there is a version: 4.0.12rc3 I (or Red Hat) decide to build an RPM based on that, so my version is: 4.0.12rc3-1 Then, Rainer releases a 4.0.12rc4. It has too many new features in it, so I decide not to move to it. HOWEVER, it also has one crucial bugfix that I need. So I take that patch and backport it into my version. Now, my new *local* version is: 4.0.12rc3-2 So, I do like the idea of attaching rc* (because that makes it clear the changes are from the developer), but not a dash-number (-1, etc.) to the version, if at all possible, because those tend to be localized/internal version number changes. johnn From rgerhards at hq.adiscon.com Fri Mar 28 17:58:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 17:58:58 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1625.9030305@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C91@grfint2.intern.adiscon.com> And this sounds very reasonable, too. Also -rc seems to alert some folks and makes clear what they are doing ;) Please keep the feedback flowing, very good discussion. Just a quick explanation why I would like to settle this out of the sudden quickly. It's a somewhat political issue. While I have no distro-preferrence (rsyslog should run great on as many platforms as possible), I can get the new versioning scheme into Fedora 9 if I do it by mid next week. That is a very nice incentive ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 28, 2008 5:01 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Raoul Bhatia [IPAX] wrote: > > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > > like a build number which is used by some linux distributions. > > Actually, Red Hat-based distros (at least) use that "-1" for > internal changes. > > Let's say there is a version: 4.0.12rc3 > > I (or Red Hat) decide to build an RPM based on that, so my > version is: > 4.0.12rc3-1 > > > Then, Rainer releases a 4.0.12rc4. It has too many new > features in it, so I decide not to move to it. HOWEVER, it > also has one crucial bugfix that I need. So I take that > patch and backport it into my version. Now, my new *local* > version is: > 4.0.12rc3-2 > > So, I do like the idea of attaching rc* (because that makes > it clear the changes are from the developer), but not a > dash-number (-1, etc.) to the version, if at all possible, > because those tend to be localized/internal version number > changes. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 28 18:45:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 18:45:58 +0100 Subject: [rsyslog] rsyslog 3.12.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C93@grfint2.intern.adiscon.com> Hi All, rsyslog 3.12.5 has just been released. It is mostly a bug-fixing and cleanup release. Some changed defaults make it easier to use advanced features. For example, high-precision timestamps are now enabled by default when writing log files and "last message repeated n times" processing, a typical trouble spot, is turned off by default. The documentation has been improved and the v2-backward compatibility layer has been completed. This version is a candidate to become the next stable release. Changelog: http://www.rsyslog.com/Article199.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-91.phtml I suggest to (re-) read ./doc/v3compatibility.html, which also talks about the new defaults. Also available online at: http://www.rsyslog.com/doc-v3compatibility.html Feedback on this release is appreciated. I hope it will be useful. Rainer From rgerhards at hq.adiscon.com Sat Mar 29 10:44:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 29 Mar 2008 10:44:26 +0100 Subject: [rsyslog] debug setting in 3.12.5 active Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C96@grfint2.intern.adiscon.com> Hi all, I just noticed that a debug setting is active in 3.12.5. This disables the actual dlclose. It should not have too bad side effects. I will fix this in 3.12.6, with a few other cleanups, which I hopefully release later today. So you may want to hold updating if you have not done so already. Sorry for the hassle, Rainer From mbiebl at gmail.com Sat Mar 29 12:47:44 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 12:47:44 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1625.9030305@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: 2008/3/28, Johnny Tan : > Raoul Bhatia [IPAX] wrote: > > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > > like a build number which is used by some linux distributions. > > > Actually, Red Hat-based distros (at least) use that "-1" for > internal changes. Yeah, please don't use "-1" as version suffix, this is usually reserved for distros. -rc? would be ok though. Rainer, could you please summarise, what your current idea for the version numbering is. With all the different proposals I somehow lost track about the latest preferred one. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sat Mar 29 14:56:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 29 Mar 2008 14:56:30 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com><200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> I needed some time to digest all this myself ;) I've now done that and put it all into a HTML page. That doesn't imply it is the scheme we will finally use, but if so, it saves me some time. I have also described the development process a bit. I think this is vital for understanding why I need certain release numbers. The info is here (with the now irrelevant text at the end of the doc): http://www.rsyslog.com/doc-version_naming.html I am still a bit undecided if we really need to go v4 or can start this (or whatever else) in the v3 branch... Feedback appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Saturday, March 29, 2008 12:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/28, Johnny Tan : > > Raoul Bhatia [IPAX] wrote: > > > besides that, we could skip the "rc" string and simply use 4.0.12- > 1 - > > > like a build number which is used by some linux distributions. > > > > > > Actually, Red Hat-based distros (at least) use that "-1" for > > internal changes. > > Yeah, please don't use "-1" as version suffix, this is usually > reserved for distros. > -rc? would be ok though. > > Rainer, could you please summarise, what your current idea for the > version numbering is. > With all the different proposals I somehow lost track about the latest > preferred one. > > Cheers, > Michael > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Sat Mar 29 15:13:03 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 15:13:03 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: 2008/3/29, Rainer Gerhards : > I needed some time to digest all this myself ;) I've now done that and > put it all into a HTML page. That doesn't imply it is the scheme we will > finally use, but if so, it saves me some time. I have also described the > development process a bit. I think this is vital for understanding why I > need certain release numbers. The info is here (with the now irrelevant > text at the end of the doc): > > http://www.rsyslog.com/doc-version_naming.html > > I am still a bit undecided if we really need to go v4 or can start this > (or whatever else) in the v3 branch... > > Feedback appreciated. What I'm missing from the document, is a distinction between stable and unstable releases (or are all releases with -rc? unstable and without stable?) Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Sat Mar 29 15:16:18 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 15:16:18 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: 2008/3/29, Michael Biebl : > 2008/3/29, Rainer Gerhards : > > > I needed some time to digest all this myself ;) I've now done that and > > put it all into a HTML page. That doesn't imply it is the scheme we will > > finally use, but if so, it saves me some time. I have also described the > > development process a bit. I think this is vital for understanding why I > > need certain release numbers. The info is here (with the now irrelevant > > text at the end of the doc): > > > > http://www.rsyslog.com/doc-version_naming.html > > > > I am still a bit undecided if we really need to go v4 or can start this > > (or whatever else) in the v3 branch... > > > > Feedback appreciated. > > > What I'm missing from the document, is a distinction between stable > and unstable releases > (or are all releases with -rc? unstable and without stable?) I.e., are -mf releases to be considered stable or not? Would the current 3.12.5 be an rc release in the new versioning scheme? Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sun Mar 30 11:01:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 30 Mar 2008 11:01:22 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> [Mmmhhh. I thought I had sent a reply from the pda. Well, please don't wonder if it shows up some time later ;) So here is the full response:] Any version with a devstate designation (-mf, -rc) is unstable. Every version without it is stable (this will be far less versions as we reach stable only every now and then). So what would the current version be named? Mhh... It's not an easy one-to-one mapping. Because the process changes a bit. Today, I add new major and minor features at the same time while stabilizing old features. This is also the primary reason why we so far have very infrequently new features in the stable release: whenever I stabilize an "older" feature, I introduce new bugs while working on the new ones. That'll change a bit with the new scheme, as I then will freeze code once the focus feature has been implemented. When I start with the next focus feature, patches will be applied to both the "older" code as well as the one I am working on. As such, features become stable during the process. It obviously is some more work to do for me, as I now need to apply patches to multiple code paths (to keep this reasonable, I limited the stable releases to only the latest minor version). The plus is that we get much more feature-rich stable releases, so I think the additional development time is well spent. And for the ultra-conservative folks, there is always the old, feature-less previous version stable (v2 in this case). After having said this, the current release would probably have two names: 3.12.0-rc3 --> the stabilized module loader release 3.13.0-mf2 --> the new relp-enabled release which has not yet real relp support I am also now of the thought that we do not necessarily need to move to 4.0.0 to cover this new scheme. I'd say I can simply release a 3.13.0 stable next week (based on the current 3.12.5) and then continue to work on relp in 3.14.0-rc0 (there will probably be no -mf for that release as relp support is already quite far). Question now: how can I just declare the release to be stable even though it contains the RELP code? Answer: I am lucky ;) the relp code *inside* rsyslog is just two slim inout/output plugins. No core changes. The actual relp code comes via librelp, which is external. Thanks to this coincidence, I did not really introduce any new features since the last feature focus and so the rest of rsyslog could mature. At no other point in the v3 tree I would have been able to do this. That is yet another reason that I'd like to settle the issue by mid next week - the initial version of relp is almost done and I am eager to move on. Again, feedback appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Saturday, March 29, 2008 3:16 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/29, Michael Biebl : > > 2008/3/29, Rainer Gerhards : > > > > > I needed some time to digest all this myself ;) I've now done that > and > > > put it all into a HTML page. That doesn't imply it is the scheme > we will > > > finally use, but if so, it saves me some time. I have also > described the > > > development process a bit. I think this is vital for > understanding why I > > > need certain release numbers. The info is here (with the now > irrelevant > > > text at the end of the doc): > > > > > > http://www.rsyslog.com/doc-version_naming.html > > > > > > I am still a bit undecided if we really need to go v4 or can > start this > > > (or whatever else) in the v3 branch... > > > > > > Feedback appreciated. > > > > > > What I'm missing from the document, is a distinction between stable > > and unstable releases > > (or are all releases with -rc? unstable and without stable?) > > I.e., are -mf releases to be considered stable or not? > Would the current 3.12.5 be an rc release in the new versioning scheme? > > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 11:20:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 11:20:58 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> I'll do a new release according to the new versioning scheme (3.13.0-rc1) later today if nobody objects. It's still not finalized, but doing so makes everybody aware and is also a test for me (plus, we could move to 3.13.0 [stable] on April, 2nd). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Sunday, March 30, 2008 11:01 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > [Mmmhhh. I thought I had sent a reply from the pda. Well, please don't > wonder if it shows up some time later ;) So here is the full response:] > > Any version with a devstate designation (-mf, -rc) is unstable. Every > version without it is stable (this will be far less versions as we > reach > stable only every now and then). > > So what would the current version be named? Mhh... It's not an easy > one-to-one mapping. Because the process changes a bit. Today, I add new > major and minor features at the same time while stabilizing old > features. This is also the primary reason why we so far have very > infrequently new features in the stable release: whenever I stabilize > an > "older" feature, I introduce new bugs while working on the new ones. > > That'll change a bit with the new scheme, as I then will freeze code > once the focus feature has been implemented. When I start with the next > focus feature, patches will be applied to both the "older" code as well > as the one I am working on. As such, features become stable during the > process. It obviously is some more work to do for me, as I now need to > apply patches to multiple code paths (to keep this reasonable, I > limited > the stable releases to only the latest minor version). The plus is that > we get much more feature-rich stable releases, so I think the > additional > development time is well spent. And for the ultra-conservative folks, > there is always the old, feature-less previous version stable (v2 in > this case). > > After having said this, the current release would probably have two > names: > > 3.12.0-rc3 --> the stabilized module loader release > 3.13.0-mf2 --> the new relp-enabled release which has not yet real relp > support > > I am also now of the thought that we do not necessarily need to move to > 4.0.0 to cover this new scheme. I'd say I can simply release a 3.13.0 > stable next week (based on the current 3.12.5) and then continue to > work > on relp in 3.14.0-rc0 (there will probably be no -mf for that release > as > relp support is already quite far). > > Question now: how can I just declare the release to be stable even > though it contains the RELP code? Answer: I am lucky ;) the relp code > *inside* rsyslog is just two slim inout/output plugins. No core > changes. > The actual relp code comes via librelp, which is external. Thanks to > this coincidence, I did not really introduce any new features since the > last feature focus and so the rest of rsyslog could mature. At no other > point in the v3 tree I would have been able to do this. That is yet > another reason that I'd like to settle the issue by mid next week - the > initial version of relp is almost done and I am eager to move on. > > Again, feedback appreciated. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > > Sent: Saturday, March 29, 2008 3:16 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog version numbering > > > > 2008/3/29, Michael Biebl : > > > 2008/3/29, Rainer Gerhards : > > > > > > > I needed some time to digest all this myself ;) I've now done > that > > and > > > > put it all into a HTML page. That doesn't imply it is the > scheme > > we will > > > > finally use, but if so, it saves me some time. I have also > > described the > > > > development process a bit. I think this is vital for > > understanding why I > > > > need certain release numbers. The info is here (with the now > > irrelevant > > > > text at the end of the doc): > > > > > > > > http://www.rsyslog.com/doc-version_naming.html > > > > > > > > I am still a bit undecided if we really need to go v4 or can > > start this > > > > (or whatever else) in the v3 branch... > > > > > > > > Feedback appreciated. > > > > > > > > > What I'm missing from the document, is a distinction between stable > > > and unstable releases > > > (or are all releases with -rc? unstable and without stable?) > > > > I.e., are -mf releases to be considered stable or not? > > Would the current 3.12.5 be an rc release in the new versioning > scheme? > > > > Michael > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 12:48:00 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 12:48:00 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > I'll do a new release according to the new versioning scheme > (3.13.0-rc1) later today if nobody objects. It's still not finalized, > but doing so makes everybody aware and is also a test for me (plus, we > could move to 3.13.0 [stable] on April, 2nd). > Ok, a few comments: I thinks the distinction between mf? (minor features) and rc? (major features) is a bit artifical and counterintuitive imho. For me, rc means "release candidate", meaning that we are close to a stable release (contrary to your proposal, where rc means major new feature, possibly destabilizing a lot). Also, if I read your proposal correctly, as soon as there was an rc1 release, there can be no more mf? releases (even if the added features would be minor) I'd rather use something like -pre1, -pre2 ... (no matter if you have a major or minor feature) and if we are nearing a stable release switch to -rc Or you could do it like the kernel, and only use -rc?. Another proposal for the version scheme (slightly adopted from KDE or Gnome): As an example, your currently stable release is 3.1.x, eg. 3.1.4. Now you want to work against the next minor release, which would be 3.2 So you start with 3.1.80 (80 meaning alpha/beta quality) and increment the last digit for every new release, eg. 3.1.81 (if you reach 89, you start numbering 89.1..) When you enter RC stage, you switch to 3.1.90 and increment for each new release. The final release being 3.2. If you are working towards a next major release, you would start with 3.80 -> 3.90 -> 4.0 Cheers, Michael [1] If you run out -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 12:57:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 12:57:56 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Quick comment... Others please also comment... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 12:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > I'll do a new release according to the new versioning scheme > > (3.13.0-rc1) later today if nobody objects. It's still not > finalized, > > but doing so makes everybody aware and is also a test for me (plus, > we > > could move to 3.13.0 [stable] on April, 2nd). > > > > Ok, a few comments: > I thinks the distinction between mf? (minor features) and rc? (major > features) is a bit artifical and counterintuitive imho. > For me, rc means "release candidate", meaning that we are close to a > stable release (contrary to your proposal, where rc means major new > feature, possibly destabilizing a lot). Also, if I read your proposal > correctly, as soon as there was an rc1 release, there can be no more > mf? releases (even if the added features would be minor) > I'd rather use something like -pre1, -pre2 ... (no matter if you have > a major or minor feature) and if we are nearing a stable release > switch to -rc It would be fine with me to call all -pre1, -pre2 etc... > Or you could do it like the kernel, and only use -rc?. To me, it is quite illogical to call something a -rc, when it not even contains the feature that it shall be a candidate for ;) That was the primary driver for -mf. > > Another proposal for the version scheme (slightly adopted from KDE or > Gnome): > As an example, your currently stable release is > 3.1.x, eg. 3.1.4. > Now you want to work against the next minor release, which would be 3.2 > So you start with > 3.1.80 (80 meaning alpha/beta quality) and increment the last digit > for every new release, eg. 3.1.81 (if you reach 89, you start > numbering 89.1..) > When you enter RC stage, you switch to > 3.1.90 and increment for each new release. The final release being 3.2. Well, that is somewhat along the lines of what I currently do. Remember, I start at 3.10.x for deve builds, then (in the original scheme), I'd go to 3.0, continue with e.g. 3.12.3, which becomes 3.1. The problem were the bouncing numbers. But maybe using patchlevel for that solves the issues. Let's use an actual sample. 3.13.0 is the current stable. The next major feature is RELP, which has been completed, but is not yet stable. I am now working on plain tcp TLS. So would this be the release numbers: 3.13.5 stable 3.13.87 relp 3.14.82 relp/tcp Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.13.88 relp 3.14.83 rep/tcp Once relp is stable, we will have 3.13.6 deprecated 3.14.0 stable 3.14.83 rep/tcp unstable So actually the 3.14.0 release happens possibly much later than 3.14.80? And in that scheme we never have a development designation but the patch level contains it (being >= 80 implies being unstable, while being less implies being stable)? >From what I see, I think that would work for me. Rainer > > If you are working towards a next major release, you would start with > 3.80 -> 3.90 -> 4.0 > > Cheers, > Michael > > [1] If you run out > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 13:02:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:02:56 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Question ;) > If you are working towards a next major release, you would start with > 3.80 -> 3.90 -> 4.0 That also means once we have reached 3.80, we can/will no longer apply the new features to the 3.x stable tree, but rather only the 4.x stable tree? I am asking because I just realized we would otherwise be back to the up and down bouncing version numbers - this happens to be exactly the scheme I used so far... Rainer From mbiebl at gmail.com Mon Mar 31 13:10:07 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 13:10:07 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Question ;) > > > > If you are working towards a next major release, you would start with > > 3.80 -> 3.90 -> 4.0 > > > That also means once we have reached 3.80, we can/will no longer apply > the new features to the 3.x stable tree, but rather only the 4.x stable > tree? I am asking because I just realized we would otherwise be back to > the up and down bouncing version numbers - this happens to be exactly > the scheme I used so far... I don't see the problem here. Say you have a current 3.2.1 stable release. Then you start working for the next major release 4. After 4.0 has been released, you want to add a new feature to the old stable branch, so you release 3.3.0. That's something completely different than renaming 3.12.x to 3.1 when it becomes stable. Michaell -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 13:12:55 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 13:12:55 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Quick comment... Others please also comment... > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > > > Sent: Monday, March 31, 2008 12:48 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog version numbering > > > > > 2008/3/31, Rainer Gerhards : > > > I'll do a new release according to the new versioning scheme > > > (3.13.0-rc1) later today if nobody objects. It's still not > > finalized, > > > but doing so makes everybody aware and is also a test for me (plus, > > we > > > could move to 3.13.0 [stable] on April, 2nd). > > > > > > > Ok, a few comments: > > I thinks the distinction between mf? (minor features) and rc? (major > > features) is a bit artifical and counterintuitive imho. > > For me, rc means "release candidate", meaning that we are close to a > > stable release (contrary to your proposal, where rc means major new > > feature, possibly destabilizing a lot). Also, if I read your proposal > > correctly, as soon as there was an rc1 release, there can be no more > > mf? releases (even if the added features would be minor) > > I'd rather use something like -pre1, -pre2 ... (no matter if you have > > a major or minor feature) and if we are nearing a stable release > > switch to -rc > > > It would be fine with me to call all -pre1, -pre2 etc... > > > > Or you could do it like the kernel, and only use -rc?. > > > To me, it is quite illogical to call something a -rc, when it not even > contains the feature that it shall be a candidate for ;) That was the > primary driver for -mf. The point is, that the don't try to draw a line where they start calling something alpha, beta then rc. The simply use one, rc, and increment that. > > > > > Another proposal for the version scheme (slightly adopted from KDE or > > Gnome): > > As an example, your currently stable release is > > 3.1.x, eg. 3.1.4. > > Now you want to work against the next minor release, which would be > 3.2 > > So you start with > > 3.1.80 (80 meaning alpha/beta quality) and increment the last digit > > for every new release, eg. 3.1.81 (if you reach 89, you start > > numbering 89.1..) > > When you enter RC stage, you switch to > > 3.1.90 and increment for each new release. The final release being > 3.2. > > > Well, that is somewhat along the lines of what I currently do. Remember, > I start at 3.10.x for deve builds, then (in the original scheme), I'd go > to 3.0, continue with e.g. 3.12.3, which becomes 3.1. The problem were The problem is, that for package management systems (and for people) 3.12.x > 3.1.x > the bouncing numbers. But maybe using patchlevel for that solves the > issues. > > Let's use an actual sample. > > 3.13.0 is the current stable. The next major feature is RELP, which has > been completed, but is not yet stable. I am now working on plain tcp > TLS. So would this be the release numbers: > > 3.13.5 stable > 3.13.87 relp > 3.14.82 relp/tcp > > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.13.88 relp > 3.14.83 rep/tcp > > Once relp is stable, we will have > > 3.13.6 deprecated > 3.14.0 stable > 3.14.83 rep/tcp unstable > > So actually the 3.14.0 release happens possibly much later than 3.14.80? This only happens, if you have two unstable branches at the same time. Do you expect that to happen that often? Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From Gerrard.Geldenhuis at datacash.com Mon Mar 31 13:28:26 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Mon, 31 Mar 2008 12:28:26 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: I have not really followed the thread but thought I would give a quick opinion. My requirements would be: Easy to follow numbering. I should be able to see at a glance what the newer version is. There should not be confusing things like 3.12.3 which becomes 3.1 as Michael highligted. Other than that I am not really bothered. Stable and unstable numbering is not realy relevant in my opinion anymore. I would rather look at the website or an rss feed to ascertain which version is the latest stable of a specific product. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: 31 March 2008 12:13 > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > Quick comment... Others please also comment... > > From rgerhards at hq.adiscon.com Mon Mar 31 13:39:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:39:57 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> > > So actually the 3.14.0 release happens possibly much later than > 3.14.80? > > This only happens, if you have two unstable branches at the same time. > Do you expect that to happen that often? I'd say that's the usual case given the current pace of development. I could even envision three unstables at one time... Rainer From rgerhards at hq.adiscon.com Mon Mar 31 13:51:39 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:51:39 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 1:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > Question ;) > > > > > > > If you are working towards a next major release, you would start > with > > > 3.80 -> 3.90 -> 4.0 > > > > > > That also means once we have reached 3.80, we can/will no longer > apply > > the new features to the 3.x stable tree, but rather only the 4.x > stable > > tree? I am asking because I just realized we would otherwise be back > to > > the up and down bouncing version numbers - this happens to be > exactly > > the scheme I used so far... > > I don't see the problem here. > Say you have a current 3.2.1 stable release. > Then you start working for the next major release 4. > After 4.0 has been released, you want to add a new feature to the old > stable branch, so you release 3.3.0. > That's something completely different than renaming 3.12.x to 3.1 when > it becomes stable. I was thinking about the .80+ case. Back to the sample, this time just assuming that relp is scheduled for the next major release. 3.13.0 is the current stable. 3.13.5 stable 3.80.5 relp 3.81.2 relp/tcp Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.80.6 relp 3.81.3 rep/tcp Once relp is stable, we will have... well, what will we have? RELP is stable, but the v4 goal is not yet reached? I thought it means it'll become 3.14.0: 3.13.6 deprecated 3.14.0 stable relp 3.81.3 rep/tcp unstable But now we are back to a case wher 3.81.6 (higher version number) is the same a 3.14.0. And now let's consider: 3.13.6 deprecated 3.14.0 stable relp (based on 3.80.6) 3.81.3 relp/tcp unstable 3.82.0 relp w/ tls unstable Now tcp becomes stable: 3.13.6 deprecated 3.14.0 stable relp (based on 3.80.6) 3.15.0 relp/tcp stable (based on 3.81.3) 3.82.0 relp w/ tls unstable But now we have the situation that 3.15 is more capable than 3.80. This is what I meant. It's exactly the scheme rsyslog used so far, except that I started unstable builds at .10 and not .80. If things working towards a new release shall start with .80, we need to either hold development on the stable (just as I did so far) or we need to have more capable builds with fewer IDs. Maybe this whole stable/unstable thing is not really worth the trouble and I should instead provide the feature stability matrix instead - where for each feature is shown how mature it is. Actually, I'd think that would probably make up for better information. Comments please. Rainer From rgerhards at hq.adiscon.com Mon Mar 31 13:59:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:59:23 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAC@grfint2.intern.adiscon.com> > > > So actually the 3.14.0 release happens possibly much later than > > 3.14.80? > > > > This only happens, if you have two unstable branches at the same > time. > > Do you expect that to happen that often? > > I'd say that's the usual case given the current pace of development. I > could even envision three unstables at one time... This was maybe a bit brief. Let me add the reasoning, it may not be obvious... What makes a stable a stable? Bug fixes? Well, kind of. But actually it is *TIME*. It takes some time to find the "initial bug set". So the most bug reports come in in the 2 to 3 weeks after release. Depending on the feature complexity, it may take even longer (just think about the queue engine and its worker-thread system). So in essence, to get to a stable, I need to have the version just sit around and wait for bugs and their fixes. This will start whenever I have implemented a feature initially. So to continue, I'll immediately fork and work on the next feature set - else I would be idling for at least a couple of weeks. And, depending on how things go, I may even finish that one feature (minor release) and go to the next one before even the version before it matures (especially if my confidence in that code for some reason is reduced or the code is quite complex). During that course, it may happen that a minor feature will never become stable, but only the next one will (if that's quicker stabilized than the previous one). Rainer From r.bhatia at ipax.at Mon Mar 31 15:18:32 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 15:18:32 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> Message-ID: <47F0E4A8.50207@ipax.at> Rainer Gerhards wrote: i clearly object this numbering scheme as it gives much trouble to both ppl and package maintainers. moreover, i would see the 3.x.y branch as a branch depending on some "old" kind of version numbering and start with a fresh version scheme from 4.x.y onwards. coming back to your example, what would you think of: > 3.13.0 is the current stable. > 3.13.5 stable 3.13.5-relp5 (3.80.5 relp) 3.13.5-relp+tcp2 (3.81.2 relp/tcp) > Now let's assume I add a bugfix for the core engine. Would that bring us > to > 3.13.6 stable 3.13.6-relp6 (3.80.6 relp) 3.13.5-relp+tcp3 (3.81.3 rep/tcp) > Once relp is stable, we will have... well, what will we have? RELP is > stable, but the v4 goal is not yet reached? I thought it means it'll > become 3.14.0: 3.13.6 deprecated 3.14.0 stable relp 3.14.0-tcp3 (3.81.3 rep/tcp unstable) 3.13.6 deprecated 3.14.0 stable relp 3.14.3-tcp3 (3.81.3 relp/tcp unstable) 3.14.0-tls (3.82.0 relp w/ tls unstable) > Now tcp becomes stable: 3.13.6 deprecated 3.14.0 stable relp 3.15.0 relp/tcp stable 3.15.0-tls (3.82.0 relp w/ tls unstable) this means we're basically working with patches which can be applied on top of the current stable release. in git you can of course maintain them as branches and merge as to your liking. cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Mon Mar 31 15:36:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 15:36:31 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F0E4A8.50207@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> This has some subtleties for me, too (well, maybe its not really subtleties but simply frightening me ;)). Let me try to get this back to a simple path. How about a simple major.minor.patchlevel-dev So basically the "Sunday scheme" without -mf/-rc but instead just -dev. I guess the scheme was good for all, but the -mf/-rc was deemed confusing. In the sample, it would look like: 3.13.5 stable 3.14.0-dev6 (relp) 3.15.0-dev3 (relp/tcp) Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.14.0-dev7 (relp) 3.15.0-dev4 (relp/tcp) Once relp is stable, we have 3.13.6 deprecated 3.14.0 stable relp 3.15.0-dev4 (relp/tcp) TLS begun: 3.13.6 deprecated 3.14.0 stable relp 3.15.0-dev4 (relp/tcp) 3.16.0-dev0 (tls) Now tcp becomes stable: 3.13.6 deprecated 3.14.0 deprecated 3.15.0 stable (relp/tcp) 3.16.0-dev0 (tls) Bottom line: versions with -dev are unstable, all others are stable. Does this sound acceptable? Rainer From rgerhards at hq.adiscon.com Mon Mar 31 16:25:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 16:25:54 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com><47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB2@grfint2.intern.adiscon.com> FYI: I'll release as 3.13.0-dev0 soon to check out if that scheme brings some problems to me in regard of the systems where I enter the version number. Again, nothing is decided yet. But I think it is better to do that experiment in the v3 tree than to potentially lose a v4 version number ;) (and, as a side note, my v4 release goal is not yet reached, so I'd prefer to stay at v3 if that's possible). Please let us know if you don't like that 3.13.0-dev0 scheme so that we can move to something different before I need to release the next stable. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, March 31, 2008 3:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > This has some subtleties for me, too (well, maybe its not really > subtleties but simply frightening me ;)). Let me try to get this back > to > a simple path. How about a simple > > major.minor.patchlevel-dev > > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) > > > Now let's assume I add a bugfix for the core engine. Would that bring > us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) > > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) > > Now tcp becomes stable: > > 3.13.6 deprecated > 3.14.0 deprecated > 3.15.0 stable (relp/tcp) > 3.16.0-dev0 (tls) > > Bottom line: versions with -dev are unstable, all others are > stable. > > Does this sound acceptable? > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From pvrabec at redhat.com Mon Mar 31 16:51:12 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Mon, 31 Mar 2008 16:51:12 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <200803311651.12753.pvrabec@redhat.com> Hi, > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) > > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) There is a great disadvatage of this scheme. 3.14.0-dev7 > 3.14.0, BUT 3.14.0 is newer. From linuxweb at gmail.com Mon Mar 31 16:53:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 10:53:22 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F0FAE2.3070405@gmail.com> Rainer Gerhards wrote: > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed Since many folks are already familiar with alpha, beta, and rc, I think it'd be good to stick with those as much as possible. > confusing. In the sample, it would look like: > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) 3.13.5 stable 3.14.0-beta6 (relp) 3.15.0-alpha3 (relp/tcp) > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.0-beta7 (relp) 3.15.0-alpha4 (relp/tcp) > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) 3.13.6 deprecated 3.14.0-rc1 (?) relp becoming stable 3.15.0-alpha4 (relp/tcp) > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) 3.13.6 deprecated 3.14.0 stable relp 3.15.0-beta1 (relp/tcp) (3.15 now becomes beta) 3.16.0-alpha1 (tls) (the new alpha/dev branch) In a nutshell: alpha - very (b)leading edge dev release branch beta - dev branch that has been out for awhile but still being worked on rc - the beta/dev branch finally getting ready to be declared stable, pretty much bug-fix-only at this point stable deprecated From r.bhatia at ipax.at Mon Mar 31 16:58:02 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 16:58:02 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F0FBFA.3020802@ipax.at> Rainer Gerhards wrote: > This has some subtleties for me, too (well, maybe its not really > subtleties but simply frightening me ;)). Let me try to get this back to > a simple path. How about a simple > > major.minor.patchlevel-dev > > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) what i'm not getting is, that you a) want to maintain some work-in-progress-patches as seperate trees. what happens if you want to implement feature-x (ssl/tls) first (which would then be 3.17.0) and this becomes stable faster than relp/other stuff? then we have 3.13.5 3.14.0-devX 3.15.0-devY 3.16.0 stable uhm ... ? what happens if a patch will be unstable for ages as it was some kind of paid-for-feature but time ran out? or if a feature becomes obsolete because of another feature? b) why do you want to stick to some in-between-version - especially with the v3.x scheme which i still see as progressing as 3.14.0, 3.15.0, 3.16.0 as in the old-style so nobody is confused. therefore i call 3.14.0 3.15.0 etc. in-betwee-versions, if 3.16 is under development) mhm, now an idea begins to take hold of me ;) do you want to totally switch to the new scheme now? or do you want to keep some kind of legacy numbering for v3? maybe my problem is, that i somehow see the versioning this way: v3.pl or rather v3.subminor.pl and v4.minor.pl or rather v4.minor.subminor.pl (such as we can see with kernel v2.6.24.4) so 3.15.0 is translated to 3.0.15 in my mind. so the next step would be 3.0.16 which would become 3.16.x in the "old" release style. perhaps you can enlighten me/us? :) and maybe, if you want to switch to the final version scheme, we should jump over a couple of releases and start with 3.20.0 to make things clear? cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Mon Mar 31 17:01:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:01:35 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <200803311651.12753.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> Peter, Ah, so you compare the strings... makes sense. But also makes clear we have a problem with any such scheme... ... that means it applies to Johnn's suggestion, too :( Next try, back to odd/even. So any odd version number is unstable, any even one is stable. Once something gets stable, it is incremented to the even version. -rc and similar things may (may!) be added to alert users. Again into the sample case: 3.14.0 stable 3.15.6 unstable (relp) 3.17.2 unstable (relp/tcp) Now let's assume I add a bugfix for the core engine. Would that bring us to 3.14.1 stable 3.15.7 unstable (relp) 3.17.3 unstable (relp/tcp) Once relp is stable, we have 3.14.1 deprecated 3.16.0 stable (relp) 3.17.3 unstable (relp/tcp) TLS begins: 3.14.1 deprecated 3.16.0 stable (relp) 3.17.3 unstable (relp/tcp) 3.19.0 unstable (tls) Now tcp becomes stable: 3.14.1 deprecated 3.16.0 deprecated 3.18.0 stable (relp/tcp) 3.19.0 unstable (tls) In all cases stable > devel, at least as far as the some feature set is concerned. There may be gaps in the minor version numbers when versions are not yet stable. Does that work? Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Monday, March 31, 2008 4:51 PM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] rsyslog version numbering > > Hi, > > > 3.13.6 stable > > 3.14.0-dev7 (relp) > > 3.15.0-dev4 (relp/tcp) > > > > Once relp is stable, we have > > > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > There is a great disadvatage of this scheme. > 3.14.0-dev7 > 3.14.0, BUT 3.14.0 is newer. From friedl at hq.adiscon.com Mon Mar 31 17:08:57 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 31 Mar 2008 17:08:57 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Hi all, I have just release rsyslog 3.13.0-dev0, which will hopefully become the next stable release by mid-week. It fixes two bugs, one is a potential segfault in the syslog/plain tcp receiver. The other one is removal of some debug instrumentation that accidently made it into 3.12.5. There is also a new ./configure option (--enable/disable-rsyslogd) which permits to build just specific plugins without the main binary. This has been added to support source-based packaging systems (like in FreeBSD). This is also a test for one potential new versioning scheme. 3.13.0-dev0 is recommended for all v3 branch users. Feedback on it is highly appreciated. Changelog: http://www.rsyslog.com/Article201.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-92.phtml Florian Riedl From linuxweb at gmail.com Mon Mar 31 17:21:19 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 11:21:19 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> Message-ID: <47F1016F.6010200@gmail.com> Rainer Gerhards wrote: > Peter, > > Ah, so you compare the strings... makes sense. But also makes clear we > have a problem with any such scheme... > > ... that means it applies to Johnn's suggestion, too :( No, I made a mistake of keeping the numbering when my main point was more about, if you are going to use appendages, to stick with the well-known ones (pre, alpha, beta, rc, stable, patch). It should work with *any* numbering scheme (yours, Michael's, the even/odd, etc.). The value-added benefit is it gives people an instant knowledge of where the branch stands. My company doesn't mind using rc's in production if we absolutely need the features it provides. But using an alpha would be much harder to justify. Granted, in the end, these are somewhat all arbitrary (even declaring something stable). But I think, like you said, most devs (or the communities that surround a project) have a general sense that the passage of time and what actually gets added (bug-fix-only, etc.) to a branch help determine these labels. That's generally good enough for most people who end up using the software. johnn From rgerhards at hq.adiscon.com Mon Mar 31 17:25:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:25:57 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F1016F.6010200@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> <47F1016F.6010200@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB7@grfint2.intern.adiscon.com> I couldn't agree more. I have a maturity matrix on my mind. I guess I'll do a sample of it. Maybe that helps find a solution. The core thing is that with a truly modular design like we have in rsyslog nothing is stable or unstable per se. Some components are extremely stable, while others are bleeding edge. As long as you don't use the bleeding edge ones (and they did not touch core features) the last ironed-out alpha devel version is rock solid. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Monday, March 31, 2008 5:21 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Rainer Gerhards wrote: > > Peter, > > > > Ah, so you compare the strings... makes sense. But also makes clear > we > > have a problem with any such scheme... > > > > ... that means it applies to Johnn's suggestion, too :( > > No, I made a mistake of keeping the numbering when my main > point was more about, if you are going to use appendages, to > stick with the well-known ones (pre, alpha, beta, rc, > stable, patch). > > It should work with *any* numbering scheme (yours, > Michael's, the even/odd, etc.). > > > The value-added benefit is it gives people an instant > knowledge of where the branch stands. > > My company doesn't mind using rc's in production if we > absolutely need the features it provides. But using an alpha > would be much harder to justify. > > Granted, in the end, these are somewhat all arbitrary (even > declaring something stable). But I think, like you said, > most devs (or the communities that surround a project) have > a general sense that the passage of time and what actually > gets added (bug-fix-only, etc.) to a branch help determine > these labels. That's generally good enough for most people > who end up using the software. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 17:32:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:32:06 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F0FBFA.3020802@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F0FBFA.3020802@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB8@grfint2.intern.adiscon.com> Well... First thing is that I have urgent need to release -- there are a couple of things in the queue. If I don't release them this week, I'll probably don't have a chance to get to a stable any time soon. So I will release, even at the price that the version number scheme may be less than optimum this time. But now on to the real meat (and thanks for sticking around on this topic! ;))... > Rainer Gerhards wrote: > > This has some subtleties for me, too (well, maybe its not really > > subtleties but simply frightening me ;)). Let me try to get this back > to > > a simple path. How about a simple > > > > major.minor.patchlevel-dev > > > > So basically the "Sunday scheme" without -mf/-rc but instead just - > dev. > > I guess the scheme was good for all, but the -mf/-rc was deemed > > confusing. In the sample, it would look like: > > > > 3.13.5 stable > > 3.14.0-dev6 (relp) > > 3.15.0-dev3 (relp/tcp) > > what i'm not getting is, that you > a) want to maintain some work-in-progress-patches as seperate trees. > what happens if you want to implement feature-x (ssl/tls) first > (which would then be 3.17.0) and this becomes stable faster than > relp/other stuff? > > then we have > 3.13.5 > 3.14.0-devX > 3.15.0-devY > 3.16.0 stable > > uhm ... ? [I am answering this in the context of the version numbers above. After Peter's note on the strcmp() of version numbers, I think that we need to use something like odd/even where stable is compares always greater than unstable.] In this case, there will never be a 3.14 and 3.15 stable. This scenario may happen. In my current scheme, we'd probably never seen a 3.11 stable (queue engine) but a 3.12, because the queue engine needed much more time to mature. > what happens if a patch will be unstable for ages as it was some kind > of > paid-for-feature but time ran out? or if a feature becomes obsolete > because of another feature? Maybe the problem is that I am NOT branching off for new features. I branch off stable releases. I fear that branching of for new features will be much more time consuming at the current pace of development and has high potential for conflict. Maybe I am wrong ;) To the cases above: the paid-for-feature would never be released. That case is not fully thought out as I did not yet find someone who want's to pay. My perception may change when this happens ;) If a feature that is not yet stable is obsoleted, that version will never become stable (gap in release history). > > b) why do you want to stick to some in-between-version - especially > with > the v3.x scheme > > which i still see as progressing as 3.14.0, 3.15.0, > 3.16.0 as in the old-style so nobody is confused. therefore i call > 3.14.0 3.15.0 etc. in-betwee-versions, if 3.16 is under development) I have a major release focus for v3: full modularization. Not reached yet ;) Of course, that can be sacrificed. But as long as we experiment (and we seem to do), I prefer to do that in v3 instead of risking to lose v4, too (when we go to something else) ;) > > mhm, now an idea begins to take hold of me ;) > > do you want to totally switch to the new scheme now? or do you want to > keep some kind of legacy numbering for v3? > > maybe my problem is, that i somehow see the versioning this way: > > v3.pl or rather v3.subminor.pl and > v4.minor.pl or rather v4.minor.subminor.pl (such as we can see with > kernel v2.6.24.4) > > so 3.15.0 is translated to 3.0.15 in my mind. so the next step would be > 3.0.16 which would become 3.16.x in the "old" release style. > > perhaps you can enlighten me/us? :) Well, *if* I'd taken proper stable branchpoints, we would now probably be at 3.3 or 3.4 speaking stable-wise (3.0 intial release and input plugins, 3.1 queue (probably never delivered), 3.2 expression support, ...). So far, rsyslog did a stable release once every 12 to 24 month. From there on, everything was experimental. We can keep with that scheme (less work for me), but the drawback is that it takes an immense amount of time before those looking for stable can find new features. IMHO this hasn't worked out well. > > and maybe, if you want to switch to the final version scheme, we should > jump over a couple of releases and start with 3.20.0 to make things > clear? I agree, maybe even v4 - but then the version scheme itself must have reached stable state ;) Rainer > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Mon Mar 31 17:44:43 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 11:44:43 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F106EB.6090504@gmail.com> Rainer Gerhards wrote: > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) Ok, let me take one last swipe at this, with numbering AND labels: 3.13.5 stable 3.14.6-beta 3.15.3-alpha > > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.7-beta 3.15.4-alpha > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.8-rc 3.15.5-alpha > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) 3.13.6 deprecated 3.14.9 stable 3.15.5-beta 3.16.0-alpha > Now tcp becomes stable: > > 3.13.6 deprecated > 3.14.0 deprecated > 3.15.0 stable (relp/tcp) > 3.16.0-dev0 (tls) Well, this is kind of a big jump, but assuming it goes through all the proper alpha/beta/rc phases: 3.13.6 deprecated 3.14.9 deprecated 3.15.6 stable 3.16.1-alpha So, you increment the patchlevels, as you've been doing, but you use -alpha, -beta, -rc (with no numbers) to designate the newness/readiness of the branch. From rgerhards at hq.adiscon.com Mon Mar 31 18:06:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 18:06:34 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F106EB.6090504@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F106EB.6090504@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> Please have a look at: http://www.rsyslog.com/doc-maturity.html It's not complete yet, because it requires some work. If it is done parallel to the releases, it's not much work. I personally would find this much more useful than a single binary switch for all of the modules (which never is really right ;)). That, of course, doesn't solve the version numbering question ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Monday, March 31, 2008 5:45 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Rainer Gerhards wrote: > > confusing. In the sample, it would look like: > > > > 3.13.5 stable > > 3.14.0-dev6 (relp) > > 3.15.0-dev3 (relp/tcp) > > Ok, let me take one last swipe at this, with numbering AND > labels: > > 3.13.5 stable > 3.14.6-beta > 3.15.3-alpha > > > > > > Now let's assume I add a bugfix for the core engine. Would that bring > us > > to > > > > 3.13.6 stable > > 3.14.0-dev7 (relp) > > 3.15.0-dev4 (relp/tcp) > > 3.13.6 stable > 3.14.7-beta > 3.15.4-alpha > > > > Once relp is stable, we have > > > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > 3.13.6 stable > 3.14.8-rc > 3.15.5-alpha > > > > TLS begun: > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > 3.16.0-dev0 (tls) > > 3.13.6 deprecated > 3.14.9 stable > 3.15.5-beta > 3.16.0-alpha > > > > Now tcp becomes stable: > > > > 3.13.6 deprecated > > 3.14.0 deprecated > > 3.15.0 stable (relp/tcp) > > 3.16.0-dev0 (tls) > > Well, this is kind of a big jump, but assuming it goes > through all the proper alpha/beta/rc phases: > > 3.13.6 deprecated > 3.14.9 deprecated > 3.15.6 stable > 3.16.1-alpha > > So, you increment the patchlevels, as you've been doing, but > you use -alpha, -beta, -rc (with no numbers) to designate > the newness/readiness of the branch. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 19:15:59 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 19:15:59 +0200 Subject: [rsyslog] openssl vs rsyslog Message-ID: Hi Rainer, I noticed that you added an openssl configure check. That probably means, that you plan to add native SSL support via openssl. Please note that OpenSSL and the GPL are incompatible [1], such packages are usually rejected from Debian. You have two options: - Either add a special exemption for OpenSSL - Use another SSL implementation like GnuTLS [2] or NSS [3] Cheers, Michael [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html [2] http://www.gnu.org/software/gnutls/ [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 19:52:37 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 19:52:37 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Florian Riedl : > Hi all, > > I have just release rsyslog 3.13.0-dev0, which will hopefully become the > next stable release by mid-week. > > It fixes two bugs, one is a potential segfault in the syslog/plain tcp > receiver. The other one is removal of some debug instrumentation that > accidently made it into 3.12.5. There is also a new ./configure option > (--enable/disable-rsyslogd) which permits to build just specific plugins Hm, the default of enable_rsyslogd should imho be "yes" > without the main binary. This has been added to support source-based > packaging systems (like in FreeBSD). This is also a test for one > potential new versioning scheme. I don't see the use case yet. Why would someone want to build the plugins without the rsyslogd binary? Does it actually work if you mix different versions of plugins and rsyslogd? If someone is interested in only a specific plugin, why not use "make -C plugins/foo"? Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 20:04:18 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 20:04:18 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Michael Biebl : > 2008/3/31, Florian Riedl : > > > Hi all, > > > > I have just release rsyslog 3.13.0-dev0, which will hopefully become the > > next stable release by mid-week. > > > > It fixes two bugs, one is a potential segfault in the syslog/plain tcp > > receiver. The other one is removal of some debug instrumentation that > > accidently made it into 3.12.5. There is also a new ./configure option > > (--enable/disable-rsyslogd) which permits to build just specific plugins > > > Hm, the default of enable_rsyslogd should imho be "yes" > > > > without the main binary. This has been added to support source-based > > packaging systems (like in FreeBSD). This is also a test for one > > potential new versioning scheme. > > > I don't see the use case yet. Why would someone want to build the > plugins without the rsyslogd binary? Does it actually work if you mix > different versions of plugins and rsyslogd? > If someone is interested in only a specific plugin, why not use "make > -C plugins/foo"? Forgot to add: If you intend to build rsyslogd and rfc3195d conditionally, the corresponding man pages should also be installed conditionally. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 20:14:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 20:14:01 +0200 Subject: [rsyslog] openssl vs rsyslog Message-ID: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) Can you point me to the problem description? What would i need to exempt? Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... Any advise is deeply appreciated. Thanks, rainer ----- Urspr?ngliche Nachricht ----- Von: "Michael Biebl" An: "rsyslog-users" Gesendet: 31.03.08 19:16 Betreff: [rsyslog] openssl vs rsyslog Hi Rainer, I noticed that you added an openssl configure check. That probably means, that you plan to add native SSL support via openssl. Please note that OpenSSL and the GPL are incompatible [1], such packages are usually rejected from Debian. You have two options: - Either add a special exemption for OpenSSL - Use another SSL implementation like GnuTLS [2] or NSS [3] Cheers, Michael [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html [2] http://www.gnu.org/software/gnutls/ [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 20:34:40 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 20:34:40 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) > > Can you point me to the problem description? What would i need to exempt? Google for "openssl gpl" and you should find relevant information. E.g. [1] tries to summarize the problem. It also has a proposal for such a OpensSSL exemption. You should add this addendum to COPYING and probably also ship the original OpenSSL COPYING file (e.g. as COPYING.OpenSSL) > > Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... A prominent example of a project using GnuTLS is e.g. samba (if it's highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. A prominent example of a project using NSS is mozilla/firefox/thunderbird. I think the more widely used library is GnuTLS. I really don't have that much experience with either of these libraries, so it's hard to give a recommendation. Cheers, Michael > [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html > [2] http://www.gnu.org/software/gnutls/ > [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 20:47:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 20:47:03 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 8:35 PM > To: rsyslog-users > Subject: Re: [rsyslog] openssl vs rsyslog > > 2008/3/31, Rainer Gerhards : > > Oh, excellent feedback. And I thought that openssl is the > workhorse of os tls - wrong preception obviously ;) > > > > Can you point me to the problem description? What would i > need to exempt? > > Google for "openssl gpl" and you should find relevant information. > E.g. [1] tries to summarize the problem. > It also has a proposal for such a OpensSSL exemption. You should add > this addendum to COPYING and probably also ship the original OpenSSL > COPYING file (e.g. as COPYING.OpenSSL) Yes - sorry. I overlooked the link on the PDA. Will mow read through it. But if openssl means trouble, I'll prefer to stay away from it unless there is *very* good reason. You wake-up call was *right* in time. I planned to do some major hacking tomorrow - nothing lost so far :) [and, in case you wonder, yes, initial RELP is done ;)]. > > Is any of the alternates suitable for production use, > especially in a highly threaded environment? GNU, from the > name, sounds appealing... > > > A prominent example of a project using GnuTLS is e.g. samba (if it's > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > A prominent example of a project using NSS is > mozilla/firefox/thunderbird. > > I think the more widely used library is GnuTLS. I really don't have > that much experience with either of these libraries, so it's hard to > give a recommendation. At least it doesn't sound too bad ;) Anyone with experience please comment. Thanks again, Rainer > > Cheers, > Michael > > > > [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html > > [2] http://www.gnu.org/software/gnutls/ > > [3] http://www.mozilla.org/projects/security/pki/nss/ > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From milton at calnek.com Mon Mar 31 21:00:11 2008 From: milton at calnek.com (Milton Calnek) Date: Mon, 31 Mar 2008 13:00:11 -0600 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> Message-ID: <47F134BB.9090705@calnek.com> Don't know if this helps, or if you got the same from COPYING.Openssl. http://www.openssl.org/support/faq.html#LEGAL1 Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl >> Sent: Monday, March 31, 2008 8:35 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] openssl vs rsyslog >> >> 2008/3/31, Rainer Gerhards : >>> Oh, excellent feedback. And I thought that openssl is the >> workhorse of os tls - wrong preception obviously ;) >>> Can you point me to the problem description? What would i >> need to exempt? >> >> Google for "openssl gpl" and you should find relevant information. >> E.g. [1] tries to summarize the problem. >> It also has a proposal for such a OpensSSL exemption. You should add >> this addendum to COPYING and probably also ship the original OpenSSL >> COPYING file (e.g. as COPYING.OpenSSL) > > Yes - sorry. I overlooked the link on the PDA. Will mow read through it. > But if openssl means trouble, I'll prefer to stay away from it unless > there is *very* good reason. You wake-up call was *right* in time. I > planned to do some major hacking tomorrow - nothing lost so far :) [and, > in case you wonder, yes, initial RELP is done ;)]. > >>> Is any of the alternates suitable for production use, >> especially in a highly threaded environment? GNU, from the >> name, sounds appealing... >> >> >> A prominent example of a project using GnuTLS is e.g. samba (if it's >> highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. >> A prominent example of a project using NSS is >> mozilla/firefox/thunderbird. >> >> I think the more widely used library is GnuTLS. I really don't have >> that much experience with either of these libraries, so it's hard to >> give a recommendation. > > At least it doesn't sound too bad ;) > > Anyone with experience please comment. > > Thanks again, > Rainer >> Cheers, >> Michael >> >> >>> [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html >>> [2] http://www.gnu.org/software/gnutls/ >>> [3] http://www.mozilla.org/projects/security/pki/nss/ >> >> -- >> Why is it that all of the instruments seeking intelligent life in the >> universe are pointed away from Earth? >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From skvidal at fedoraproject.org Mon Mar 31 20:38:19 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 14:38:19 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: <1206988699.9240.38.camel@cutter> On Mon, 2008-03-31 at 20:34 +0200, Michael Biebl wrote: > 2008/3/31, Rainer Gerhards : > > Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) > > > > Can you point me to the problem description? What would i need to exempt? > > Google for "openssl gpl" and you should find relevant information. > E.g. [1] tries to summarize the problem. > It also has a proposal for such a OpensSSL exemption. You should add > this addendum to COPYING and probably also ship the original OpenSSL > COPYING file (e.g. as COPYING.OpenSSL) > > > > > Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... > > > A prominent example of a project using GnuTLS is e.g. samba (if it's > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > A prominent example of a project using NSS is mozilla/firefox/thunderbird. > > I think the more widely used library is GnuTLS. I really don't have > that much experience with either of these libraries, so it's hard to > give a recommendation. > the library that a lot of folks inside red hat and fedora are driving people to is the mozilla nss library. It handles all the bits openssl does and w/o the licensing problems. -sv From rgerhards at hq.adiscon.com Mon Mar 31 21:10:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:10:04 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206988699.9240.38.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> Hi sv, > the library that a lot of folks inside red hat and fedora are driving > people to is the mozilla nss library. It handles all the bits openssl > does and w/o the licensing problems. Sounds promising. I just found http://www.gnu.org/software/gnutls/comparison.html And that makes GNU TLS quite appealing (no wonder given the source ;)). Do you happen to have a link which tells why use nss? That would be excellent (but I am of course searching myself). Thanks, Rainer From r.bhatia at ipax.at Mon Mar 31 21:16:16 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 21:16:16 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F106EB.6090504@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> Message-ID: <47F13880.3020506@ipax.at> Rainer Gerhards wrote: > Please have a look at: > > http://www.rsyslog.com/doc-maturity.html > > It's not complete yet, because it requires some work. If it is done > parallel to the releases, it's not much work. I personally would find > this much more useful than a single binary switch for all of the modules > (which never is really right ;)). That, of course, doesn't solve the > version numbering question ;) as a short input if you want to provide a feature matrix: http://linux-vserver.org/Feature_Matrix cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From mbiebl at gmail.com Mon Mar 31 21:18:33 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 21:18:33 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206988699.9240.38.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: 2008/3/31, seth vidal : > > A prominent example of a project using GnuTLS is e.g. samba (if it's > > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > > A prominent example of a project using NSS is mozilla/firefox/thunderbird. > > > > I think the more widely used library is GnuTLS. I really don't have > > that much experience with either of these libraries, so it's hard to > > give a recommendation. > > > > > the library that a lot of folks inside red hat and fedora are driving > people to is the mozilla nss library. It handles all the bits openssl > does and w/o the licensing problems. Interesting to know. Do you know any technical advantages of NSS over GnuTLS (stability, features, nicer API, etc)? At least on Debian I had the impression that GnuTLS was the preferred solution. E.g. samba, openldap and exim4 are originally using openssl but contain a patch to use GnuTLS in Debian. One reason might be, that libgnutls is quite a bit smaller: gnutls: installed size (on i386): 920 kb [1] libnss3: installed size (on i386): 2472 kb, also requires the netscape portable runtime library libnpr (588 kb) => ~3Mb [2] Which imho is a plus for GnuTLS. [1] http://packages.debian.org/sid/libgnutls26 [2] http://packages.debian.org/sid/libnss3-1d -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From skvidal at fedoraproject.org Mon Mar 31 21:21:26 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 15:21:26 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> Message-ID: <1206991286.9240.43.camel@cutter> On Mon, 2008-03-31 at 21:10 +0200, Rainer Gerhards wrote: > Hi sv, > > > the library that a lot of folks inside red hat and fedora are driving > > people to is the mozilla nss library. It handles all the bits openssl > > does and w/o the licensing problems. > > Sounds promising. I just found > > http://www.gnu.org/software/gnutls/comparison.html > > > And that makes GNU TLS quite appealing (no wonder given the source ;)). > Do you happen to have a link which tells why use nss? That would be > excellent (but I am of course searching myself). > nss is fips certified: http://www.mozilla.org/projects/security/pki/nss/fips/ which makes it easier to use in large gov't agencies. gnutls is not certified. nss is also where the fedora/red hat crypto consolidation is going.... there is also an api-helper library for transitioning from openssl to nss. if at all possible, go with nss. -sv From skvidal at fedoraproject.org Mon Mar 31 21:22:19 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 15:22:19 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: <1206991339.9240.45.camel@cutter> On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > Interesting to know. Do you know any technical advantages of NSS over > GnuTLS (stability, features, nicer API, etc)? openssl transition library fips certification. http://www.mozilla.org/projects/security/pki/nss/fips/ the second one being the most important, I think. -sv From mbiebl at gmail.com Mon Mar 31 21:26:51 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 21:26:51 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206991339.9240.45.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> <1206991339.9240.45.camel@cutter> Message-ID: 2008/3/31, seth vidal : > On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > > Interesting to know. Do you know any technical advantages of NSS over > > GnuTLS (stability, features, nicer API, etc)? > > > openssl transition library > fips certification. > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > the second one being the most important, I think. Yeah, the openssl transition library doesn't concern us. Why is the fips certification important for rsyslog? Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 21:26:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:26:59 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206991339.9240.45.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com><1206988699.9240.38.camel@cutter> <1206991339.9240.45.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC7@grfint2.intern.adiscon.com> > > Interesting to know. Do you know any technical advantages > of NSS over > > GnuTLS (stability, features, nicer API, etc)? > > openssl transition library > fips certification. > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > the second one being the most important, I think. > > -sv FIPS is a strong selling point, I have to admit. Especially as rsyslog is heading towards the compliance space and trying to be as tamper-proof as possible. On a quick lock, GNUTLS looks a bit nicer to me (the NSPL doesn't sound really good whereas the GNUTLS zlib implementations solves something I intended to do). But FIPS... Rainer From rgerhards at hq.adiscon.com Mon Mar 31 21:29:52 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:29:52 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com><1206988699.9240.38.camel@cutter><1206991339.9240.45.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC8@grfint2.intern.adiscon.com> Whatever I do - I'll try to create a small driver shim so that hopefully different libraries could be used together with rsyslog. It depends on the APIs, but in general that should not be too hard (but practice often tells the difference). On FIPS I made I comment, where I think the mails crossed. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 9:27 PM > To: rsyslog-users > Subject: Re: [rsyslog] openssl vs rsyslog > > 2008/3/31, seth vidal : > > On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > > > Interesting to know. Do you know any technical > advantages of NSS over > > > GnuTLS (stability, features, nicer API, etc)? > > > > > > openssl transition library > > fips certification. > > > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > > > the second one being the most important, I think. > > Yeah, the openssl transition library doesn't concern us. > > Why is the fips certification important for rsyslog? > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From leitstelle at bnet.li Mon Mar 3 15:07:23 2008 From: leitstelle at bnet.li (leitstelle at bnet.li) Date: Mon, 03 Mar 2008 15:07:23 +0100 Subject: [rsyslog] Selector and MySQL Message-ID: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> Hi all, i tried to write data into a DB with a selector in rsyslog.conf: ... !dhcpd $template test, "...",sql :msg, contains, "dinfo" >127.0.0.1,db,user,pw;test #eof one of the messages, which rsyslog is like: 2008-03-03 14:14.20 dhcpd : dinfo:, myipadress and mac So from my point of view the selector should write this message to the DB. BUT it doesn`t. If i replace the selector e.g. with *.info it works (but then all message are written to the DB) Thanks for your support Andre From rgerhards at hq.adiscon.com Mon Mar 3 15:17:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 3 Mar 2008 15:17:21 +0100 Subject: [rsyslog] Selector and MySQL In-Reply-To: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> References: <20080303150723.5h0cdcxbeygwcwoo@webmail.bnet.li> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AC0@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of leitstelle at bnet.li > Sent: Monday, March 03, 2008 3:07 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Selector and MySQL > > Hi all, > > i tried to write data into a DB with a selector > > in rsyslog.conf: > ... > !dhcpd > $template test, "...",sql > :msg, contains, "dinfo" >127.0.0.1,db,user,pw;test > #eof > > one of the messages, which rsyslog is like: > 2008-03-03 14:14.20 dhcpd : dinfo:, myipadress and mac Umm... That message format looks strange. I guess the parser gets confused by it. Can you please run rsyslogd in interactively in debug mode (-d -n) and post the resulting log while such a message is received. Thanks, Rainer > > So from my point of view the selector should write this message to the > DB. > BUT it doesn`t. > > If i replace the selector e.g. with *.info it works (but then all > message are written to the DB) > > Thanks for your support > Andre > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 3 16:27:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 3 Mar 2008 16:27:30 +0100 Subject: [rsyslog] Solaris Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> Hi all, is there anybody on this list with (open) Solaris? I installed a system and tried to compile rsyslog on it, but I can't get the environment ready :( I somebody is in the know, advise would deeply be appreciated (if not, that's a good indication I still do not need to care about Solaris, which also solves my issue ;)). Thanks, Rainer From Mark.Capelle at pcmc.com Tue Mar 4 15:43:19 2008 From: Mark.Capelle at pcmc.com (Capelle, Mark (PCMC-GB)) Date: Tue, 4 Mar 2008 08:43:19 -0600 Subject: [rsyslog] Recording IP address of sender in logs Message-ID: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> I have been using syslogd 1.4.1 for quite some time to gather syslogs from multiple hosts which I then have a script which separates out the logs based on the source host. I just rebuilt my syslog server and took the opportunity to use rsyslogd 2.0.2 instead of the syslogd, but I noticed that my syslogs no longer record the address of the source host which will break my scripts. Is there any way to change this behavior? Examples - Syslogd: Mar 4 08:05:47 10.10.1.1 Mar 04 2008 08:05:47: %ASA-4-106023: Deny tcp src inside:x.x.x.x/3713 dst outside:x.x.x.x/1021 by access-group "inside_access_out" [0x0, 0x0] ^^^^^^^^^ Rsyslogd: Mar 4 08:41:17 Mar 04 2008 08:41:25: %ASA-4-106023: Deny tcp src inside:x.x.x.x/2125 dst outside:x.x.x.x/9018 by access-group "inside_access_out" [0x0, 0x0] Regards, Mark From rgerhards at hq.adiscon.com Tue Mar 4 16:58:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 4 Mar 2008 16:58:23 +0100 Subject: [rsyslog] Recording IP address of sender in logs In-Reply-To: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> References: <12ED24B2537575458993F162D4CB3F6C43EA1D@PcmcMxsUsr01.BWCINC.ORG> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308AD4@grfint2.intern.adiscon.com> Mark, it looks like message format received is somewhat invalid. Rsyslog tries to parse according to rfc 3164 but also applies legacy legacy parsing if it finds a way to do so. If you could provide me a wireshark capture of the message, I could see if I can modify the parser. HOWEVER, that shouldn't limit to do you what you want. Nothing is fixed in rsyslog and you can modify the format via a template. It is just the default template that gives you the result below. Please see the rsyslog doc on how to define other template. If you have a hard time, let me know and I'll figure it out for you, but I am currently very busy with new work, so I can't promise on the timeline. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Capelle, Mark (PCMC-GB) > Sent: Tuesday, March 04, 2008 3:43 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Recording IP address of sender in logs > > I have been using syslogd 1.4.1 for quite some time to gather syslogs > from multiple hosts which I then have a script which separates out the > logs based on the source host. I just rebuilt my syslog server and > took > the opportunity to use rsyslogd 2.0.2 instead of the syslogd, but I > noticed that my syslogs no longer record the address of the source host > which will break my scripts. Is there any way to change this behavior? > > > > Examples - > > > > Syslogd: > > Mar 4 08:05:47 10.10.1.1 Mar 04 2008 08:05:47: > %ASA-4-106023: Deny tcp src inside:x.x.x.x/3713 dst > outside:x.x.x.x/1021 > by access-group "inside_access_out" [0x0, 0x0] > > ^^^^^^^^^ > > > > Rsyslogd: > > Mar 4 08:41:17 Mar 04 2008 08:41:25: %ASA-4-106023: Deny > tcp src inside:x.x.x.x/2125 dst outside:x.x.x.x/9018 by access-group > "inside_access_out" [0x0, 0x0] > > > > > > Regards, > > Mark > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Thu Mar 6 01:49:46 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 05 Mar 2008 19:49:46 -0500 Subject: [rsyslog] local/remote logging Message-ID: <47CF3FAA.6050302@gmail.com> I want to have logs written locally (the way they usually are) as well as sent to a central logger. The two situations that I want local logs for are: 1) in case the central logger is not available for any reason, I don't lose messages 2) if I need to troubleshoot something, I can just look at the local logs Obviously, I will rotate the local logs and have older ones discarded. Whereas the central logger will archive them for longer periods. Can I accomplish this with rsyslog? AIUI, I can, and I have to move to at least v3.11.x -- is that correct? Assuming yes, I already got the remote logging working. What do I do to get the simultaneous local logging to work too? I can't find any wiki or faq on this. If I get it working, I'd be happy to post it on the site. johnn From mbiebl at gmail.com Thu Mar 6 02:24:12 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Thu, 6 Mar 2008 02:24:12 +0100 Subject: [rsyslog] Solaris In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308AC4@grfint2.intern.adiscon.com> Message-ID: 2008/3/3, Rainer Gerhards : > Hi all, > > is there anybody on this list with (open) Solaris? I installed a system > and tried to compile rsyslog on it, but I can't get the environment > ready :( > > I somebody is in the know, advise would deeply be appreciated (if not, > that's a good indication I still do not need to care about Solaris, > which also solves my issue ;)). http://blogs.sun.com/AngeloJ/entry/virtualbox_and_opensolaris_indiana ? -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Wed Mar 5 19:16:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 05 Mar 2008 19:16:18 +0100 Subject: [rsyslog] local/remote logging In-Reply-To: <47CF3FAA.6050302@gmail.com> References: <47CF3FAA.6050302@gmail.com> Message-ID: <1204740978.567.17.camel@localhost.localdomain> Hi Johnn, On Wed, 2008-03-05 at 19:49 -0500, Johnny Tan wrote: > I want to have logs written locally (the way they usually > are) as well as sent to a central logger. > > The two situations that I want local logs for are: > > 1) in case the central logger is not available for any > reason, I don't lose messages You don't need to lose message when the remote host is unavailable. Use queued mode. This is currently documented for the database, but works for any other action as well: http://www.rsyslog.com/doc-rsyslog_high_database_rate.html > > 2) if I need to troubleshoot something, I can just look at > the local logs > > Obviously, I will rotate the local logs and have older ones > discarded. Whereas the central logger will archive them for > longer periods. > > > Can I accomplish this with rsyslog? AIUI, I can, and I have > to move to at least v3.11.x -- is that correct? Indeed. But don't use anything earlier than 3.12.0 - if you do, you'll run into a couple of bugs. > Assuming yes, I already got the remote logging working. What > do I do to get the simultaneous local logging to work too? Rsyslog carries out whatever you configure. So just add a rule for local logging, e.g. *.* -/var/log/local-logfile That's it ;) > I can't find any wiki or faq on this. If I get it working, > I'd be happy to post it on the site. That would be much appreciated! HTH Rainer From rgerhards at hq.adiscon.com Thu Mar 6 12:24:16 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 12:24:16 +0100 Subject: [rsyslog] rsyslog on HP UX Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B02@grfint2.intern.adiscon.com> Hi all, thanks to HP's testdrive, I got rsyslog to compile and basically run on HP UX. My testing abilities on the testdrive are limited (non-root access), but I was successfully able to receive a UDP syslog message and store it to a file. So the engine and the plugins seem to run quite OK. If anybody has HP UX and is interested in trying rsyslog out, please let me know. Obviously, you'll receive great support if you try ;) The HP UX "support" will be part of 3.12.1, which I hope to release soon (today or tomorrow). During the compile, there is a set of warnings that I do not fully understand. If somebody can shed some light on them, I'd deeply appreciate that: net.c: In function `AddAllowedSender': net.c:203: warning: cast increases required alignment of target type net.c:212: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:280: warning: cast increases required alignment of target type net.c:290: warning: cast increases required alignment of target type net.c:294: warning: cast increases required alignment of target type net.c:295: warning: cast increases required alignment of target type net.c:296: warning: cast increases required alignment of target type net.c: In function `MaskCmp': net.c:491: warning: cast increases required alignment of target type net.c:492: warning: cast increases required alignment of target type net.c:502: warning: cast increases required alignment of target type net.c:503: warning: cast increases required alignment of target type net.c:512: warning: cast increases required alignment of target type net.c:513: warning: cast increases required alignment of target type net.c:513: warning: cast increases required alignment of target type net.c:516: warning: cast increases required alignment of target type net.c:517: warning: cast increases required alignment of target type net.c: In function `debugListenInfo': net.c:744: warning: cast increases required alignment of target type net.c:749: warning: cast increases required alignment of target type No suffix list. They seem to be related to casting IPv4/IPv6 structures. Feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Thu Mar 6 16:39:10 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 16:39:10 +0100 Subject: [rsyslog] rsyslog 3.12.1 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B0E@grfint2.intern.adiscon.com> Hi all, Rsyslog 3.12.1 is available immediately. There has been a log going on under the hood, code has been cleaned up and become more modular (you'll see more of that in the upcoming releases) and a library plugin-interface has been added (which so far is not really visible to the end user). We also worked a bit on portability and rsyslogd now compiles under HP UX. We are looking for testers, so if you have HP UX, you are invited to try it out. Feature-wise, a new syslog mib as been added, runtime debugging support enhanced and the ability to discard timestamps received via unix sockets has now been added (and is the default). This is necessary to keep time information consistent even if local programs log with wrong time information (what they do in practice). The gssapi code has greatly been changed and uses the new dynamic plugin library system. Consequently, some trouble in this regard cannot be ruled out ;). Finally, there is an important bugfix: failed actions were not properly retried, causing message loss in a destination-failure scenario. This is now corrected. Download: http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml Change Log: http://www.rsyslog.com/Article187.phtml As always, feedback is appreciated. Rainer Gerhards From stephen.carville at gmail.com Thu Mar 6 18:54:06 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Thu, 6 Mar 2008 09:54:06 -0800 Subject: [rsyslog] Log watch software Message-ID: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> I have a cenltralized repository usng rsyslogm and syslog to mirror /var/log/messages, /var/log/secure ,and information messages from cfengine. In the near future I hope to get auditd reporting to a central server. My immedate taks is to add some log analysis software on teh central server. I've started modifiying LogWatch to work with MySQL -- thats pretty straightforward -- but I'm curious what other solutions there may be out there. FOSS is preferred but a I'm not against a reasonably priced commercial product. So far everything Google has returned are commercial products for Windows sytems. -- Stephen Carville From rgerhards at hq.adiscon.com Thu Mar 6 18:55:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 18:55:51 +0100 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> I am not so involved with logwatch. Let me ask feature-wise: what capabilities do you need to do the job? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: Thursday, March 06, 2008 6:54 PM > To: rsyslog-users > Subject: [rsyslog] Log watch software > > I have a cenltralized repository usng rsyslogm and syslog to mirror > /var/log/messages, /var/log/secure ,and information messages from > cfengine. In the near future I hope to get auditd reporting to a > central server. My immedate taks is to add some log analysis software > on teh central server. I've started modifiying LogWatch to work with > MySQL -- thats pretty straightforward -- but I'm curious what other > solutions there may be out there. FOSS is preferred but a I'm not > against a reasonably priced commercial product. So far everything > Google has returned are commercial products for Windows sytems. > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From Gerrard.Geldenhuis at datacash.com Thu Mar 6 18:57:35 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Thu, 6 Mar 2008 17:57:35 -0000 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> Message-ID: Hi Stephen, Did you mean to ask a question... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > Sent: 06 March 2008 17:54 > To: rsyslog-users > Subject: [rsyslog] Log watch software > > I have a cenltralized repository usng rsyslogm and syslog to mirror > /var/log/messages, /var/log/secure ,and information messages from > cfengine. In the near future I hope to get auditd reporting to a > central server. My immedate taks is to add some log analysis software > on teh central server. I've started modifiying LogWatch to work with > MySQL -- thats pretty straightforward -- but I'm curious what other > solutions there may be out there. FOSS is preferred but a I'm not > against a reasonably priced commercial product. So far everything > Google has returned are commercial products for Windows sytems. > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From stephen.carville at gmail.com Thu Mar 6 20:44:11 2008 From: stephen.carville at gmail.com (Stephen Carville) Date: Thu, 6 Mar 2008 11:44:11 -0800 Subject: [rsyslog] Log watch software In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> Message-ID: <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> On Thu, Mar 6, 2008 at 9:55 AM, Rainer Gerhards wrote: > I am not so involved with logwatch. Let me ask feature-wise: what > capabilities do you need to do the job? About 99% of what's in messages or secure is trivia. JoeBob logged in, ran a sudo command and logged off. An authenticated mount request was received from ip.add.re.ss. That sort of thing. What I'm looking for is a parser that can pick out the (hopefully) rare messages that indicates a problem like a disk drive is reporting errors. I can modify big brother and logwatch to do this but I am curious if anyone has a favorite package I haven't heard of yet. > Rainer > > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > > Sent: Thursday, March 06, 2008 6:54 PM > > To: rsyslog-users > > Subject: [rsyslog] Log watch software > > > > I have a cenltralized repository usng rsyslogm and syslog to mirror > > /var/log/messages, /var/log/secure ,and information messages from > > cfengine. In the near future I hope to get auditd reporting to a > > central server. My immedate taks is to add some log analysis software > > on teh central server. I've started modifiying LogWatch to work with > > MySQL -- thats pretty straightforward -- but I'm curious what other > > solutions there may be out there. FOSS is preferred but a I'm not > > against a reasonably priced commercial product. So far everything > > Google has returned are commercial products for Windows sytems. > > > > -- > > Stephen Carville > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Stephen Carville From rgerhards at hq.adiscon.com Thu Mar 6 21:27:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 21:27:13 +0100 Subject: [rsyslog] Log watch software In-Reply-To: <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com> <2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> Hi, I am seeing where you come from. That's the million-dollar question ;) I suggest you also post to the loganalysis list, that's probably a better place than over here: http://www.loganalysis.org/mailman/listinfo/loganalysis Let me hijack this thread to share an idea. Rsyslog has a lot of infrastructure in place. Once I am finished with the essentials (which will of course be in a few month...), I'd like to put that infrastructure to better use than just drive the simple outputs we currently have. One thing I have on my mind is an output plugin which stores (hashes) of all message within a timeframe (e.g. last 7 days). Then, when a new message comes in, it compares it to all previous messages and emits a special message itself if the message occured less than "n" times in the past. I think this goes into the direction of what you are looking for. But would it generally be considered to be a useful idea? Even though we are months away from an implementation, feedack would be very valuable to me as it helps me shape my mid- to long-term direction. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Stephen Carville > Sent: Thursday, March 06, 2008 8:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] Log watch software > > On Thu, Mar 6, 2008 at 9:55 AM, Rainer Gerhards > wrote: > > I am not so involved with logwatch. Let me ask feature-wise: what > > capabilities do you need to do the job? > > About 99% of what's in messages or secure is trivia. JoeBob logged > in, ran a sudo command and logged off. An authenticated mount request > was received from ip.add.re.ss. That sort of thing. What I'm looking > for is a parser that can pick out the (hopefully) rare messages that > indicates a problem like a disk drive is reporting errors. > > I can modify big brother and logwatch to do this but I am curious if > anyone has a favorite package I haven't heard of yet. > > > Rainer > > > > > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville > > > Sent: Thursday, March 06, 2008 6:54 PM > > > To: rsyslog-users > > > Subject: [rsyslog] Log watch software > > > > > > I have a cenltralized repository usng rsyslogm and > syslog to mirror > > > /var/log/messages, /var/log/secure ,and information messages from > > > cfengine. In the near future I hope to get auditd reporting to a > > > central server. My immedate taks is to add some log > analysis software > > > on teh central server. I've started modifiying LogWatch > to work with > > > MySQL -- thats pretty straightforward -- but I'm curious > what other > > > solutions there may be out there. FOSS is preferred but a I'm not > > > against a reasonably priced commercial product. So far > everything > > > Google has returned are commercial products for Windows sytems. > > > > > > -- > > > Stephen Carville > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > -- > Stephen Carville > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Thu Mar 6 21:47:47 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 06 Mar 2008 15:47:47 -0500 Subject: [rsyslog] local/remote logging In-Reply-To: <1204740978.567.17.camel@localhost.localdomain> References: <47CF3FAA.6050302@gmail.com> <1204740978.567.17.camel@localhost.localdomain> Message-ID: <47D05873.60300@gmail.com> Rainer Gerhards wrote: > Hi Johnn, > > On Wed, 2008-03-05 at 19:49 -0500, Johnny Tan wrote: >> I want to have logs written locally (the way they usually >> are) as well as sent to a central logger. >> >> The two situations that I want local logs for are: >> >> 1) in case the central logger is not available for any >> reason, I don't lose messages > > You don't need to lose message when the remote host is unavailable. Use > queued mode. This is currently documented for the database, but works > for any other action as well: Hmm... and this is doable even with 2.x, right? So I could have it queued in memory buffer long enough to bring it back up or bring up another one with same ip to accept the messages? >> Assuming yes, I already got the remote logging working. What >> do I do to get the simultaneous local logging to work too? > > Rsyslog carries out whatever you configure. So just add a rule for local > logging, e.g. > > *.* -/var/log/local-logfile Awesome! Thanks. I'll post on the wiki once I get it working. johnn From rgerhards at hq.adiscon.com Thu Mar 6 21:53:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 6 Mar 2008 21:53:59 +0100 Subject: [rsyslog] local/remote logging In-Reply-To: <47D05873.60300@gmail.com> References: <47CF3FAA.6050302@gmail.com><1204740978.567.17.camel@localhost.localdomain> <47D05873.60300@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B18@grfint2.intern.adiscon.com> > >> 1) in case the central logger is not available for any > >> reason, I don't lose messages > > > > You don't need to lose message when the remote host is > unavailable. Use > > queued mode. This is currently documented for the database, > but works > > for any other action as well: > > Hmm... and this is doable even with 2.x, right? Sorry, Dude, no such queues in v2 :-( > So I could have it queued in memory buffer long enough to > bring it back up or bring up another one with same ip to > accept the messages? It's *not* a main memory buffer. If you configure it to use disk *and* you have enough disk space, it can queue for ages. It's pretty advanced, the full doc on queues is here: http://www.rsyslog.com/doc-queues.html The disk, of course, is only touched if needed. If the destination server is only offline for a brief period, messages may be buffered in memory only. But if the volume is too large or you stop rsyslogd for a while, it is written disk. So if you have enough disk space and your destination dies on Friday and you are too lazy to come in over the weekend, rsyslogd will send all queued elements after the restart on Monday. The same, of course, works if you are on vacation ;) Rainer From Gerrard.Geldenhuis at datacash.com Fri Mar 7 10:34:17 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Fri, 7 Mar 2008 09:34:17 -0000 Subject: [rsyslog] Log watch software In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> References: <2428c0380803060954h14664406u2cdd28892fa13075@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B15@grfint2.intern.adiscon.com><2428c0380803061144p784f3b93x44382fc4534b040a@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B16@grfint2.intern.adiscon.com> Message-ID: > Let me hijack this thread to share an idea. Rsyslog has a lot of > infrastructure in place. Once I am finished with the essentials (which > will of course be in a few month...), I'd like to put that > infrastructure to better use than just drive the simple outputs we > currently have. One thing I have on my mind is an output plugin which > stores (hashes) of all message within a timeframe (e.g. last 7 days). > Then, when a new message comes in, it compares it to all previous > messages and emits a special message itself if the message occured less > than "n" times in the past. I think this goes into the direction of what > you are looking for. > > But would it generally be considered to be a useful idea? Even though we > are months away from an implementation, feedack would be very valuable > to me as it helps me shape my mid- to long-term direction. > > Rainer Just thinking out loud... it would be very cool if one could build in some AI in such a plugin. You could then spend time "training" the plugin and buying "trained" AI's to regonize certain patterns in the logs etc. Regards From linuxweb at gmail.com Sat Mar 8 00:22:11 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 07 Mar 2008 18:22:11 -0500 Subject: [rsyslog] wiki with various application logging examples Message-ID: <47D1CE23.2010005@gmail.com> Just wondering if there is a wiki with examples of how to log to syslog/rsyslog for various apps. For my needs, I was specifically looking for apache, mysql, tomcat, puppet, etc. For apache, I already figured out error_log can just point to syslog, but access_log, you do: CustomLog "| /usr/bin/logger -p local0.info" combined Do I do something similar for any application where I can specify the logfile destination? johnn From rgerhards at hq.adiscon.com Sat Mar 8 13:54:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 8 Mar 2008 13:54:19 +0100 Subject: [rsyslog] wiki with various application logging examples Message-ID: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> I think nobody has added descriptions so far, but maybe that changes now ;) On the file logs - I suggest you look into imfule, that sound much cleaner and reliable to me than the logger trick. rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 08.03.08 00:22 Betreff: [rsyslog] wiki with various application logging examples Just wondering if there is a wiki with examples of how to log to syslog/rsyslog for various apps. For my needs, I was specifically looking for apache, mysql, tomcat, puppet, etc. For apache, I already figured out error_log can just point to syslog, but access_log, you do: CustomLog "| /usr/bin/logger -p local0.info" combined Do I do something similar for any application where I can specify the logfile destination? johnn _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 8 17:04:18 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 8 Mar 2008 17:04:18 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Just to facilitate things: this is what I am thinking about: http://www.rsyslog.com/doc-imfile.html It can monitor all application log files natively. Did I understand your need right? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Rainer Gerhards > Sent: Saturday, March 08, 2008 1:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > I think nobody has added descriptions so far, but maybe that > changes now ;) > > On the file logs - I suggest you look into imfule, that sound > much cleaner and reliable to me than the logger trick. > > rainer > > ----- Urspr?ngliche Nachricht ----- > Von: "Johnny Tan" > An: "rsyslog-users" > Gesendet: 08.03.08 00:22 > Betreff: [rsyslog] wiki with various application logging examples > > Just wondering if there is a wiki with examples of how to > log to syslog/rsyslog for various apps. > > For my needs, I was specifically looking for apache, mysql, > tomcat, puppet, etc. > > For apache, I already figured out error_log can just point > to syslog, but access_log, you do: > CustomLog "| /usr/bin/logger -p local0.info" combined > > Do I do something similar for any application where I can > specify the logfile destination? > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Sun Mar 9 01:14:47 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Sat, 08 Mar 2008 19:14:47 -0500 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: <47D32BF7.5030108@gmail.com> Rainer Gerhards wrote: > Just to facilitate things: this is what I am thinking > about: > > http://www.rsyslog.com/doc-imfile.html > > It can monitor all application log files natively. Did I > understand your need right? Yes, that's exactly what I was looking for. Thanks, really really awesome! One quick error -- in your Sample, you use: $InputFilePollingInterval 10 It should be $InputFilePollInterval (no -ing on Poll) Thanks again! johnn From janfrode at tanso.net Sun Mar 9 12:56:15 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Mar 2008 12:56:15 +0100 Subject: [rsyslog] wiki with various application logging examples References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: On 2008-03-08, Rainer Gerhards wrote: > Just to facilitate things: this is what I am thinking about: > > http://www.rsyslog.com/doc-imfile.html > > It can monitor all application log files natively. Did I understand your need right? > Seems like "imfile" could be trivially implemented trough: while sleep $InputFilePollInterval ; do logtail $InputFileName $InputFileStateFile | \ logger -t $InputFileTag -p $InputFileFacility.$InputFileSeverity done http://debian.stro.at/bzr-test/logtail/ Or for realtime monitoring: tail --follow=name --retry $InputFileName | \ logger -t $InputFileTag -p $InputFileFacility.$InputFileSeverity IMHO rsyslogd might be trying to do too much internally. When will it start reading mail ? That might not sound too far fetched.. :-) "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." -- jwz -jf From mbiebl at gmail.com Sun Mar 9 15:02:04 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sun, 9 Mar 2008 15:02:04 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: 2008/3/9, Jan-Frode Myklebust : > IMHO rsyslogd might be trying to do too much internally. When will it start > reading mail ? That might not sound too far fetched.. :-) > > "Every program attempts to expand until it can read mail. Those programs > which cannot so expand are replaced by ones which can." -- jwz To some extent, I share your concerns. I very much believe in the Unix philosophy of small tools, which do one job, and do that right and can be chained/combined easily. rsyslog should keeps it's focus and stay what it's primary focus is, a syslog daemon. Taking your email example: Having the functionality to send out email notifications on certain events is certainly useful. But we should not build the email sending functionality into rsyslog itself, but simply hand out (pipe) the data to external tools (mail/sendmail). The point is, what we can expect to be available on a given system or not ("logtail" probably isn't, "mail" very likely is). Also, as it's implemented as a loadable module, it doesn't clutter the main rsyslog code and so keeps the code base maintainable. But again ,I think your concern is valid and we should evaluate carefully what to implement directly into rsyslog and where it makes more sense to utilise external tools. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sun Mar 9 18:38:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 9 Mar 2008 18:38:00 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Hi all, and especially JF and Michael, Good discussion, please keep the thoughts flowing :) Let me start by tell you I am guilty of a much broader view of what a syslogd can do than most people probably have. To me, a syslogd processes *system events* and that's much more than just those pieces sent via syslog. I know that I should probably stopped the rsyslog project at some point in time and fork an eventlogd from it. But, why would this have made any sense? To me, the core point is maintainability of code and the freedom for people to use only (and exactly) what they need (and want!) to use. After all, there are also lots of folks how say that database support has nothing to do with a syslogd and thus should not be part of if (and that was the birthplace of rsyslog... ;)). So for me the natural thing was to make rsyslog pluggable AND keep only in the actual core what is ultimately needed. I got very serious about this last summer. There are a couple of posts from last August in my blog and I'd like to highlight this one: http://rgerhards.blogspot.com/2007/08/on-importance-of-plug-ins-for-rsys log.html We have now reached a point where we have a quite well implemented plugin interface and rsyslogd has become much more modular. Still, there are things that people need to run even if they never need them (like the syslog-protocol message parser and a couple of "standard" outputs that come with sysklogd). I am continuously working to remove these dependencies. For example, on Friday I moved regex support out to a separate module. So you no longer need to load regexpes if you do not need them. Now to the plugins. What is a plugin? Good question. A plugin is *a project in itself*. Just for convenience, it is part of the main tarball (we had a couple of real headaches when we tried to separate the initial plugins and came to the conclusion that it is by far more convenient to deliver them as part of the main tarball). Plugins need not be written by the rsyslog team. Maybe somebody has already written an email reception plugin and just not told us - may very well be (though I doubt ;)). Now let me take on the imfile example. The key point - at least IMHO - is that there is no single line in rsyslog core's code that has been added in support of imfile. And, more importantly, if imfile would go away, not a single line of code could be removed. So the imfile plugin (project) does neither add complexity nor code nor other overhead to rsyslog core. So can it be evil? Imfile was added in response of a user request. When I heared it, I thought it would be useful to have such a plugin. Did I know about the logger approach? Yes - and I have read some many folks complain over this or that aspect of a logger-approach that I knew I didn't want it. Maybe these were all false claims, maybe people simply did not understand to do it right. I don't know and I honestly admit that I don't care. The presence of many people feeling bugged about that approach made me not like it (if you look for actual samples, I suggest going through the loganalysis mailing list archive). Did I know about logtail? No. Had I written imfile if I did? Not sure: from the small shell script you provided, it looks like there is a problem if a) script is in sleep period b) data is appended to text file c) text file is rotated d) new lines are written to text file e) script awake for new polling loop On a quick look, it looks like the data written in b) will never make it to the syslogd. Imfile handles that. But even if logtail (or the script) handles it - as I said, I did not know about it. So I took roughly a day of my time to create an initial imfile. If logtail handles the situation described above, would be available on the platforms I was asked for and I knew about it, I'd probably refrained from implementing imfile. But, again, no harm happened. If you don't like imfile, simply don't use it. It adds nothing to the rsyslog core. You can still use the logtail approach. IMHO this is what we want: freedom of choice. And rsyslog provides this freedom. On the mail output case (though I need to be a bit brief as dinner is approaching ;)): I actually intend to add an email output plugin. It's on the list for long a time. I personally see some benefit in the ability to talk to a tightly coupled, in-memory, very performant accessible way to send mail message (just like with the SNMP traps). On the other hand, you can see that I didn't implement it for a long time - simply because I did not consider it important enough given what else needs to be done. But I now need to speak up, because I have actually begun to seriously look into implementing it before this discussion begun ;) Before I do, I will make sure I understand that other options and if there is some benefit in doing so. But if I implement it for one reason or the other, nothing bad happens to rsyslog: if you don't like it, you don't need to use it. It doesn't add any overhead to the core. And now please feedback. Am I overlooking something real evil? What is the bad thing in offering exotic functionality that only few will ever need? Thanks again for the discussion, I am *really* interested in getting more feedback on this topic. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Sunday, March 09, 2008 3:02 PM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > 2008/3/9, Jan-Frode Myklebust : > > IMHO rsyslogd might be trying to do too much internally. When will > it start > > reading mail ? That might not sound too far fetched.. :-) > > > > "Every program attempts to expand until it can read mail. > Those programs > > which cannot so expand are replaced by ones which can." -- > jwz > > To some extent, I share your concerns. > I very much believe in the Unix philosophy of small tools, which do > one job, and do that right and can be chained/combined easily. > > rsyslog should keeps it's focus and stay what it's primary focus is, a > syslog daemon. > Taking your email example: > Having the functionality to send out email notifications on certain > events is certainly useful. But we should not build the email sending > functionality into rsyslog itself, but simply hand out (pipe) the data > to external tools (mail/sendmail). > > The point is, what we can expect to be available on a given system or > not ("logtail" probably isn't, "mail" very likely is). Also, as it's > implemented as a loadable module, it doesn't clutter the main rsyslog > code and so keeps the code base maintainable. > > But again ,I think your concern is valid and we should evaluate > carefully what to implement directly into rsyslog and where it makes > more sense to utilise external tools. > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Sun Mar 9 23:05:10 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Sun, 9 Mar 2008 23:05:10 +0100 Subject: [rsyslog] wiki with various application logging examples References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Message-ID: On 2008-03-09, Rainer Gerhards wrote: > > Now let me take on the imfile example. The key point - at least IMHO - > is that there is no single line in rsyslog core's code that has been > added in support of imfile. And, more importantly, if imfile would go > away, not a single line of code could be removed. So the imfile plugin > (project) does neither add complexity nor code nor other overhead to > rsyslog core. So can it be evil? It's evil that you're forcing me to upgrade to rsyslog v3.x to take advantage of it ;-) That's what triggered my previous post.. I want to be tracking non-syslog logfiles, and if imfile/plugins where more in the unix philosophy of small tools that chain easily.. I might have been able to pick it from v3.x and use it on stable systems. > from the small shell script you provided, it looks like there is a > problem if > > a) script is in sleep period > b) data is appended to text file > c) text file is rotated > d) new lines are written to text file > e) script awake for new polling loop > > On a quick look, it looks like the data written in b) will never make it > to the syslogd. Imfile handles that. Yes, you're right. Cool that imfile handles it. > On the mail output case (though I need to be a bit brief as dinner is > approaching ;)): I actually intend to add an email output plugin. The quote was about being able to *read* mail: "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." and I was thinking it not too far fetched since "splunk" can do it (it can download email messages via IMAP, index them and create alerts on suspicious content). It might seem like you want to take rsyslog in that direction, i.e. your complete eventlogd&alertSystem fork of rsyslogd that can read any input (syslog, other-logfile, email, snmptraps) and analyze and alert on the data. That's not what *I* want from a syslog server. I just want it to reliably collect and store the logs in an organized manner. Then I'll use other tools to read and analyze them. -jf From rgerhards at hq.adiscon.com Mon Mar 10 08:52:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Mar 2008 08:52:46 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B39@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B3B@grfint2.intern.adiscon.com> Hi JF, thanks for the note. Of course, it triggers a couple of responses (see below). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Sunday, March 09, 2008 11:05 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] wiki with various application logging examples > > On 2008-03-09, Rainer Gerhards wrote: > > > > Now let me take on the imfile example. The key point - at least IMHO > - > > is that there is no single line in rsyslog core's code that has been > > added in support of imfile. And, more importantly, if imfile would go > > away, not a single line of code could be removed. So the imfile > plugin > > (project) does neither add complexity nor code nor other overhead to > > rsyslog core. So can it be evil? > > It's evil that you're forcing me to upgrade to rsyslog v3.x to take > advantage of it ;-) That's what triggered my previous post.. I want > to be tracking non-syslog logfiles, and if imfile/plugins where more > in the unix philosophy of small tools that chain easily.. I might have > been able to pick it from v3.x and use it on stable systems. I've see the smily but let me do a couple of comments. First of all, the "philosophy of small tools that chain easily" implies that they chain via a pipe. This is great for a lot of applications, but it has its drawbacks. A plain pipe is a simplex, relatively loosely coupled IPC method. So if one part of the pipe dies, other parts will learn about it eventually, but not at the same instant and, most importantly, they do not exactly know what was processed and what not. It's pretty much the same thing as plain TCP transport, which pretends reliability but still has a few windows of exposure where message loss may occur (see http://www.monitorware.com/en/workinprogress/selp.txt section 2.4 and, yes, rsyslog is victim to this as long as we don't have a full RFC 3195 implementation). So relying on plain pipes is not exactly my premier communications method if I would like to implement a reliable syslogd (and "reliable" is the "r" in rsyslog). One can circumvent the problem by defining an app-level protocol on top of the pipe, working with app-level acks. HOWEVER, with that you would run into the pretty same situation, namely that you could not use a new version together with an old engine. One can circumvent that, too, but only as far as the old engine has implemented such methodology. And v1/2 do not have this (due to time constraints, rsyslog is still only about half finished...). There is some other reason that makes me avoid pipes. If I do an app-level ack, I need to do at least 4 system calls to pass a single message: 1. input write to pipe 2. engine read msg from pipe 3. engine post processing stat (ACK) to pipe 4. input read ACK from pipe If you do a little bit of math, you'll see with how many user/kernel space transitions you end up, plus how many cycles are needed to run the necessary housekeeping code inside the kernel and libraries. I don't sum them up now, but I am pretty sure that I can completely process the message in less time than is need just for the IPC in that case. That probably is fine if you look at a low-end workstation syslogd, but you don't want to have this overhead if you aim for an enterprise solutions capable of handling massive data input. Also, if everything goes to the system log socket, there is little you can multithread. But we need to be able to multithread as much as possible. If I write a new engine (as I currently do), I would like to see it well working for at least the next 10 years. If I think about hardware trends during that period, it is clear that a single core will become not much faster than it is today. But the number of cores will greatly increase. In order to utilize that, an application must be able to run on as many threads as possible - all with reasonable overhead, of course. So my conclusion is that rsyslog must be able to run massively multithreaded for the high-end use case. This also prohibits using primary interfaces which cannot easily be multithreaded. Finally there is the issue of flow control. Rsyslog *does* flow control, and will do more advanced flow control in the future. Especially with world-dominating UDP syslog it is vitally important to do flow control, because UDP cannot be flow controlled. Sound strange? Well... If we can not flow control UDP syslog, we need to apply intelligent and adoptable flow control that flow controls sources who can (tcp syslog, rfc 3195 and of course file data!) so that buffer space is kept for those precious UDP message which are lost if we can not buffer them at the right instant. If you think this through, you'll see that this requires different level and methods of flow control, depending on the source [so far, I see three levels: can not (UDP), can somewhat (TCP, local sockets), can easily (log files and other sources that generate data themselves)]. Having a native interface greatly reduces code complexity and thus program reliability when it comes at implementing these features. Also, it would require specialized plugins in any case, you couldn't do it with a simple "pipe me in approach" (well, ... some things yes, but at a complexity cost). Also, while I too believe in the Unix approach of small tools, I also think it is important that the average user is able to configure it. Rsyslog aims not only at being enterprise-class but at the some time aims at being easy to use for the novice. Novices don't understand complex scripting to get the job done. I think relying too much on complex glue doesn't help getting the job done. To come back to your orginal post, rsyslogd *should* of course support the Unix way of piping. It looks like there is a small plugin missing to read natively from a pipe. However, I never got a request to implement it. I guess most people use the logger trick to accomplish that task. I know syslog-ng can natively read from pipes, but have not yet considered this important enough given the lack of requests from the community. If you like such a plugin, it's probably a good idea to speak up now ;) > > > > from the small shell script you provided, it looks like there is a > > problem if > > > > a) script is in sleep period > > b) data is appended to text file > > c) text file is rotated > > d) new lines are written to text file > > e) script awake for new polling loop > > > > On a quick look, it looks like the data written in b) will never make > it > > to the syslogd. Imfile handles that. > > Yes, you're right. Cool that imfile handles it. > > > > On the mail output case (though I need to be a bit brief as dinner is > > approaching ;)): I actually intend to add an email output plugin. > > The quote was about being able to *read* mail: I was replying here to Michael Biebel. It looks I was a bit too much in hurry to point this out. Sorry... > > "Every program attempts to expand until it can read mail. > Those programs which cannot so expand are replaced by ones > which can." > > and I was thinking it not too far fetched since "splunk" can do it (it > can > download email messages via IMAP, index them and create alerts on > suspicious > content). In short: receiving email is very low on my agenda. Keep in mind that I already have architecture and at least partly written such a beast on Windows: http://www.monitorware.com/en/Product/product_comparision.php The email question never was in much demand. But if demand comes up, it for sure is not a big thing to add it... (and, of course, you already can do it today with a bit of scripting, the right mailbox rules and logger -- but that isn't appealing to most folks and is one reason I tend to write plugins ;)). > It might seem like you want to take rsyslog in that > direction, i.e. > your complete eventlogd&alertSystem fork of rsyslogd that can read any > input > (syslog, other-logfile, email, snmptraps) and analyze and alert on the > data. > > That's not what *I* want from a syslog server. I just want it to > reliably > collect and store the logs in an organized manner. Then I'll use other > tools to read and analyze them. ... and this is of course perfectly fine with me, too. However, if you look at the core engine needs, you'll see that the "do it all" eventprocessor and the "plain simple syslogd" have exactly the same needs - at least if you would like to extend the syslogd to be enterprise class. Take the queue engine in v3. Its complex. Actually horribly complex. I didn't like to include that complex beast, but it was the only clean solution to the need of being massively concurrent AND being able to queue data while a destination is down. Any other alternative IMHO would have been dirty and hackish - and in the long term much less maintainable. So I just did the right thing (hopefully), even though it was a really big effort and even though it probably will need a few more firedrills before it is really ready for prime time in all scenarios. Another alternative would have been to use one of the big enterprise class message queueing projects. But that would have created a dependency for such a system on each desktop - ouch... I hope I made the right compromise. Currently, the full queue engine is part of the core. If that turns out to be a problem, I can outsource that to a plugin, but that isn't currently very appealing to me. After all, it's "just" some memory overhead - if you don't need the advanced features, no code is executed to do that. The queue uses an internal driver model and simply configuration means simple code. Thus, the v3 queue engine is as reliable as v2, except if you use all those bells and whistles where I am sure currently a few bugs wait to be detected (even though the situation has much improved recently and will improve with each new feedback I receive). Another good example is the config file: of course, there is no need to have a scriptable configuration for a simple syslogd. But while thinking about the (necessary) expression support and a lot of user requests for a better to read config file format, I came to the conclusion that creating a scriptable format is actually the right route to take: http://rgerhards.blogspot.com/2008/02/introducing-rainerscript-and-some. html Anything else (IMHO) would again be less clean, less maintainable and, in this case, would even take longer to implement. So one might think it is evil to include a virtual machine inside a syslogd, but to me it is actually the least effort to implement things. Of course, you can rightly argue that all of this is over the board if you just want to have a plain local logger that takes messages from the local log socket, maybe UDP syslog and store it to local files. You are probably right. But in this case, you can still continue to use sysklogd. After all, rsyslog was spawned from it to create and advanced syslogd. So it comes at no surprise that I am adding features which may not be required for the simple use cases. ;) Let me conclude with two core points: - rsyslog core is as slim as possible, plugins are separate projects that extend the core; This means nobody is forced to run more code than actually required for his job - both a simple, but fairly enhanced syslogd as well as a full network Event processor share the same root engine needs Thus, rsyslog implements this core engine and I occasionally add a plugin here and there to take advantage of the core. Right now, rsyslog core is far from being finished, as are the plugins. At this time, I am working on getting the core right and doing the most requested plugins. When I am done with that, I'll look at the *real* advanced plugins for all kinds of things that users have interest in. I don't see any need to fork of a separate core project for that. In fact, I think it would be counter-productive as I would need to maintain two code bases and the newly forked project would always be able to do what rsyslogd does. So what would be the motivation to maintain another, feature-less project...? Anyhow, I may be totally wrong. Feedback on this topic is still highly appreciated (be it brief or elaborate ;)). I am probably abusing this thread to also tell you a bit about design decisions I have not yet communicated (sorry for that, so much to do, so few time...;)). Rainer > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 10 08:54:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 10 Mar 2008 08:54:42 +0100 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <47D32BF7.5030108@gmail.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <47D32BF7.5030108@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> Hi Johnn, I'd really appreciate if you could any actual samples (even brief) to the wiki at http://wiki.rsyslog.com They would definitely be of great benefit for other users, too. As you have probably seen in my other mails, I am pretty busy with developing right now so I hope that users contribute use cases. Thanks, Raiener > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Sunday, March 09, 2008 1:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] wiki with various application logging examples > > Rainer Gerhards wrote: > > Just to facilitate things: this is what I am thinking > > about: > > > > http://www.rsyslog.com/doc-imfile.html > > > > It can monitor all application log files natively. Did I > > understand your need right? > > Yes, that's exactly what I was looking for. Thanks, really > really awesome! > > One quick error -- in your Sample, you use: > $InputFilePollingInterval 10 > > It should be $InputFilePollInterval (no -ing on Poll) > > Thanks again! > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Mon Mar 10 17:01:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 10 Mar 2008 12:01:22 -0400 Subject: [rsyslog] wiki with various application logging examples In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> References: <005801c8811b$88a23ab6$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B38@grfint2.intern.adiscon.com> <47D32BF7.5030108@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B3C@grfint2.intern.adiscon.com> Message-ID: <47D55B52.7010905@gmail.com> Rainer Gerhards wrote: > Hi Johnn, > > I'd really appreciate if you could any actual samples (even brief) to > the wiki at > > http://wiki.rsyslog.com > > They would definitely be of great benefit for other users, too. As you > have probably seen in my other mails, I am pretty busy with developing > right now so I hope that users contribute use cases. Yes, absolutely! I'm a rsyslog convert :). Since we're going live this week, I've got a ton of things on my plate. But I fully intend to put up my samples (however basic they are) on the wiki soon after launch! Thanks for a great program! johnn From maurizio.rottin at gmail.com Tue Mar 11 13:01:57 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Tue, 11 Mar 2008 13:01:57 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log Message-ID: Hi all, i set up an apache webserver wich logs with directives: ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined" then syslog forwards them with local5.err @10.10.10.89 local6.info @10.10.10.89 and it works good! now on 10.10.10.89 i set up an rsyslog server with directives $template ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" local5.err -?ApacheRemoteErr $template ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Custom.log" local6.info -?ApacheRemoteCustom the result is that i get these files in /var/log/rsyslog/Apache/ ./www.mysite.com:/Custom.log with customLog ./error_www.mysite.com:/Err.log with ErroLog ./message/Err.log with lines like Mar 11 12:32:52 last message repeated 9 times Mar 11 12:32:52 last message repeated 5 times (Note that this is very annoying!!!) Now this a simple way i have found to divide logs per VHost. I don't know if there is a better way, but the real question is different: In the Custom or Error logs i have lines like: Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part do i have to use a double template setup? one for the file and one for the content? the template in the file seems to be this: $template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" but i need just $template TheOneINeed,%msg%\n" i want only the %msg%\n in my file Thanks in advance -- mr From linuxweb at gmail.com Tue Mar 11 19:08:06 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 14:08:06 -0400 Subject: [rsyslog] rsyslog v3 and selinux Message-ID: <47D6CA86.6060501@gmail.com> I took Fedora 8's rsyslog v2.0.2 SRPM and rebuilt it for v3.12.1 I'm using the same init script. When I used that init script (i.e., "/etc/init.d/rsyslog start") on v2.0.2, the only SELinux problem I had was the domain transition, as documented here: http://tanso.net/rsyslog/ However, when I rebuilt the RPM for v3.12.1, and used the same init script to run it, I get many SELinux errors. Eventually, I worked out all the things I had to allow for rsyslog v3.12.1 to run properly. It's enclosed below as a semanage module. Just curious if there was some change to rsyslog between versions 2 and 3 which would make rsyslog, even when running properly in domain syslogd_exec_t, to cause so many SELinux denials, including not being able to do TCP bind?? Thanks for any clues, johnn p.s. Here's the module that finally worked. Will document on wiki when all is done. ========== module rsyslog 1.0; require { class dir search; class file { getattr read write }; class filesystem remount; class tcp_socket { create accept read setopt bind name_bind node_bind listen }; type boot_t; type auditd_log_t; type var_log_t; type syslogd_t; type syslogd_port_t; type port_t; type mount_t; type system_map_t; type inaddr_any_node_t; type unspec_node_t; role system_r; }; allow syslogd_t boot_t:dir search; allow syslogd_t auditd_log_t:dir search; allow syslogd_t auditd_log_t:file { getattr read }; allow syslogd_t self:tcp_socket { create accept read setopt bind listen }; allow syslogd_t syslogd_port_t:tcp_socket name_bind; allow syslogd_t port_t:tcp_socket name_bind; allow syslogd_t system_map_t:file { read getattr }; allow syslogd_t inaddr_any_node_t:tcp_socket node_bind; allow syslogd_t unspec_node_t:tcp_socket node_bind; ========== From rgerhards at hq.adiscon.com Tue Mar 11 20:05:13 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Mar 2008 20:05:13 +0100 Subject: [rsyslog] rsyslog v3 and selinux Message-ID: <005c01c883aa$d9cecd2e$060013ac@intern.adiscon.com> I have absolutely no idea about selinux ... But: may it be that the fact that now plugins - separate libraries - are loaded by rsyslogd to do the job? rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 11.03.08 19:08 Betreff: [rsyslog] rsyslog v3 and selinux I took Fedora 8's rsyslog v2.0.2 SRPM and rebuilt it for v3.12.1 I'm using the same init script. When I used that init script (i.e., "/etc/init.d/rsyslog start") on v2.0.2, the only SELinux problem I had was the domain transition, as documented here: http://tanso.net/rsyslog/ However, when I rebuilt the RPM for v3.12.1, and used the same init script to run it, I get many SELinux errors. Eventually, I worked out all the things I had to allow for rsyslog v3.12.1 to run properly. It's enclosed below as a semanage module. Just curious if there was some change to rsyslog between versions 2 and 3 which would make rsyslog, even when running properly in domain syslogd_exec_t, to cause so many SELinux denials, including not being able to do TCP bind?? Thanks for any clues, johnn p.s. Here's the module that finally worked. Will document on wiki when all is done. ========== module rsyslog 1.0; require { class dir search; class file { getattr read write }; class filesystem remount; class tcp_socket { create accept read setopt bind name_bind node_bind listen }; type boot_t; type auditd_log_t; type var_log_t; type syslogd_t; type syslogd_port_t; type port_t; type mount_t; type system_map_t; type inaddr_any_node_t; type unspec_node_t; role system_r; }; allow syslogd_t boot_t:dir search; allow syslogd_t auditd_log_t:dir search; allow syslogd_t auditd_log_t:file { getattr read }; allow syslogd_t self:tcp_socket { create accept read setopt bind listen }; allow syslogd_t syslogd_port_t:tcp_socket name_bind; allow syslogd_t port_t:tcp_socket name_bind; allow syslogd_t system_map_t:file { read getattr }; allow syslogd_t inaddr_any_node_t:tcp_socket node_bind; allow syslogd_t unspec_node_t:tcp_socket node_bind; ========== _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 11 21:45:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 11 Mar 2008 21:45:00 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > Maurizio Rottin > Sent: Tuesday, March 11, 2008 1:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog with apache and per vhost log > > Hi all, > i set up an apache webserver wich logs with directives: > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com > " "combined" > > then syslog forwards them with > local5.err @10.10.10.89 > local6.info @10.10.10.89 > and it works good! > > now on 10.10.10.89 i set up an rsyslog server with directives > $template > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" > local5.err -?ApacheRemoteErr > $template > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C > ustom.log" > local6.info -?ApacheRemoteCustom > > the result is that i get these files in /var/log/rsyslog/Apache/ > ./www.mysite.com:/Custom.log > with customLog > > ./error_www.mysite.com:/Err.log > with ErroLog > > ./message/Err.log > with lines like > Mar 11 12:32:52 last message repeated 9 times > Mar 11 12:32:52 last message repeated 5 times > (Note that this is very annoying!!!) Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd compatibility, -e stand for "log *e*very message") > > > Now this a simple way i have found to divide logs per VHost. > I don't know if there is a better way, but the real question > is different: > In the Custom or Error logs i have lines like: > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.7.13) Gecko/20060607" > > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part > do i have to use a double template setup? one for the file and one for > the content? > This is caused by the default template, which is what most peiople expect in their syslog files. But, as you write, the soluton is quite easy, just use another template ;) > the template in the file seems to be this: > $template TraditionalFormat,%timegenerated% %HOSTNAME% > %syslogtag%%msg%\n" > but i need just > $template TheOneINeed,%msg%\n" > i want only the %msg%\n in my file > So, use local6.info -?ApacheRemoteCustom;TheOneINeed HTH, feel free to ask any follow-up questions. :) Rainer > > Thanks in advance > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 00:42:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 19:42:22 -0400 Subject: [rsyslog] segmentation fault Message-ID: <47D718DE.90603@gmail.com> Ahh, so close yet so far. I've setup a central rsyslog server and am setting up my first client to point to it. I am going over stunnel. I set it up pretty much like it is in the secure rsyslog article. There are some existing logs on the client box. I load imfile.so module to convert those logs to syslog and send them over TCP/stunnel to my central logger. Everytime I try to start it, it dies within seconds. When I turn on debug mode, I notice the client gets a segmentation fault. I have pasted the tail end of the debug output here: http://pastebin.com/m9a83946 Is it just the sheer number of messages that is causing it to segfault, or is something else going on? johnn From linuxweb at gmail.com Wed Mar 12 03:09:00 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 11 Mar 2008 22:09:00 -0400 Subject: [rsyslog] segmentation fault In-Reply-To: <47D718DE.90603@gmail.com> References: <47D718DE.90603@gmail.com> Message-ID: <47D73B3C.1000509@gmail.com> After MUCH trial & error, I've found the problem. If there is a blank line in one of the log files being read by imfile, then rsyslog dies. I filed a bug: http://bugzilla.adiscon.com/show_bug.cgi?id=49 johnn From rgerhards at hq.adiscon.com Wed Mar 12 08:00:40 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 08:00:40 +0100 Subject: [rsyslog] segmentation fault In-Reply-To: <47D73B3C.1000509@gmail.com> References: <47D718DE.90603@gmail.com> <47D73B3C.1000509@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B64@grfint2.intern.adiscon.com> Thanks for finding out. Looks like we got a bug-fixing day today... Will update the bug tracker as things progress. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 3:09 AM > To: rsyslog-users > Subject: Re: [rsyslog] segmentation fault > > After MUCH trial & error, I've found the problem. If there > is a blank line in one of the log files being read by > imfile, then rsyslog dies. > > I filed a bug: > http://bugzilla.adiscon.com/show_bug.cgi?id=49 > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Wed Mar 12 10:47:31 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 10:47:31 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> Message-ID: 2008/3/11, Rainer Gerhards : > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > > Maurizio Rottin > > Sent: Tuesday, March 11, 2008 1:02 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] rsyslog with apache and per vhost log > > > > Hi all, > > i set up an apache webserver wich logs with directives: > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com > > " "combined" > > > > then syslog forwards them with > > local5.err @10.10.10.89 > > local6.info @10.10.10.89 > > and it works good! > > > > now on 10.10.10.89 i set up an rsyslog server with directives > > $template > > ApacheRemoteErr,%msg%"/var/log/rsyslog/Apache/%syslogtag%/Err.log" > > local5.err -?ApacheRemoteErr > > $template > > ApacheRemoteCustom,%msg%"/var/log/rsyslog/Apache/%syslogtag%/C > > ustom.log" > > local6.info -?ApacheRemoteCustom > > > > the result is that i get these files in /var/log/rsyslog/Apache/ > > ./www.mysite.com:/Custom.log > > with customLog > > > > ./error_www.mysite.com:/Err.log > > with ErroLog > > > > ./message/Err.log > > with lines like > > Mar 11 12:32:52 last message repeated 9 times > > Mar 11 12:32:52 last message repeated 5 times > > (Note that this is very annoying!!!) > > > Add -e to rsyslogd startup and they go away ;) (this is part of sysklogd > compatibility, -e stand for "log *e*very message") And i red about the -e options a lot of times...thank! > > > > > > > Now this a simple way i have found to divide logs per VHost. > > I don't know if there is a better way, but the real question > > is different: > > In the Custom or Error logs i have lines like: > > Mar 11 12:32:53 php03 www.mysite.com: 10.10.10.2 - - > > [11/Mar/2008:12:35:07 +0100] "GET /pagina.php?id=61 HTTP/1.1" 200 93 > > "http://www.mysite.com/home.php" "Mozilla/5.0 (X11; U; Linux i686; > > en-US; rv:1.7.13) Gecko/20060607" > > > > i don't want the "Mar 11 12:32:53 php03 www.mysite.com:" part > > do i have to use a double template setup? one for the file and one for > > the content? > > > > This is caused by the default template, which is what most peiople > expect in their syslog files. But, as you write, the soluton is quite > easy, just use another template ;) > > > > the template in the file seems to be this: > > $template TraditionalFormat,%timegenerated% %HOSTNAME% > > %syslogtag%%msg%\n" > > but i need just > > $template TheOneINeed,%msg%\n" > > i want only the %msg%\n in my file > > > > > So, use > > local6.info -?ApacheRemoteCustom;TheOneINeed works great man! Thanks! > HTH, feel free to ask any follow-up questions. :) > just one question, do you know any way to pass the website name in "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an other program...i know only logger... -- mr From rgerhards at hq.adiscon.com Wed Mar 12 10:50:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 10:50:06 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> [happy snip] > just one question, do you know any way to pass the website name in > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > other program...i know only logger... Well, you could use the imfile plugin to read the log files directly: http://www.rsyslog.com/doc-imfile.html Then, I'd set the site name as tag and so you can process it from the tag value. HTH Rainer From maurizio.rottin at gmail.com Wed Mar 12 11:24:08 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 11:24:08 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > [happy snip] > > > just one question, do you know any way to pass the website name in > > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > > other program...i know only logger... > > > Well, you could use the imfile plugin to read the log files directly: > > http://www.rsyslog.com/doc-imfile.html > > Then, I'd set the site name as tag and so you can process it from the > tag value. > > HTH > > Rainer > mmmmm...i'll check that better but at a glance it seems a heavy process with big logs. -- mr From rgerhards at hq.adiscon.com Wed Mar 12 11:25:53 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 11:25:53 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> > mmmmm...i'll check that better but at a glance it seems a heavy > process with big logs. Sorry, I don't really understand what you mean ;) But the problem is that I probably do not really know how what you do with logger really works. It is part of the apache config, is this right? Maybe you could post a full config sample? Rainer From janfrode at tanso.net Wed Mar 12 12:39:04 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 12 Mar 2008 12:39:04 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> Message-ID: On 2008-03-11, Johnny Tan wrote: > > However, when I rebuilt the RPM for v3.12.1, and used the > same init script to run it, I get many SELinux errors. > Eventually, I worked out all the things I had to allow for > rsyslog v3.12.1 to run properly. It's enclosed below as a > semanage module. These seems very strange: > allow syslogd_t boot_t:dir search; > allow syslogd_t system_map_t:file { read getattr }; Why does it want to read /boot/System.map* ? > allow syslogd_t auditd_log_t:dir search; > allow syslogd_t auditd_log_t:file { getattr read }; And why read /var/log/audit/* ? -jf From rgerhards at hq.adiscon.com Wed Mar 12 13:48:42 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 13:48:42 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B6D@grfint2.intern.adiscon.com> That's part of the klog(d) code I inherited from sysklogd. I've never questioned if there is another way to do it. It seems to use these to obtain kernel symbols. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Wednesday, March 12, 2008 12:39 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] rsyslog v3 and selinux > > On 2008-03-11, Johnny Tan wrote: > > > > However, when I rebuilt the RPM for v3.12.1, and used the > > same init script to run it, I get many SELinux errors. > > Eventually, I worked out all the things I had to allow for > > rsyslog v3.12.1 to run properly. It's enclosed below as a > > semanage module. > > These seems very strange: > > > allow syslogd_t boot_t:dir search; > > allow syslogd_t system_map_t:file { read getattr }; > > Why does it want to read /boot/System.map* ? > > > allow syslogd_t auditd_log_t:dir search; > > allow syslogd_t auditd_log_t:file { getattr read }; > > And why read /var/log/audit/* ? > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Wed Mar 12 14:37:25 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Wed, 12 Mar 2008 14:37:25 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > > mmmmm...i'll check that better but at a glance it seems a heavy > > process with big logs. > > > Sorry, I don't really understand what you mean ;) But the problem is > that I probably do not really know how what you do with logger really > works. It is part of the apache config, is this right? Maybe you could > post a full config sample? > but i did... this is the log part in the directive ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " "combined" in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with facility "-p local*.xxx" with tag "-t www.mysite.com" then i can use the tag to split logs by website in the remote rsyslog machine ;) -- mr From rgerhards at hq.adiscon.com Wed Mar 12 14:44:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 14:44:31 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Wednesday, March 12, 2008 2:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/12, Rainer Gerhards : > > > mmmmm...i'll check that better but at a glance it seems a heavy > > > process with big logs. > > > > > > Sorry, I don't really understand what you mean ;) But the problem is > > that I probably do not really know how what you do with logger > really > > works. It is part of the apache config, is this right? Maybe you > could > > post a full config sample? > > > but i did... > this is the log part in the directive > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > "combined" > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > facility "-p local*.xxx" with tag "-t www.mysite.com" > > then i can use the tag to split logs by website in the remote rsyslog > machine ;) Ah, OK, I didn't realize this was actual Apache config. So your original question > just one question, do you know any way to pass the website name in > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > other program...i know only logger... was how you could use the same apache config command for each log, so that you don't need to have separate apache config statements for each virtual server? Please bear with me if I am getting it wrong, I probably have too much rsyslog on my mind. I am trying to help but I don't get a grip at what the real problem is... Rainer From linuxweb at gmail.com Wed Mar 12 15:08:17 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 10:08:17 -0400 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> Message-ID: <47D7E3D1.9010309@gmail.com> Jan-Frode Myklebust wrote: >> allow syslogd_t auditd_log_t:dir search; >> allow syslogd_t auditd_log_t:file { getattr read }; > > And why read /var/log/audit/* ? This is custom. I prefer to install auditd and have the SELinux logs separated out from /var/log/messages. Then I use imfile to convert the audit logs to syslog in order to send it over to the central logger. Yes, it's a waste of a syslog "local" facility, but I'm not aware of any other way unless I remove auditd and have those go back into /var/log/messages. johnn From linuxweb at gmail.com Wed Mar 12 18:16:56 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 13:16:56 -0400 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <47D81008.3090302@gmail.com> I wanted to setup my rsyslog clients such that when the rsyslog server is down and then comes back up, the clients will resume forwarding messages to it. I'm running into this same bug as another user: http://bugzilla.adiscon.com/show_bug.cgi?id=45 According to that bug, the fix is in 3.12.1, which is what I'm using on client and server. But I still can't seem to get it to work. Prior to doing this, I simply had this line: *.* @@127.0.0.1:61514 Am I supposed to replace that line entirely with the ActionQueue stanza, such as this one (the == is for clarity, not actually part of my config): == $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 30 $ActionResumeRetryCount -1 # infinite retries on insert failure & @@127.0.0.1:61514 == If I do, rsyslog dies immediately on restart. So I changed it so look like this instead: == *.* @@127.0.0.1:61514 $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 & @@127.0.0.1:61514 == This allows rsyslog to work again, but then it doesn't actually do any resume. Just as the OP states in the bug, if the rsyslog server is stopped, the client simply logs messages locally. When the rsyslog server is started again, the client once again logs locally as well as to the server. But all the messages from the client between stop and start on the server are not forwarded. I checked action.c and it does have the code-fix listed in the forum post tied to the bug. I also understand that one message WILL be lost, which is a TCP syslog protocol problem. So I sent multiple messages, none of them came through. johnn From rgerhards at hq.adiscon.com Wed Mar 12 18:22:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:22:12 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D81008.3090302@gmail.com> References: <47D81008.3090302@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B81@grfint2.intern.adiscon.com> I need to check for the restart problem (currently working on something else). Please also follow this bug: http://bugzilla.adiscon.com/show_bug.cgi?id=48 This is the "one message lost" problem, and I have created at least some cure for it today (to be verified). Will check your bug report when I verify it. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 6:17 PM > To: rsyslog-users > Subject: [rsyslog] actionqueue in front of tcp forward > > I wanted to setup my rsyslog clients such that when the > rsyslog server is down and then comes back up, the clients > will resume forwarding messages to it. > > I'm running into this same bug as another user: > http://bugzilla.adiscon.com/show_bug.cgi?id=45 > > According to that bug, the fix is in 3.12.1, which is what > I'm using on client and server. > > > But I still can't seem to get it to work. Prior to doing > this, I simply had this line: > > *.* @@127.0.0.1:61514 > > > > Am I supposed to replace that line entirely with the > ActionQueue stanza, such as this one (the == is for clarity, > not actually part of my config): > > == > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 30 > $ActionResumeRetryCount -1 # infinite retries on insert failure > & @@127.0.0.1:61514 > == > > If I do, rsyslog dies immediately on restart. So I changed > it so look like this instead: > > == > *.* @@127.0.0.1:61514 > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 10 > $ActionResumeRetryCount -1 > & @@127.0.0.1:61514 > == > > This allows rsyslog to work again, but then it doesn't > actually do any resume. Just as the OP states in the bug, if > the rsyslog server is stopped, the client simply logs > messages locally. When the rsyslog server is started again, > the client once again logs locally as well as to the server. > But all the messages from the client between stop and start > on the server are not forwarded. > > I checked action.c and it does have the code-fix listed in > the forum post tied to the bug. > > I also understand that one message WILL be lost, which is a > TCP syslog protocol problem. So I sent multiple messages, > none of them came through. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 18:32:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:32:35 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D81008.3090302@gmail.com> References: <47D81008.3090302@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Argh, some config parsing seems to be wrong. There is a problem with the config file. The right config format is this: == $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 == & in column one is to chain multiple actions together, but it can never be at the start of the chain. Did you take it from some rsyslog documentation? If so and you still know which, please let me know, I'd like to fix that. If I think correctly, the queued action is now chained to the regular action, and probably never executed because the regular action fails ... or something along these lines. Please try the config above, I think it will work (at least it does in my lab). And of course I'll look into the config parser and co ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 6:17 PM > To: rsyslog-users > Subject: [rsyslog] actionqueue in front of tcp forward > > I wanted to setup my rsyslog clients such that when the > rsyslog server is down and then comes back up, the clients > will resume forwarding messages to it. > > I'm running into this same bug as another user: > http://bugzilla.adiscon.com/show_bug.cgi?id=45 > > According to that bug, the fix is in 3.12.1, which is what > I'm using on client and server. > > > But I still can't seem to get it to work. Prior to doing > this, I simply had this line: > > *.* @@127.0.0.1:61514 > > > > Am I supposed to replace that line entirely with the > ActionQueue stanza, such as this one (the == is for clarity, > not actually part of my config): > > == > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 30 > $ActionResumeRetryCount -1 # infinite retries on insert failure > & @@127.0.0.1:61514 > == > > If I do, rsyslog dies immediately on restart. So I changed > it so look like this instead: > > == > *.* @@127.0.0.1:61514 > $WorkDirectory /tmp/rsyslogwork > $ActionQueueType LinkedList > $ActionQueueFileName dqueue > $ActionQueueSaveOnShutdown on > $ActionQueueDiscardSeverity 7 > $ActionResumeInterval 10 > $ActionResumeRetryCount -1 > & @@127.0.0.1:61514 > == > > This allows rsyslog to work again, but then it doesn't > actually do any resume. Just as the OP states in the bug, if > the rsyslog server is stopped, the client simply logs > messages locally. When the rsyslog server is started again, > the client once again logs locally as well as to the server. > But all the messages from the client between stop and start > on the server are not forwarded. > > I checked action.c and it does have the code-fix listed in > the forum post tied to the bug. > > I also understand that one message WILL be lost, which is a > TCP syslog protocol problem. So I sent multiple messages, > none of them came through. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 18:50:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 18:50:01 +0100 Subject: [rsyslog] rsyslog 2.0.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> Hi all, Rsyslog 2.0.3, a release of the stable branch, has been released today. It is a purely bug-fixing release, solving mostly bugs that occur either seldom or due to mistakes in the configuration file. It fixes some potential segfaults in those cases. Rsyslog 2.0.3 is a recommended update for all users of the rsyslog v2 branch. Download: http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml Changelog: http://www.rsyslog.com/Article189.phtml As always, feedback is appreciated. Rainer Gerhards From linuxweb at gmail.com Wed Mar 12 19:27:38 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 14:27:38 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> References: <47D81008.3090302@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Message-ID: <47D8209A.3070908@gmail.com> Rainer Gerhards wrote: > & in column one is to chain multiple actions together, but it can never > be at the start of the chain. Did you take it from some rsyslog > documentation? If so and you still know which, please let me know, I'd > like to fix that. No, I copied it from the bug report I listed. The OP must've had a regular action that wasn't listed. Anyway, I fixed it, restarted client rsyslog, but the behavior is the same. I don't need any special config on the server side, right? Well, if it helps, here's the top of my client /etc/rsyslog.conf, followed by the top of my server /etc/rsyslog.conf: == client /etc/rsyslog.conf == # Modules $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 # Standard syslog configuration (for local logging) [snip] == == server /etc/rsyslog.conf == # Modules $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so # TCP port $InputTCPServerRun 61514 # Global Directives $AllowedSender UDP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $AllowedSender TCP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $DirOwner root $DirGroup root $DirCreateMode 0755 # Templates [snip] # Standard syslog configuration [snip] == From linuxweb at gmail.com Wed Mar 12 19:32:15 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 14:32:15 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> References: <47D81008.3090302@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B82@grfint2.intern.adiscon.com> Message-ID: <47D821AF.9030107@gmail.com> Rainer Gerhards wrote: > $WorkDirectory /tmp/rsyslogwork Also, I should also point out that I NEVER see anything show up in my WorkDirectory. That directory above is owned by root:root and mode 0755. I turned off SELinux. Nothing EVER shows up there. johnn From milton at calnek.com Wed Mar 12 19:22:10 2008 From: milton at calnek.com (Milton Calnek) Date: Wed, 12 Mar 2008 12:22:10 -0600 Subject: [rsyslog] rsyslog 2.0.3 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B83@grfint2.intern.adiscon.com> Message-ID: <47D81F52.2050701@calnek.com> The link in the web site does not work. The requested URL /rsyslog/rsyslog-2.0.23tar.gz was not found on this server. Thanks. Rainer Gerhards wrote: > Hi all, > > Rsyslog 2.0.3, a release of the stable branch, has been released today. > It is a purely bug-fixing release, solving mostly bugs that occur either > seldom or due to mistakes in the configuration file. It fixes some > potential segfaults in those cases. Rsyslog 2.0.3 is a recommended > update for all users of the rsyslog v2 branch. > > Download: > http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml > > Changelog: > http://www.rsyslog.com/Article189.phtml > > As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From rgerhards at hq.adiscon.com Wed Mar 12 19:48:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:48:45 +0100 Subject: [rsyslog] rsyslog 2.0.3 released Message-ID: <006101c88471$b85b588f$060013ac@intern.adiscon.com> Oops, as a quick fix http://download.rsyslog.com/rsyslog/rsyslog-2.0.3.tar.gz If i have not mistyped again ;) rainer ----- Urspr?ngliche Nachricht ----- Von: "Milton Calnek" An: "rsyslog-users" Gesendet: 12.03.08 19:37 Betreff: Re: [rsyslog] rsyslog 2.0.3 released The link in the web site does not work. The requested URL /rsyslog/rsyslog-2.0.23tar.gz was not found on this server. Thanks. Rainer Gerhards wrote: > Hi all, > > Rsyslog 2.0.3, a release of the stable branch, has been released today. > It is a purely bug-fixing release, solving mostly bugs that occur either > seldom or due to mistakes in the configuration file. It fixes some > potential segfaults in those cases. Rsyslog 2.0.3 is a recommended > update for all users of the rsyslog v2 branch. > > Download: > http://www.rsyslog.com/Downloads-req-getit-lid-85.phtml > > Changelog: > http://www.rsyslog.com/Article189.phtml > > As always, feedback is appreciated. > > Rainer Gerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 19:50:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:50:55 +0100 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <006201c88472$0590e0ed$060013ac@intern.adiscon.com> That's fine, files are only created if the mem queue is filling up. By default thats after 8,000 msgs queued. And, no server needs no special config. More later, i am on a pda and can not really check out anything right now ;) rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 12.03.08 19:32 Betreff: Re: [rsyslog] actionqueue in front of tcp forward Rainer Gerhards wrote: > $WorkDirectory /tmp/rsyslogwork Also, I should also point out that I NEVER see anything show up in my WorkDirectory. That directory above is owned by root:root and mode 0755. I turned off SELinux. Nothing EVER shows up there. johnn _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Wed Mar 12 19:52:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 19:52:43 +0100 Subject: [rsyslog] actionqueue in front of tcp forward Message-ID: <006301c88472$46360ab0$060013ac@intern.adiscon.com> Can you send me a ful debug output from the sender when it fails (maybe via mail) - if thats possible... rainer ----- Urspr?ngliche Nachricht ----- Von: "Johnny Tan" An: "rsyslog-users" Gesendet: 12.03.08 19:28 Betreff: Re: [rsyslog] actionqueue in front of tcp forward Rainer Gerhards wrote: > & in column one is to chain multiple actions together, but it can never > be at the start of the chain. Did you take it from some rsyslog > documentation? If so and you still know which, please let me know, I'd > like to fix that. No, I copied it from the bug report I listed. The OP must've had a regular action that wasn't listed. Anyway, I fixed it, restarted client rsyslog, but the behavior is the same. I don't need any special config on the server side, right? Well, if it helps, here's the top of my client /etc/rsyslog.conf, followed by the top of my server /etc/rsyslog.conf: == client /etc/rsyslog.conf == # Modules $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so $WorkDirectory /tmp/rsyslogwork $ActionQueueType LinkedList $ActionQueueFileName dqueue $ActionQueueSaveOnShutdown on $ActionQueueDiscardSeverity 7 $ActionResumeInterval 10 $ActionResumeRetryCount -1 *.* @@127.0.0.1:61514 # Standard syslog configuration (for local logging) [snip] == == server /etc/rsyslog.conf == # Modules $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imuxsock.so $ModLoad imklog.so $ModLoad imfile.so # TCP port $InputTCPServerRun 61514 # Global Directives $AllowedSender UDP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $AllowedSender TCP, 127.0.0.1, 172.16.0.0/16, 192.168.32.0/24 $DirOwner root $DirGroup root $DirCreateMode 0755 # Templates [snip] # Standard syslog configuration [snip] == _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Wed Mar 12 20:07:34 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 15:07:34 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <006301c88472$46360ab0$060013ac@intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> Message-ID: <47D829F6.8070404@gmail.com> Rainer Gerhards wrote: > Can you send me a ful debug output from the sender when > it fails (maybe via mail) - if thats possible... I sent it via YouSendIt. Thanks for looking into this! johnn From rgerhards at hq.adiscon.com Wed Mar 12 21:40:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 21:40:22 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D829F6.8070404@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> I have received the file and had a chance to look at it. It's quite confusing. The send fails, but each retry succeeds... It looks like you use stunnel. I currently doubt that stunnel accepts each send and only reports the error after it can not connect to the remote side. So in fact we continously run into the situation that exactly *that* one message is lost. But I am still puzzled. Could you, for a test, run the same without stunnel and tell me if the problem persists or goes away? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 8:08 PM > To: rsyslog-users > Subject: Re: [rsyslog] actionqueue in front of tcp forward > > Rainer Gerhards wrote: > > Can you send me a ful debug output from the sender when > > it fails (maybe via mail) - if thats possible... > > I sent it via YouSendIt. > Thanks for looking into this! > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 22:05:54 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 17:05:54 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> Message-ID: <47D845B2.30701@gmail.com> Rainer Gerhards wrote: > I have received the file and had a chance to look at it. It's quite > confusing. The send fails, but each retry succeeds... It looks like you > use stunnel. I currently doubt that stunnel accepts each send and only > reports the error after it can not connect to the remote side. So in > fact we continously run into the situation that exactly *that* one > message is lost. But I am still puzzled. Could you, for a test, run the > same without stunnel and tell me if the problem persists or goes away? Yes, I had the same thought about stunnel being the problem and was running some tests. It works! I now only lose the one message (known problem). But if I go over stunnel, then I lose all messages during the rsyslog server downtime. I can do without the stunnel for now. Thanks once again! johnn From rgerhards at hq.adiscon.com Wed Mar 12 22:20:15 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 12 Mar 2008 22:20:15 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D845B2.30701@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Wednesday, March 12, 2008 10:06 PM > To: rsyslog-users > Subject: Re: [rsyslog] actionqueue in front of tcp forward > > Rainer Gerhards wrote: > > I have received the file and had a chance to look at it. It's quite > > confusing. The send fails, but each retry succeeds... It > looks like you > > use stunnel. I currently doubt that stunnel accepts each > send and only > > reports the error after it can not connect to the remote side. So in > > fact we continously run into the situation that exactly *that* one > > message is lost. But I am still puzzled. Could you, for a > test, run the > > same without stunnel and tell me if the problem persists or > goes away? > > Yes, I had the same thought about stunnel being the problem > and was running some tests. > > It works! I now only lose the one message (known problem). > But if I go over stunnel, then I lose all messages during > the rsyslog server downtime. I have the strong feeling that it is time to do something against this plain old ack-less syslog tcp protocol... Maybe I add a half-duplex mode for starters. That's low, but quick to implement and ultra-reliable. I'll also see that I get more serious with RFC 3195 re-enabling. I've already done some basic thinking in regard to 3195 and the new syslog engine and doing it ultra-reliable will require a little bit of work. So there won't be an immediate cure - but defenitely the right route to take. How about half-duplex mode? Would that work for you? It means that each message must be acked before the next one is sent, so tcp's streaming features will almost be disabled. I'd expect a drop to at most 50% (more probable 40%) of the performance compared to what we currently run (half-duplex would obviously need to be an option...). So it would be a large performance hit. > I can do without the stunnel for now. For encryption, you could also look into the GSSAPI modules. It's contributed code, and I currently unfortunately have limited insight into it. But varmojfekoj, the contributor, has done a great job. > Thanks once again! I am very intersted in real-life experience. After all, the engine is still quite new. So I need feedback from the field to make it ultra-solid - a lab is a lab, is a lab ;) So keep the thoughts flowing. A side-note: we are rewriting phpLogCon, the web interface to syslog data. Any chance you happen to have some interest in that? ;) Rainer > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From linuxweb at gmail.com Wed Mar 12 23:10:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 12 Mar 2008 18:10:22 -0400 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> Message-ID: <47D854CE.8020209@gmail.com> Rainer Gerhards wrote: > I have the strong feeling that it is time to do something against this > plain old ack-less syslog tcp protocol... Maybe I add a half-duplex mode > for starters. That's low, but quick to implement and ultra-reliable. > I'll also see that I get more serious with RFC 3195 re-enabling. I've > already done some basic thinking in regard to 3195 and the new syslog > engine and doing it ultra-reliable will require a little bit of work. So > there won't be an immediate cure - but defenitely the right route to > take. > > How about half-duplex mode? Would that work for you? It means that each > message must be acked before the next one is sent, so tcp's streaming > features will almost be disabled. I'd expect a drop to at most 50% (more > probable 40%) of the performance compared to what we currently run > (half-duplex would obviously need to be an option...). So it would be a > large performance hit. Just curious: do either of the above solutions (3195 and half-duplex) resolve the issue of losing all messages when the server goes down and you are going over stunnel? It sounds like the half-duplex does, I don't know enough about the 3195 to know. In answer to your question, I probably don't have as heavy a load as some folks out there, so it's easy for me to say, yes, half-duplex works for me, at least for the moment. >> I can do without the stunnel for now. > > For encryption, you could also look into the GSSAPI modules. It's > contributed code, and I currently unfortunately have limited insight > into it. But varmojfekoj, the contributor, has done a great job. Thanks! I'll have to look more closely at this in a staging environment down the road, as I can't afford potentially running into another stumbling block at the moment. > A side-note: we are rewriting phpLogCon, the web interface to syslog > data. Any chance you happen to have some interest in that? ;) It'll be a couple weeks before I really start looking into this, but yes, I was in fact planning to setup phpLogCon, too. johnn From rgerhards at hq.adiscon.com Thu Mar 13 08:11:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 08:11:58 +0100 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <47D854CE.8020209@gmail.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com> <47D854CE.8020209@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> > Rainer Gerhards wrote: > > I have the strong feeling that it is time to do something against > this > > plain old ack-less syslog tcp protocol... Maybe I add a half-duplex > mode > > for starters. That's low, but quick to implement and ultra-reliable. > > I'll also see that I get more serious with RFC 3195 re-enabling. I've > > already done some basic thinking in regard to 3195 and the new syslog > > engine and doing it ultra-reliable will require a little bit of work. > So > > there won't be an immediate cure - but defenitely the right route to > > take. > > > > How about half-duplex mode? Would that work for you? It means that > each > > message must be acked before the next one is sent, so tcp's streaming > > features will almost be disabled. I'd expect a drop to at most 50% > (more > > probable 40%) of the performance compared to what we currently run > > (half-duplex would obviously need to be an option...). So it would be > a > > large performance hit. > > Just curious: do either of the above solutions (3195 and > half-duplex) resolve the issue of losing all messages when > the server goes down and you are going over stunnel? > > It sounds like the half-duplex does, I don't know enough > about the 3195 to know. > > > In answer to your question, I probably don't have as heavy a > load as some folks out there, so it's easy for me to say, > yes, half-duplex works for me, at least for the moment. I had a somewhat sleepless night, which is good news because it usually results in good ideas ;) This problem really bugged me, but I finally made up my mind. Instead of wasting time on fixing broken plain tcp syslog transport mode (e.g. by implementing half-duplex, which isn't standard anyhow), I'll do "the right thing". I thought rfc 3195 is the right thing. But it carries a lot of overhead that I really don't need. And, most importantly, any standard additions takes ages to go through the IETF (I know what I am talking about, have finally succeeded to get a better syslog rfc though it in "just" 4 (or 5?) years -- and it is still waiting to be published...). So instead of adding on any of these existing protocols, I'll do a new, lightweight but capable protocol specifically designed to solve the shortcomings we currently experience. Please welcome RELP, the "reliable event logging protocol" (name based on the quite successful selp [simple event logging protocol] effort: http://www.monitorware.com/en/workinprogress/selp.txt). Relp will evolve over time. I hope to do something useful relatively soon, and it will be extended as the project progresses. The ultimate goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog communications. I'll don't care about the outside world, so I can do the best of breed solution. For the rest of the world, rsyslog will of course continue to support plain tcp syslog and will get better support for rfc 3195. But if you wanna go hardcore on high-reliability, high-performance event logging, relp will be your choice. Technically, I'll split this off into rsyslog relp input and output plugins AND an independent librelp, which provides core protocol functionality (just in case somebody else wants to support it in the long term, so this will not be tied to rsyslog). > > >> I can do without the stunnel for now. > > > > For encryption, you could also look into the GSSAPI modules. It's > > contributed code, and I currently unfortunately have limited insight > > into it. But varmojfekoj, the contributor, has done a great job. > > Thanks! I'll have to look more closely at this in a staging > environment down the road, as I can't afford potentially > running into another stumbling block at the moment. > > > > A side-note: we are rewriting phpLogCon, the web interface to syslog > > data. Any chance you happen to have some interest in that? ;) > > It'll be a couple weeks before I really start looking into > this, but yes, I was in fact planning to setup phpLogCon, too. Excellent. Please keep an eye on its beta announcements - various stages of v2 will probably be available in a few weeks and I promise it will be much better than what is currently there. In fact, it tries to become the best-ever log web viewer. If there are some things you'd like to see in such a viewer, it would be great to hear your opinion. Rainer From maurizio.rottin at gmail.com Thu Mar 13 09:03:16 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 13 Mar 2008 09:03:16 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> Message-ID: 2008/3/12, Rainer Gerhards : > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Wednesday, March 12, 2008 2:37 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > 2008/3/12, Rainer Gerhards : > > > > mmmmm...i'll check that better but at a glance it seems a heavy > > > > process with big logs. > > > > > > > > > Sorry, I don't really understand what you mean ;) But the problem is > > > that I probably do not really know how what you do with logger > > really > > > works. It is part of the apache config, is this right? Maybe you > > could > > > post a full config sample? > > > > > but i did... > > this is the log part in the directive > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > "combined" > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > > facility "-p local*.xxx" with tag "-t www.mysite.com" > > > > then i can use the tag to split logs by website in the remote rsyslog > > machine ;) > > > Ah, OK, I didn't realize this was actual Apache config. So your original > question > > > > just one question, do you know any way to pass the website name in > > "/usr/bin/logger -p local5.err -t error_www.mysite.com", even using an > > other program...i know only logger... > > > was how you could use the same apache config command for each log, so > that you don't need to have separate apache config statements for each > virtual server? > > Please bear with me if I am getting it wrong, I probably have too much > rsyslog on my mind. I am trying to help but I don't get a grip at what > the real problem is... > > Not really...i don't want to use logger, it becomes a heavy process under high traffic load....but this should be a question for the apache ML... -- mr From rgerhards at hq.adiscon.com Thu Mar 13 09:06:09 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 09:06:09 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> [sniiiiip] > Not really...i don't want to use logger, it becomes a heavy process > under high traffic load....but this should be a question for the > apache ML... OK, I don't know what makes it a heavy process... anyhow... If you come up with an explanation or what would be less resource-intense (from an apache point of view), please let me know. Maybe I can add a simple little tool to do the job. But obviously I need to know what's wrong with logger before even thinking about that ;) Rainer From Gerrard.Geldenhuis at datacash.com Thu Mar 13 10:04:11 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Thu, 13 Mar 2008 09:04:11 -0000 Subject: [rsyslog] actionqueue in front of tcp forward In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com><47D854CE.8020209@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> Message-ID: > > I had a somewhat sleepless night, which is good news because it usually > results in good ideas ;) This problem really bugged me, but I finally > made up my mind. Instead of wasting time on fixing broken plain tcp > syslog transport mode (e.g. by implementing half-duplex, which isn't > standard anyhow), I'll do "the right thing". I thought rfc 3195 is the > right thing. But it carries a lot of overhead that I really don't need. > And, most importantly, any standard additions takes ages to go through > the IETF (I know what I am talking about, have finally succeeded to get > a better syslog rfc though it in "just" 4 (or 5?) years -- and it is > still waiting to be published...). > > So instead of adding on any of these existing protocols, I'll do a new, > lightweight but capable protocol specifically designed to solve the > shortcomings we currently experience. Please welcome RELP, the "reliable > event logging protocol" (name based on the quite successful selp [simple > event logging protocol] effort: > http://www.monitorware.com/en/workinprogress/selp.txt). > > Relp will evolve over time. I hope to do something useful relatively > soon, and it will be extended as the project progresses. The ultimate > goal is to have a good, very reliable, protocol for rsyslog-to-rsyslog > communications. I'll don't care about the outside world, so I can do the > best of breed solution. For the rest of the world, rsyslog will of > course continue to support plain tcp syslog and will get better support > for rfc 3195. But if you wanna go hardcore on high-reliability, > high-performance event logging, relp will be your choice. > Maybe in another 5 years RELP will replace rfc 3195 especially. Now that rsyslog is part of official redhat release it should gain a lot of momentum, the anouncement for RHEL5.2 beta contained rsyslog. Regards From rgerhards at hq.adiscon.com Thu Mar 13 10:59:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 10:59:30 +0100 Subject: [rsyslog] RELP - was: actionqueue in front of tcp forward In-Reply-To: References: <006301c88472$46360ab0$060013ac@intern.adiscon.com> <47D829F6.8070404@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B84@grfint2.intern.adiscon.com> <47D845B2.30701@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B85@grfint2.intern.adiscon.com><47D854CE.8020209@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308B86@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B8E@grfint2.intern.adiscon.com> > Maybe in another 5 years RELP will replace rfc 3195 especially. Doesn't really matter ;) For the technially interested folks: I have drafted an intial spec. Still a lot is missing (of course), but it sufficient to solve the stunnel problem that we saw yesterday (and a couple of others...): http://git.adiscon.com/?p=librelp.git;a=blob;f=src/relp.c;h=a9aff8b005d2 3a8dc66aed1ffa66f41e0ba7b41c;hb=master > Now that > rsyslog is part of official redhat release it should gain a lot of > momentum, the anouncement for RHEL5.2 beta contained rsyslog. Oh, nice to hear. This is good news :) Rainer From friedl at hq.adiscon.com Thu Mar 13 12:59:00 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 13 Mar 2008 12:59:00 +0100 Subject: [rsyslog] rsyslog 3.12.2 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308B96@grfint2.intern.adiscon.com> Hi all, the development branch also sees a new released. Rsyslog 3.12.2 is available immediately. It is primarily a code cleanup and bug fixing release. Feature-wise, it aims at some better recovery of aborted TCP sessions. It contains a number of important fixes, which, for example, prevent a segfault when imfile tries to read empty log file lines. Internally, loadable library modules are now automatically unloaded if they are no longer used. Rsyslog 3.12.2 is a recommended update for all v3 branch users. Download: http://www.rsyslog.com/Downloads-req-getit-lid-87.phtml Changelog: http://www.rsyslog.com/Article191.phtml Feedback is, as always, appreciated. Thank you. Florian Riedl From linuxweb at gmail.com Thu Mar 13 20:27:07 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 13 Mar 2008 15:27:07 -0400 Subject: [rsyslog] imfile with existing files, splits files on server side Message-ID: <47D9800B.40704@gmail.com> I tested this on both rsyslog 3.12.1 and this morning's 3.12.2 release. I ran a couple tests with existing log files. E.g., I have an httpd access_log which is 1.3MB and about 5,000 lines -- nowhere near as large as my real existing log, which is 136MB and over 511,000 lines. With the smaller file, as soon as I started rsyslog on the client side, it correctly started to show up on the server side in: /var/log/client1/httpd_access_log And the hostname shows up in the prefix for the message: Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip] But after a few hundred lines, it suddenly starts logging to: /var/log/httpd_access_log And the hostname no longer shows up in the prefix: Mar 13 15:12:20 tag_access_log:76.8.67.2[snip] Then, thousands of lines later, it goes back to logging to: /var/log/client1/httpd_access_log The other thing is, it never actually finishes the entire 5000 lines. It does 4000-something and then stops. My gut feeling is this is related to ActionQueue settings. So I'm going to experiment with a few different parameters. But the part that I don't get is why it's not all being logged to the same %HOSTNAME% directory? I don't know what's useful to send for diagnostics -- the original files from the client side, the split files from the server side, the client debug, the server debug. Etc. But I'd be happy to run any requested tests. Meanwhile, I'll tweak ActionQueue params. johnn From rgerhards at hq.adiscon.com Thu Mar 13 20:58:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 13 Mar 2008 20:58:45 +0100 Subject: [rsyslog] imfile with existing files, splits files on server side In-Reply-To: <47D9800B.40704@gmail.com> References: <47D9800B.40704@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> Mmmhhhh... So the 5,000 line file doesn't make it completely to the server? That's strange... With the larger file, I'd assume that there are some flow control issues, but with the small one... Maybe it is related to the other problem with the hostname. Kind of sounds like it looses sync some time in between reading the files. I guess it is probably a good idea if you can send me the 5000 line file, so I can run it in my lab tomorrow. Also, please confirm that the small file actually doesn't make it to the server. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Thursday, March 13, 2008 8:27 PM > To: rsyslog-users > Subject: [rsyslog] imfile with existing files, splits files > on server side > > I tested this on both rsyslog 3.12.1 and this morning's > 3.12.2 release. > > I ran a couple tests with existing log files. E.g., I have > an httpd access_log which is 1.3MB and about 5,000 lines -- > nowhere near as large as my real existing log, which is > 136MB and over 511,000 lines. > > With the smaller file, as soon as I started rsyslog on the > client side, it correctly started to show up on the server > side in: > /var/log/client1/httpd_access_log > > And the hostname shows up in the prefix for the message: > Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip] > > > But after a few hundred lines, it suddenly starts logging to: > /var/log/httpd_access_log > > And the hostname no longer shows up in the prefix: > Mar 13 15:12:20 tag_access_log:76.8.67.2[snip] > > > Then, thousands of lines later, it goes back to logging to: > /var/log/client1/httpd_access_log > > > > The other thing is, it never actually finishes the entire > 5000 lines. It does 4000-something and then stops. > > My gut feeling is this is related to ActionQueue settings. > So I'm going to experiment with a few different parameters. > > But the part that I don't get is why it's not all being > logged to the same %HOSTNAME% directory? > > > I don't know what's useful to send for diagnostics -- the > original files from the client side, the split files from > the server side, the client debug, the server debug. Etc. > > But I'd be happy to run any requested tests. > > Meanwhile, I'll tweak ActionQueue params. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From janfrode at tanso.net Fri Mar 14 00:28:29 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 00:28:29 +0100 Subject: [rsyslog] rsyslog on RHEL5u2 ? Message-ID: Looks like rsyslogd will be available in RHEL5u2 ! Congratulations Rainer! https://www.redhat.com/archives/rhelv5-announce/2008-March/msg00000.html """ This beta release includes the following improvements: * Improved Audit and Logging + Add rsyslog logging facility New package: rsyslog-2.0.0-11.el5 Enhanced system logging and kernel message trapping daemons """ -jf From janfrode at tanso.net Fri Mar 14 00:58:55 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 00:58:55 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> Message-ID: On 2008-03-12, Johnny Tan wrote: > > This is custom. I prefer to install auditd and have the > SELinux logs separated out from /var/log/messages. Then I > use imfile to convert the audit logs to syslog in order to > send it over to the central logger. > > Yes, it's a waste of a syslog "local" facility, but I'm not > aware of any other way unless I remove auditd and have those > go back into /var/log/messages. Hmmm.. I have been wondering about how to collect the auditd logs centrally. Could you please share how you do it ? Would be great if you manage to keep the format on the receiving host.. -jf From rgerhards at hq.adiscon.com Fri Mar 14 10:49:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 10:49:45 +0100 Subject: [rsyslog] Feedback requested - length of syslog tag Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Hi all, I got the following bug report (and patch - thx) for situation where the syslog tag is larger than 32 characters. http://bugzilla.adiscon.com/show_bug.cgi?id=50 Syslog rfcs prohibit such oversize tags. However, I think seeing them in practice triggers the question if we should really follow RFC 3164 and the newer rfcs in this case. Or may it be better to have at least an option to allow tags of more than 32 characters? If you have an opinion, please voice it. Thanks, Rainer From pvrabec at redhat.com Fri Mar 14 12:15:15 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Fri, 14 Mar 2008 12:15:15 +0100 Subject: [rsyslog] HUP problem Message-ID: <200803141215.15887.pvrabec@redhat.com> Hi folks, we have some problems here :) rsyslog-3.12.1 ------------------ https://bugzilla.redhat.com/show_bug.cgi?id=437393 imklog.so + HUP generates memory leaks use: HUPs and #watch -d=cumulative cat /proc/`pgrep rsyslog`/status rsyslog-3.12.2 ------------------ is even worse, SIGSEGV occurs after every HUP :( From Gerrard.Geldenhuis at datacash.com Fri Mar 14 12:36:43 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Fri, 14 Mar 2008 11:36:43 -0000 Subject: [rsyslog] Feedback requested - length of syslog tag In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Message-ID: It's a tough one. Software that follows standards generally work well with other software and just makes life less complex. However a lot of these RFC's are very dated and have not kept track with time and 2001 when rfc3164 was published is a long time ago. If there is a flag which is not set by default and which is documented as breaking the standard it might be acceptable to allow longer tags. Users should break at own risk... Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 14 March 2008 09:50 > To: rsyslog-users > Subject: [rsyslog] Feedback requested - length of syslog tag > > Hi all, > > I got the following bug report (and patch - thx) for situation where the > syslog tag is larger than 32 characters. > > http://bugzilla.adiscon.com/show_bug.cgi?id=50 > > Syslog rfcs prohibit such oversize tags. However, I think seeing them in > practice triggers the question if we should really follow RFC 3164 and > the newer rfcs in this case. Or may it be better to have at least an > option to allow tags of more than 32 characters? > > If you have an opinion, please voice it. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Mar 14 12:57:41 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 12:57:41 +0100 Subject: [rsyslog] Feedback requested - length of syslog tag References: <577465F99B41C842AAFBE9ED71E70ABA308BA7@grfint2.intern.adiscon.com> Message-ID: On 2008-03-14, Rainer Gerhards wrote: > > I got the following bug report (and patch - thx) for situation where the > syslog tag is larger than 32 characters. > > http://bugzilla.adiscon.com/show_bug.cgi?id=50 > > Syslog rfcs prohibit such oversize tags. However, I think seeing them in > practice triggers the question if we should really follow RFC 3164 and > the newer rfcs in this case. Or may it be better to have at least an > option to allow tags of more than 32 characters? > > If you have an opinion, please voice it. > I think rsyslog should "be liberal in what it receives and conservative in what it send". So, maybe allow receiving messages with large tags, but don't send them out to other rfc3164-expecting servers. Sending large tags should maybe only be done in RELP.. ? -jf From linuxweb at gmail.com Fri Mar 14 15:15:26 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 14 Mar 2008 10:15:26 -0400 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> Message-ID: <47DA887E.9020601@gmail.com> Jan-Frode Myklebust wrote: > Hmmm.. I have been wondering about how to collect the auditd logs > centrally. Could you please share how you do it ? I use imfile, this is my client stanza: == # auditd audit.log $InputFileName /var/log/audit/audit.log $InputFileTag tag_audit_log: $InputFileStateFile audit_log $InputFileSeverity info $InputFileFacility local6 $InputRunFileMonitor == And on the server side: == $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" local6.* ?HostAudit == > Would be great > if you manage to keep the format on the receiving host.. The format is NOT kept due to the syslog prefix. However audit2allow can still read and process the file. If I need the original format (like for ausearch, and maybe other tools), I would need to run something like this on the server side, in the directory for the host I'm wanting to do the analysis on: sed 's/^.*tag_audit_log://' audit_log | ausearch -i Is that what you meant? johnn From rgerhards at hq.adiscon.com Fri Mar 14 15:29:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 15:29:44 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: <47DA887E.9020601@gmail.com> References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> <47DA887E.9020601@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> I guess you can keep the format if you use a template with just %msg% inside it: $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" $template auditFormat, "%msg%" local6.* ?HostAudit;auditFormat I have not tested this, just guessing. If you try it out and it does not work, I'd appreciate if you could post me a line as written by rsyslog and an original line. I'll see what I can do with these ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 14, 2008 3:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog v3 and selinux > > Jan-Frode Myklebust wrote: > > Hmmm.. I have been wondering about how to collect the auditd logs > > centrally. Could you please share how you do it ? > > I use imfile, this is my client stanza: > == > # auditd audit.log > $InputFileName /var/log/audit/audit.log > $InputFileTag tag_audit_log: > $InputFileStateFile audit_log > $InputFileSeverity info > $InputFileFacility local6 > $InputRunFileMonitor > == > > > And on the server side: > == > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > local6.* ?HostAudit > == > > > > Would be great > > if you manage to keep the format on the receiving host.. > > The format is NOT kept due to the syslog prefix. However > audit2allow can still read and process the file. > > If I need the original format (like for ausearch, and maybe > other tools), I would need to run something like this on the > server side, in the directory for the host I'm wanting to do > the analysis on: > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > > Is that what you meant? > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 14 15:33:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 15:33:43 +0100 Subject: [rsyslog] rsyslog v3 and selinux In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com><47DA887E.9020601@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308BB2@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BB3@grfint2.intern.adiscon.com> Ummm... I forgot two important chraceters, else you'll have no line feeds and all in one line... (see the \n after %msg$)! $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" $template auditFormat, "%msg%\n" local6.* ?HostAudit;auditFormat > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, March 14, 2008 3:30 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog v3 and selinux > > I guess you can keep the format if you use a template with just %msg% > inside it: > > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > $template auditFormat, "%msg%" > local6.* ?HostAudit;auditFormat > > I have not tested this, just guessing. If you try it out and it does > not > work, I'd appreciate if you could post me a line as written by rsyslog > and an original line. I'll see what I can do with these ;) > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > > Sent: Friday, March 14, 2008 3:15 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog v3 and selinux > > > > Jan-Frode Myklebust wrote: > > > Hmmm.. I have been wondering about how to collect the auditd logs > > > centrally. Could you please share how you do it ? > > > > I use imfile, this is my client stanza: > > == > > # auditd audit.log > > $InputFileName /var/log/audit/audit.log > > $InputFileTag tag_audit_log: > > $InputFileStateFile audit_log > > $InputFileSeverity info > > $InputFileFacility local6 > > $InputRunFileMonitor > > == > > > > > > And on the server side: > > == > > $template HostAudit, "/var/log/rsyslog/%HOSTNAME%/audit_log" > > local6.* ?HostAudit > > == > > > > > > > Would be great > > > if you manage to keep the format on the receiving host.. > > > > The format is NOT kept due to the syslog prefix. However > > audit2allow can still read and process the file. > > > > If I need the original format (like for ausearch, and maybe > > other tools), I would need to run something like this on the > > server side, in the directory for the host I'm wanting to do > > the analysis on: > > > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > > > > > Is that what you meant? > > johnn > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From janfrode at tanso.net Fri Mar 14 15:29:46 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 14 Mar 2008 15:29:46 +0100 Subject: [rsyslog] rsyslog v3 and selinux References: <47D6CA86.6060501@gmail.com> <47D7E3D1.9010309@gmail.com> <47DA887E.9020601@gmail.com> Message-ID: On 2008-03-14, Johnny Tan wrote: >== > The format is NOT kept due to the syslog prefix. However > audit2allow can still read and process the file. > > If I need the original format (like for ausearch, and maybe > other tools), I would need to run something like this on the > server side, in the directory for the host I'm wanting to do > the analysis on: > > sed 's/^.*tag_audit_log://' audit_log | ausearch -i > > Is that what you meant? Yes, thanks! -jf From rgerhards at hq.adiscon.com Fri Mar 14 18:09:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 18:09:35 +0100 Subject: [rsyslog] rsyslog on RHEL5u2 ? In-Reply-To: References: Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BBA@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, March 14, 2008 12:28 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog on RHEL5u2 ? > > Looks like rsyslogd will be available in RHEL5u2 ! Congratulations > Rainer! Thanks, much appreciated :) Let's hope to get v3 stable ready for RHEL 6 ;) Rainer > > > https://www.redhat.com/archives/rhelv5-announce/2008- > March/msg00000.html > > > """ > This beta release includes the following improvements: > > > > * Improved Audit and Logging > + Add rsyslog logging facility > > > > New package: rsyslog-2.0.0-11.el5 > Enhanced system logging and kernel message trapping daemons > > """ > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 14 18:29:40 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 14 Mar 2008 18:29:40 +0100 Subject: [rsyslog] HUP problem In-Reply-To: <200803141215.15887.pvrabec@redhat.com> References: <200803141215.15887.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BBB@grfint2.intern.adiscon.com> I was busy with the other things today but had a quick look. I can repro the hup problem. Looks like it is related to the new object loader (and specifically the unloader), I guess it will be a relatively easy fix ... on Monday ;) Will look into the memleak after that. Thanks for reporting. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > Sent: Friday, March 14, 2008 12:15 PM > To: rsyslog-users > Subject: [rsyslog] HUP problem > > Hi folks, > > we have some problems here :) > > rsyslog-3.12.1 > ------------------ > https://bugzilla.redhat.com/show_bug.cgi?id=437393 > imklog.so + HUP generates memory leaks > use: > HUPs and #watch -d=cumulative cat /proc/`pgrep rsyslog`/status > > rsyslog-3.12.2 > ------------------ > is even worse, SIGSEGV occurs after every HUP :( > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 11:04:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 11:04:45 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Hi all, once again, I am in need for feedback. In order to be backwards-compatible with sysklogd, rsyslogd supported the "last message repeated n times" message compression feature. However, this feature is prone to causing user trouble. Some even think that it is a design flaw (see this discussion on the loganalysis mailing list, this posting is probably a good entry point into a lengthy thread: http://www.loganalysis.org/pipermail/loganalysis/2008-January/000547.htm l ). >From the rsyslog core engine point of view, "last message repeated n times" is quite costly in terms of code complexity and even performance. There is a -e command line switch to turn it off, which most users seem to do (and those that don't do it often seem to run into troubles). I am very tempted to DROP this feature from future builds. That would result in a great code complexity reduction (really, it takes a lot of effort...) and probably also rid us of a standard trouble spot. However, it also means that its compression features are no longer available. Question now: does anybody actually need this feature? If so, why is it good? Please provide feedback. Thanks, Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 11:15:09 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 10:15:09 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: Hi I personnally find it not very usefull to put it politely. I'd rather have every message and every time stamp for the sake of acuracy and precision then an attempt from the software to be intelligent. And if you get the benefit of a code clean up then double thumbs up from me. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 10:05 > To: rsyslog-users > Subject: [rsyslog] Feedback requested "Last message logged n times"... > > Hi all, > > once again, I am in need for feedback. In order to be > backwards-compatible with sysklogd, rsyslogd supported the "last message > repeated n times" message compression feature. However, this feature is > prone to causing user trouble. Some even think that it is a design flaw > (see this discussion on the loganalysis mailing list, this posting is > probably a good entry point into a lengthy thread: > http://www.loganalysis.org/pipermail/loganalysis/2008-January/000547.htm > l ). > > >From the rsyslog core engine point of view, "last message repeated n > times" is quite costly in terms of code complexity and even performance. > There is a -e command line switch to turn it off, which most users seem > to do (and those that don't do it often seem to run into troubles). > > I am very tempted to DROP this feature from future builds. That would > result in a great code complexity reduction (really, it takes a lot of > effort...) and probably also rid us of a standard trouble spot. However, > it also means that its compression features are no longer available. > > Question now: does anybody actually need this feature? If so, why is it > good? > > Please provide feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 11:23:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 11:23:37 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gerrard Geldenhuis > Sent: Tuesday, March 18, 2008 11:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n > times"... > > Hi > I personnally find it not very usefull to put it politely. I'd rather > have every message and every time stamp for the sake of acuracy and > precision then an attempt from the software to be intelligent. > > And if you get the benefit of a code clean up then double thumbs up > from > me. It's actually a *big* cleanup, because in order to support that feature, we need to have A) some background tick processing to flush the "repeated message" every few (30 or 60) seconds (it prevents rsyslogd from being tickles - think powertop, green IT and all that) B) requires rsyslogd to keep multiple message objects alive for an extended period of time, because the message object is needed for comparison when a new message comes in. Among others, this prevents some advanced flow control options which would depend on message destruction *when the message is done* and not *when the next message arrives*. C) [estimate] almost half the code of the main action calling loop, and most of the complexity, is related to "last message repeated n times" D) it prevents future end-to-end ACK capabilities (e.g. message is only acked after being written to DB) E) requires additional synchronization slowing down parallel processing (granted, just a little bit, but...) F) requires a "duplicate message capability" (some more code) G) ... and probably a few more spots ... I should have added this information to the initial message, too. These are *my* primary drivers for the idea to get rid of it ;) Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 12:12:02 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 11:12:02 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> Message-ID: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 10:24 > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n times"... > > > It's actually a *big* cleanup, because in order to support that feature, > we need to have > I have asked the question on a local lug to get some more opinions. I will post the results if any here. Regards From pvrabec at redhat.com Tue Mar 18 12:26:27 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 18 Mar 2008 12:26:27 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> Message-ID: <200803181226.27878.pvrabec@redhat.com> On Tuesday 18 March 2008 12:12:02 pm Gerrard Geldenhuis wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: 18 March 2008 10:24 > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > times"... > > > It's actually a *big* cleanup, because in order to support that > > feature, > > > we need to have > > I have asked the question on a local lug to get some more opinions. I > will post the results if any here. I did the same. :) The only arguments against removal I have received until now were: 1. DoS (but nobody explained how) 2. cleanness of logs (relative) I'm thinking about forwarding discussion to fedora-devel :) > > Regards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From raimund.sacherer at gmail.com Tue Mar 18 12:37:35 2008 From: raimund.sacherer at gmail.com (Raimund Sacherer) Date: Tue, 18 Mar 2008 12:37:35 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> Message-ID: <41963B0D-1D02-4C60-99E1-CF52AFF0ABCB@gmail.com> > > Question now: does anybody actually need this feature? If so, why is > it > good? > > Please provide feedback. Hi all, i think it's a biest of the past, remember those days where every byte, every bit was really valuable? i remember programing on my C64 with byte values, because they could hold 256 states and it was just using one (!, today i have to smile) byte ... Or the times i had my first 40 MB Harddrive in one of my first PC's ... i guess compressing a message which is exactly the same but only of the timestamp was virtually a lifesaver those days ... as far as i can say, from the bottom of my heart: get rid of it! disc- space is cheap, i *love* consistant logs, personally i can not think of any answer that would justify today a) log-file inconsinstency b) a known trouble-spot and c) such a future-development-stopper as this code seems to be (after reading your problems with this piece of code. again, from me: go for it best Raimund > > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 12:42:11 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 12:42:11 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803181226.27878.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com> <200803181226.27878.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Good discussion, thanks all :) ... some more points below ... > > I have asked the question on a local lug to get some more opinions. I > > will post the results if any here. > I did the same. :) > > The only arguments against removal I have received until now were: > > 1. DoS (but nobody explained how) I think this is along the lines of making it easy to flood the log with similar messages. If that's the case, I think it is a (too) weak argument because an attacker could easily include a random pattern inside the message. HOWEVER, the current logic indeed prevents flooding the log from equal messages, e.g. if a process runs wild. > 2. cleanness of logs (relative) > > I'm thinking about forwarding discussion to fedora-devel :) Excellent. Maybe there are also alternatives to the current behavior, something in between. Right now, the core engine does this, so the reduction capability is inherent to every action (except if an action specifically forbids it, because it can not intelligently handle it - e.g. database writers). So it applies to forwarding, snmp, whatever, ... One alternative would be to remove it from the core engine and move it into the *file write* output plugin (omfile). This has its subtleties, too, so I don't like to take that approach lightly. Plus, it than means that the network may become spammed. I think one core requirement is to find people who actually *intentionally* use this feature and ask for their reasoning. That will probably be the best way to tell us if its really needed. And, of course, we need to weigh the negative effects of it against these con-points. Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 14:11:15 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 13:11:15 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: The only arguments for keeping the feature that I got on my lug was the preservartion of disk/network IO. I think to prevent DOS attacks is a valid argument but as you said can be easily circumvented by randomizing messages. To safeguard against dos attacks you could have a monitor that monitors for extra ordinary amount of traffic and then generate a snmp trap. Whether that should be a rsyslog plugin or part of other software is open to debate. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: 18 March 2008 11:42 > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message logged n times"... > > Good discussion, thanks all :) ... some more points below ... > > > > I have asked the question on a local lug to get some more opinions. > I > > > will post the results if any here. > > I did the same. :) > > > > The only arguments against removal I have received until now were: > > > > 1. DoS (but nobody explained how) > > I think this is along the lines of making it easy to flood the log with > similar messages. If that's the case, I think it is a (too) weak > argument because an attacker could easily include a random pattern > inside the message. > > HOWEVER, the current logic indeed prevents flooding the log from equal > messages, e.g. if a process runs wild. > > > 2. cleanness of logs (relative) > > > > I'm thinking about forwarding discussion to fedora-devel :) > > Excellent. > > Maybe there are also alternatives to the current behavior, something in > between. Right now, the core engine does this, so the reduction > capability is inherent to every action (except if an action specifically > forbids it, because it can not intelligently handle it - e.g. database > writers). So it applies to forwarding, snmp, whatever, ... One > alternative would be to remove it from the core engine and move it into > the *file write* output plugin (omfile). This has its subtleties, too, > so I don't like to take that approach lightly. Plus, it than means that > the network may become spammed. I think one core requirement is to find > people who actually *intentionally* use this feature and ask for their > reasoning. That will probably be the best way to tell us if its really > needed. And, of course, we need to weigh the negative effects of it > against these con-points. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 18 14:22:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 18 Mar 2008 14:22:30 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> > The only arguments for keeping the feature that I got on my lug was the > preservartion of disk/network IO. Did you get any "feeling" of how important this is being considered? > > I think to prevent DOS attacks is a valid argument but as you said can > be easily circumvented by randomizing messages. > > To safeguard against dos attacks you could have a monitor that monitors > for extra ordinary amount of traffic and then generate a snmp trap. > Whether that should be a rsyslog plugin or part of other software is > open to debate. This may (m-a-y it's far too early) be part of the flow control logic or an exception detector or a rate-limiting feature... Even for non-DoS cases it might be interesting to know who is sending most messages... mmmh... maybe this points into a direction on how to solve the need that is behind "last message repeated n times". Probably that need is not even fully understood... mmmhh. More thoughts are appreciated ;) Rainer From Gerrard.Geldenhuis at datacash.com Tue Mar 18 14:41:48 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Tue, 18 Mar 2008 13:41:48 -0000 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BD6@grfint2.intern.adiscon.com><200803181226.27878.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BDD@grfint2.intern.adiscon.com> Message-ID: > > > The only arguments for keeping the feature that I got on my lug was > the > > preservartion of disk/network IO. > > Did you get any "feeling" of how important this is being considered? I think the general feeling is: "don't care" or not important at all but then I did not get many replies. > > Even for non-DoS cases it might be interesting to know who is sending > most messages... mmmh... maybe this points into a direction on how to > solve the need that is behind "last message repeated n times". Probably > that need is not even fully understood... mmmhh. More thoughts are > appreciated ;) When I asked about which applications has caused these type of repeat messages one culprit mentioned was the kernel. Regards From friedl at hq.adiscon.com Tue Mar 18 15:05:41 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Tue, 18 Mar 2008 15:05:41 +0100 Subject: [rsyslog] rsyslog 3.12.3 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308BDF@grfint2.intern.adiscon.com> Hi all, rsyslog 3.12.3 has been released today. It is primarily a bug-fixing release, but offers new, advanced flow control options. So far, only imfile utilizes these, but other modules will do so soon. The release contains important bug fixes, for example for a segfault on hup, improper framing for syslog/tcp communication and processing empty lines in the log file reader. Rsyslog 3.12.3 is a recommended update to all v3 branch users and users of 3.12.2 are urged to apply the update (some bugs where introduced in that release). Download: http://www.rsyslog.com/Downloads-req-getit-lid-88.phtml Changelog: http://www.rsyslog.com/Article193.phtml As always, feedback is appreciated. Florian Riedl From pvrabec at redhat.com Tue Mar 18 15:23:52 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 18 Mar 2008 15:23:52 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> Message-ID: <200803181523.52321.pvrabec@redhat.com> On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > The only arguments for keeping the feature that I got on my lug was the > preservartion of disk/network IO. > > I think to prevent DOS attacks is a valid argument but as you said can > be easily circumvented by randomizing messages. I'm afraid it's not true in all cases. What if you do DOS attach not directly to do rsyslog, but via other daemon. In situation when you can't send message directly to syslog, but you can make some daemon generate message for you. This message would be probably always the same content. > To safeguard against dos attacks you could have a monitor that monitors > for extra ordinary amount of traffic and then generate a snmp trap. > Whether that should be a rsyslog plugin or part of other software is > open to debate. > > Regards > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > Sent: 18 March 2008 11:42 > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > times"... > > > Good discussion, thanks all :) ... some more points below ... > > > > > > I have asked the question on a local lug to get some more > > opinions. > > > I > > > > > > will post the results if any here. > > > > > > I did the same. :) > > > > > > The only arguments against removal I have received until now were: > > > > > > 1. DoS (but nobody explained how) > > > > I think this is along the lines of making it easy to flood the log > > with > > > similar messages. If that's the case, I think it is a (too) weak > > argument because an attacker could easily include a random pattern > > inside the message. > > > > HOWEVER, the current logic indeed prevents flooding the log from equal > > messages, e.g. if a process runs wild. > > > > > 2. cleanness of logs (relative) > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > Excellent. > > > > Maybe there are also alternatives to the current behavior, something > > in > > > between. Right now, the core engine does this, so the reduction > > capability is inherent to every action (except if an action > > specifically > > > forbids it, because it can not intelligently handle it - e.g. database > > writers). So it applies to forwarding, snmp, whatever, ... One > > alternative would be to remove it from the core engine and move it > > into > > > the *file write* output plugin (omfile). This has its subtleties, too, > > so I don't like to take that approach lightly. Plus, it than means > > that > > > the network may become spammed. I think one core requirement is to > > find > > > people who actually *intentionally* use this feature and ask for their > > reasoning. That will probably be the best way to tell us if its really > > needed. And, of course, we need to weigh the negative effects of it > > against these con-points. > > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Tue Mar 18 22:12:39 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Tue, 18 Mar 2008 17:12:39 -0400 Subject: [rsyslog] imfile with existing files, splits files on server side In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> References: <47D9800B.40704@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA1CF8D4@grfint2.intern.adiscon.com> Message-ID: <47E03047.3090801@gmail.com> Rainer Gerhards wrote: > So the 5,000 line file doesn't make it completely to the server? That's > strange... With the larger file, I'd assume that there are some flow > control issues, but with the small one... Just wanted to follow-up... thanks to Rainer's tireless poring over my excessively long logfiles and debug outputs, this issue is finally resolved. It seems like it was a combination of tcp flow control issues (fixed with the new 3.12.3) and the server by default discarding messages above severity 4. Thanks again to Rainer for not giving up on this! johnn From rgerhards at hq.adiscon.com Thu Mar 20 11:04:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 11:04:19 +0100 Subject: [rsyslog] update on RELP Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C16@grfint2.intern.adiscon.com> Hi all, I thought I provide a quick update on RELP, the protocol rsyslog will use in the future for reliable rsyslog-to-rsyslog communication. I am currently very actively working on both the protocol and librelp, an implementation for it. The RELP "spec" is also growing. It is now available at http://www.rsyslog.com/doc-relp.html Those with some deep technical interest might want to have a peek at it. I have also setup a RELP-specific mailing list. Find it at http://lists.adiscon.net/mailman/listinfo/relp The next major feature in rsyslogd will be relp support. Until this is finished, I am focused on bug fixing. Rainer From maurizio.rottin at gmail.com Thu Mar 20 15:20:17 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:20:17 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> Message-ID: 2008/3/13, Rainer Gerhards : wow, > [sniiiiip] > > > Not really...i don't want to use logger, it becomes a heavy process > > under high traffic load....but this should be a question for the > > apache ML... > > > OK, I don't know what makes it a heavy process... anyhow... If you come > up with an explanation or what would be less resource-intense (from an > apache point of view), please let me know. Maybe I can add a simple > little tool to do the job. But obviously I need to know what's wrong > with logger before even thinking about that ;) > > ok my ideas are a little more clear now, as you said: > was how you could use the same apache config command for each log, so > that you don't need to have separate apache config statements for each > virtual server? > > Please bear with me if I am getting it wrong, I probably have too much > rsyslog on my mind. I am trying to help but I don't get a grip at what > the real problem is... > > you were right...this is what i need! just one command for ErrorLog and one for CustomLog so as i posted now i'm using: > > this is the log part in the directive > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > "combined" > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger with > > facility "-p local*.xxx" with tag "-t www.mysite.com" for each virtualhost, but this mean that i have 2 loggers running for every virtualhost. This is why logger becomes a heavy process. But if i use one default directive for every vhost, i don't know how to split the log per vhost in the remote machine which collects the logs... Hope now is a little bit clearer... now, new config in httpd.conf "LogLevel warn ErrorLog "|/usr/bin/logger -p local5.err" LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined CustomLog "|/usr/bin/logger -p local6.info " vcombined" and the %v gives the virtualhost name, so now i've got only two loggers. At this point how can i say to rsyslog to split the files according to the first word which is in this case the vhost name? Thanks for being patient! -- mr From rgerhards at hq.adiscon.com Thu Mar 20 15:25:09 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 15:25:09 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B62@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Can you send me a handful of the logline to play with? Probably not this week, but next... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:20 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/13, Rainer Gerhards : > > wow, > > [sniiiiip] > > > > > Not really...i don't want to use logger, it becomes a heavy process > > > under high traffic load....but this should be a question for the > > > apache ML... > > > > > > OK, I don't know what makes it a heavy process... anyhow... If you > come > > up with an explanation or what would be less resource-intense (from > an > > apache point of view), please let me know. Maybe I can add a simple > > little tool to do the job. But obviously I need to know what's wrong > > with logger before even thinking about that ;) > > > > > > ok my ideas are a little more clear now, as you said: > > was how you could use the same apache config command for each log, so > > that you don't need to have separate apache config statements for > each > > virtual server? > > > > Please bear with me if I am getting it wrong, I probably have too > much > > rsyslog on my mind. I am trying to help but I don't get a grip at > what > > the real problem is... > > > > > > you were right...this is what i need! > > just one command for ErrorLog and one for CustomLog > > so as i posted now i'm using: > > > this is the log part in the directive > > > ErrorLog "|/usr/bin/logger -p local5.err -t error_www.mysite.com" > > > CustomLog "|/usr/bin/logger -p local6.info -t www.mysite.com " > > > "combined" > > > > > > in practice i pass the ErrorLog and CustomLog to /usr/bin/logger > with > > > facility "-p local*.xxx" with tag "-t www.mysite.com" > > for each virtualhost, but this mean that i have 2 loggers running for > every virtualhost. > This is why logger becomes a heavy process. > > But if i use one default directive for every vhost, i don't know how > to split the log per vhost in the remote machine which collects the > logs... > > Hope now is a little bit clearer... > > now, new config in httpd.conf > "LogLevel warn > ErrorLog "|/usr/bin/logger -p local5.err" > LogFormat "%v %h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" > vcombined > CustomLog "|/usr/bin/logger -p local6.info " vcombined" > > and the %v gives the virtualhost name, so now i've got only two > loggers. > > At this point how can i say to rsyslog to split the files according to > the first word which is in this case the vhost name? > > Thanks for being patient! > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 15:46:32 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:46:32 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Message-ID: 2008/3/20, Rainer Gerhards : > Can you send me a handful of the logline to play with? Probably not this > week, but next... > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" i'm trying to use the regexp but with no success in this way: $template MsgFormat,"%msg%\n" $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ [a-z,\.]*--end%_az.log" local6.info -?ApacheRemoteCustom;MsgFormat from the documentation: "the property replacer will return the part of the property text that matches the regular expression" which should be " www.mysite.com" but i get a file named _az.log -- mr From rgerhards at hq.adiscon.com Thu Mar 20 15:49:21 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 15:49:21 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B6A@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Let me try to avoid the regexp (its expensive and I can not debug it now ;)): so you search for the string that is at the start of the msg and delimited by the first space? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:47 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > 2008/3/20, Rainer Gerhards : > > Can you send me a handful of the logline to play with? Probably not > this > > week, but next... > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" > > i'm trying to use the regexp but with no success in this way: > $template MsgFormat,"%msg%\n" > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > [a-z,\.]*--end%_az.log" > local6.info -?ApacheRemoteCustom;MsgFormat > > from the documentation: "the property replacer will return the part of > the property text that matches the regular expression" which should be > " www.mysite.com" > but i get a file named _az.log > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 15:59:02 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 15:59:02 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Message-ID: yes! but actually there is a space at the beginning and hostname can contain the dash -, numbers, and letters. 2008/3/20, Rainer Gerhards : > Let me try to avoid the regexp (its expensive and I can not debug it now > ;)): so you search for the string that is at the start of the msg and > delimited by the first space? > > > > Rainer > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Thursday, March 20, 2008 3:47 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > 2008/3/20, Rainer Gerhards : > > > Can you send me a handful of the logline to play with? Probably not > > this > > > week, but next... > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > /images/wm001.jpg HTTP/1.1" 304 - "http://www.mysite.com/webmail.htm" > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060607" > > > > i'm trying to use the regexp but with no success in this way: > > $template MsgFormat,"%msg%\n" > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > [a-z,\.]*--end%_az.log" > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > from the documentation: "the property replacer will return the part of > > the property text that matches the regular expression" which should be > > " www.mysite.com" > > but i get a file named _az.log > > > > -- > > mr > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- mr From rgerhards at hq.adiscon.com Thu Mar 20 16:09:24 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 16:09:24 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Should work with fields (much faster). I can't try it out due to relp work, but try: %msg:F,32:2% [32 is USASCII SP, the delimiter here] But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that should work... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 3:59 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > yes! but actually there is a space at the beginning and hostname can > contain the dash -, numbers, and letters. > > 2008/3/20, Rainer Gerhards : > > Let me try to avoid the regexp (its expensive and I can not debug it > now > > ;)): so you search for the string that is at the start of the msg > and > > delimited by the first space? > > > > > > > > Rainer > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > Can you send me a handful of the logline to play with? Probably > not > > > this > > > > week, but next... > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > > /images/wm001.jpg HTTP/1.1" 304 - > "http://www.mysite.com/webmail.htm" > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > Gecko/20060607" > > > > > > i'm trying to use the regexp but with no success in this way: > > > $template MsgFormat,"%msg%\n" > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > [a-z,\.]*--end%_az.log" > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > from the documentation: "the property replacer will return the > part of > > > the property text that matches the regular expression" which > should be > > > " www.mysite.com" > > > but i get a file named _az.log > > > > > > -- > > > mr > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From maurizio.rottin at gmail.com Thu Mar 20 16:22:11 2008 From: maurizio.rottin at gmail.com (Maurizio Rottin) Date: Thu, 20 Mar 2008 16:22:11 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Message-ID: wow! %msg:F,32:2% works! i completely misunderstood how Fielf works...i thought the first number was the FromChar and second the ToChar... Thanks a lot!!! 2008/3/20, Rainer Gerhards : > Should work with fields (much faster). I can't try it out due to relp > work, but try: > > %msg:F,32:2% [32 is USASCII SP, the delimiter here] > > But maybe %msg:F,32:1% - you need to experiment a bit. In any case, that > should work... > > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > Sent: Thursday, March 20, 2008 3:59 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > yes! but actually there is a space at the beginning and hostname can > > contain the dash -, numbers, and letters. > > > > 2008/3/20, Rainer Gerhards : > > > Let me try to avoid the regexp (its expensive and I can not debug it > > now > > > ;)): so you search for the string that is at the start of the msg > > and > > > delimited by the first space? > > > > > > > > > > > > Rainer > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > > Can you send me a handful of the logline to play with? Probably > > not > > > > this > > > > > week, but next... > > > > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] "GET > > > > /images/wm001.jpg HTTP/1.1" 304 - > > "http://www.mysite.com/webmail.htm" > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > > Gecko/20060607" > > > > > > > > i'm trying to use the regexp but with no success in this way: > > > > $template MsgFormat,"%msg%\n" > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > > [a-z,\.]*--end%_az.log" > > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > > > from the documentation: "the property replacer will return the > > part of > > > > the property text that matches the regular expression" which > > should be > > > > " www.mysite.com" > > > > but i get a file named _az.log > > > > > > > > -- > > > > mr > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > -- > > mr > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- mr From rgerhards at hq.adiscon.com Thu Mar 20 16:39:49 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 20 Mar 2008 16:39:49 +0100 Subject: [rsyslog] rsyslog with apache and per vhost log In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308B70@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308B87@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C19@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1C@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C1D@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C1E@grfint2.intern.adiscon.com> Excellent, good to hear :) Just a side-note: when I am done with relp, I'll hopefully have time to come back to the config format. Then we'll also see much more powerful ways to extract values... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > Sent: Thursday, March 20, 2008 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > wow! > %msg:F,32:2% works! > i completely misunderstood how Fielf works...i thought the first > number was the FromChar and second the ToChar... > Thanks a lot!!! > > 2008/3/20, Rainer Gerhards : > > Should work with fields (much faster). I can't try it out due to relp > > work, but try: > > > > %msg:F,32:2% [32 is USASCII SP, the delimiter here] > > > > But maybe %msg:F,32:1% - you need to experiment a bit. In any case, > that > > should work... > > > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > Sent: Thursday, March 20, 2008 3:59 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > yes! but actually there is a space at the beginning and hostname > can > > > contain the dash -, numbers, and letters. > > > > > > 2008/3/20, Rainer Gerhards : > > > > Let me try to avoid the regexp (its expensive and I can not > debug it > > > now > > > > ;)): so you search for the string that is at the start of the > msg > > > and > > > > delimited by the first space? > > > > > > > > > > > > > > > > Rainer > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Maurizio Rottin > > > > > > > > > Sent: Thursday, March 20, 2008 3:47 PM > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] rsyslog with apache and per vhost log > > > > > > > > > > > > > > 2008/3/20, Rainer Gerhards : > > > > > > Can you send me a handful of the logline to play with? > Probably > > > not > > > > > this > > > > > > week, but next... > > > > > > > > > > > > > > > > www.mysite.com 192.168.242.2 [20/Mar/2008:15:41:10 +0100] > "GET > > > > > /images/wm001.jpg HTTP/1.1" 304 - > > > "http://www.mysite.com/webmail.htm" > > > > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > > > Gecko/20060607" > > > > > > > > > > i'm trying to use the regexp but with no success in this way: > > > > > $template MsgFormat,"%msg%\n" > > > > > $template ApacheRemoteCustom,"/var/log/apachelog/%msg:R:^\ > > > > > [a-z,\.]*--end%_az.log" > > > > > local6.info -?ApacheRemoteCustom;MsgFormat > > > > > > > > > > from the documentation: "the property replacer will return > the > > > part of > > > > > the property text that matches the regular expression" which > > > should be > > > > > " www.mysite.com" > > > > > but i get a file named _az.log > > > > > > > > > > -- > > > > > mr > > > > > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > > > > > > -- > > > mr > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > -- > mr > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 06:47:10 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 07:47:10 +0200 Subject: [rsyslog] Property-Based Filters Message-ID: <47E49D5E.5060408@pengooin.net> Hi, I'm having trouble setting up some properties-based filters. I want to see if some property is equal to "" (empty). I'm doing this: :FROMHOST, isequal, "" *.info;mail.none;authpriv.none;cron.none ?messages-per-host4 But it still matches everything it should not. Am i doing something wrong? Thanks, Radu From rgerhards at hq.adiscon.com Sat Mar 22 10:26:47 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 10:26:47 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E49D5E.5060408@pengooin.net> References: <47E49D5E.5060408@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> Hi Radu, I will look into the issue, smells like a bug. But FROMHOST can never be empty... Is it just a sample? If so, which property you are looking it (I wonder what may be empty, thus the question...). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:47 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Property-Based Filters > > Hi, > > I'm having trouble setting up some properties-based filters. I want to > see if some property is equal to "" (empty). > I'm doing this: > :FROMHOST, isequal, "" > *.info;mail.none;authpriv.none;cron.none ?messages-per- > host4 > > But it still matches everything it should not. Am i doing something > wrong? > > Thanks, > Radu > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:03:28 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:03:28 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> References: <47E49D5E.5060408@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> Message-ID: <47E4F590.8010306@pengooin.net> Hi, I'm using stable. I'm trying to create some central log machine. I made some tests .. and it seems that when i start rsyslog on the central logging machine, it creates some messages regarding rsyslog version, for those messages FROMHOST is empty. If you want i can provide you with full rsyslog.conf. Please test and reply. Thanks, Radu Gheorghiu Rainer Gerhards wrote: > Hi Radu, > > I will look into the issue, smells like a bug. But FROMHOST can never be > empty... Is it just a sample? If so, which property you are looking it > (I wonder what may be empty, thus the question...). > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:47 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] Property-Based Filters >> >> Hi, >> >> I'm having trouble setting up some properties-based filters. I want to >> see if some property is equal to "" (empty). >> I'm doing this: >> :FROMHOST, isequal, "" >> *.info;mail.none;authpriv.none;cron.none ?messages-per- >> host4 >> >> But it still matches everything it should not. Am i doing something >> wrong? >> >> Thanks, >> Radu >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 13:06:00 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 13:06:00 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E4F590.8010306@pengooin.net> References: <47E49D5E.5060408@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> <47E4F590.8010306@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> > Hi, > I'm using stable. I'm trying to create some central log machine. > I made some tests .. and it seems that when i start rsyslog on the > central logging machine, it creates some messages regarding rsyslog > version, for those messages FROMHOST is empty. That *is* a bug. FROMHOST should not be empty. And now that I know it'll probably won't be empty in the future ;) > If you want i can > provide > you with full rsyslog.conf. Please test and reply. So you want to filter out the rsyslog startup and shutdown messages? Please provide me a few samples of what the messages look in your log files. Thanks, Rainer > > Thanks, > Radu Gheorghiu > > Rainer Gerhards wrote: > > Hi Radu, > > > > I will look into the issue, smells like a bug. But FROMHOST can never > be > > empty... Is it just a sample? If so, which property you are looking > it > > (I wonder what may be empty, thus the question...). > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 6:47 AM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] Property-Based Filters > >> > >> Hi, > >> > >> I'm having trouble setting up some properties-based filters. I want > to > >> see if some property is equal to "" (empty). > >> I'm doing this: > >> :FROMHOST, isequal, "" > >> *.info;mail.none;authpriv.none;cron.none ?messages- > per- > >> host4 > >> > >> But it still matches everything it should not. Am i doing something > >> wrong? > >> > >> Thanks, > >> Radu > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:21:44 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:21:44 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> References: <47E49D5E.5060408@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C28@grfint2.intern.adiscon.com> <47E4F590.8010306@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C29@grfint2.intern.adiscon.com> Message-ID: <47E4F9D8.1040004@pengooin.net> Hi, Well as is said i needed a central log solution. So i did this: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" and for testing purposes (and debug): $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" And here is the rest: *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName i look in /var/log/hosts: 2008 clog-he-de messages clog-he-de is the localhost .. and 2008 + messages is created because FROMHOST is empty. proof: cat messages TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart note the "dd" after "TheHost:" . i put it there to be sure there's nothing between the two "d". Thanks, Radu Gheorghiu Rainer Gerhards wrote: >> Hi, >> I'm using stable. I'm trying to create some central log machine. >> I made some tests .. and it seems that when i start rsyslog on the >> central logging machine, it creates some messages regarding rsyslog >> version, for those messages FROMHOST is empty. >> > > That *is* a bug. FROMHOST should not be empty. And now that I know it'll > probably won't be empty in the future ;) > > >> If you want i can >> provide >> you with full rsyslog.conf. Please test and reply. >> > > So you want to filter out the rsyslog startup and shutdown messages? > Please provide me a few samples of what the messages look in your log > files. > > Thanks, > Rainer > > >> Thanks, >> Radu Gheorghiu >> >> Rainer Gerhards wrote: >> >>> Hi Radu, >>> >>> I will look into the issue, smells like a bug. But FROMHOST can >>> > never > >> be >> >>> empty... Is it just a sample? If so, which property you are looking >>> >> it >> >>> (I wonder what may be empty, thus the question...). >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 6:47 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Property-Based Filters >>>> >>>> Hi, >>>> >>>> I'm having trouble setting up some properties-based filters. I want >>>> >> to >> >>>> see if some property is equal to "" (empty). >>>> I'm doing this: >>>> :FROMHOST, isequal, "" >>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>> >> per- >> >>>> host4 >>>> >>>> But it still matches everything it should not. Am i doing something >>>> wrong? >>>> >>>> Thanks, >>>> Radu >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 13:46:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 13:46:51 +0100 Subject: [rsyslog] Property-Based Filters Message-ID: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> Ah, ok, so the problem actually is that fromhost is empty... ----- Urspr?ngliche Nachricht ----- Von: "Radu Gheorghiu" An: "rsyslog-users" Gesendet: 22.03.08 13:21 Betreff: Re: [rsyslog] Property-Based Filters Hi, Well as is said i needed a central log solution. So i did this: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" and for testing purposes (and debug): $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" And here is the rest: *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName i look in /var/log/hosts: 2008 clog-he-de messages clog-he-de is the localhost .. and 2008 + messages is created because FROMHOST is empty. proof: cat messages TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart note the "dd" after "TheHost:" . i put it there to be sure there's nothing between the two "d". Thanks, Radu Gheorghiu Rainer Gerhards wrote: >> Hi, >> I'm using stable. I'm trying to create some central log machine. >> I made some tests .. and it seems that when i start rsyslog on the >> central logging machine, it creates some messages regarding rsyslog >> version, for those messages FROMHOST is empty. >> > > That *is* a bug. FROMHOST should not be empty. And now that I know it'll > probably won't be empty in the future ;) > > >> If you want i can >> provide >> you with full rsyslog.conf. Please test and reply. >> > > So you want to filter out the rsyslog startup and shutdown messages? > Please provide me a few samples of what the messages look in your log > files. > > Thanks, > Rainer > > >> Thanks, >> Radu Gheorghiu >> >> Rainer Gerhards wrote: >> >>> Hi Radu, >>> >>> I will look into the issue, smells like a bug. But FROMHOST can >>> > never > >> be >> >>> empty... Is it just a sample? If so, which property you are looking >>> >> it >> >>> (I wonder what may be empty, thus the question...). >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 6:47 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Property-Based Filters >>>> >>>> Hi, >>>> >>>> I'm having trouble setting up some properties-based filters. I want >>>> >> to >> >>>> see if some property is equal to "" (empty). >>>> I'm doing this: >>>> :FROMHOST, isequal, "" >>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>> >> per- >> >>>> host4 >>>> >>>> But it still matches everything it should not. Am i doing something >>>> wrong? >>>> >>>> Thanks, >>>> Radu >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 13:54:43 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 14:54:43 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> Message-ID: <47E50193.4020904@pengooin.net> Yes, it is. And that wouldn't be a problem if property based filter would work fine. I tried to filter messages with FROMHOST empty and use HOSTNAME instead. It worked for localhost generated messages, but didn't work for messages coming from other host on the network. Here's sample: $template messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" $template messages-per-host3,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%-%$DAY%/messages" $template messages-per-host4,"/var/log/hosts/%HOSTNAME%/messages" $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: %msg%\n" # first we check if FROMHOST is empty . if it is empty .. we use HOSTNAME :FROMHOST, isequal, "" *.info;mail.none;authpriv.none;cron.none ?messages-per-host4;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host3;MyTemplateName # we drop the messages with FROMHOST empty, and we log everything else based on FROMHOST . :FROMHOST, isequal, "" ~ *.info;mail.none;authpriv.none;cron.none ?messages-per-host1;MyTemplateName *.info;mail.none;authpriv.none;cron.none ?messages-per-host2;MyTemplateName Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? Well.. it looks like there are several messages that don't contain the HOSTNAME field and this breaks everything. I'm not sure these are all bugs. They may be some result of some human error of some kind (my error). Waiting for your confirmation of the above, Radu Gheorghiu Rainer Gerhards wrote: > Ah, ok, so the problem actually is that fromhost is empty... > > ----- Urspr?ngliche Nachricht ----- > Von: "Radu Gheorghiu" > An: "rsyslog-users" > Gesendet: 22.03.08 13:21 > Betreff: Re: [rsyslog] Property-Based Filters > > Hi, > > Well as is said i needed a central log solution. > So i did this: > > $template > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%-%$DAY%/messages" > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > > and for testing purposes (and debug): > > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: > %msg%\n" > > And here is the rest: > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host1;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host2;MyTemplateName > > i look in /var/log/hosts: > 2008 clog-he-de messages > clog-he-de is the localhost .. > and 2008 + messages is created because FROMHOST is empty. > proof: > > cat messages > TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" > x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo > udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] restart > > note the "dd" after "TheHost:" . i put it there to be sure there's > nothing between the two "d". > > Thanks, > Radu Gheorghiu > > Rainer Gerhards wrote: > >>> Hi, >>> I'm using stable. I'm trying to create some central log machine. >>> I made some tests .. and it seems that when i start rsyslog on the >>> central logging machine, it creates some messages regarding rsyslog >>> version, for those messages FROMHOST is empty. >>> >>> >> That *is* a bug. FROMHOST should not be empty. And now that I know it'll >> probably won't be empty in the future ;) >> >> >> >>> If you want i can >>> provide >>> you with full rsyslog.conf. Please test and reply. >>> >>> >> So you want to filter out the rsyslog startup and shutdown messages? >> Please provide me a few samples of what the messages look in your log >> files. >> >> Thanks, >> Rainer >> >> >> >>> Thanks, >>> Radu Gheorghiu >>> >>> Rainer Gerhards wrote: >>> >>> >>>> Hi Radu, >>>> >>>> I will look into the issue, smells like a bug. But FROMHOST can >>>> >>>> >> never >> >> >>> be >>> >>> >>>> empty... Is it just a sample? If so, which property you are looking >>>> >>>> >>> it >>> >>> >>>> (I wonder what may be empty, thus the question...). >>>> >>>> Rainer >>>> >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>>> Sent: Saturday, March 22, 2008 6:47 AM >>>>> To: rsyslog at lists.adiscon.com >>>>> Subject: [rsyslog] Property-Based Filters >>>>> >>>>> Hi, >>>>> >>>>> I'm having trouble setting up some properties-based filters. I want >>>>> >>>>> >>> to >>> >>> >>>>> see if some property is equal to "" (empty). >>>>> I'm doing this: >>>>> :FROMHOST, isequal, "" >>>>> *.info;mail.none;authpriv.none;cron.none ?messages- >>>>> >>>>> >>> per- >>> >>> >>>>> host4 >>>>> >>>>> But it still matches everything it should not. Am i doing something >>>>> wrong? >>>>> >>>>> Thanks, >>>>> Radu >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From rgerhards at hq.adiscon.com Sat Mar 22 14:07:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 14:07:46 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50193.4020904@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2A@grfint2.intern.adiscon.com> Ahh... That sample is most helpful. I think there is also a misunderstanding. I couldn't run a lab yet and will probably not before after easter, but (read inline below) ... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 1:55 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Yes, it is. > And that wouldn't be a problem if property based filter would work > fine. > > I tried to filter messages with FROMHOST empty and use HOSTNAME > instead. > It worked for localhost generated messages, but didn't work for > messages > coming from other host on the network. Here's sample: > > $template > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > $template > messages-per-host3,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > $template messages-per-host4,"/var/log/hosts/%HOSTNAME%/messages" > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% TheMsg: > %msg%\n" > > # first we check if FROMHOST is empty . if it is empty .. we use > HOSTNAME > :FROMHOST, isequal, "" Property based filters are just regular filters. So you need to specify what shall happen when the filter matches. In the above line, there is no action (sorry, looks like I overlooked that in the first message you send). So to discard these message, you'd need to do: :FROMHOST, isequal, "" ~ Filters (in v2) do NOT combine, except for the BSD-style filters. This is a feature of v3. > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host4;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host3;MyTemplateName > # we drop the messages with FROMHOST empty, and we log everything else > based on FROMHOST . > :FROMHOST, isequal, "" ~ That, of course, should work... > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host1;MyTemplateName > *.info;mail.none;authpriv.none;cron.none > ?messages-per-host2;MyTemplateName > > Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? > Well.. it looks like there are several messages that don't contain the > HOSTNAME field and this breaks everything. > > I'm not sure these are all bugs. At may very well be. I suspect that they all have the same root cause, and that is that some message properties are not correctly being set for internally-generated messages. So one trouble spot with multiple problems resulting from it. Could you do me a favor and run rsyslogd with -d -n options interactively and send me the resulting debug log? Rainer > They may be some result of some human > error of some kind (my error). > > Waiting for your confirmation of the above, > Radu Gheorghiu > > Rainer Gerhards wrote: > > Ah, ok, so the problem actually is that fromhost is empty... > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "Radu Gheorghiu" > > An: "rsyslog-users" > > Gesendet: 22.03.08 13:21 > > Betreff: Re: [rsyslog] Property-Based Filters > > > > Hi, > > > > Well as is said i needed a central log solution. > > So i did this: > > > > $template > > messages-per-host1,"/var/log/hosts/%FROMHOST%/%$YEAR%/%$MONTH%- > %$DAY%/messages" > > $template messages-per-host2,"/var/log/hosts/%FROMHOST%/messages" > > > > and for testing purposes (and debug): > > > > $template MyTemplateName,"TheHost:d%FROMHOST%d %syslogseverity% > TheMsg: > > %msg%\n" > > > > And here is the rest: > > *.info;mail.none;authpriv.none;cron.none > > ?messages-per-host1;MyTemplateName > > *.info;mail.none;authpriv.none;cron.none > > ?messages-per-host2;MyTemplateName > > > > i look in /var/log/hosts: > > 2008 clog-he-de messages > > clog-he-de is the localhost .. > > and 2008 + messages is created because FROMHOST is empty. > > proof: > > > > cat messages > > TheHost:dd 6 TheMsg: [origin software="rsyslogd" swVersion="2.0.3" > > x-pid="12215" x-info="http://www.rsyslog.com"][x-configInfo > > udpReception="Yes" udpPort="514" tcpReception="No" tcpPort="0"] > restart > > > > note the "dd" after "TheHost:" . i put it there to be sure there's > > nothing between the two "d". > > > > Thanks, > > Radu Gheorghiu > > > > Rainer Gerhards wrote: > > > >>> Hi, > >>> I'm using stable. I'm trying to create some central log machine. > >>> I made some tests .. and it seems that when i start rsyslog on the > >>> central logging machine, it creates some messages regarding rsyslog > >>> version, for those messages FROMHOST is empty. > >>> > >>> > >> That *is* a bug. FROMHOST should not be empty. And now that I know > it'll > >> probably won't be empty in the future ;) > >> > >> > >> > >>> If you want i can > >>> provide > >>> you with full rsyslog.conf. Please test and reply. > >>> > >>> > >> So you want to filter out the rsyslog startup and shutdown messages? > >> Please provide me a few samples of what the messages look in your > log > >> files. > >> > >> Thanks, > >> Rainer > >> > >> > >> > >>> Thanks, > >>> Radu Gheorghiu > >>> > >>> Rainer Gerhards wrote: > >>> > >>> > >>>> Hi Radu, > >>>> > >>>> I will look into the issue, smells like a bug. But FROMHOST can > >>>> > >>>> > >> never > >> > >> > >>> be > >>> > >>> > >>>> empty... Is it just a sample? If so, which property you are > looking > >>>> > >>>> > >>> it > >>> > >>> > >>>> (I wonder what may be empty, thus the question...). > >>>> > >>>> Rainer > >>>> > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >>>>> Sent: Saturday, March 22, 2008 6:47 AM > >>>>> To: rsyslog at lists.adiscon.com > >>>>> Subject: [rsyslog] Property-Based Filters > >>>>> > >>>>> Hi, > >>>>> > >>>>> I'm having trouble setting up some properties-based filters. I > want > >>>>> > >>>>> > >>> to > >>> > >>> > >>>>> see if some property is equal to "" (empty). > >>>>> I'm doing this: > >>>>> :FROMHOST, isequal, "" > >>>>> *.info;mail.none;authpriv.none;cron.none > ?messages- > >>>>> > >>>>> > >>> per- > >>> > >>> > >>>>> host4 > >>>>> > >>>>> But it still matches everything it should not. Am i doing > something > >>>>> wrong? > >>>>> > >>>>> Thanks, > >>>>> Radu > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> > >>>>> > >>>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > >> > >> > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 22 14:14:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 14:14:55 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50193.4020904@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> > Now you may ask why i didn't use HOSTNAME and only HOSTNAME after all? > Well.. it looks like there are several messages that don't contain the > HOSTNAME field and this breaks everything. I had a quick look at the code and I can't see why HOSTNAME should not be filled. Could you provide me a sample of such a message? That would be most interesting. I think I've also identified why FROMHOST is missing, but I'd like to evaluate it a bit more. If you can provide a sample of missing hostnames, it would very helpful to have a debug log from such a message being received and processed - but the most important thing is to see what the source of this all is... Thanks, Rainer From radu at pengooin.net Sat Mar 22 14:47:39 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 15:47:39 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> Message-ID: <47E50DFB.40505@pengooin.net> I don't think that the HOSTNAME problem is a rsyslog problem. I think that the remote host is not sending the HOSTNAME. Remote host is using the classic syslog . and it is configured like this: *.* @central-logger on the central logger .. i modify my template for debug: $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% TheMsg: %msg%\n" and in the logs: TheHost:dlastd 5 TheMsg: repeated 8 times Seems like the message had no hostname field. and rsyslog thinks "last" is the hostname. Radu From rgerhards at hq.adiscon.com Sat Mar 22 18:09:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:09:04 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E50DFB.40505@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> Well... Rsyslog tries all kind of things to get hold of the real host. So if you could send me a copy of the one causing problems, I may (may ;)) be able to do something against it. HOSTNAME should always contain something usable, but as you say ... it depends ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 2:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > I don't think that the HOSTNAME problem is a rsyslog problem. > I think that the remote host is not sending the HOSTNAME. > Remote host is using the classic syslog . > and it is configured like this: > *.* @central-logger > > on the central logger .. i modify my template for debug: > $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% TheMsg: > %msg%\n" > > and in the logs: > TheHost:dlastd 5 TheMsg: repeated 8 times > > Seems like the message had no hostname field. and rsyslog thinks > "last" > is the hostname. > > Radu > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 18:13:33 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:13:33 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> Message-ID: <47E53E3D.90606@pengooin.net> Hmm i think i don't understand . Haven't you received the file that i sent you? Or did i miss something? Radu Rainer Gerhards wrote: > Well... Rsyslog tries all kind of things to get hold of the real host. > So if you could send me a copy of the one causing problems, I may (may > ;)) be able to do something against it. HOSTNAME should always contain > something usable, but as you say ... it depends ;) > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 2:48 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> I don't think that the HOSTNAME problem is a rsyslog problem. >> I think that the remote host is not sending the HOSTNAME. >> Remote host is using the classic syslog . >> and it is configured like this: >> *.* >> > @central-logger > >> on the central logger .. i modify my template for debug: >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >> > TheMsg: > >> %msg%\n" >> >> and in the logs: >> TheHost:dlastd 5 TheMsg: repeated 8 times >> >> Seems like the message had no hostname field. and rsyslog thinks >> "last" >> is the hostname. >> >> Radu >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 18:17:27 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:17:27 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E53E3D.90606@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> Replying as I can ;) I've not been able to go in-depth through the file, but I ... doh ;) I guess it's all sitting in there. Lol - sorry. As an apology, here is the FROMHOST bugfix ;) http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2. 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Hmm i think i don't understand . Haven't you received the file that i > sent you? Or did i miss something? > > Radu > > Rainer Gerhards wrote: > > Well... Rsyslog tries all kind of things to get hold of the real > host. > > So if you could send me a copy of the one causing problems, I may > (may > > ;)) be able to do something against it. HOSTNAME should always > contain > > something usable, but as you say ... it depends ;) > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 2:48 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> I don't think that the HOSTNAME problem is a rsyslog problem. > >> I think that the remote host is not sending the HOSTNAME. > >> Remote host is using the classic syslog . > >> and it is configured like this: > >> *.* > >> > > @central-logger > > > >> on the central logger .. i modify my template for debug: > >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >> > > TheMsg: > > > >> %msg%\n" > >> > >> and in the logs: > >> TheHost:dlastd 5 TheMsg: repeated 8 times > >> > >> Seems like the message had no hostname field. and rsyslog thinks > >> "last" > >> is the hostname. > >> > >> Radu > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Sat Mar 22 18:25:19 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:25:19 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E53E3D.90606@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> Ah, I've now reviewed it, but there is no message in it from a host that does not send a HOSTNAME (The last message... case). What syslogd are your running on the clients? Sysklogd? And which version? I am asking because I could check the code and see what exactly it generates. I already begin to get some bad feelings about what it sends ;) What I see is rsyslog's bug (the one just fixed) and an error message telling you that rsyslog is discarding a selector line because of no actions. That is the one where you had use the filter but without actions. Filters work only in front of actions in v2. So if you don't provide an action, nothing happens except that startup error message. Please note that the doc just talks about the property based filter. But it doesn't mean you can use it without an action. Filters only work with actions. In v3, things are already a bit different and will be much more different soon. V3 offers full expression support, so you can do Boolean operations inside filters. Also, v3 will be scriptable. HTH, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Hmm i think i don't understand . Haven't you received the file that i > sent you? Or did i miss something? > > Radu > > Rainer Gerhards wrote: > > Well... Rsyslog tries all kind of things to get hold of the real > host. > > So if you could send me a copy of the one causing problems, I may > (may > > ;)) be able to do something against it. HOSTNAME should always > contain > > something usable, but as you say ... it depends ;) > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 2:48 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> I don't think that the HOSTNAME problem is a rsyslog problem. > >> I think that the remote host is not sending the HOSTNAME. > >> Remote host is using the classic syslog . > >> and it is configured like this: > >> *.* > >> > > @central-logger > > > >> on the central logger .. i modify my template for debug: > >> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >> > > TheMsg: > > > >> %msg%\n" > >> > >> and in the logs: > >> TheHost:dlastd 5 TheMsg: repeated 8 times > >> > >> Seems like the message had no hostname field. and rsyslog thinks > >> "last" > >> is the hostname. > >> > >> Radu > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 18:32:42 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:32:42 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> Message-ID: <47E542BA.4040503@pengooin.net> Ok thanks :) I only wanted to know if you received it . But if you also fixed it, it's even better :) Rainer Gerhards wrote: > Replying as I can ;) I've not been able to go in-depth through the file, > but I ... doh ;) I guess it's all sitting in there. Lol - sorry. > > As an apology, here is the FROMHOST bugfix ;) > > http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2. > 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> Hmm i think i don't understand . Haven't you received the file that i >> sent you? Or did i miss something? >> >> Radu >> >> Rainer Gerhards wrote: >> >>> Well... Rsyslog tries all kind of things to get hold of the real >>> >> host. >> >>> So if you could send me a copy of the one causing problems, I may >>> >> (may >> >>> ;)) be able to do something against it. HOSTNAME should always >>> >> contain >> >>> something usable, but as you say ... it depends ;) >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 2:48 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Property-Based Filters >>>> >>>> I don't think that the HOSTNAME problem is a rsyslog problem. >>>> I think that the remote host is not sending the HOSTNAME. >>>> Remote host is using the classic syslog . >>>> and it is configured like this: >>>> *.* >>>> >>>> >>> @central-logger >>> >>> >>>> on the central logger .. i modify my template for debug: >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >>>> >>>> >>> TheMsg: >>> >>> >>>> %msg%\n" >>>> >>>> and in the logs: >>>> TheHost:dlastd 5 TheMsg: repeated 8 times >>>> >>>> Seems like the message had no hostname field. and rsyslog thinks >>>> "last" >>>> is the hostname. >>>> >>>> Radu >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From radu at pengooin.net Sat Mar 22 18:34:09 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 19:34:09 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2E@grfint2.intern.adiscon.com> Message-ID: <47E54311.1080502@pengooin.net> I will prepare a new file + details, in the next 30mins :) Rainer Gerhards wrote: > Ah, I've now reviewed it, but there is no message in it from a host that > does not send a HOSTNAME (The last message... case). What syslogd are > your running on the clients? Sysklogd? And which version? I am asking > because I could check the code and see what exactly it generates. I > already begin to get some bad feelings about what it sends ;) > > What I see is rsyslog's bug (the one just fixed) and an error message > telling you that rsyslog is discarding a selector line because of no > actions. That is the one where you had use the filter but without > actions. > > Filters work only in front of actions in v2. So if you don't provide an > action, nothing happens except that startup error message. Please note > that the doc just talks about the property based filter. But it doesn't > mean you can use it without an action. Filters only work with actions. > In v3, things are already a bit different and will be much more > different soon. V3 offers full expression support, so you can do > Boolean operations inside filters. Also, v3 will be scriptable. > > HTH, > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >> Sent: Saturday, March 22, 2008 6:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Property-Based Filters >> >> Hmm i think i don't understand . Haven't you received the file that i >> sent you? Or did i miss something? >> >> Radu >> >> Rainer Gerhards wrote: >> >>> Well... Rsyslog tries all kind of things to get hold of the real >>> >> host. >> >>> So if you could send me a copy of the one causing problems, I may >>> >> (may >> >>> ;)) be able to do something against it. HOSTNAME should always >>> >> contain >> >>> something usable, but as you say ... it depends ;) >>> >>> Rainer >>> >>> >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu >>>> Sent: Saturday, March 22, 2008 2:48 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] Property-Based Filters >>>> >>>> I don't think that the HOSTNAME problem is a rsyslog problem. >>>> I think that the remote host is not sending the HOSTNAME. >>>> Remote host is using the classic syslog . >>>> and it is configured like this: >>>> *.* >>>> >>>> >>> @central-logger >>> >>> >>>> on the central logger .. i modify my template for debug: >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% >>>> >>>> >>> TheMsg: >>> >>> >>>> %msg%\n" >>>> >>>> and in the logs: >>>> TheHost:dlastd 5 TheMsg: repeated 8 times >>>> >>>> Seems like the message had no hostname field. and rsyslog thinks >>>> "last" >>>> is the hostname. >>>> >>>> Radu >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > > From rgerhards at hq.adiscon.com Sat Mar 22 18:34:43 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 22 Mar 2008 18:34:43 +0100 Subject: [rsyslog] Property-Based Filters In-Reply-To: <47E542BA.4040503@pengooin.net> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> <47E542BA.4040503@pengooin.net> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> No problem at all. The fix was rather trivial once I knew what I was looking for. Please let me know if it works for you. The original bug report - property based filters - however, was no bug, but a misunderstanding on how they work. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > Sent: Saturday, March 22, 2008 6:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] Property-Based Filters > > Ok thanks :) I only wanted to know if you received it . But if you also > fixed it, it's even better :) > > Rainer Gerhards wrote: > > Replying as I can ;) I've not been able to go in-depth through the > file, > > but I ... doh ;) I guess it's all sitting in there. Lol - sorry. > > > > As an apology, here is the FROMHOST bugfix ;) > > > > > http://rsyslog.cvs.sourceforge.net/rsyslog/rsyslog/syslogd.c?r1=1.380.2 > . > > 21&r2=1.380.2.22&pathrev=Branch_Stable_2-0 > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >> Sent: Saturday, March 22, 2008 6:14 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Property-Based Filters > >> > >> Hmm i think i don't understand . Haven't you received the file that > i > >> sent you? Or did i miss something? > >> > >> Radu > >> > >> Rainer Gerhards wrote: > >> > >>> Well... Rsyslog tries all kind of things to get hold of the real > >>> > >> host. > >> > >>> So if you could send me a copy of the one causing problems, I may > >>> > >> (may > >> > >>> ;)) be able to do something against it. HOSTNAME should always > >>> > >> contain > >> > >>> something usable, but as you say ... it depends ;) > >>> > >>> Rainer > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghiu > >>>> Sent: Saturday, March 22, 2008 2:48 PM > >>>> To: rsyslog-users > >>>> Subject: Re: [rsyslog] Property-Based Filters > >>>> > >>>> I don't think that the HOSTNAME problem is a rsyslog problem. > >>>> I think that the remote host is not sending the HOSTNAME. > >>>> Remote host is using the classic syslog . > >>>> and it is configured like this: > >>>> *.* > >>>> > >>>> > >>> @central-logger > >>> > >>> > >>>> on the central logger .. i modify my template for debug: > >>>> $template MyTemplateName,"TheHost:d%HOSTNAME%d %syslogseverity% > >>>> > >>>> > >>> TheMsg: > >>> > >>> > >>>> %msg%\n" > >>>> > >>>> and in the logs: > >>>> TheHost:dlastd 5 TheMsg: repeated 8 times > >>>> > >>>> Seems like the message had no hostname field. and rsyslog thinks > >>>> "last" > >>>> is the hostname. > >>>> > >>>> Radu > >>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From radu at pengooin.net Sat Mar 22 19:41:38 2008 From: radu at pengooin.net (Radu Gheorghiu) Date: Sat, 22 Mar 2008 20:41:38 +0200 Subject: [rsyslog] Property-Based Filters In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> References: <001501c88c1a$cf025ef8$060013ac@intern.adiscon.com> <47E50193.4020904@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2B@grfint2.intern.adiscon.com> <47E50DFB.40505@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2C@grfint2.intern.adiscon.com> <47E53E3D.90606@pengooin.net><577465F99B41C842AAFBE9ED71E70ABA308C2D@grfint2.intern.adiscon.com> <47E542BA.4040503@pengooin.net> <577465F99B41C842AAFBE9ED71E70ABA308C2F@grfint2.intern.adiscon.com> Message-ID: <47E552E2.3040907@pengooin.net> I just applied the patch and the problem with the empty HOSTNAME seems to be solved now. Congrats :) And thank you for your effort. Radu Rainer Gerhards wrote: > No problem at all. The fix was rather trivial once I knew what I was > looking for. Please let me know if it works for you. The original bug > report - property based filters - however, was no bug, but a > misunderstanding on how they work. > > Rainer > > From rgerhards at hq.adiscon.com Mon Mar 24 22:25:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 24 Mar 2008 22:25:55 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803181523.52321.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308BDA@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> Hi all, I now have a use case where the "last message..." prevents sources from flooding syslog. But now on to a different use case: does someone actually use this feature to reduce the size of files or network traffic? Except, of course, in cases where there is a message flood due to either attact or a misbehaving source (these need to be treated differentely - I am right now after the *destination* message compression...). Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > Sent: Tuesday, March 18, 2008 3:24 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested "Last message > logged n times"... > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > The only arguments for keeping the feature that I got on my > lug was the > > preservartion of disk/network IO. > > > > I think to prevent DOS attacks is a valid argument but as > you said can > > be easily circumvented by randomizing messages. > I'm afraid it's not true in all cases. What if you do DOS > attach not directly > to do rsyslog, but via other daemon. In situation when you > can't send message > directly to syslog, but you can make some daemon generate > message for you. > This message would be probably always the same content. > > > To safeguard against dos attacks you could have a monitor > that monitors > > for extra ordinary amount of traffic and then generate a snmp trap. > > Whether that should be a rsyslog plugin or part of other software is > > open to debate. > > > > Regards > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > Sent: 18 March 2008 11:42 > > > To: rsyslog-users > > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > > > times"... > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > I have asked the question on a local lug to get some more > > > > opinions. > > > > > I > > > > > > > > will post the results if any here. > > > > > > > > I did the same. :) > > > > > > > > The only arguments against removal I have received > until now were: > > > > > > > > 1. DoS (but nobody explained how) > > > > > > I think this is along the lines of making it easy to flood the log > > > > with > > > > > similar messages. If that's the case, I think it is a (too) weak > > > argument because an attacker could easily include a random pattern > > > inside the message. > > > > > > HOWEVER, the current logic indeed prevents flooding the > log from equal > > > messages, e.g. if a process runs wild. > > > > > > > 2. cleanness of logs (relative) > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > Excellent. > > > > > > Maybe there are also alternatives to the current > behavior, something > > > > in > > > > > between. Right now, the core engine does this, so the reduction > > > capability is inherent to every action (except if an action > > > > specifically > > > > > forbids it, because it can not intelligently handle it - > e.g. database > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > alternative would be to remove it from the core engine and move it > > > > into > > > > > the *file write* output plugin (omfile). This has its > subtleties, too, > > > so I don't like to take that approach lightly. Plus, it than means > > > > that > > > > > the network may become spammed. I think one core requirement is to > > > > find > > > > > people who actually *intentionally* use this feature and > ask for their > > > reasoning. That will probably be the best way to tell us > if its really > > > needed. And, of course, we need to weigh the negative > effects of it > > > against these con-points. > > > > > > Rainer > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From pvrabec at redhat.com Tue Mar 25 10:46:55 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Tue, 25 Mar 2008 10:46:55 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> Message-ID: <200803251046.55919.pvrabec@redhat.com> Hi folks, I talked with some people, and nobody is happy with removing this thing from rsyslog. Won't be possible to move it to separate plug-in? On Monday 24 March 2008 10:25:55 pm Rainer Gerhards wrote: > Hi all, > > I now have a use case where the "last message..." prevents sources from > flooding syslog. But now on to a different use case: does someone > actually use this feature to reduce the size of files or network > traffic? Except, of course, in cases where there is a message flood due > to either attact or a misbehaving source (these need to be treated > differentely - I am right now after the *destination* message > compression...). > > Thanks, > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter Vrabec > > Sent: Tuesday, March 18, 2008 3:24 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Feedback requested "Last message > > logged n times"... > > > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > > The only arguments for keeping the feature that I got on my > > > > lug was the > > > > > preservartion of disk/network IO. > > > > > > I think to prevent DOS attacks is a valid argument but as > > > > you said can > > > > > be easily circumvented by randomizing messages. > > > > I'm afraid it's not true in all cases. What if you do DOS > > attach not directly > > to do rsyslog, but via other daemon. In situation when you > > can't send message > > directly to syslog, but you can make some daemon generate > > message for you. > > This message would be probably always the same content. > > > > > To safeguard against dos attacks you could have a monitor > > > > that monitors > > > > > for extra ordinary amount of traffic and then generate a snmp trap. > > > Whether that should be a rsyslog plugin or part of other software is > > > open to debate. > > > > > > Regards > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > Sent: 18 March 2008 11:42 > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] Feedback requested "Last message logged n > > > > > > times"... > > > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > > > I have asked the question on a local lug to get some more > > > > > > opinions. > > > > > > > I > > > > > > > > > > will post the results if any here. > > > > > > > > > > I did the same. :) > > > > > > > > > > The only arguments against removal I have received > > > > until now were: > > > > > 1. DoS (but nobody explained how) > > > > > > > > I think this is along the lines of making it easy to flood the log > > > > > > with > > > > > > > similar messages. If that's the case, I think it is a (too) weak > > > > argument because an attacker could easily include a random pattern > > > > inside the message. > > > > > > > > HOWEVER, the current logic indeed prevents flooding the > > > > log from equal > > > > > > messages, e.g. if a process runs wild. > > > > > > > > > 2. cleanness of logs (relative) > > > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > > > Excellent. > > > > > > > > Maybe there are also alternatives to the current > > > > behavior, something > > > > > in > > > > > > > between. Right now, the core engine does this, so the reduction > > > > capability is inherent to every action (except if an action > > > > > > specifically > > > > > > > forbids it, because it can not intelligently handle it - > > > > e.g. database > > > > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > > alternative would be to remove it from the core engine and move it > > > > > > into > > > > > > > the *file write* output plugin (omfile). This has its > > > > subtleties, too, > > > > > > so I don't like to take that approach lightly. Plus, it than means > > > > > > that > > > > > > > the network may become spammed. I think one core requirement is to > > > > > > find > > > > > > > people who actually *intentionally* use this feature and > > > > ask for their > > > > > > reasoning. That will probably be the best way to tell us > > > > if its really > > > > > > needed. And, of course, we need to weigh the negative > > > > effects of it > > > > > > against these con-points. > > > > > > > > Rainer > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Tue Mar 25 10:52:29 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 10:52:29 +0100 Subject: [rsyslog] Feedback requested "Last message logged n times"... In-Reply-To: <200803251046.55919.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308BD3@grfint2.intern.adiscon.com> <200803181523.52321.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C3A@grfint2.intern.adiscon.com> <200803251046.55919.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C3D@grfint2.intern.adiscon.com> Pluging... well, that depends on the use cases. I am trying to identify the actual use cases, so far I found the one with flooding/attack. The problem is that the current logic can not be moved to a plugin because it is right at the most inner workings of the engine. If we can identify the actual use cases, things will probably be much easier to work with. For example, I currently think that I can build an optional interface that enables flood-controlling inputs and reducing messages if some condition is given there. Note, however, that this is different from current logic, where the flood control only applies to some outputs, because the output decides if it can work with the repeated lines. The bottom line is that I think there are some valid use cases, we just don't know about them. The current implementation/idea is soooo bad because we don't know what we really need to do. So identifying the actual use cases is the priority and then doing a good design that covers them. I'll probably follow advise from the loganalysis mailing list and change the default to not do the "last message..." processing. Then, see who complains and if nobody does drop the feature. It would also be extremely useful to know *why* people are unhappy (uses cases ;)), instead of just knowing they are... Can you shed some light on that? Thanks for all your help, Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Tuesday, March 25, 2008 10:47 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] Feedback requested "Last message logged n > times"... > > Hi folks, > > I talked with some people, and nobody is happy with removing this thing > from > rsyslog. Won't be possible to move it to separate plug-in? > > On Monday 24 March 2008 10:25:55 pm Rainer Gerhards wrote: > > Hi all, > > > > I now have a use case where the "last message..." prevents sources > from > > flooding syslog. But now on to a different use case: does someone > > actually use this feature to reduce the size of files or network > > traffic? Except, of course, in cases where there is a message flood > due > > to either attact or a misbehaving source (these need to be treated > > differentely - I am right now after the *destination* message > > compression...). > > > > Thanks, > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com > > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Peter > Vrabec > > > Sent: Tuesday, March 18, 2008 3:24 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] Feedback requested "Last message > > > logged n times"... > > > > > > On Tuesday 18 March 2008 02:11:15 pm Gerrard Geldenhuis wrote: > > > > The only arguments for keeping the feature that I got on my > > > > > > lug was the > > > > > > > preservartion of disk/network IO. > > > > > > > > I think to prevent DOS attacks is a valid argument but as > > > > > > you said can > > > > > > > be easily circumvented by randomizing messages. > > > > > > I'm afraid it's not true in all cases. What if you do DOS > > > attach not directly > > > to do rsyslog, but via other daemon. In situation when you > > > can't send message > > > directly to syslog, but you can make some daemon generate > > > message for you. > > > This message would be probably always the same content. > > > > > > > To safeguard against dos attacks you could have a monitor > > > > > > that monitors > > > > > > > for extra ordinary amount of traffic and then generate a snmp > trap. > > > > Whether that should be a rsyslog plugin or part of other software > is > > > > open to debate. > > > > > > > > Regards > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > > > > > Sent: 18 March 2008 11:42 > > > > > To: rsyslog-users > > > > > Subject: Re: [rsyslog] Feedback requested "Last message logged > n > > > > > > > > times"... > > > > > > > > > Good discussion, thanks all :) ... some more points below ... > > > > > > > > > > > > I have asked the question on a local lug to get some more > > > > > > > > opinions. > > > > > > > > > I > > > > > > > > > > > > will post the results if any here. > > > > > > > > > > > > I did the same. :) > > > > > > > > > > > > The only arguments against removal I have received > > > > > > until now were: > > > > > > 1. DoS (but nobody explained how) > > > > > > > > > > I think this is along the lines of making it easy to flood the > log > > > > > > > > with > > > > > > > > > similar messages. If that's the case, I think it is a (too) > weak > > > > > argument because an attacker could easily include a random > pattern > > > > > inside the message. > > > > > > > > > > HOWEVER, the current logic indeed prevents flooding the > > > > > > log from equal > > > > > > > > messages, e.g. if a process runs wild. > > > > > > > > > > > 2. cleanness of logs (relative) > > > > > > > > > > > > I'm thinking about forwarding discussion to fedora-devel :) > > > > > > > > > > Excellent. > > > > > > > > > > Maybe there are also alternatives to the current > > > > > > behavior, something > > > > > > > in > > > > > > > > > between. Right now, the core engine does this, so the reduction > > > > > capability is inherent to every action (except if an action > > > > > > > > specifically > > > > > > > > > forbids it, because it can not intelligently handle it - > > > > > > e.g. database > > > > > > > > writers). So it applies to forwarding, snmp, whatever, ... One > > > > > alternative would be to remove it from the core engine and move > it > > > > > > > > into > > > > > > > > > the *file write* output plugin (omfile). This has its > > > > > > subtleties, too, > > > > > > > > so I don't like to take that approach lightly. Plus, it than > means > > > > > > > > that > > > > > > > > > the network may become spammed. I think one core requirement is > to > > > > > > > > find > > > > > > > > > people who actually *intentionally* use this feature and > > > > > > ask for their > > > > > > > > reasoning. That will probably be the best way to tell us > > > > > > if its really > > > > > > > > needed. And, of course, we need to weigh the negative > > > > > > effects of it > > > > > > > > against these con-points. > > > > > > > > > > Rainer > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > From friedl at hq.adiscon.com Tue Mar 25 11:31:39 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Tue, 25 Mar 2008 11:31:39 +0100 Subject: [rsyslog] rsyslog 3.12.4 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C41@grfint2.intern.adiscon.com> Hi all, rsyslog 3.12.4 has been released today. Thanks to a changed default, it now offers much faster log file writing. Also, advanced flow control has now been added to all inputs, giving an even more reliable system. There are some other small enhancements (see change log). There are also a number of bug fixes, the most important one being a fix for a massive memory leak in imklog. It lost several hundred kilobytes on each hup. Rsyslog 3.12.4 is a recommended update for all v3 branch users. Download: http://www.rsyslog.com/Downloads-req-getit-lid-89.phtml Changelog: http://www.rsyslog.com/Article195.phtml As always, feedback is appreciated. Florian Riedl From rgerhards at hq.adiscon.com Tue Mar 25 16:33:55 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 16:33:55 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> Hi all, a question on the forum (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a bit asking around finally made me think about the possibility of a v3 stable version, something that dangled on my mind for a while now... While it is just three month since v2 stable, we have gone a long way and during that time the new queue engine has much matured. As "development" state is most often a show-stopper for putting code into production, I think about releasing a 3.0.0 stable (based on the most recent 3.12.x version WITHOUT RELP support). >From a support perspective, I'd like to continue support v2 stable and obviously v3 stable and development concurrently. Development would continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug fixes. Some time from now, 3.. would become 3.1.0 stable. At this point, I would drop support/fixing for 3.0. and ask users with problems in the branch to update to 3.1.. Does this sound reasonable? The big problem is that I must be careful not to create too many version which I must support, but at the same time allow new, matured, features to go into broader production. And, of course, 3.0.0 stable would probably be a bit less stable than 2.0.4, but only if you use features that are not present in v2. All in all, the queue engine has received more than a fire drill and is ready for production - though probably not fully trouble free. But we won't see that until it gets more used. Feedback is appreciated. Thanks, Rainer From r.bhatia at ipax.at Tue Mar 25 17:04:10 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Tue, 25 Mar 2008 17:04:10 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> Message-ID: <47E9227A.5010903@ipax.at> Rainer Gerhards wrote: > Hi all, > > a question on the forum > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a bit > asking around finally made me think about the possibility of a v3 stable > version, something that dangled on my mind for a while now... While it > is just three month since v2 stable, we have gone a long way and during > that time the new queue engine has much matured. As "development" state > is most often a show-stopper for putting code into production, I think > about releasing a 3.0.0 stable (based on the most recent 3.12.x version > WITHOUT RELP support). > >>From a support perspective, I'd like to continue support v2 stable and > obviously v3 stable and development concurrently. Development would > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > fixes. Some time from now, 3.. would become 3.1.0 > stable. At this point, I would drop support/fixing for 3.0. > and ask users with problems in the branch to update to 3.1.. i guess this will cause serious headake because people tend to think like this: 3.0.x is less mature/was released before 3.1.x is less mature/was released before 3.12.x is less mature/was released before 3.13.x i guess there should be a deliberate versioning scheme. examples are: - versioning similar to the linux kernel - release 3.12.x as 3.12.x-stable and continue development without this tag. - something else but i guess its no good to jump from 3.12.x back to 3.0.x and from a future revision such as 3.40.x back to 3.1.x cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Tue Mar 25 17:06:52 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 17:06:52 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <47E9227A.5010903@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> Hi Raoul, OK, good feedback on the version number scheme - it has been sitting this (confusing) way all time since the early days of rsyslog... Probably a sufficiently good cleanup would be to go to 4.x and the follow a reasonable scheme from then on... But besides that: how do you feel about the facts other then the version numbering? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Tuesday, March 25, 2008 5:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.0.0 stable? - feedback, please > > Rainer Gerhards wrote: > > Hi all, > > > > a question on the forum > > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a > bit > > asking around finally made me think about the possibility of a v3 > stable > > version, something that dangled on my mind for a while now... While > it > > is just three month since v2 stable, we have gone a long way and > during > > that time the new queue engine has much matured. As "development" > state > > is most often a show-stopper for putting code into production, I > think > > about releasing a 3.0.0 stable (based on the most recent 3.12.x > version > > WITHOUT RELP support). > > > >>From a support perspective, I'd like to continue support v2 stable > and > > obviously v3 stable and development concurrently. Development would > > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > > fixes. Some time from now, 3.. would become 3.1.0 > > stable. At this point, I would drop support/fixing for 3.0. > > and ask users with problems in the branch to update to > 3.1.. > > i guess this will cause serious headake because people tend to think > like this: > > 3.0.x is less mature/was released before > 3.1.x is less mature/was released before > 3.12.x is less mature/was released before > 3.13.x > > i guess there should be a deliberate versioning scheme. examples are: > > - versioning similar to the linux kernel > - release 3.12.x as 3.12.x-stable and continue development without this > tag. > - something else > > but i guess its no good to jump from 3.12.x back to 3.0.x and from a > future revision such as 3.40.x back to 3.1.x > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Tue Mar 25 18:05:04 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Tue, 25 Mar 2008 18:05:04 +0100 Subject: [rsyslog] rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C4C@grfint2.intern.adiscon.com> Message-ID: <47E930C0.6020903@ipax.at> hi, Rainer Gerhards wrote: > OK, good feedback on the version number scheme - it has been sitting > this (confusing) way all time since the early days of rsyslog... > Probably a sufficiently good cleanup would be to go to 4.x and the > follow a reasonable scheme from then on... > > But besides that: how do you feel about the facts other then the version > numbering? sorry, i cannot quote on this as i am not using v3 as of now. this might change in the next couple of weeks but as of now we're still on version v1.x, as this version is not giving much trouble. cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Tue Mar 25 18:16:05 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 25 Mar 2008 18:16:05 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <47E9227A.5010903@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> Hi all, I have thought about the version numbering. Obviously, the old scheme (http://www.rsyslog.com/doc-version_naming.html ) is confusing and should be dropped. Rsyslog currently follows a three-number version scheme: major.minor.patchlevel I can follow an odd/even scheme on major for unstable/stable branches. However, with major features being continuously introduced may bring us to rsyslog 10.x.x by the end of the year. While I have no hard concerns, it seems to be unusual for open source to increase the major version that fast. On the other hand, I need minor.patchlevel to handle the smaller fixes and new major feature. I increase minor for each new feature (like queue engine, expression-based filters, relp, to name a few actual ones) and increase patchlevel for smaller changes. The three number scheme works well, but, again, keeps the major number very quickly growing given the current fast pace of development. An alternative would be to use politicalmajor.major.minor.patchlevel Where politicalmajor would only be incremented every now and then and actually without much technical reasoning for doing so - only then when we "feel" it is time for a new major release. It would probably turn out to be incremented around once a year, just to keep the corporate folks happy and the major version number reasonable low. Question now: which scheme should I follow? Is there any other one? If so, why should I follow that other scheme? I would appreciate quick feedback, as I'd like to settle this issue before releasing RELP support, which currently looks like some time (very) early April. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX] > Sent: Tuesday, March 25, 2008 5:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog 3.0.0 stable? - feedback, please > > Rainer Gerhards wrote: > > Hi all, > > > > a question on the forum > > (http://www.rsyslog.com/PNphpBB2-viewtopic-p-1113.phtml#1113 ) and a > bit > > asking around finally made me think about the possibility of a v3 > stable > > version, something that dangled on my mind for a while now... While > it > > is just three month since v2 stable, we have gone a long way and > during > > that time the new queue engine has much matured. As "development" > state > > is most often a show-stopper for putting code into production, I > think > > about releasing a 3.0.0 stable (based on the most recent 3.12.x > version > > WITHOUT RELP support). > > > >>From a support perspective, I'd like to continue support v2 stable > and > > obviously v3 stable and development concurrently. Development would > > continue with 3.13.0 (RELP support), and 3.0.0 would only receive bug > > fixes. Some time from now, 3.. would become 3.1.0 > > stable. At this point, I would drop support/fixing for 3.0. > > and ask users with problems in the branch to update to > 3.1.. > > i guess this will cause serious headake because people tend to think > like this: > > 3.0.x is less mature/was released before > 3.1.x is less mature/was released before > 3.12.x is less mature/was released before > 3.13.x > > i guess there should be a deliberate versioning scheme. examples are: > > - versioning similar to the linux kernel > - release 3.12.x as 3.12.x-stable and continue development without this > tag. > - something else > > but i guess its no good to jump from 3.12.x back to 3.0.x and from a > future revision such as 3.40.x back to 3.1.x > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From pvrabec at redhat.com Wed Mar 26 11:47:21 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Wed, 26 Mar 2008 11:47:21 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> Message-ID: <200803261147.21726.pvrabec@redhat.com> Hi, politicalmajor.major.minor.patchlevel seems good to me. politicalmajor - something great happened, like v2 and v3 major - odd, even - developer/stable release minor - new features patchlevel - patchlevel so current 3.12.4 becomes 3.0.12.4 development starts with 3.1.0.0 and continues to 3.1.0.1, 3.1.0.2, 3.1.1.0, 3.1.2.0, ... 3.1.6.10 than we decide to release stable 3.2.6.0, 3.2.6.1, 3.2.6.2 or 3.2.0.0, 3.2.0.1, 3.2.0.2? development 3.3.0.0, ... 3.3.x.y There is one problem 3.12.4 > 3.0.12.4 == problem with upgrade solutions: 1) 3.12.4 -> 3.14.0.0 2) 3.12.4 -> 3.0.12.4 and I'll start new epoch of rsyslog package (Epoch: 2 in specfile, don't know how is it in other distros) On Tuesday 25 March 2008 06:16:05 pm Rainer Gerhards wrote: > Hi all, > > I have thought about the version numbering. Obviously, the old scheme > (http://www.rsyslog.com/doc-version_naming.html ) is confusing and > should be dropped. > > Rsyslog currently follows a three-number version scheme: > > major.minor.patchlevel > > I can follow an odd/even scheme on major for unstable/stable branches. > However, with major features being continuously introduced may bring us > to rsyslog 10.x.x by the end of the year. While I have no hard concerns, > it seems to be unusual for open source to increase the major version > that fast. On the other hand, I need minor.patchlevel to handle the > smaller fixes and new major feature. I increase minor for each new > feature (like queue engine, expression-based filters, relp, to name a > few actual ones) and increase patchlevel for smaller changes. The three > number scheme works well, but, again, keeps the major number very > quickly growing given the current fast pace of development. > > An alternative would be to use > > politicalmajor.major.minor.patchlevel > > Where politicalmajor would only be incremented every now and then and > actually without much technical reasoning for doing so - only then when > we "feel" it is time for a new major release. It would probably turn out > to be incremented around once a year, just to keep the corporate folks > happy and the major version number reasonable low. > > Question now: which scheme should I follow? Is there any other one? If > so, why should I follow that other scheme? > I would appreciate quick feedback, as I'd like to settle this issue > before releasing RELP support, which currently looks like some time > (very) early April. > > Thanks, > Rainer From linuxweb at gmail.com Thu Mar 27 02:28:46 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Wed, 26 Mar 2008 21:28:46 -0400 Subject: [rsyslog] central logserver log rotation Message-ID: <47EAF84E.4010909@gmail.com> Thanks to Rainer, I've got my shiny new central log server up & running. I'd like to rotate the logs on there. If I've got a separate directory for each host, though (i.e., /var/log/rsyslog/%HOSTNAME%), how can I do this? Does something like this work with logrotate: /var/log/rsyslog/*/* { weekly notifempty missingok } What are people doing for logrotation on their CENTRAL logging server? johnn From rgerhards at hq.adiscon.com Thu Mar 27 10:00:46 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 10:00:46 +0100 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <200803261147.21726.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> Hi Peter, thanks for the feedback. So it looks like we go for the 4-number scheme. If I look at the migration case, I am tempted to do a one-time move to 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as interim devel release) to prevent any confusion about version numbers. Changing from a 3-number to a 4-number scheme within the same major release sounds confusing in itself to me. Any concerns with such an approach? Anyone? Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Wednesday, March 26, 2008 11:47 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] rsyslog version numbering - was: RE: rsyslog > 3.0.0 stable? - feedback, please > > Hi, > > politicalmajor.major.minor.patchlevel seems good to me. > > politicalmajor - something great happened, like v2 and v3 > > major - odd, even - developer/stable release > > minor - new features > > patchlevel - patchlevel > > so current 3.12.4 becomes 3.0.12.4 > > development starts with 3.1.0.0 and continues to > > 3.1.0.1, 3.1.0.2, 3.1.1.0, 3.1.2.0, ... 3.1.6.10 > > than we decide to release stable > > 3.2.6.0, 3.2.6.1, 3.2.6.2 > > or > > 3.2.0.0, 3.2.0.1, 3.2.0.2? > > development 3.3.0.0, ... 3.3.x.y > > There is one problem 3.12.4 > 3.0.12.4 == problem with upgrade > > solutions: > > 1) 3.12.4 -> 3.14.0.0 > > 2) 3.12.4 -> 3.0.12.4 and I'll start new epoch of rsyslog package > > (Epoch: 2 in specfile, don't know how is it in other distros) > > On Tuesday 25 March 2008 06:16:05 pm Rainer Gerhards wrote: > > > Hi all, > > > > > > I have thought about the version numbering. Obviously, the old scheme > > > (http://www.rsyslog.com/doc-version_naming.html ) is confusing and > > > should be dropped. > > > > > > Rsyslog currently follows a three-number version scheme: > > > > > > major.minor.patchlevel > > > > > > I can follow an odd/even scheme on major for unstable/stable > branches. > > > However, with major features being continuously introduced may bring > us > > > to rsyslog 10.x.x by the end of the year. While I have no hard > concerns, > > > it seems to be unusual for open source to increase the major version > > > that fast. On the other hand, I need minor.patchlevel to handle the > > > smaller fixes and new major feature. I increase minor for each new > > > feature (like queue engine, expression-based filters, relp, to name a > > > few actual ones) and increase patchlevel for smaller changes. The > three > > > number scheme works well, but, again, keeps the major number very > > > quickly growing given the current fast pace of development. > > > > > > An alternative would be to use > > > > > > politicalmajor.major.minor.patchlevel > > > > > > Where politicalmajor would only be incremented every now and then and > > > actually without much technical reasoning for doing so - only then > when > > > we "feel" it is time for a new major release. It would probably turn > out > > > to be incremented around once a year, just to keep the corporate > folks > > > happy and the major version number reasonable low. > > > > > > Question now: which scheme should I follow? Is there any other one? > If > > > so, why should I follow that other scheme? > > > I would appreciate quick feedback, as I'd like to settle this issue > > > before releasing RELP support, which currently looks like some time > > > (very) early April. > > > > > > Thanks, > > > Rainer From rgerhards at hq.adiscon.com Thu Mar 27 10:13:25 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 10:13:25 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: <47EAF84E.4010909@gmail.com> References: <47EAF84E.4010909@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Hi Johnn, the typical solution seems to be to include a date-specific part in the filename: /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% And clean up the directories every now and then... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Thursday, March 27, 2008 2:29 AM > To: rsyslog-users > Subject: [rsyslog] central logserver log rotation > > Thanks to Rainer, I've got my shiny new central log server > up & running. I'd like to rotate the logs on there. > > If I've got a separate directory for each host, though > (i.e., /var/log/rsyslog/%HOSTNAME%), how can I do this? > > Does something like this work with logrotate: > > /var/log/rsyslog/*/* { > weekly > notifempty > missingok > } > > > What are people doing for logrotation on their CENTRAL > logging server? > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Thu Mar 27 15:39:11 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Thu, 27 Mar 2008 10:39:11 -0400 Subject: [rsyslog] rsyslog version numbering - was: RE: rsyslog 3.0.0 stable? - feedback, please In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> Message-ID: <47EBB18F.1040803@gmail.com> Rainer Gerhards wrote: > Hi Peter, > > thanks for the feedback. So it looks like we go for the 4-number scheme. > > If I look at the migration case, I am tempted to do a one-time move to > 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as > interim devel release) to prevent any confusion about version numbers. > Changing from a 3-number to a 4-number scheme within the same major > release sounds confusing in itself to me. I support the suggestion above. Trying to number within v3 is going to wreak havoc on attempts to update properly. p.s. To be perfectly honest, I didn't think a move to 4-number was even necessary, but I think you'd know better than anyone else what is in the pipeline for rsyslog, and if those are major enough to require "major" number to be changed, then, yeah, it wouldn't make sense to see 10.x.x within a year or two. johnn From rgerhards at hq.adiscon.com Thu Mar 27 16:39:45 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 27 Mar 2008 16:39:45 +0100 Subject: [rsyslog] timestamp format - why use the old one? Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C74@grfint2.intern.adiscon.com> Hi all, I got an interesting question from another discussion list. As you (hopefully) are aware, rsyslog supports high precision timestamps. However, the default format does not output them in order to avoid confusing log parsers. That, in turn, leads to few people actually ever using it. So in the end run, rsyslog's high-precision timestamps have never been used much in the past 3+ years. One suggestion to solve the log parser problem was that we should begin to use the high-precision stamp by default and work with converters/filters when data is fed to log analysers. What is your opinion on this topic? Is it time to change the default output format for timestamps? Rainer From friedl at hq.adiscon.com Thu Mar 27 17:32:56 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Thu, 27 Mar 2008 17:32:56 +0100 Subject: [rsyslog] rsyslog 2.0.4 stable released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C78@grfint2.intern.adiscon.com> Hi all, rsyslog 2.0.4, a stable release version, has been released today. It contains some bug fixes, the most important one is that FROMHOST was not properly set on internal messages. There are also two new properties ($HHOUR and $QHOUR) which can be used in dynamic log file generation. Rsyslog 2.0.4 is a recommended update for all users of the stable v2 branch. Changelog: http://www.rsyslog.com/Article197.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-90.phtml Florian Riedl From janfrode at tanso.net Fri Mar 28 08:51:22 2008 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 28 Mar 2008 08:51:22 +0100 Subject: [rsyslog] central logserver log rotation References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: On 2008-03-27, Rainer Gerhards wrote: > the typical solution seems to be to include a date-specific part in the > filename: > > /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% > > And clean up the directories every now and then... /etc/rsyslog.conf: ##################################################### # Log everything to a per host daily logfile # ##################################################### $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log" *.* -?DailyPerHostLogs /etc/cron.hourly/syslog-bzip2: find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec bzip2 '{}' \; -jf From rgerhards at hq.adiscon.com Fri Mar 28 10:17:44 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 10:17:44 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: References: <47EAF84E.4010909@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C82@grfint2.intern.adiscon.com> Hi JF, thanks, I've made a wiki page out of it: http://wiki.rsyslog.com/index.php/DailyLogRotation Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust > Sent: Friday, March 28, 2008 8:51 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] central logserver log rotation > > On 2008-03-27, Rainer Gerhards wrote: > > the typical solution seems to be to include a date-specific part in > the > > filename: > > > > /var/log/rsyslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME% > > > > And clean up the directories every now and then... > > /etc/rsyslog.conf: > > ##################################################### > # Log everything to a per host daily logfile # > ##################################################### > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/me > ssages.log" > *.* -?DailyPerHostLogs > > /etc/cron.hourly/syslog-bzip2: > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec > bzip2 '{}' \; > > > > -jf > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 28 12:58:37 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 12:58:37 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47EBB18F.1040803@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> > > If I look at the migration case, I am tempted to do a one-time move > to > > 4.0.x.x (stable) and 4.1.x.x (devel) and drop the v3 tree (flagged as > > interim devel release) to prevent any confusion about version > numbers. > > Changing from a 3-number to a 4-number scheme within the same major > > release sounds confusing in itself to me. > > I support the suggestion above. Trying to number within v3 > is going to wreak havoc on attempts to update properly. > > p.s. To be perfectly honest, I didn't think a move to > 4-number was even necessary, I thought quite a while over your message. I have to admit that I personally would prefer a three-number scheme. Maybe there is a solution without going overboard on the major version number increments. First of all, things become stable because bugs are fixed - and no new ones added. "No new ones added" means every stable version is a dead-end when it comes to features. So, 2.0.4 may become 2.0.9 or even 2.0.9999, but we won't see a 2.1.x as that would imply a sufficiently large new feature. This is also a quite hard limit as it means the stable version will always starve for new features. But that's how it works today. In devel, the minor is updated whenever something cool enough is added to justify that. Which means typically every 2 to 6 weeks on the current schedule. So 3.10.0 has become 3.10.2, later 3.11.0 and we are now (soon;)) at 3.12.5. The interesting thing is what happens next: 3.12.5 is quite stable now. There are some feature introduced in 3.10.0 being very stable, some from 3.11.0 which are the same and there as some new ones in 3.12.0 introduced which are not as stable as others (because of limited exposure to practice). Anyhow, I'll soon convert that whole version to 4.0.0 (maybe 4.0.0.0) stable. Then I'll start a new devel branch and the stable build again begins to mature but also become feature-less compared to the devel - which has various stages of maturity some time later. I now got a new school of thinking: let's stick with the 3-digit scheme: major.minor.patchlevel Major will be incremented whenever there is sufficiently large change (typically some inner workings must be changed quite a bit). Stabel and Devel will coexist within the same major release. For minor, we have even - stable and odd - devel. Patchlevel again is smaller magnitude changes within the minor (or evolution of a new feature not yet announced to be usable). But that doesn't solve my need to sometimes have two or three devel versions with new features in a row. To solve that, I suggest that we simply skip the stable releases. For example: 4.0.0 has been released, 4.1.0 is the new stable. I add relp support in 4.1.0. Then I do some changes and end up at 4.1.4. Then, I decide to do the new config file format. That'll be 4.3.0. Note that there now is a stable version missing: we have 4.0.0, 4.1.4 and 4.3.0. I work some more on devel. Then comes a new feature (let's guess TLS support...). So we finally have 4.0.0, 4.1.4, 4.3.x and 4.5.x. After a while, I am at 4.5.5 and also got enough bug fixes to think that the feature introduced in 4.3.x is now ready for prime time. Then, I'll release 4.4.0, based on the 4.3.x codebase (and missing the new 4.5.x feature). With the release of 4.4.0, bugfixing for 4.0.x will end, as we now have a new stable version of the v4 branch (but fixes would still be applied to stable v2). So we end up with 4.0.x (deadend), 4.1.x (dead), NO 4.2.x, 4.3.x (dead), 4.4.x (current stable v4) and 4.5.x (current stable v5). The key thing is that I left out 4.2.x. If that seems to be acceptable, we do not need the 4-number scheme. I know it's pretty wild with all these version numbers... I hope I still could get the idea over. Feedback is highly appreciated. If there is none and no objection by next week Tuesday, I'll go for that scheme. I intend to release 4.0.0(.0) next Wednesday. Today's release of 3.12.5 will set the stage for it. > but I think you'd know better > than anyone else what is in the pipeline for rsyslog, and if > those are major enough to require "major" number to be > changed, then, yeah, it wouldn't make sense to see 10.x.x > within a year or two. Just some thoughts of the pipeline. No guarantees on the order, but the order is current thinking: * relp support * relp with TLS support (followed by relp/GSSAPI) * config file format (scriptable, RainerScript) * email notification output * native TCP w. TLS/syslog-transport-tls internet draft * cryptographically signed messages * support for other message types but syslog * tamperproof (compliance-compliant ;)) system that forces rsyslogd to work with a centralized configuration and provides the ability to detect unauthorized client configuration changes (where they can't be prevented) ... and probably much more ;) Rainer From r.bhatia at ipax.at Fri Mar 28 14:26:14 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 28 Mar 2008 14:26:14 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> Message-ID: <47ECF1F6.8030909@ipax.at> hi, what about the following major.minor.patchlevel with 3.x.y as an exception, used as: STABLE RELEASES =============== 1) release 3.x.y (e.g. 3.12.0) as stable. we might continue providing patches, fixes, etc. in a fashion like: - 3.12.0 (= stable) - 3.13.0 (= major fix) - 3.14.0 (= major fix) - 3.15.0 (= small fix) ... 2) continue development with 4.0.x. release 4.0.14 as stable with: - 4.0.14 (= stable) - 4.0.15 (= fix) - 4.0.16 (= fix) - 4.0.17 (= fix) ... 3) continue development with 4.1.x, release e.g. 4.1.25 as stable with: - 4.1.25 (= stable) - 4.1.26 (= fix) - 4.1.27 (= fix) - 4.1.28 (= fix) ... that's for the handling of stable releases. ad 1) as a more consistent use of the major.minor.patchlevel, we could use: - 3.12.0 - 3.12.1 - 3.12.2 ... - 3.12.1255 DEVELOPMENT SUGGESTION A ======================== inside the development tree, one could do: 4.1.0 = new dev release 4.1.1-rc1 = major new features - exerimental - only for hardcore testers 4.1.1-rc2 = getting more stable 4.1.1-rc3 = getting more stable 4.1.1 = let more ppl test this release with its new features 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not be continued/patched/etc. DEVELOPMENT SUGGESTION B ======================== do not release that many devel releases but put out nightlies with a git "short" tag (i think i've seen this on hg). [1] this would mean: 4.1.0 = new dev tree is started. 4.1.1-d12080bf3d99 snapshot/interim release 2008-03-28 big new features! 4.1.1-c377eb7b2a22 snapshot/interim release 2008-03-29 fixing 4.1.1-b7b2a2515d3a snapshot/interim release 2008-03-30 fixing 4.1.1 - now we're confident enough to make a release for testing. 4.1.2-10e7d626f05c - big new features snapshot/interim release ... 4.1.2 - we're confident - public testers wanted ... 4.1.25 - now we're stable - declare stable, no more features. ;) of course, such interim releases must not happen on a daily basis. everyone should be free to do a recent checkout and those interim releases could be limited to rainer feeling confident enough ;) so, what about this suggestion? cheers, raoul [1] how to shorten the git revisions. long: 4.1.1-d120806a4549ae6c377eb7b2a25172251fbf3d99 4.1.1-d12080(6a4549ae6c377eb7b2a25172251f)bf3d99 4.1.1-d12080...bf3d99 4.1.1-d12080.bf3d99 4.1.1-d12080bf3d99 -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Fri Mar 28 14:56:51 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 14:56:51 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ECF1F6.8030909@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> > what about the following major.minor.patchlevel with 3.x.y as an > exception, used as: > > STABLE RELEASES > =============== > 1) release 3.x.y (e.g. 3.12.0) as stable. > we might continue providing patches, fixes, etc. in a fashion like: > - 3.12.0 (= stable) > - 3.13.0 (= major fix) > - 3.14.0 (= major fix) > - 3.15.0 (= small fix) > ... > > 2) continue development with 4.0.x. release 4.0.14 as stable with: > - 4.0.14 (= stable) > - 4.0.15 (= fix) > - 4.0.16 (= fix) > - 4.0.17 (= fix) > ... > > 3) continue development with 4.1.x, release e.g. 4.1.25 as stable with: > - 4.1.25 (= stable) > - 4.1.26 (= fix) > - 4.1.27 (= fix) > - 4.1.28 (= fix) > ... > > that's for the handling of stable releases. So in essence, at one patchlevel a major.minor becomes stable and a new (major.minor) is begun - that would work for me and sound quite logical... A key point still would be that there could be a 4.1.24 unstable, while I begin to work on 4.2.0. After all, things are unstable at first ;) So we have two unstable in a row. After a while (we may be at 4.2.6) we can release 4.1.25 (stable) as it has matured enough. Any more comments on that? > ad 1) as a more consistent use of the major.minor.patchlevel, we could > use: > - 3.12.0 > - 3.12.1 > - 3.12.2 > ... > - 3.12.1255 > > > > DEVELOPMENT SUGGESTION A > ======================== > inside the development tree, one could do: > 4.1.0 = new dev release > 4.1.1-rc1 = major new features - exerimental - only for hardcore > testers > 4.1.1-rc2 = getting more stable > 4.1.1-rc3 = getting more stable > 4.1.1 = let more ppl test this release with its new features > > 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not be > continued/patched/etc. I see where you are coming from. This is actually a three and a half version numbering scheme - but also makes sense to me. Looks like a good solution. I like this much more than suggestion B, I have to admit ;) -- the main reason is that I tend to commit very frequently, but a nightly release will probably bad to follow up on. Overall, these suggestions look easier to follow than mine. So if nobody objects, I'll implement them by mid next week. BUT... Feedback is still most welcome! Rainer > > > > DEVELOPMENT SUGGESTION B > ======================== > do not release that many devel releases but put out nightlies with > a git "short" tag (i think i've seen this on hg). [1] > > this would mean: > 4.1.0 = new dev tree is started. > 4.1.1-d12080bf3d99 snapshot/interim release 2008-03-28 big new > features! > 4.1.1-c377eb7b2a22 snapshot/interim release 2008-03-29 fixing > 4.1.1-b7b2a2515d3a snapshot/interim release 2008-03-30 fixing > 4.1.1 - now we're confident enough to make a release for testing. > > 4.1.2-10e7d626f05c - big new features snapshot/interim release > ... > 4.1.2 - we're confident - public testers wanted > ... > 4.1.25 - now we're stable - declare stable, no more features. ;) > > of course, such interim releases must not happen on a daily basis. > everyone should be free to do a recent checkout and those interim > releases could be limited to rainer feeling confident enough ;) > > > > so, what about this suggestion? > > cheers, > raoul > > [1] how to shorten the git revisions. > long: 4.1.1-d120806a4549ae6c377eb7b2a25172251fbf3d99 > 4.1.1-d12080(6a4549ae6c377eb7b2a25172251f)bf3d99 > 4.1.1-d12080...bf3d99 > 4.1.1-d12080.bf3d99 > 4.1.1-d12080bf3d99 > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Fri Mar 28 16:03:05 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 11:03:05 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> Message-ID: <47ED08A9.40500@gmail.com> Rainer Gerhards wrote: >> inside the development tree, one could do: >> 4.1.0 = new dev release >> 4.1.1-rc1 = major new features - exerimental - only for hardcore >> testers >> 4.1.1-rc2 = getting more stable >> 4.1.1-rc3 = getting more stable >> 4.1.1 = let more ppl test this release with its new features >> >> 4.1.2-rc1 = new features, new fixes, based on 4.1.1. 4.1.1. will not > be >> continued/patched/etc. > > I see where you are coming from. This is actually a three and a half > version numbering scheme - but also makes sense to me. Looks like a good > solution. I like this much more than suggestion B, I have to admit ;) -- > the main reason is that I tend to commit very frequently, but a nightly > release will probably bad to follow up on. I like this idea too (or the general direction of it, anyway -- I'm sure once you start down it, you will likely tweak it to fit actual needs). It lets you keep the 3-number scheme, while still providing for interim releases that don't overload the version number. And a few other projects seem to use this scheme too (consistency is good). > Overall, these suggestions look easier to follow than mine. So if nobody > objects, I'll implement them by mid next week. BUT... Feedback is still > most welcome! So, starting next week, there will be 3 branches? * 2.0.x -- stable, this is the one that systems like RHEL will track for awhile * 3.12.x -- what is this called? this is the "new" stable, for those that need the new 3.x features, but also want a relatively stabilized version, once the numbering is sufficiently high * 4.0.x -- devel, this is where the new major features start going in I think this works. Actually, taking a look at mysql just for comparison sake, they appear to have something similar: 5.0.x (for super-stable, like RHEL), 5.1.x (for those who need full NDB support, but mostly stable once it comes out of RC), and 6.0.x (alpha). johnn From linuxweb at gmail.com Fri Mar 28 16:06:48 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 11:06:48 -0400 Subject: [rsyslog] central logserver log rotation In-Reply-To: References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> Message-ID: <47ED0988.2060705@gmail.com> Jan-Frode Myklebust wrote: > /etc/rsyslog.conf: > > ##################################################### > # Log everything to a per host daily logfile # > ##################################################### > $template DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log" > *.* -?DailyPerHostLogs > > /etc/cron.hourly/syslog-bzip2: > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec bzip2 '{}' \; Thanks for the example. And ... I was just thinking of doing some gzip/bzip on my morning commute today, so your cron example comes just in time too :). johnn From rgerhards at hq.adiscon.com Fri Mar 28 16:25:12 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 16:25:12 +0100 Subject: [rsyslog] central logserver log rotation In-Reply-To: <47ED0988.2060705@gmail.com> References: <47EAF84E.4010909@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C63@grfint2.intern.adiscon.com> <47ED0988.2060705@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C8C@grfint2.intern.adiscon.com> In the long term (think tank!) it may be possible do some automagic with dynafiles. Let's say specify a timeout after which, when no longer accessed, some (external) rotation logic is invoked. The good thing, probably, would be that rsyslog knows best when it can and need to do so. It could even invoke it somewhat quicker if it runs out of disk space ;) Again, nothing to be implemented soon, but if you have an opinion on it, I'd like to hear about it. Side-Note: we are getting close to an initial release of the web interface. It will not only work with databases, but also with text files. I have this, too, on my mind when I think about rotation... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 28, 2008 4:07 PM > To: rsyslog-users > Subject: Re: [rsyslog] central logserver log rotation > > Jan-Frode Myklebust wrote: > > /etc/rsyslog.conf: > > > > ##################################################### > > # Log everything to a per host daily logfile # > > ##################################################### > > $template > DailyPerHostLogs,"/var/log/syslog/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/me > ssages.log" > > *.* -?DailyPerHostLogs > > > > /etc/cron.hourly/syslog-bzip2: > > > > find /var/log/syslog/2008 -type f -mtime +1 -name "*.log" -exec > bzip2 '{}' \; > > Thanks for the example. And ... I was just thinking of doing > some gzip/bzip on my morning commute today, so your cron > example comes just in time too :). > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From r.bhatia at ipax.at Fri Mar 28 16:48:16 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 28 Mar 2008 16:48:16 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> Message-ID: <47ED1340.8030300@ipax.at> Rainer Gerhards wrote: > A key point still would be that there could be a 4.1.24 unstable, while > I begin to work on 4.2.0. After all, things are unstable at first ;) So > we have two unstable in a row. After a while (we may be at 4.2.6) we can > release 4.1.25 (stable) as it has matured enough. i see no problem with this. there even does not need to be a 4.1.x which is declared as stable. > I see where you are coming from. This is actually a three and a half > version numbering scheme - but also makes sense to me. Looks like a good > solution. I like this much more than suggestion B, I have to admit ;) -- > the main reason is that I tend to commit very frequently, but a nightly > release will probably bad to follow up on. there need not be a nightly release. after ordering things in my mind, the main difference is: do we need to map -rcX to git snapshot 45b6c18f1d13be2ff89ee62024ba... or do we allready have this information somewhere accessible. besides that, we could skip the "rc" string and simply use 4.0.12-1 - like a build number which is used by some linux distributions. another nice aspect would be to integrate the git hash into the initial startup message or in some status output. see: > # cat /proc/drbd > version: 8.0.8 (api:86/proto:86) > GIT-hash: bd3e2c922f95c4fa0dca57a4f8c24bf8b249cc02 build by root at webcluster01, 2008-01-29 18:36:06 > Mar 23 06:25:27 webcluster02 rsyslogd: [origin software="rsyslogd" > swVersion="1.19.7" x-pid="2525"][x-configInfo udpReception="No" gitHash="bd3e2c922f95c4fa0dca57a4f8c24bf8b249cc02" > udpPort="514" tcpReception="No" tcpPort="0"] > Overall, these suggestions look easier to follow than mine. So if nobody > objects, I'll implement them by mid next week. BUT... Feedback is still > most welcome! maybe it would be good to make a final statement before implementing and releasing it. so we do not have to revert/skip/create the version 5 and 6 and discard 3/4 as a "versioning experiment" ;) cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From linuxweb at gmail.com Fri Mar 28 17:00:37 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Fri, 28 Mar 2008 12:00:37 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1340.8030300@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> Message-ID: <47ED1625.9030305@gmail.com> Raoul Bhatia [IPAX] wrote: > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > like a build number which is used by some linux distributions. Actually, Red Hat-based distros (at least) use that "-1" for internal changes. Let's say there is a version: 4.0.12rc3 I (or Red Hat) decide to build an RPM based on that, so my version is: 4.0.12rc3-1 Then, Rainer releases a 4.0.12rc4. It has too many new features in it, so I decide not to move to it. HOWEVER, it also has one crucial bugfix that I need. So I take that patch and backport it into my version. Now, my new *local* version is: 4.0.12rc3-2 So, I do like the idea of attaching rc* (because that makes it clear the changes are from the developer), but not a dash-number (-1, etc.) to the version, if at all possible, because those tend to be localized/internal version number changes. johnn From rgerhards at hq.adiscon.com Fri Mar 28 17:58:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 17:58:58 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1625.9030305@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47E9227A.5010903@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C91@grfint2.intern.adiscon.com> And this sounds very reasonable, too. Also -rc seems to alert some folks and makes clear what they are doing ;) Please keep the feedback flowing, very good discussion. Just a quick explanation why I would like to settle this out of the sudden quickly. It's a somewhat political issue. While I have no distro-preferrence (rsyslog should run great on as many platforms as possible), I can get the new versioning scheme into Fedora 9 if I do it by mid next week. That is a very nice incentive ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Friday, March 28, 2008 5:01 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Raoul Bhatia [IPAX] wrote: > > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > > like a build number which is used by some linux distributions. > > Actually, Red Hat-based distros (at least) use that "-1" for > internal changes. > > Let's say there is a version: 4.0.12rc3 > > I (or Red Hat) decide to build an RPM based on that, so my > version is: > 4.0.12rc3-1 > > > Then, Rainer releases a 4.0.12rc4. It has too many new > features in it, so I decide not to move to it. HOWEVER, it > also has one crucial bugfix that I need. So I take that > patch and backport it into my version. Now, my new *local* > version is: > 4.0.12rc3-2 > > So, I do like the idea of attaching rc* (because that makes > it clear the changes are from the developer), but not a > dash-number (-1, etc.) to the version, if at all possible, > because those tend to be localized/internal version number > changes. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Fri Mar 28 18:45:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 28 Mar 2008 18:45:58 +0100 Subject: [rsyslog] rsyslog 3.12.5 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C93@grfint2.intern.adiscon.com> Hi All, rsyslog 3.12.5 has just been released. It is mostly a bug-fixing and cleanup release. Some changed defaults make it easier to use advanced features. For example, high-precision timestamps are now enabled by default when writing log files and "last message repeated n times" processing, a typical trouble spot, is turned off by default. The documentation has been improved and the v2-backward compatibility layer has been completed. This version is a candidate to become the next stable release. Changelog: http://www.rsyslog.com/Article199.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-91.phtml I suggest to (re-) read ./doc/v3compatibility.html, which also talks about the new defaults. Also available online at: http://www.rsyslog.com/doc-v3compatibility.html Feedback on this release is appreciated. I hope it will be useful. Rainer From rgerhards at hq.adiscon.com Sat Mar 29 10:44:26 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 29 Mar 2008 10:44:26 +0100 Subject: [rsyslog] debug setting in 3.12.5 active Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C96@grfint2.intern.adiscon.com> Hi all, I just noticed that a debug setting is active in 3.12.5. This disables the actual dlclose. It should not have too bad side effects. I will fix this in 3.12.6, with a few other cleanups, which I hopefully release later today. So you may want to hold updating if you have not done so already. Sorry for the hassle, Rainer From mbiebl at gmail.com Sat Mar 29 12:47:44 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 12:47:44 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47ED1625.9030305@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com> <200803261147.21726.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: 2008/3/28, Johnny Tan : > Raoul Bhatia [IPAX] wrote: > > besides that, we could skip the "rc" string and simply use 4.0.12-1 - > > like a build number which is used by some linux distributions. > > > Actually, Red Hat-based distros (at least) use that "-1" for > internal changes. Yeah, please don't use "-1" as version suffix, this is usually reserved for distros. -rc? would be ok though. Rainer, could you please summarise, what your current idea for the version numbering is. With all the different proposals I somehow lost track about the latest preferred one. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sat Mar 29 14:56:30 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 29 Mar 2008 14:56:30 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C53@grfint2.intern.adiscon.com><200803261147.21726.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> I needed some time to digest all this myself ;) I've now done that and put it all into a HTML page. That doesn't imply it is the scheme we will finally use, but if so, it saves me some time. I have also described the development process a bit. I think this is vital for understanding why I need certain release numbers. The info is here (with the now irrelevant text at the end of the doc): http://www.rsyslog.com/doc-version_naming.html I am still a bit undecided if we really need to go v4 or can start this (or whatever else) in the v3 branch... Feedback appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Saturday, March 29, 2008 12:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/28, Johnny Tan : > > Raoul Bhatia [IPAX] wrote: > > > besides that, we could skip the "rc" string and simply use 4.0.12- > 1 - > > > like a build number which is used by some linux distributions. > > > > > > Actually, Red Hat-based distros (at least) use that "-1" for > > internal changes. > > Yeah, please don't use "-1" as version suffix, this is usually > reserved for distros. > -rc? would be ok though. > > Rainer, could you please summarise, what your current idea for the > version numbering is. > With all the different proposals I somehow lost track about the latest > preferred one. > > Cheers, > Michael > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Sat Mar 29 15:13:03 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 15:13:03 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C60@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: 2008/3/29, Rainer Gerhards : > I needed some time to digest all this myself ;) I've now done that and > put it all into a HTML page. That doesn't imply it is the scheme we will > finally use, but if so, it saves me some time. I have also described the > development process a bit. I think this is vital for understanding why I > need certain release numbers. The info is here (with the now irrelevant > text at the end of the doc): > > http://www.rsyslog.com/doc-version_naming.html > > I am still a bit undecided if we really need to go v4 or can start this > (or whatever else) in the v3 branch... > > Feedback appreciated. What I'm missing from the document, is a distinction between stable and unstable releases (or are all releases with -rc? unstable and without stable?) Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Sat Mar 29 15:16:18 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Sat, 29 Mar 2008 15:16:18 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47EBB18F.1040803@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com> <47ECF1F6.8030909@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: 2008/3/29, Michael Biebl : > 2008/3/29, Rainer Gerhards : > > > I needed some time to digest all this myself ;) I've now done that and > > put it all into a HTML page. That doesn't imply it is the scheme we will > > finally use, but if so, it saves me some time. I have also described the > > development process a bit. I think this is vital for understanding why I > > need certain release numbers. The info is here (with the now irrelevant > > text at the end of the doc): > > > > http://www.rsyslog.com/doc-version_naming.html > > > > I am still a bit undecided if we really need to go v4 or can start this > > (or whatever else) in the v3 branch... > > > > Feedback appreciated. > > > What I'm missing from the document, is a distinction between stable > and unstable releases > (or are all releases with -rc? unstable and without stable?) I.e., are -mf releases to be considered stable or not? Would the current 3.12.5 be an rc release in the new versioning scheme? Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Sun Mar 30 11:01:22 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 30 Mar 2008 11:01:22 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> [Mmmhhh. I thought I had sent a reply from the pda. Well, please don't wonder if it shows up some time later ;) So here is the full response:] Any version with a devstate designation (-mf, -rc) is unstable. Every version without it is stable (this will be far less versions as we reach stable only every now and then). So what would the current version be named? Mhh... It's not an easy one-to-one mapping. Because the process changes a bit. Today, I add new major and minor features at the same time while stabilizing old features. This is also the primary reason why we so far have very infrequently new features in the stable release: whenever I stabilize an "older" feature, I introduce new bugs while working on the new ones. That'll change a bit with the new scheme, as I then will freeze code once the focus feature has been implemented. When I start with the next focus feature, patches will be applied to both the "older" code as well as the one I am working on. As such, features become stable during the process. It obviously is some more work to do for me, as I now need to apply patches to multiple code paths (to keep this reasonable, I limited the stable releases to only the latest minor version). The plus is that we get much more feature-rich stable releases, so I think the additional development time is well spent. And for the ultra-conservative folks, there is always the old, feature-less previous version stable (v2 in this case). After having said this, the current release would probably have two names: 3.12.0-rc3 --> the stabilized module loader release 3.13.0-mf2 --> the new relp-enabled release which has not yet real relp support I am also now of the thought that we do not necessarily need to move to 4.0.0 to cover this new scheme. I'd say I can simply release a 3.13.0 stable next week (based on the current 3.12.5) and then continue to work on relp in 3.14.0-rc0 (there will probably be no -mf for that release as relp support is already quite far). Question now: how can I just declare the release to be stable even though it contains the RELP code? Answer: I am lucky ;) the relp code *inside* rsyslog is just two slim inout/output plugins. No core changes. The actual relp code comes via librelp, which is external. Thanks to this coincidence, I did not really introduce any new features since the last feature focus and so the rest of rsyslog could mature. At no other point in the v3 tree I would have been able to do this. That is yet another reason that I'd like to settle the issue by mid next week - the initial version of relp is almost done and I am eager to move on. Again, feedback appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Saturday, March 29, 2008 3:16 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/29, Michael Biebl : > > 2008/3/29, Rainer Gerhards : > > > > > I needed some time to digest all this myself ;) I've now done that > and > > > put it all into a HTML page. That doesn't imply it is the scheme > we will > > > finally use, but if so, it saves me some time. I have also > described the > > > development process a bit. I think this is vital for > understanding why I > > > need certain release numbers. The info is here (with the now > irrelevant > > > text at the end of the doc): > > > > > > http://www.rsyslog.com/doc-version_naming.html > > > > > > I am still a bit undecided if we really need to go v4 or can > start this > > > (or whatever else) in the v3 branch... > > > > > > Feedback appreciated. > > > > > > What I'm missing from the document, is a distinction between stable > > and unstable releases > > (or are all releases with -rc? unstable and without stable?) > > I.e., are -mf releases to be considered stable or not? > Would the current 3.12.5 be an rc release in the new versioning scheme? > > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 11:20:58 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 11:20:58 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47EBB18F.1040803@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C84@grfint2.intern.adiscon.com><47ECF1F6.8030909@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> I'll do a new release according to the new versioning scheme (3.13.0-rc1) later today if nobody objects. It's still not finalized, but doing so makes everybody aware and is also a test for me (plus, we could move to 3.13.0 [stable] on April, 2nd). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Sunday, March 30, 2008 11:01 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > [Mmmhhh. I thought I had sent a reply from the pda. Well, please don't > wonder if it shows up some time later ;) So here is the full response:] > > Any version with a devstate designation (-mf, -rc) is unstable. Every > version without it is stable (this will be far less versions as we > reach > stable only every now and then). > > So what would the current version be named? Mhh... It's not an easy > one-to-one mapping. Because the process changes a bit. Today, I add new > major and minor features at the same time while stabilizing old > features. This is also the primary reason why we so far have very > infrequently new features in the stable release: whenever I stabilize > an > "older" feature, I introduce new bugs while working on the new ones. > > That'll change a bit with the new scheme, as I then will freeze code > once the focus feature has been implemented. When I start with the next > focus feature, patches will be applied to both the "older" code as well > as the one I am working on. As such, features become stable during the > process. It obviously is some more work to do for me, as I now need to > apply patches to multiple code paths (to keep this reasonable, I > limited > the stable releases to only the latest minor version). The plus is that > we get much more feature-rich stable releases, so I think the > additional > development time is well spent. And for the ultra-conservative folks, > there is always the old, feature-less previous version stable (v2 in > this case). > > After having said this, the current release would probably have two > names: > > 3.12.0-rc3 --> the stabilized module loader release > 3.13.0-mf2 --> the new relp-enabled release which has not yet real relp > support > > I am also now of the thought that we do not necessarily need to move to > 4.0.0 to cover this new scheme. I'd say I can simply release a 3.13.0 > stable next week (based on the current 3.12.5) and then continue to > work > on relp in 3.14.0-rc0 (there will probably be no -mf for that release > as > relp support is already quite far). > > Question now: how can I just declare the release to be stable even > though it contains the RELP code? Answer: I am lucky ;) the relp code > *inside* rsyslog is just two slim inout/output plugins. No core > changes. > The actual relp code comes via librelp, which is external. Thanks to > this coincidence, I did not really introduce any new features since the > last feature focus and so the rest of rsyslog could mature. At no other > point in the v3 tree I would have been able to do this. That is yet > another reason that I'd like to settle the issue by mid next week - the > initial version of relp is almost done and I am eager to move on. > > Again, feedback appreciated. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > > Sent: Saturday, March 29, 2008 3:16 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog version numbering > > > > 2008/3/29, Michael Biebl : > > > 2008/3/29, Rainer Gerhards : > > > > > > > I needed some time to digest all this myself ;) I've now done > that > > and > > > > put it all into a HTML page. That doesn't imply it is the > scheme > > we will > > > > finally use, but if so, it saves me some time. I have also > > described the > > > > development process a bit. I think this is vital for > > understanding why I > > > > need certain release numbers. The info is here (with the now > > irrelevant > > > > text at the end of the doc): > > > > > > > > http://www.rsyslog.com/doc-version_naming.html > > > > > > > > I am still a bit undecided if we really need to go v4 or can > > start this > > > > (or whatever else) in the v3 branch... > > > > > > > > Feedback appreciated. > > > > > > > > > What I'm missing from the document, is a distinction between stable > > > and unstable releases > > > (or are all releases with -rc? unstable and without stable?) > > > > I.e., are -mf releases to be considered stable or not? > > Would the current 3.12.5 be an rc release in the new versioning > scheme? > > > > Michael > > > > -- > > Why is it that all of the instruments seeking intelligent life in the > > universe are pointed away from Earth? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 12:48:00 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 12:48:00 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com> <47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > I'll do a new release according to the new versioning scheme > (3.13.0-rc1) later today if nobody objects. It's still not finalized, > but doing so makes everybody aware and is also a test for me (plus, we > could move to 3.13.0 [stable] on April, 2nd). > Ok, a few comments: I thinks the distinction between mf? (minor features) and rc? (major features) is a bit artifical and counterintuitive imho. For me, rc means "release candidate", meaning that we are close to a stable release (contrary to your proposal, where rc means major new feature, possibly destabilizing a lot). Also, if I read your proposal correctly, as soon as there was an rc1 release, there can be no more mf? releases (even if the added features would be minor) I'd rather use something like -pre1, -pre2 ... (no matter if you have a major or minor feature) and if we are nearing a stable release switch to -rc Or you could do it like the kernel, and only use -rc?. Another proposal for the version scheme (slightly adopted from KDE or Gnome): As an example, your currently stable release is 3.1.x, eg. 3.1.4. Now you want to work against the next minor release, which would be 3.2 So you start with 3.1.80 (80 meaning alpha/beta quality) and increment the last digit for every new release, eg. 3.1.81 (if you reach 89, you start numbering 89.1..) When you enter RC stage, you switch to 3.1.90 and increment for each new release. The final release being 3.2. If you are working towards a next major release, you would start with 3.80 -> 3.90 -> 4.0 Cheers, Michael [1] If you run out -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 12:57:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 12:57:56 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Quick comment... Others please also comment... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 12:48 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > I'll do a new release according to the new versioning scheme > > (3.13.0-rc1) later today if nobody objects. It's still not > finalized, > > but doing so makes everybody aware and is also a test for me (plus, > we > > could move to 3.13.0 [stable] on April, 2nd). > > > > Ok, a few comments: > I thinks the distinction between mf? (minor features) and rc? (major > features) is a bit artifical and counterintuitive imho. > For me, rc means "release candidate", meaning that we are close to a > stable release (contrary to your proposal, where rc means major new > feature, possibly destabilizing a lot). Also, if I read your proposal > correctly, as soon as there was an rc1 release, there can be no more > mf? releases (even if the added features would be minor) > I'd rather use something like -pre1, -pre2 ... (no matter if you have > a major or minor feature) and if we are nearing a stable release > switch to -rc It would be fine with me to call all -pre1, -pre2 etc... > Or you could do it like the kernel, and only use -rc?. To me, it is quite illogical to call something a -rc, when it not even contains the feature that it shall be a candidate for ;) That was the primary driver for -mf. > > Another proposal for the version scheme (slightly adopted from KDE or > Gnome): > As an example, your currently stable release is > 3.1.x, eg. 3.1.4. > Now you want to work against the next minor release, which would be 3.2 > So you start with > 3.1.80 (80 meaning alpha/beta quality) and increment the last digit > for every new release, eg. 3.1.81 (if you reach 89, you start > numbering 89.1..) > When you enter RC stage, you switch to > 3.1.90 and increment for each new release. The final release being 3.2. Well, that is somewhat along the lines of what I currently do. Remember, I start at 3.10.x for deve builds, then (in the original scheme), I'd go to 3.0, continue with e.g. 3.12.3, which becomes 3.1. The problem were the bouncing numbers. But maybe using patchlevel for that solves the issues. Let's use an actual sample. 3.13.0 is the current stable. The next major feature is RELP, which has been completed, but is not yet stable. I am now working on plain tcp TLS. So would this be the release numbers: 3.13.5 stable 3.13.87 relp 3.14.82 relp/tcp Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.13.88 relp 3.14.83 rep/tcp Once relp is stable, we will have 3.13.6 deprecated 3.14.0 stable 3.14.83 rep/tcp unstable So actually the 3.14.0 release happens possibly much later than 3.14.80? And in that scheme we never have a development designation but the patch level contains it (being >= 80 implies being unstable, while being less implies being stable)? >From what I see, I think that would work for me. Rainer > > If you are working towards a next major release, you would start with > 3.80 -> 3.90 -> 4.0 > > Cheers, > Michael > > [1] If you run out > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 13:02:56 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:02:56 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C87@grfint2.intern.adiscon.com><47ED1340.8030300@ipax.at> <47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Question ;) > If you are working towards a next major release, you would start with > 3.80 -> 3.90 -> 4.0 That also means once we have reached 3.80, we can/will no longer apply the new features to the 3.x stable tree, but rather only the 4.x stable tree? I am asking because I just realized we would otherwise be back to the up and down bouncing version numbers - this happens to be exactly the scheme I used so far... Rainer From mbiebl at gmail.com Mon Mar 31 13:10:07 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 13:10:07 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Question ;) > > > > If you are working towards a next major release, you would start with > > 3.80 -> 3.90 -> 4.0 > > > That also means once we have reached 3.80, we can/will no longer apply > the new features to the 3.x stable tree, but rather only the 4.x stable > tree? I am asking because I just realized we would otherwise be back to > the up and down bouncing version numbers - this happens to be exactly > the scheme I used so far... I don't see the problem here. Say you have a current 3.2.1 stable release. Then you start working for the next major release 4. After 4.0 has been released, you want to add a new feature to the old stable branch, so you release 3.3.0. That's something completely different than renaming 3.12.x to 3.1 when it becomes stable. Michaell -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 13:12:55 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 13:12:55 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47ED1625.9030305@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Quick comment... Others please also comment... > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > > > Sent: Monday, March 31, 2008 12:48 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog version numbering > > > > > 2008/3/31, Rainer Gerhards : > > > I'll do a new release according to the new versioning scheme > > > (3.13.0-rc1) later today if nobody objects. It's still not > > finalized, > > > but doing so makes everybody aware and is also a test for me (plus, > > we > > > could move to 3.13.0 [stable] on April, 2nd). > > > > > > > Ok, a few comments: > > I thinks the distinction between mf? (minor features) and rc? (major > > features) is a bit artifical and counterintuitive imho. > > For me, rc means "release candidate", meaning that we are close to a > > stable release (contrary to your proposal, where rc means major new > > feature, possibly destabilizing a lot). Also, if I read your proposal > > correctly, as soon as there was an rc1 release, there can be no more > > mf? releases (even if the added features would be minor) > > I'd rather use something like -pre1, -pre2 ... (no matter if you have > > a major or minor feature) and if we are nearing a stable release > > switch to -rc > > > It would be fine with me to call all -pre1, -pre2 etc... > > > > Or you could do it like the kernel, and only use -rc?. > > > To me, it is quite illogical to call something a -rc, when it not even > contains the feature that it shall be a candidate for ;) That was the > primary driver for -mf. The point is, that the don't try to draw a line where they start calling something alpha, beta then rc. The simply use one, rc, and increment that. > > > > > Another proposal for the version scheme (slightly adopted from KDE or > > Gnome): > > As an example, your currently stable release is > > 3.1.x, eg. 3.1.4. > > Now you want to work against the next minor release, which would be > 3.2 > > So you start with > > 3.1.80 (80 meaning alpha/beta quality) and increment the last digit > > for every new release, eg. 3.1.81 (if you reach 89, you start > > numbering 89.1..) > > When you enter RC stage, you switch to > > 3.1.90 and increment for each new release. The final release being > 3.2. > > > Well, that is somewhat along the lines of what I currently do. Remember, > I start at 3.10.x for deve builds, then (in the original scheme), I'd go > to 3.0, continue with e.g. 3.12.3, which becomes 3.1. The problem were The problem is, that for package management systems (and for people) 3.12.x > 3.1.x > the bouncing numbers. But maybe using patchlevel for that solves the > issues. > > Let's use an actual sample. > > 3.13.0 is the current stable. The next major feature is RELP, which has > been completed, but is not yet stable. I am now working on plain tcp > TLS. So would this be the release numbers: > > 3.13.5 stable > 3.13.87 relp > 3.14.82 relp/tcp > > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.13.88 relp > 3.14.83 rep/tcp > > Once relp is stable, we will have > > 3.13.6 deprecated > 3.14.0 stable > 3.14.83 rep/tcp unstable > > So actually the 3.14.0 release happens possibly much later than 3.14.80? This only happens, if you have two unstable branches at the same time. Do you expect that to happen that often? Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From Gerrard.Geldenhuis at datacash.com Mon Mar 31 13:28:26 2008 From: Gerrard.Geldenhuis at datacash.com (Gerrard Geldenhuis) Date: Mon, 31 Mar 2008 12:28:26 +0100 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: I have not really followed the thread but thought I would give a quick opinion. My requirements would be: Easy to follow numbering. I should be able to see at a glance what the newer version is. There should not be confusing things like 3.12.3 which becomes 3.1 as Michael highligted. Other than that I am not really bothered. Stable and unstable numbering is not realy relevant in my opinion anymore. I would rather look at the website or an rss feed to ascertain which version is the latest stable of a specific product. Regards > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: 31 March 2008 12:13 > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > Quick comment... Others please also comment... > > From rgerhards at hq.adiscon.com Mon Mar 31 13:39:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:39:57 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> > > So actually the 3.14.0 release happens possibly much later than > 3.14.80? > > This only happens, if you have two unstable branches at the same time. > Do you expect that to happen that often? I'd say that's the usual case given the current pace of development. I could even envision three unstables at one time... Rainer From rgerhards at hq.adiscon.com Mon Mar 31 13:51:39 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:51:39 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 1:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > 2008/3/31, Rainer Gerhards : > > Question ;) > > > > > > > If you are working towards a next major release, you would start > with > > > 3.80 -> 3.90 -> 4.0 > > > > > > That also means once we have reached 3.80, we can/will no longer > apply > > the new features to the 3.x stable tree, but rather only the 4.x > stable > > tree? I am asking because I just realized we would otherwise be back > to > > the up and down bouncing version numbers - this happens to be > exactly > > the scheme I used so far... > > I don't see the problem here. > Say you have a current 3.2.1 stable release. > Then you start working for the next major release 4. > After 4.0 has been released, you want to add a new feature to the old > stable branch, so you release 3.3.0. > That's something completely different than renaming 3.12.x to 3.1 when > it becomes stable. I was thinking about the .80+ case. Back to the sample, this time just assuming that relp is scheduled for the next major release. 3.13.0 is the current stable. 3.13.5 stable 3.80.5 relp 3.81.2 relp/tcp Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.80.6 relp 3.81.3 rep/tcp Once relp is stable, we will have... well, what will we have? RELP is stable, but the v4 goal is not yet reached? I thought it means it'll become 3.14.0: 3.13.6 deprecated 3.14.0 stable relp 3.81.3 rep/tcp unstable But now we are back to a case wher 3.81.6 (higher version number) is the same a 3.14.0. And now let's consider: 3.13.6 deprecated 3.14.0 stable relp (based on 3.80.6) 3.81.3 relp/tcp unstable 3.82.0 relp w/ tls unstable Now tcp becomes stable: 3.13.6 deprecated 3.14.0 stable relp (based on 3.80.6) 3.15.0 relp/tcp stable (based on 3.81.3) 3.82.0 relp w/ tls unstable But now we have the situation that 3.15 is more capable than 3.80. This is what I meant. It's exactly the scheme rsyslog used so far, except that I started unstable builds at .10 and not .80. If things working towards a new release shall start with .80, we need to either hold development on the stable (just as I did so far) or we need to have more capable builds with fewer IDs. Maybe this whole stable/unstable thing is not really worth the trouble and I should instead provide the feature stability matrix instead - where for each feature is shown how mature it is. Actually, I'd think that would probably make up for better information. Comments please. Rainer From rgerhards at hq.adiscon.com Mon Mar 31 13:59:23 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 13:59:23 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA8@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAA@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CAC@grfint2.intern.adiscon.com> > > > So actually the 3.14.0 release happens possibly much later than > > 3.14.80? > > > > This only happens, if you have two unstable branches at the same > time. > > Do you expect that to happen that often? > > I'd say that's the usual case given the current pace of development. I > could even envision three unstables at one time... This was maybe a bit brief. Let me add the reasoning, it may not be obvious... What makes a stable a stable? Bug fixes? Well, kind of. But actually it is *TIME*. It takes some time to find the "initial bug set". So the most bug reports come in in the 2 to 3 weeks after release. Depending on the feature complexity, it may take even longer (just think about the queue engine and its worker-thread system). So in essence, to get to a stable, I need to have the version just sit around and wait for bugs and their fixes. This will start whenever I have implemented a feature initially. So to continue, I'll immediately fork and work on the next feature set - else I would be idling for at least a couple of weeks. And, depending on how things go, I may even finish that one feature (minor release) and go to the next one before even the version before it matures (especially if my confidence in that code for some reason is reduced or the code is quite complex). During that course, it may happen that a minor feature will never become stable, but only the next one will (if that's quicker stabilized than the previous one). Rainer From r.bhatia at ipax.at Mon Mar 31 15:18:32 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 15:18:32 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> Message-ID: <47F0E4A8.50207@ipax.at> Rainer Gerhards wrote: i clearly object this numbering scheme as it gives much trouble to both ppl and package maintainers. moreover, i would see the 3.x.y branch as a branch depending on some "old" kind of version numbering and start with a fresh version scheme from 4.x.y onwards. coming back to your example, what would you think of: > 3.13.0 is the current stable. > 3.13.5 stable 3.13.5-relp5 (3.80.5 relp) 3.13.5-relp+tcp2 (3.81.2 relp/tcp) > Now let's assume I add a bugfix for the core engine. Would that bring us > to > 3.13.6 stable 3.13.6-relp6 (3.80.6 relp) 3.13.5-relp+tcp3 (3.81.3 rep/tcp) > Once relp is stable, we will have... well, what will we have? RELP is > stable, but the v4 goal is not yet reached? I thought it means it'll > become 3.14.0: 3.13.6 deprecated 3.14.0 stable relp 3.14.0-tcp3 (3.81.3 rep/tcp unstable) 3.13.6 deprecated 3.14.0 stable relp 3.14.3-tcp3 (3.81.3 relp/tcp unstable) 3.14.0-tls (3.82.0 relp w/ tls unstable) > Now tcp becomes stable: 3.13.6 deprecated 3.14.0 stable relp 3.15.0 relp/tcp stable 3.15.0-tls (3.82.0 relp w/ tls unstable) this means we're basically working with patches which can be applied on top of the current stable release. in git you can of course maintain them as branches and merge as to your liking. cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Mon Mar 31 15:36:31 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 15:36:31 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F0E4A8.50207@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> This has some subtleties for me, too (well, maybe its not really subtleties but simply frightening me ;)). Let me try to get this back to a simple path. How about a simple major.minor.patchlevel-dev So basically the "Sunday scheme" without -mf/-rc but instead just -dev. I guess the scheme was good for all, but the -mf/-rc was deemed confusing. In the sample, it would look like: 3.13.5 stable 3.14.0-dev6 (relp) 3.15.0-dev3 (relp/tcp) Now let's assume I add a bugfix for the core engine. Would that bring us to 3.13.6 stable 3.14.0-dev7 (relp) 3.15.0-dev4 (relp/tcp) Once relp is stable, we have 3.13.6 deprecated 3.14.0 stable relp 3.15.0-dev4 (relp/tcp) TLS begun: 3.13.6 deprecated 3.14.0 stable relp 3.15.0-dev4 (relp/tcp) 3.16.0-dev0 (tls) Now tcp becomes stable: 3.13.6 deprecated 3.14.0 deprecated 3.15.0 stable (relp/tcp) 3.16.0-dev0 (tls) Bottom line: versions with -dev are unstable, all others are stable. Does this sound acceptable? Rainer From rgerhards at hq.adiscon.com Mon Mar 31 16:25:54 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 16:25:54 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com><47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB2@grfint2.intern.adiscon.com> FYI: I'll release as 3.13.0-dev0 soon to check out if that scheme brings some problems to me in regard of the systems where I enter the version number. Again, nothing is decided yet. But I think it is better to do that experiment in the v3 tree than to potentially lose a v4 version number ;) (and, as a side note, my v4 release goal is not yet reached, so I'd prefer to stay at v3 if that's possible). Please let us know if you don't like that 3.13.0-dev0 scheme so that we can move to something different before I need to release the next stable. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, March 31, 2008 3:37 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > This has some subtleties for me, too (well, maybe its not really > subtleties but simply frightening me ;)). Let me try to get this back > to > a simple path. How about a simple > > major.minor.patchlevel-dev > > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) > > > Now let's assume I add a bugfix for the core engine. Would that bring > us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) > > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) > > Now tcp becomes stable: > > 3.13.6 deprecated > 3.14.0 deprecated > 3.15.0 stable (relp/tcp) > 3.16.0-dev0 (tls) > > Bottom line: versions with -dev are unstable, all others are > stable. > > Does this sound acceptable? > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From pvrabec at redhat.com Mon Mar 31 16:51:12 2008 From: pvrabec at redhat.com (Peter Vrabec) Date: Mon, 31 Mar 2008 16:51:12 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <200803311651.12753.pvrabec@redhat.com> Hi, > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) > > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) There is a great disadvatage of this scheme. 3.14.0-dev7 > 3.14.0, BUT 3.14.0 is newer. From linuxweb at gmail.com Mon Mar 31 16:53:22 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 10:53:22 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F0FAE2.3070405@gmail.com> Rainer Gerhards wrote: > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed Since many folks are already familiar with alpha, beta, and rc, I think it'd be good to stick with those as much as possible. > confusing. In the sample, it would look like: > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) 3.13.5 stable 3.14.0-beta6 (relp) 3.15.0-alpha3 (relp/tcp) > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.0-beta7 (relp) 3.15.0-alpha4 (relp/tcp) > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) 3.13.6 deprecated 3.14.0-rc1 (?) relp becoming stable 3.15.0-alpha4 (relp/tcp) > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) 3.13.6 deprecated 3.14.0 stable relp 3.15.0-beta1 (relp/tcp) (3.15 now becomes beta) 3.16.0-alpha1 (tls) (the new alpha/dev branch) In a nutshell: alpha - very (b)leading edge dev release branch beta - dev branch that has been out for awhile but still being worked on rc - the beta/dev branch finally getting ready to be declared stable, pretty much bug-fix-only at this point stable deprecated From r.bhatia at ipax.at Mon Mar 31 16:58:02 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 16:58:02 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F0FBFA.3020802@ipax.at> Rainer Gerhards wrote: > This has some subtleties for me, too (well, maybe its not really > subtleties but simply frightening me ;)). Let me try to get this back to > a simple path. How about a simple > > major.minor.patchlevel-dev > > So basically the "Sunday scheme" without -mf/-rc but instead just -dev. > I guess the scheme was good for all, but the -mf/-rc was deemed > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) what i'm not getting is, that you a) want to maintain some work-in-progress-patches as seperate trees. what happens if you want to implement feature-x (ssl/tls) first (which would then be 3.17.0) and this becomes stable faster than relp/other stuff? then we have 3.13.5 3.14.0-devX 3.15.0-devY 3.16.0 stable uhm ... ? what happens if a patch will be unstable for ages as it was some kind of paid-for-feature but time ran out? or if a feature becomes obsolete because of another feature? b) why do you want to stick to some in-between-version - especially with the v3.x scheme which i still see as progressing as 3.14.0, 3.15.0, 3.16.0 as in the old-style so nobody is confused. therefore i call 3.14.0 3.15.0 etc. in-betwee-versions, if 3.16 is under development) mhm, now an idea begins to take hold of me ;) do you want to totally switch to the new scheme now? or do you want to keep some kind of legacy numbering for v3? maybe my problem is, that i somehow see the versioning this way: v3.pl or rather v3.subminor.pl and v4.minor.pl or rather v4.minor.subminor.pl (such as we can see with kernel v2.6.24.4) so 3.15.0 is translated to 3.0.15 in my mind. so the next step would be 3.0.16 which would become 3.16.x in the "old" release style. perhaps you can enlighten me/us? :) and maybe, if you want to switch to the final version scheme, we should jump over a couple of releases and start with 3.20.0 to make things clear? cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From rgerhards at hq.adiscon.com Mon Mar 31 17:01:35 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:01:35 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <200803311651.12753.pvrabec@redhat.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> Peter, Ah, so you compare the strings... makes sense. But also makes clear we have a problem with any such scheme... ... that means it applies to Johnn's suggestion, too :( Next try, back to odd/even. So any odd version number is unstable, any even one is stable. Once something gets stable, it is incremented to the even version. -rc and similar things may (may!) be added to alert users. Again into the sample case: 3.14.0 stable 3.15.6 unstable (relp) 3.17.2 unstable (relp/tcp) Now let's assume I add a bugfix for the core engine. Would that bring us to 3.14.1 stable 3.15.7 unstable (relp) 3.17.3 unstable (relp/tcp) Once relp is stable, we have 3.14.1 deprecated 3.16.0 stable (relp) 3.17.3 unstable (relp/tcp) TLS begins: 3.14.1 deprecated 3.16.0 stable (relp) 3.17.3 unstable (relp/tcp) 3.19.0 unstable (tls) Now tcp becomes stable: 3.14.1 deprecated 3.16.0 deprecated 3.18.0 stable (relp/tcp) 3.19.0 unstable (tls) In all cases stable > devel, at least as far as the some feature set is concerned. There may be gaps in the minor version numbers when versions are not yet stable. Does that work? Rainer > -----Original Message----- > From: Peter Vrabec [mailto:pvrabec at redhat.com] > Sent: Monday, March 31, 2008 4:51 PM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: [rsyslog] rsyslog version numbering > > Hi, > > > 3.13.6 stable > > 3.14.0-dev7 (relp) > > 3.15.0-dev4 (relp/tcp) > > > > Once relp is stable, we have > > > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > There is a great disadvatage of this scheme. > 3.14.0-dev7 > 3.14.0, BUT 3.14.0 is newer. From friedl at hq.adiscon.com Mon Mar 31 17:08:57 2008 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 31 Mar 2008 17:08:57 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Hi all, I have just release rsyslog 3.13.0-dev0, which will hopefully become the next stable release by mid-week. It fixes two bugs, one is a potential segfault in the syslog/plain tcp receiver. The other one is removal of some debug instrumentation that accidently made it into 3.12.5. There is also a new ./configure option (--enable/disable-rsyslogd) which permits to build just specific plugins without the main binary. This has been added to support source-based packaging systems (like in FreeBSD). This is also a test for one potential new versioning scheme. 3.13.0-dev0 is recommended for all v3 branch users. Feedback on it is highly appreciated. Changelog: http://www.rsyslog.com/Article201.phtml Download: http://www.rsyslog.com/Downloads-req-getit-lid-92.phtml Florian Riedl From linuxweb at gmail.com Mon Mar 31 17:21:19 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 11:21:19 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com> <577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> Message-ID: <47F1016F.6010200@gmail.com> Rainer Gerhards wrote: > Peter, > > Ah, so you compare the strings... makes sense. But also makes clear we > have a problem with any such scheme... > > ... that means it applies to Johnn's suggestion, too :( No, I made a mistake of keeping the numbering when my main point was more about, if you are going to use appendages, to stick with the well-known ones (pre, alpha, beta, rc, stable, patch). It should work with *any* numbering scheme (yours, Michael's, the even/odd, etc.). The value-added benefit is it gives people an instant knowledge of where the branch stands. My company doesn't mind using rc's in production if we absolutely need the features it provides. But using an alpha would be much harder to justify. Granted, in the end, these are somewhat all arbitrary (even declaring something stable). But I think, like you said, most devs (or the communities that surround a project) have a general sense that the passage of time and what actually gets added (bug-fix-only, etc.) to a branch help determine these labels. That's generally good enough for most people who end up using the software. johnn From rgerhards at hq.adiscon.com Mon Mar 31 17:25:57 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:25:57 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F1016F.6010200@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <200803311651.12753.pvrabec@redhat.com><577465F99B41C842AAFBE9ED71E70ABA308CB5@grfint2.intern.adiscon.com> <47F1016F.6010200@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB7@grfint2.intern.adiscon.com> I couldn't agree more. I have a maturity matrix on my mind. I guess I'll do a sample of it. Maybe that helps find a solution. The core thing is that with a truly modular design like we have in rsyslog nothing is stable or unstable per se. Some components are extremely stable, while others are bleeding edge. As long as you don't use the bleeding edge ones (and they did not touch core features) the last ironed-out alpha devel version is rock solid. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Monday, March 31, 2008 5:21 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Rainer Gerhards wrote: > > Peter, > > > > Ah, so you compare the strings... makes sense. But also makes clear > we > > have a problem with any such scheme... > > > > ... that means it applies to Johnn's suggestion, too :( > > No, I made a mistake of keeping the numbering when my main > point was more about, if you are going to use appendages, to > stick with the well-known ones (pre, alpha, beta, rc, > stable, patch). > > It should work with *any* numbering scheme (yours, > Michael's, the even/odd, etc.). > > > The value-added benefit is it gives people an instant > knowledge of where the branch stands. > > My company doesn't mind using rc's in production if we > absolutely need the features it provides. But using an alpha > would be much harder to justify. > > Granted, in the end, these are somewhat all arbitrary (even > declaring something stable). But I think, like you said, > most devs (or the communities that surround a project) have > a general sense that the passage of time and what actually > gets added (bug-fix-only, etc.) to a branch help determine > these labels. That's generally good enough for most people > who end up using the software. > > johnn > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From rgerhards at hq.adiscon.com Mon Mar 31 17:32:06 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 17:32:06 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F0FBFA.3020802@ipax.at> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F0FBFA.3020802@ipax.at> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CB8@grfint2.intern.adiscon.com> Well... First thing is that I have urgent need to release -- there are a couple of things in the queue. If I don't release them this week, I'll probably don't have a chance to get to a stable any time soon. So I will release, even at the price that the version number scheme may be less than optimum this time. But now on to the real meat (and thanks for sticking around on this topic! ;))... > Rainer Gerhards wrote: > > This has some subtleties for me, too (well, maybe its not really > > subtleties but simply frightening me ;)). Let me try to get this back > to > > a simple path. How about a simple > > > > major.minor.patchlevel-dev > > > > So basically the "Sunday scheme" without -mf/-rc but instead just - > dev. > > I guess the scheme was good for all, but the -mf/-rc was deemed > > confusing. In the sample, it would look like: > > > > 3.13.5 stable > > 3.14.0-dev6 (relp) > > 3.15.0-dev3 (relp/tcp) > > what i'm not getting is, that you > a) want to maintain some work-in-progress-patches as seperate trees. > what happens if you want to implement feature-x (ssl/tls) first > (which would then be 3.17.0) and this becomes stable faster than > relp/other stuff? > > then we have > 3.13.5 > 3.14.0-devX > 3.15.0-devY > 3.16.0 stable > > uhm ... ? [I am answering this in the context of the version numbers above. After Peter's note on the strcmp() of version numbers, I think that we need to use something like odd/even where stable is compares always greater than unstable.] In this case, there will never be a 3.14 and 3.15 stable. This scenario may happen. In my current scheme, we'd probably never seen a 3.11 stable (queue engine) but a 3.12, because the queue engine needed much more time to mature. > what happens if a patch will be unstable for ages as it was some kind > of > paid-for-feature but time ran out? or if a feature becomes obsolete > because of another feature? Maybe the problem is that I am NOT branching off for new features. I branch off stable releases. I fear that branching of for new features will be much more time consuming at the current pace of development and has high potential for conflict. Maybe I am wrong ;) To the cases above: the paid-for-feature would never be released. That case is not fully thought out as I did not yet find someone who want's to pay. My perception may change when this happens ;) If a feature that is not yet stable is obsoleted, that version will never become stable (gap in release history). > > b) why do you want to stick to some in-between-version - especially > with > the v3.x scheme > > which i still see as progressing as 3.14.0, 3.15.0, > 3.16.0 as in the old-style so nobody is confused. therefore i call > 3.14.0 3.15.0 etc. in-betwee-versions, if 3.16 is under development) I have a major release focus for v3: full modularization. Not reached yet ;) Of course, that can be sacrificed. But as long as we experiment (and we seem to do), I prefer to do that in v3 instead of risking to lose v4, too (when we go to something else) ;) > > mhm, now an idea begins to take hold of me ;) > > do you want to totally switch to the new scheme now? or do you want to > keep some kind of legacy numbering for v3? > > maybe my problem is, that i somehow see the versioning this way: > > v3.pl or rather v3.subminor.pl and > v4.minor.pl or rather v4.minor.subminor.pl (such as we can see with > kernel v2.6.24.4) > > so 3.15.0 is translated to 3.0.15 in my mind. so the next step would be > 3.0.16 which would become 3.16.x in the "old" release style. > > perhaps you can enlighten me/us? :) Well, *if* I'd taken proper stable branchpoints, we would now probably be at 3.3 or 3.4 speaking stable-wise (3.0 intial release and input plugins, 3.1 queue (probably never delivered), 3.2 expression support, ...). So far, rsyslog did a stable release once every 12 to 24 month. From there on, everything was experimental. We can keep with that scheme (less work for me), but the drawback is that it takes an immense amount of time before those looking for stable can find new features. IMHO this hasn't worked out well. > > and maybe, if you want to switch to the final version scheme, we should > jump over a couple of releases and start with 3.20.0 to make things > clear? I agree, maybe even v4 - but then the version scheme itself must have reached stable state ;) Rainer > > cheers, > raoul > -- > ____________________________________________________________________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ____________________________________________________________________ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From linuxweb at gmail.com Mon Mar 31 17:44:43 2008 From: linuxweb at gmail.com (Johnny Tan) Date: Mon, 31 Mar 2008 11:44:43 -0400 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at> <577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> Message-ID: <47F106EB.6090504@gmail.com> Rainer Gerhards wrote: > confusing. In the sample, it would look like: > > 3.13.5 stable > 3.14.0-dev6 (relp) > 3.15.0-dev3 (relp/tcp) Ok, let me take one last swipe at this, with numbering AND labels: 3.13.5 stable 3.14.6-beta 3.15.3-alpha > > Now let's assume I add a bugfix for the core engine. Would that bring us > to > > 3.13.6 stable > 3.14.0-dev7 (relp) > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.7-beta 3.15.4-alpha > Once relp is stable, we have > > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) 3.13.6 stable 3.14.8-rc 3.15.5-alpha > TLS begun: > 3.13.6 deprecated > 3.14.0 stable relp > 3.15.0-dev4 (relp/tcp) > 3.16.0-dev0 (tls) 3.13.6 deprecated 3.14.9 stable 3.15.5-beta 3.16.0-alpha > Now tcp becomes stable: > > 3.13.6 deprecated > 3.14.0 deprecated > 3.15.0 stable (relp/tcp) > 3.16.0-dev0 (tls) Well, this is kind of a big jump, but assuming it goes through all the proper alpha/beta/rc phases: 3.13.6 deprecated 3.14.9 deprecated 3.15.6 stable 3.16.1-alpha So, you increment the patchlevels, as you've been doing, but you use -alpha, -beta, -rc (with no numbers) to designate the newness/readiness of the branch. From rgerhards at hq.adiscon.com Mon Mar 31 18:06:34 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 18:06:34 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <47F106EB.6090504@gmail.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F106EB.6090504@gmail.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> Please have a look at: http://www.rsyslog.com/doc-maturity.html It's not complete yet, because it requires some work. If it is done parallel to the releases, it's not much work. I personally would find this much more useful than a single binary switch for all of the modules (which never is really right ;)). That, of course, doesn't solve the version numbering question ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Johnny Tan > Sent: Monday, March 31, 2008 5:45 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog version numbering > > Rainer Gerhards wrote: > > confusing. In the sample, it would look like: > > > > 3.13.5 stable > > 3.14.0-dev6 (relp) > > 3.15.0-dev3 (relp/tcp) > > Ok, let me take one last swipe at this, with numbering AND > labels: > > 3.13.5 stable > 3.14.6-beta > 3.15.3-alpha > > > > > > Now let's assume I add a bugfix for the core engine. Would that bring > us > > to > > > > 3.13.6 stable > > 3.14.0-dev7 (relp) > > 3.15.0-dev4 (relp/tcp) > > 3.13.6 stable > 3.14.7-beta > 3.15.4-alpha > > > > Once relp is stable, we have > > > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > 3.13.6 stable > 3.14.8-rc > 3.15.5-alpha > > > > TLS begun: > > 3.13.6 deprecated > > 3.14.0 stable relp > > 3.15.0-dev4 (relp/tcp) > > 3.16.0-dev0 (tls) > > 3.13.6 deprecated > 3.14.9 stable > 3.15.5-beta > 3.16.0-alpha > > > > Now tcp becomes stable: > > > > 3.13.6 deprecated > > 3.14.0 deprecated > > 3.15.0 stable (relp/tcp) > > 3.16.0-dev0 (tls) > > Well, this is kind of a big jump, but assuming it goes > through all the proper alpha/beta/rc phases: > > 3.13.6 deprecated > 3.14.9 deprecated > 3.15.6 stable > 3.16.1-alpha > > So, you increment the patchlevels, as you've been doing, but > you use -alpha, -beta, -rc (with no numbers) to designate > the newness/readiness of the branch. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 19:15:59 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 19:15:59 +0200 Subject: [rsyslog] openssl vs rsyslog Message-ID: Hi Rainer, I noticed that you added an openssl configure check. That probably means, that you plan to add native SSL support via openssl. Please note that OpenSSL and the GPL are incompatible [1], such packages are usually rejected from Debian. You have two options: - Either add a special exemption for OpenSSL - Use another SSL implementation like GnuTLS [2] or NSS [3] Cheers, Michael [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html [2] http://www.gnu.org/software/gnutls/ [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 19:52:37 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 19:52:37 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Florian Riedl : > Hi all, > > I have just release rsyslog 3.13.0-dev0, which will hopefully become the > next stable release by mid-week. > > It fixes two bugs, one is a potential segfault in the syslog/plain tcp > receiver. The other one is removal of some debug instrumentation that > accidently made it into 3.12.5. There is also a new ./configure option > (--enable/disable-rsyslogd) which permits to build just specific plugins Hm, the default of enable_rsyslogd should imho be "yes" > without the main binary. This has been added to support source-based > packaging systems (like in FreeBSD). This is also a test for one > potential new versioning scheme. I don't see the use case yet. Why would someone want to build the plugins without the rsyslogd binary? Does it actually work if you mix different versions of plugins and rsyslogd? If someone is interested in only a specific plugin, why not use "make -C plugins/foo"? Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From mbiebl at gmail.com Mon Mar 31 20:04:18 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 20:04:18 +0200 Subject: [rsyslog] rsyslog 3.13.0-dev0 released In-Reply-To: References: <577465F99B41C842AAFBE9ED71E70ABA308CB6@grfint2.intern.adiscon.com> Message-ID: 2008/3/31, Michael Biebl : > 2008/3/31, Florian Riedl : > > > Hi all, > > > > I have just release rsyslog 3.13.0-dev0, which will hopefully become the > > next stable release by mid-week. > > > > It fixes two bugs, one is a potential segfault in the syslog/plain tcp > > receiver. The other one is removal of some debug instrumentation that > > accidently made it into 3.12.5. There is also a new ./configure option > > (--enable/disable-rsyslogd) which permits to build just specific plugins > > > Hm, the default of enable_rsyslogd should imho be "yes" > > > > without the main binary. This has been added to support source-based > > packaging systems (like in FreeBSD). This is also a test for one > > potential new versioning scheme. > > > I don't see the use case yet. Why would someone want to build the > plugins without the rsyslogd binary? Does it actually work if you mix > different versions of plugins and rsyslogd? > If someone is interested in only a specific plugin, why not use "make > -C plugins/foo"? Forgot to add: If you intend to build rsyslogd and rfc3195d conditionally, the corresponding man pages should also be installed conditionally. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 20:14:01 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 20:14:01 +0200 Subject: [rsyslog] openssl vs rsyslog Message-ID: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) Can you point me to the problem description? What would i need to exempt? Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... Any advise is deeply appreciated. Thanks, rainer ----- Urspr?ngliche Nachricht ----- Von: "Michael Biebl" An: "rsyslog-users" Gesendet: 31.03.08 19:16 Betreff: [rsyslog] openssl vs rsyslog Hi Rainer, I noticed that you added an openssl configure check. That probably means, that you plan to add native SSL support via openssl. Please note that OpenSSL and the GPL are incompatible [1], such packages are usually rejected from Debian. You have two options: - Either add a special exemption for OpenSSL - Use another SSL implementation like GnuTLS [2] or NSS [3] Cheers, Michael [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html [2] http://www.gnu.org/software/gnutls/ [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog From mbiebl at gmail.com Mon Mar 31 20:34:40 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 20:34:40 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: 2008/3/31, Rainer Gerhards : > Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) > > Can you point me to the problem description? What would i need to exempt? Google for "openssl gpl" and you should find relevant information. E.g. [1] tries to summarize the problem. It also has a proposal for such a OpensSSL exemption. You should add this addendum to COPYING and probably also ship the original OpenSSL COPYING file (e.g. as COPYING.OpenSSL) > > Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... A prominent example of a project using GnuTLS is e.g. samba (if it's highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. A prominent example of a project using NSS is mozilla/firefox/thunderbird. I think the more widely used library is GnuTLS. I really don't have that much experience with either of these libraries, so it's hard to give a recommendation. Cheers, Michael > [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html > [2] http://www.gnu.org/software/gnutls/ > [3] http://www.mozilla.org/projects/security/pki/nss/ -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 20:47:03 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 20:47:03 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 8:35 PM > To: rsyslog-users > Subject: Re: [rsyslog] openssl vs rsyslog > > 2008/3/31, Rainer Gerhards : > > Oh, excellent feedback. And I thought that openssl is the > workhorse of os tls - wrong preception obviously ;) > > > > Can you point me to the problem description? What would i > need to exempt? > > Google for "openssl gpl" and you should find relevant information. > E.g. [1] tries to summarize the problem. > It also has a proposal for such a OpensSSL exemption. You should add > this addendum to COPYING and probably also ship the original OpenSSL > COPYING file (e.g. as COPYING.OpenSSL) Yes - sorry. I overlooked the link on the PDA. Will mow read through it. But if openssl means trouble, I'll prefer to stay away from it unless there is *very* good reason. You wake-up call was *right* in time. I planned to do some major hacking tomorrow - nothing lost so far :) [and, in case you wonder, yes, initial RELP is done ;)]. > > Is any of the alternates suitable for production use, > especially in a highly threaded environment? GNU, from the > name, sounds appealing... > > > A prominent example of a project using GnuTLS is e.g. samba (if it's > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > A prominent example of a project using NSS is > mozilla/firefox/thunderbird. > > I think the more widely used library is GnuTLS. I really don't have > that much experience with either of these libraries, so it's hard to > give a recommendation. At least it doesn't sound too bad ;) Anyone with experience please comment. Thanks again, Rainer > > Cheers, > Michael > > > > [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html > > [2] http://www.gnu.org/software/gnutls/ > > [3] http://www.mozilla.org/projects/security/pki/nss/ > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > From milton at calnek.com Mon Mar 31 21:00:11 2008 From: milton at calnek.com (Milton Calnek) Date: Mon, 31 Mar 2008 13:00:11 -0600 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CBF@grfint2.intern.adiscon.com> Message-ID: <47F134BB.9090705@calnek.com> Don't know if this helps, or if you got the same from COPYING.Openssl. http://www.openssl.org/support/faq.html#LEGAL1 Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl >> Sent: Monday, March 31, 2008 8:35 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] openssl vs rsyslog >> >> 2008/3/31, Rainer Gerhards : >>> Oh, excellent feedback. And I thought that openssl is the >> workhorse of os tls - wrong preception obviously ;) >>> Can you point me to the problem description? What would i >> need to exempt? >> >> Google for "openssl gpl" and you should find relevant information. >> E.g. [1] tries to summarize the problem. >> It also has a proposal for such a OpensSSL exemption. You should add >> this addendum to COPYING and probably also ship the original OpenSSL >> COPYING file (e.g. as COPYING.OpenSSL) > > Yes - sorry. I overlooked the link on the PDA. Will mow read through it. > But if openssl means trouble, I'll prefer to stay away from it unless > there is *very* good reason. You wake-up call was *right* in time. I > planned to do some major hacking tomorrow - nothing lost so far :) [and, > in case you wonder, yes, initial RELP is done ;)]. > >>> Is any of the alternates suitable for production use, >> especially in a highly threaded environment? GNU, from the >> name, sounds appealing... >> >> >> A prominent example of a project using GnuTLS is e.g. samba (if it's >> highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. >> A prominent example of a project using NSS is >> mozilla/firefox/thunderbird. >> >> I think the more widely used library is GnuTLS. I really don't have >> that much experience with either of these libraries, so it's hard to >> give a recommendation. > > At least it doesn't sound too bad ;) > > Anyone with experience please comment. > > Thanks again, > Rainer >> Cheers, >> Michael >> >> >>> [1] http://www.gnome.org/~markmc/openssl-and-the-gpl.html >>> [2] http://www.gnu.org/software/gnutls/ >>> [3] http://www.mozilla.org/projects/security/pki/nss/ >> >> -- >> Why is it that all of the instruments seeking intelligent life in the >> universe are pointed away from Earth? >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > -- Milton Calnek BSc, A/Slt(Ret.) milton at calnek.com 306-717-8737 -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From skvidal at fedoraproject.org Mon Mar 31 20:38:19 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 14:38:19 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> Message-ID: <1206988699.9240.38.camel@cutter> On Mon, 2008-03-31 at 20:34 +0200, Michael Biebl wrote: > 2008/3/31, Rainer Gerhards : > > Oh, excellent feedback. And I thought that openssl is the workhorse of os tls - wrong preception obviously ;) > > > > Can you point me to the problem description? What would i need to exempt? > > Google for "openssl gpl" and you should find relevant information. > E.g. [1] tries to summarize the problem. > It also has a proposal for such a OpensSSL exemption. You should add > this addendum to COPYING and probably also ship the original OpenSSL > COPYING file (e.g. as COPYING.OpenSSL) > > > > > Is any of the alternates suitable for production use, especially in a highly threaded environment? GNU, from the name, sounds appealing... > > > A prominent example of a project using GnuTLS is e.g. samba (if it's > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > A prominent example of a project using NSS is mozilla/firefox/thunderbird. > > I think the more widely used library is GnuTLS. I really don't have > that much experience with either of these libraries, so it's hard to > give a recommendation. > the library that a lot of folks inside red hat and fedora are driving people to is the mozilla nss library. It handles all the bits openssl does and w/o the licensing problems. -sv From rgerhards at hq.adiscon.com Mon Mar 31 21:10:04 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:10:04 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206988699.9240.38.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> Hi sv, > the library that a lot of folks inside red hat and fedora are driving > people to is the mozilla nss library. It handles all the bits openssl > does and w/o the licensing problems. Sounds promising. I just found http://www.gnu.org/software/gnutls/comparison.html And that makes GNU TLS quite appealing (no wonder given the source ;)). Do you happen to have a link which tells why use nss? That would be excellent (but I am of course searching myself). Thanks, Rainer From r.bhatia at ipax.at Mon Mar 31 21:16:16 2008 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Mon, 31 Mar 2008 21:16:16 +0200 Subject: [rsyslog] rsyslog version numbering In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA308C4B@grfint2.intern.adiscon.com><47ED1625.9030305@gmail.com><577465F99B41C842AAFBE9ED71E70ABA308C98@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308C9B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA4@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA308CA9@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA308CAB@grfint2.intern.adiscon.com> <47F0E4A8.50207@ipax.at><577465F99B41C842AAFBE9ED71E70ABA308CB1@grfint2.intern.adiscon.com> <47F106EB.6090504@gmail.com> <577465F99B41C842AAFBE9ED71E70ABA308CBA@grfint2.intern.adiscon.com> Message-ID: <47F13880.3020506@ipax.at> Rainer Gerhards wrote: > Please have a look at: > > http://www.rsyslog.com/doc-maturity.html > > It's not complete yet, because it requires some work. If it is done > parallel to the releases, it's not much work. I personally would find > this much more useful than a single binary switch for all of the modules > (which never is really right ;)). That, of course, doesn't solve the > version numbering question ;) as a short input if you want to provide a feature matrix: http://linux-vserver.org/Feature_Matrix cheers, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OEG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From mbiebl at gmail.com Mon Mar 31 21:18:33 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 21:18:33 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206988699.9240.38.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: 2008/3/31, seth vidal : > > A prominent example of a project using GnuTLS is e.g. samba (if it's > > highly threaded, I don't know). OpenLDAP in Debian also uses GnuTLS. > > A prominent example of a project using NSS is mozilla/firefox/thunderbird. > > > > I think the more widely used library is GnuTLS. I really don't have > > that much experience with either of these libraries, so it's hard to > > give a recommendation. > > > > > the library that a lot of folks inside red hat and fedora are driving > people to is the mozilla nss library. It handles all the bits openssl > does and w/o the licensing problems. Interesting to know. Do you know any technical advantages of NSS over GnuTLS (stability, features, nicer API, etc)? At least on Debian I had the impression that GnuTLS was the preferred solution. E.g. samba, openldap and exim4 are originally using openssl but contain a patch to use GnuTLS in Debian. One reason might be, that libgnutls is quite a bit smaller: gnutls: installed size (on i386): 920 kb [1] libnss3: installed size (on i386): 2472 kb, also requires the netscape portable runtime library libnpr (588 kb) => ~3Mb [2] Which imho is a plus for GnuTLS. [1] http://packages.debian.org/sid/libgnutls26 [2] http://packages.debian.org/sid/libnss3-1d -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From skvidal at fedoraproject.org Mon Mar 31 21:21:26 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 15:21:26 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> <577465F99B41C842AAFBE9ED71E70ABA308CC5@grfint2.intern.adiscon.com> Message-ID: <1206991286.9240.43.camel@cutter> On Mon, 2008-03-31 at 21:10 +0200, Rainer Gerhards wrote: > Hi sv, > > > the library that a lot of folks inside red hat and fedora are driving > > people to is the mozilla nss library. It handles all the bits openssl > > does and w/o the licensing problems. > > Sounds promising. I just found > > http://www.gnu.org/software/gnutls/comparison.html > > > And that makes GNU TLS quite appealing (no wonder given the source ;)). > Do you happen to have a link which tells why use nss? That would be > excellent (but I am of course searching myself). > nss is fips certified: http://www.mozilla.org/projects/security/pki/nss/fips/ which makes it easier to use in large gov't agencies. gnutls is not certified. nss is also where the fedora/red hat crypto consolidation is going.... there is also an api-helper library for transitioning from openssl to nss. if at all possible, go with nss. -sv From skvidal at fedoraproject.org Mon Mar 31 21:22:19 2008 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 31 Mar 2008 15:22:19 -0400 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> Message-ID: <1206991339.9240.45.camel@cutter> On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > Interesting to know. Do you know any technical advantages of NSS over > GnuTLS (stability, features, nicer API, etc)? openssl transition library fips certification. http://www.mozilla.org/projects/security/pki/nss/fips/ the second one being the most important, I think. -sv From mbiebl at gmail.com Mon Mar 31 21:26:51 2008 From: mbiebl at gmail.com (Michael Biebl) Date: Mon, 31 Mar 2008 21:26:51 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206991339.9240.45.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com> <1206988699.9240.38.camel@cutter> <1206991339.9240.45.camel@cutter> Message-ID: 2008/3/31, seth vidal : > On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > > Interesting to know. Do you know any technical advantages of NSS over > > GnuTLS (stability, features, nicer API, etc)? > > > openssl transition library > fips certification. > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > the second one being the most important, I think. Yeah, the openssl transition library doesn't concern us. Why is the fips certification important for rsyslog? Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Mon Mar 31 21:26:59 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:26:59 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: <1206991339.9240.45.camel@cutter> References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com><1206988699.9240.38.camel@cutter> <1206991339.9240.45.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC7@grfint2.intern.adiscon.com> > > Interesting to know. Do you know any technical advantages > of NSS over > > GnuTLS (stability, features, nicer API, etc)? > > openssl transition library > fips certification. > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > the second one being the most important, I think. > > -sv FIPS is a strong selling point, I have to admit. Especially as rsyslog is heading towards the compliance space and trying to be as tamper-proof as possible. On a quick lock, GNUTLS looks a bit nicer to me (the NSPL doesn't sound really good whereas the GNUTLS zlib implementations solves something I intended to do). But FIPS... Rainer From rgerhards at hq.adiscon.com Mon Mar 31 21:29:52 2008 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 31 Mar 2008 21:29:52 +0200 Subject: [rsyslog] openssl vs rsyslog In-Reply-To: References: <002201c8935b$0c6287e8$060013ac@intern.adiscon.com><1206988699.9240.38.camel@cutter><1206991339.9240.45.camel@cutter> Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308CC8@grfint2.intern.adiscon.com> Whatever I do - I'll try to create a small driver shim so that hopefully different libraries could be used together with rsyslog. It depends on the APIs, but in general that should not be too hard (but practice often tells the difference). On FIPS I made I comment, where I think the mails crossed. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Monday, March 31, 2008 9:27 PM > To: rsyslog-users > Subject: Re: [rsyslog] openssl vs rsyslog > > 2008/3/31, seth vidal : > > On Mon, 2008-03-31 at 21:18 +0200, Michael Biebl wrote: > > > Interesting to know. Do you know any technical > advantages of NSS over > > > GnuTLS (stability, features, nicer API, etc)? > > > > > > openssl transition library > > fips certification. > > > > http://www.mozilla.org/projects/security/pki/nss/fips/ > > > > the second one being the most important, I think. > > Yeah, the openssl transition library doesn't concern us. > > Why is the fips certification important for rsyslog? > > Cheers, > Michael > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog >