From Ole.Rahn at t-systems.com Fri Apr 1 15:09:34 2011 From: Ole.Rahn at t-systems.com (Ole.Rahn at t-systems.com) Date: Fri, 1 Apr 2011 15:09:34 +0200 Subject: [rsyslog] (no subject) Message-ID: Hi, I am just starting to use rsyslog in a quite big network and in a quite complex environment. I would like several network elements to send their logs to a central server. *Some* messages (containing confidential information) should be encrypted via TSL, some should not be encrypted, but should be sent through plain TCP (for a certain level of reliability), in order to lower the cpu load on the network node. I have been looking through the documentation, but could not come a clear picture, e.g. this article (from 2008, see "Limitations") says, that neither network elements are authenticated, nor can TSL and plain TCP be mixed within one server instance. http://www.rsyslog.com/doc/rsyslog_tls.html Both limitations are said to be fixed, soon (since the article was written in 2008, I guess "soon" is already a few years in the past). While the limitation in regard to the authentication does not seem to exist, anymore, (http://www.rsyslog.com/doc/ns_gtls.html), I could not find information about the other one. That is why I would like to ask: * Can rsyslog listen to plain TCP and TLS-encrypted messages at the same time? * Does that require multiple instances? o If yes: Is there any guide how to setup multiple instances? o if no: Do I need 2 separate TCP ports or can rsyslog "magically" distinguish plain TCP from TSL traffic on one port? I would be glad for feedback or other hints how to implement this setup. Best regards Ole From rgerhards at hq.adiscon.com Fri Apr 1 16:41:51 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 1 Apr 2011 16:41:51 +0200 Subject: [rsyslog] (no subject) In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF1C@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Ole.Rahn at t-systems.com > Sent: Friday, April 01, 2011 3:10 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] (no subject) > > Hi, > > I am just starting to use rsyslog in a quite big network and in a quite complex > environment. > > I would like several network elements to send their logs to a central server. > *Some* messages (containing confidential information) should be encrypted > via TSL, some should not be encrypted, but should be sent through plain TCP > (for a certain level of reliability), in order to lower the cpu load on the > network node. > > I have been looking through the documentation, but could not come a clear > picture, e.g. this article (from 2008, see "Limitations") says, that neither > network elements are authenticated, nor can TSL and plain TCP be mixed > within one server instance. > http://www.rsyslog.com/doc/rsyslog_tls.html > > Both limitations are said to be fixed, soon (since the article was written in > 2008, I guess "soon" is already a few years in the past). Thanks for spotting this. Authentication is available since long. I think there is a restriction in regard to multiple listeners and/or forwarders, but I have no time at the moment to check this (paid work eating up time ;)). I'd suggest just to give it a try. If it doesn't work, the code is designed to do that, so you may be able to create a patch to enable it with relatively little effort. > > While the limitation in regard to the authentication does not seem to exist, > anymore, (http://www.rsyslog.com/doc/ns_gtls.html), I could not find > information about the other one. > > That is why I would like to ask: > > * Can rsyslog listen to plain TCP and TLS-encrypted messages at the same > time? > > * Does that require multiple instances? > > o If yes: Is there any guide how to setup multiple instances? I don't think so, but it is common sense: make sure you do not use the same pid files, input ports local log sockets etc -- nothing very rsyslog specific. > > o if no: Do I need 2 separate TCP ports or can rsyslog "magically" distinguish > plain TCP from TSL traffic on one port? You definitely need two different listeners. There is also a much more elaborate TLS guide available at http://www.rsyslog.com/doc/rsyslog_secure_tls.html This is the guide the other paper at its top recommends to read ;) Rainer > > I would be glad for feedback or other hints how to implement this setup. > > Best regards > Ole > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Ole.Rahn at t-systems.com Fri Apr 1 16:58:17 2011 From: Ole.Rahn at t-systems.com (Ole.Rahn at t-systems.com) Date: Fri, 1 Apr 2011 16:58:17 +0200 Subject: [rsyslog] (no subject) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF1C@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DDF1C@GRFEXC.intern.adiscon.com> Message-ID: Thank you for the very quick response! I will give it a try then... Ole From rgerhards at hq.adiscon.com Mon Apr 4 09:33:41 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 4 Apr 2011 09:33:41 +0200 Subject: [rsyslog] Request for Comment: rebind interval / omfwd changes Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF1F@GRFEXC.intern.adiscon.com> Hi all, we have a parameter that permits to configure omfwd to rebind to its port every n messages. I have begun to look at omfwd to optimize it. A main goal is to reduce the send() API calls to one (or few ) time(s) per batch. This conflicts with the current semantics of rebinding. My current thinking is that a rebind based on send() calls is sufficient. Does anyone has any concerns? Thanks, Rainer From david at lang.hm Mon Apr 4 09:43:30 2011 From: david at lang.hm (david at lang.hm) Date: Mon, 4 Apr 2011 00:43:30 -0700 (PDT) Subject: [rsyslog] Request for Comment: rebind interval / omfwd changes In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF1F@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DDF1F@GRFEXC.intern.adiscon.com> Message-ID: On Mon, 4 Apr 2011, Rainer Gerhards wrote: > Hi all, > > we have a parameter that permits to configure omfwd to rebind to its port > every n messages. I have begun to look at omfwd to optimize it. A main goal > is to reduce the send() API calls to one (or few ) time(s) per batch. This > conflicts with the current semantics of rebinding. My current thinking is > that a rebind based on send() calls is sufficient. > > Does anyone has any concerns? sounds reasonable, the purpose of the rebind is so that you can do load balancing of the outbound connection with a network-based load balancer. doing this a handful of times per second is pleanty (you want to keep all receivers reasonably busy without overrunning any of them, expecting a receiver to be able to handle a full second's worth of full-thottle traffic is not excessive, and cycling through all the receivers once per second should be enough to spread the load) David Lang From rgerhards at hq.adiscon.com Mon Apr 4 14:06:33 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 4 Apr 2011 14:06:33 +0200 Subject: [rsyslog] How to create a central rulebase repository for log normalization? Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF2C@GRFEXC.intern.adiscon.com> Hi all, please have a look at my post over here: http://blog.gerhards.net/2011/04/log-normalization-how-to-share.html This is a very important question, and I'd appreciate all comments and feedback. Thanks, Rainer From nikolaidis.fotis at gmail.com Mon Apr 4 14:25:03 2011 From: nikolaidis.fotis at gmail.com (Nikolaidis Fotis) Date: Mon, 04 Apr 2011 14:25:03 +0200 Subject: [rsyslog] How to create a central rulebase repository for log normalization? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF2C@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DDF2C@GRFEXC.intern.adiscon.com> Message-ID: <4D99B89F.60206@gmail.com> Hello everyone. I would say to follow a web-site approach. It's easier to submit something and the most important is that you can have a section for comments (what is missing, suggestions etc). Moreover, if the rulebase is incomplete / buggy / anything it's quite easy to withdraw it. What do you think ? cheers On 04/04/2011 02:06 PM, Rainer Gerhards wrote: > Hi all, > > please have a look at my post over here: > > http://blog.gerhards.net/2011/04/log-normalization-how-to-share.html > > This is a very important question, and I'd appreciate all comments and > feedback. > > Thanks, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > -- telnet towel.blinkenlights.nl From champ at softwink.com Mon Apr 4 16:37:56 2011 From: champ at softwink.com (Champ Clark III [Softwink]) Date: Mon, 4 Apr 2011 10:37:56 -0400 Subject: [rsyslog] How to create a central rulebase repository for log normalization? In-Reply-To: <4D99B89F.60206@gmail.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DDF2C@GRFEXC.intern.adiscon.com> <4D99B89F.60206@gmail.com> Message-ID: <20110404143756.GA16247@bundy.vistech.net> On Mon, Apr 04, 2011 at 02:25:03PM +0200, Nikolaidis Fotis wrote: > Hello everyone. > > I would say to follow a web-site approach. > It's easier to submit something and the most important is that you can > have a section for comments (what is missing, suggestions etc). > Moreover, if the rulebase is incomplete / buggy / anything it's quite > easy to withdraw it. I think you do a mixed approach. That is, you keep a "bleeding edge" copy of the rulebase via git. When Rainer feels the rules have become stable enough, then you release a tarball of them. Restart the cycle.... :) Best of both worlds. -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From trefalgar at realitybytes.net Mon Apr 4 20:20:07 2011 From: trefalgar at realitybytes.net (Jacob Steinberger) Date: Mon, 04 Apr 2011 13:20:07 -0500 Subject: [rsyslog] Configuration - disk cache on remote action failure Message-ID: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com> Is there a way for an Action Queue to buffer to disk, if none of the remote destinations are available? I don't mean a standard & /var/log/moo, I mean something that rsyslog will reprocess if any of the remote connections are reestablished. # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional *.* @@loadbalancedhostname $ActionExecOnlyWhenPreviousIsSuspended on & @@hostname1 & @@hostname2 $ActionExecOnlyWhenPreviousIsSuspended off Jacob ps. My forums account no longer works, I was quite sad. From rgerhards at hq.adiscon.com Mon Apr 4 21:12:23 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 4 Apr 2011 21:12:23 +0200 Subject: [rsyslog] Configuration - disk cache on remote action failure In-Reply-To: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com> References: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jacob Steinberger > Sent: Monday, April 04, 2011 8:20 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Configuration - disk cache on remote action failure > > Is there a way for an Action Queue to buffer to disk, if none of the > remote destinations are available? I don't mean a standard & > /var/log/moo, I mean something that rsyslog will reprocess if any of > the remote connections are reestablished. http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html > > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > *.* @@loadbalancedhostname > $ActionExecOnlyWhenPreviousIsSuspended on > & @@hostname1 > & @@hostname2 > $ActionExecOnlyWhenPreviousIsSuspended off > > Jacob > > ps. My forums account no longer works, I was quite sad. What's the account name? Rainer > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From trefalgar at realitybytes.net Mon Apr 4 21:47:27 2011 From: trefalgar at realitybytes.net (Jacob Steinberger) Date: Mon, 04 Apr 2011 14:47:27 -0500 Subject: [rsyslog] Configuration - disk cache on remote action failure In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> References: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com> <9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> Message-ID: <20110404144727.924730mltp7a0573@vincent.asmallorange.com> Quoting Rainer Gerhards : >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Jacob Steinberger >> Sent: Monday, April 04, 2011 8:20 PM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] Configuration - disk cache on remote action failure >> >> Is there a way for an Action Queue to buffer to disk, if none of the >> remote destinations are available? I don't mean a standard & >> /var/log/moo, I mean something that rsyslog will reprocess if any of >> the remote connections are reestablished. > > http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html > >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> *.* @@loadbalancedhostname >> $ActionExecOnlyWhenPreviousIsSuspended on >> & @@hostname1 >> & @@hostname2 >> $ActionExecOnlyWhenPreviousIsSuspended off >> >> Jacob >> >> ps. My forums account no longer works, I was quite sad. > > What's the account name? > > Rainer >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From roland.kamke at auconet.com Tue Apr 5 15:34:02 2011 From: roland.kamke at auconet.com (Roland Kamke) Date: Tue, 5 Apr 2011 15:34:02 +0200 Subject: [rsyslog] Missing log entries from PostgreSQL backend Message-ID: <69DFC544026A844790C013C0FB16B9E0B620DB@eta.firma.lan> A newbie question: With a simple C program I create 8000 syslog messages to rsyslog and wait then a minute before another 8000 messages will be created. This is the code to create the messages, counter is 8000: ------- snip ------- for (iy=0; iy 65), but there is a drop in the log value (64 -> 49), with the difference between 65 and 49 being 16... The messages with the numbers 17..24 and 26..32 (iy in the code) are missing. I built rsyslog from the source rpm archive rsyslog-5.6.2-1.ep.src.rpm using these steps: ./configure --prefix=/opt/rsyslog --enable-silent-rules --enable-dependency-tracking --enable-shared --enable-regexp --enable-zlib --enable-gssapi-krb5 --enable-pthreads --enable-klog --enable-inet --enable-debug --enable-rtinst --enable-memcheck --enable-diagtools --enable-pgsql --enable-snmp --enable-gnutls --enable-mail --enable-imfile --enable-imptcp --enable-omprog --enable-omudpspoof --enable-omstdout --enable-omruleset make make install Is there a way to get all messages in the database (as they appear in the log file)? Thank you, Roland From rgerhards at hq.adiscon.com Tue Apr 5 15:51:14 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 5 Apr 2011 15:51:14 +0200 Subject: [rsyslog] Missing log entries from PostgreSQL backend Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF47@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Roland Kamke > Sent: Tuesday, April 05, 2011 3:34 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Missing log entries from PostgreSQL backend > > A newbie question: > > With a simple C program I create 8000 syslog messages to rsyslog and wait > then a minute before another 8000 messages will be created. This is the code > to create the messages, counter is 8000: > ------- snip ------- > for (iy=0; iy syslog(LOG_LOCAL3 | LOG_INFO, "C test program %d%s, %d", > getpid(), count % 3 == 0 ? " (magic string)" : "", count++); > ------- snap ------- > > This is a small snippet from the configuration: > ------- snip ------- > *.info;mail.none;authpriv.none;cron.none > /var/log/messages > > if $msg contains 'C test program' then > :ompgsql:localhost,secret,secret,secret; > ------- snap ------- > After the first 8000 messages are created I would expect to find 8000 lines in > the log file and 8000 entries in the PostgreSQL database for these messages. > > But: I clean up file system and data base, start rsyslogd and my test program. > After a short while I can find all messages in the log file, but only 7984 in the > data base. This behaviour can be reproduced at will, in any attempt 16 lines > were missing. That's interesting. I unfortunately have no postgres test environment at the moment, so I cannot test this myself. > > Tracing the database I saw that there were only 7984 all inserts into the log > table. > > I ran rsyslogd with -dn option set. The log file created contains these > lines: > ------- snip ------- > 6009.255950000:414d6940: action 9 queue: entry added, size now log 60, phys > 60 entries > 6009.255964000:414d6940: action 9 queue: entry added, size now log 61, phys > 61 entries > 6009.255978000:414d6940: action 9 queue: entry added, size now log 62, phys > 62 entries > 6009.255992000:414d6940: action 9 queue: entry added, size now log 63, phys > 63 entries > 6009.256012000:414d6940: action 9 queue: entry added, size now log 64, phys > 64 entries > 6009.643753000:414d6940: action 9 queue: entry added, size now log 49, phys > 65 entries > 6009.643770000:414d6940: action 9 queue: entry added, size now log 50, phys > 66 entries > 6009.643784000:414d6940: action 9 queue: entry added, size now log 51, phys > 67 entries > 6009.643797000:414d6940: action 9 queue: entry added, size now log 52, phys > 68 entries > 6009.643811000:414d6940: action 9 queue: entry added, size now log 53, phys > 69 entries > 6009.643825000:414d6940: action 9 queue: entry added, size now log 54, phys > 70 entries > 6009.643838000:414d6940: action 9 queue: entry added, size now log 55, phys > 71 entries > 6009.643852000:414d6940: action 9 queue: entry added, size now log 56, phys > 72 entries > 6009.643865000:414d6940: action 9 queue: entry added, size now log 57, phys > 73 entries > 6009.643879000:414d6940: action 9 queue: entry added, size now log 58, phys > 74 entries > 6009.643892000:414d6940: action 9 queue: entry added, size now log 59, phys > 75 entries > ------- snap ------- > Ooops... phys increases nicely (64 -> 65), but there is a drop in the log value > (64 -> 49), with the difference between 65 and 49 being 16... That means a batch of messages has been processed. In that case, the logical (log) queue size is reduced, but the messages remain inside the physical queue as some cleanup condition is not yet met. > > The messages with the numbers 17..24 and 26..32 (iy in the code) are > missing. > > I built rsyslog from the source rpm archive rsyslog-5.6.2-1.ep.src.rpm using > these steps: > > ./configure --prefix=/opt/rsyslog --enable-silent-rules --enable-dependency- > tracking --enable-shared --enable-regexp --enable-zlib --enable-gssapi-krb5 - > -enable-pthreads --enable-klog --enable-inet --enable-debug --enable-rtinst > --enable-memcheck --enable-diagtools --enable-pgsql --enable-snmp -- > enable-gnutls --enable-mail --enable-imfile --enable-imptcp --enable- > omprog --enable-omudpspoof --enable-omstdout --enable-omruleset make > make install > > Is there a way to get all messages in the database (as they appear in the log > file)? There have been a couple of fixes recently. Even though I am not sure they help with the issue you see, it would be best if you could re-try with 5.7.10. If that doesn't help, a complete debug log would be useful. Also, can you reproduce the problem if you only submit, say, 100 messages? If not, when does it occur? Rainer From rgerhards at hq.adiscon.com Tue Apr 5 18:00:04 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 5 Apr 2011 18:00:04 +0200 Subject: [rsyslog] [Lognorm] How to create a central rulebase repository for lognormalization? Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF4C@GRFEXC.intern.adiscon.com> Thanks everyone for responding. I'll do a wrap-up on the blog, most probably tomorrow. But it looks like the overall consensus was to give the git method at least a serious try, and I'll do that :) Rainer > -----Original Message----- > From: lognorm-bounces at lists.adiscon.com [mailto:lognorm- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, April 04, 2011 2:07 PM > To: rsyslog at lists.adiscon.com; lognorm at lists.adiscon.com > Subject: [Lognorm] How to create a central rulebase repository for > lognormalization? > > Hi all, > > please have a look at my post over here: > > http://blog.gerhards.net/2011/04/log-normalization-how-to-share.html > > This is a very important question, and I'd appreciate all comments and > feedback. > > Thanks, > Rainer > _______________________________________________ > Lognorm mailing list > Lognorm at lists.adiscon.com > http://lists.adiscon.net/mailman/listinfo/lognorm From piavka at cs.bgu.ac.il Wed Apr 6 23:19:13 2011 From: piavka at cs.bgu.ac.il (Piavlo) Date: Thu, 07 Apr 2011 00:19:13 +0300 Subject: [rsyslog] high cpu spkies by rsyslog which barely does anything In-Reply-To: <4D8B51B8.7050908@cs.bgu.ac.il> References: <4D8B51B8.7050908@cs.bgu.ac.il> Message-ID: <4D9CD8D1.3080805@cs.bgu.ac.il> Anyone has an idea? Might it be due to some rsyslog compilation option? Is my question inappropriate for this list? Thanks Alex Piavlo wrote: > > Hi, > > I'd like to use rsyslog as central log collection solution. > Manwhile I've compiled and installed as rsyslog 5.6.4 with a stock > configuration as drop in replacement to local ksyslogd > But I see a high quick cpu usage bursts by rsyslog every few seconds > while it barely does anything. > Most of the cpu time is spent in user mode. > This happends on ec2 large instance with Centos-5.5. > > Below are the configs and sample strace/lsof of the rsyslogd process. > > Any idea why this happens? > > [root at www1a ~]# cat /etc/rsyslog.conf | grep -v -e '^$' -e '^#' > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > $ModLoad imklog > $ModLoad imuxsock > *.info;mail.none;authpriv.none;cron.none /var/log/messages > authpriv.* /var/log/secure > mail.* -/var/log/maillog > cron.* /var/log/cron > *.emerg * > uucp,news.crit /var/log/spooler > local7.* /var/log/boot.log > [root at www1a ~]# > [root at www1a ~]# lsof -p $(pgrep rsyslogd) > COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME > rsyslogd 29994 root cwd DIR 8,1 4096 2 / > rsyslogd 29994 root rtd DIR 8,1 4096 2 / > rsyslogd 29994 root txt REG 8,1 1961412 2378118 > /usr/local/rsyslog/5.6.4/sbin/rsyslogd > rsyslogd 29994 root mem REG 8,1 137256 1720349 > /lib64/ld-2.5.so > rsyslogd 29994 root mem REG 8,1 83344 1543492 > /usr/lib64/libz.so.1.2.3 > rsyslogd 29994 root mem REG 8,1 142696 1720364 > /lib64/libpthread-2.5.so > rsyslogd 29994 root mem REG 8,1 20424 1720336 > /lib64/libdl-2.5.so > rsyslogd 29994 root mem REG 8,1 50288 1720370 > /lib64/librt-2.5.so > rsyslogd 29994 root mem REG 8,1 1712536 1720326 > /lib64/libc-2.5.so > rsyslogd 29994 root mem REG 8,1 85956 2378093 > /usr/local/rsyslog/5.6.4/lib/rsyslog/lmnet.so > rsyslogd 29994 root mem REG 8,1 90745 2378128 > /usr/local/rsyslog/5.6.4/lib/rsyslog/imklog.so > rsyslogd 29994 root mem REG 8,1 53471 2378110 > /usr/local/rsyslog/5.6.4/lib/rsyslog/imuxsock.so > rsyslogd 29994 root 0r REG 0,3 0 4026531849 > /proc/kmsg > rsyslogd 29994 root 1w REG 8,1 5019 1048938 > /var/log/messages > rsyslogd 29994 root 2w REG 8,1 18336275 1048939 > /var/log/secure > rsyslogd 29994 root 3u unix 0xffff8800241a2a80 80601917 > /dev/log > rsyslogd 29994 root 4w REG 8,1 214769 1048943 > /var/log/cron > [root at www1a ~]# > [root at www1a ~]# strace -c -r -T -q -f -p $(pgrep rsyslogd) > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > nan 0.000000 0 4 write > nan 0.000000 0 4 select > nan 0.000000 0 4 recvfrom > nan 0.000000 0 4 gettimeofday > nan 0.000000 0 20 4 futex > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.000000 36 4 total > [root at www1a ~]# > [root at www1a ~]# strace -v -r -T -q -f -p $(pgrep rsyslogd) > [pid 29995] 0.000000 futex(0x6a5d5c, FUTEX_WAIT, 1167, NULL > > [pid 29994] 0.005107 select(1, NULL, NULL, NULL, {85286, 180000} > > [pid 29997] 0.000049 read(0, > [pid 29998] 0.000021 select(4, [3], NULL, NULL, NULL) = 1 (in [3]) > <4.329724> > [pid 29998] 4.329868 recvfrom(3, "<86>Mar 24 14:01:07 sshd[4487]: > "..., 2048, 0, NULL, NULL) = 92 <0.000015> > [pid 29998] 0.000097 gettimeofday({1300975267, 753128}, NULL) = 0 > <0.000010> > [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, 0x6a5d58, > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > [pid 29998] 0.000052 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.000021 <... futex resumed> ) = 0 <4.335124> > [pid 29995] 0.000077 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 <0.000057> > [pid 29995] 0.000203 write(2, "Mar 24 14:01:07 www1a > sshd[4487]"..., 95 > [pid 29998] 0.024438 <... select resumed> ) = 1 (in [3]) <0.024706> > [pid 29995] 0.000024 <... write resumed> ) = 95 <0.024442> > [pid 29998] 0.000247 recvfrom(3, "<86>Mar 24 14:01:07 sshd[4487]: > "..., 2048, 0, NULL, NULL) = 95 <0.000014> > [pid 29998] 0.000066 gettimeofday({1300975267, 778334}, NULL) = 0 > <0.000010> > [pid 29998] 0.000078 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.005155 write(2, "Mar 24 14:01:07 www1a > sshd[4487]"..., 98) = 98 <0.000345> > [pid 29995] 0.000489 futex(0x6a5d5c, FUTEX_WAIT, 1169, NULL > > [pid 29998] 0.107333 <... select resumed> ) = 1 (in [3]) <0.112941> > [pid 29998] 0.000053 recvfrom(3, "<86>Mar 24 14:01:07 sshd[4487]: > "..., 2048, 0, NULL, NULL) = 84 <0.000016> > [pid 29998] 0.000068 gettimeofday({1300975267, 891512}, NULL) = 0 > <0.000010> > [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, 0x6a5d58, > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.001586 <... futex resumed> ) = 0 <0.109150> > [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 <0.000046> > [pid 29995] 0.000169 write(2, "Mar 24 14:01:07 www1a > sshd[4487]"..., 87) = 87 <0.000065> > [pid 29995] 0.000194 futex(0x6a5d5c, FUTEX_WAIT, 1171, NULL > > [pid 29998] 1.420908 <... select resumed> ) = 1 (in [3]) <1.422893> > [pid 29998] 0.000049 recvfrom(3, "<86>Mar 24 14:01:09 sshd[4517]: > "..., 2048, 0, NULL, NULL) = 64 <0.000014> > [pid 29998] 0.000503 gettimeofday({1300975269, 315123}, NULL) = 0 > <0.000042> > [pid 29998] 0.000121 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, 0x6a5d58, > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} > [pid 29995] 0.000490 <... futex resumed> ) = 0 <1.422056> > [pid 29998] 0.000010 <... futex resumed> ) = 1 <0.000487> > [pid 29995] 0.000122 futex(0x6a1b30, FUTEX_WAKE, 1 > [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.000067 <... futex resumed> ) = 0 <0.000072> > [pid 29995] 0.000127 write(2, "Mar 24 14:01:09 www1a > sshd[4517]"..., 67) = 67 <0.000105> > [pid 29995] 0.001215 futex(0x6a5d5c, FUTEX_WAIT, 1173, NULL > > [pid 29998] 4.916991 <... select resumed> ) = 1 (in [3]) <4.918369> > [pid 29998] 0.000050 recvfrom(3, "<86>Mar 24 14:01:14 sshd[4518]: > "..., 2048, 0, NULL, NULL) = 92 <0.000013> > [pid 29998] 0.000060 gettimeofday({1300975274, 234392}, NULL) = 0 > <0.000009> > [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, 0x6a5d58, > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.002861 <... futex resumed> ) = 0 <4.920060> > [pid 29995] 0.000177 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 <0.000059> > [pid 29995] 0.000218 write(2, "Mar 24 14:01:14 www1a > sshd[4518]"..., 95) = 95 <0.000076> > [pid 29995] 0.000378 futex(0x6a5d5c, FUTEX_WAIT, 1175, NULL > > [pid 29998] 0.067782 <... select resumed> ) = 1 (in [3]) <0.071383> > [pid 29998] 0.000054 recvfrom(3, "<86>Mar 24 14:01:14 sshd[4518]: > "..., 2048, 0, NULL, NULL) = 95 <0.000014> > [pid 29998] 0.000069 gettimeofday({1300975274, 306060}, NULL) = 0 > <0.000011> > [pid 29998] 0.000080 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, 0x6a5d58, > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.001890 <... futex resumed> ) = 0 <0.069908> > [pid 29995] 0.000070 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 <0.000044> > [pid 29995] 0.000170 write(2, "Mar 24 14:01:14 www1a > sshd[4518]"..., 98) = 98 <0.000069> > [pid 29995] 0.000285 futex(0x6a5d5c, FUTEX_WAIT, 1177, NULL > > [pid 29998] 0.098664 <... select resumed> ) = 1 (in [3]) <0.101048> > [pid 29998] 0.001766 recvfrom(3, "<86>Mar 24 14:01:14 sshd[4518]: > "..., 2048, 0, NULL, NULL) = 84 <0.000047> > [pid 29998] 0.000125 gettimeofday({1300975274, 409161}, NULL) = 0 > <0.000039> > [pid 29998] 0.000119 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, 0x6a5d58, > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} > [pid 29995] 0.000062 <... futex resumed> ) = 0 <0.100721> > [pid 29998] 0.000009 <... futex resumed> ) = 1 <0.000058> > [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1 > [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL > [pid 29995] 0.000065 <... futex resumed> ) = 0 <0.000070> > [pid 29995] 0.000118 write(2, "Mar 24 14:01:14 www1a > sshd[4518]"..., 87) = 87 <0.000069> > [pid 29995] 0.000200 futex(0x6a5d5c, FUTEX_WAIT, 1179, NULL > > [root at www1a ~]# > > > PS. One difference I noticed between stock rsyslog and syslog-ng configs > is that rsyslog holds just one fd open to /dev/log > and select's it, while syslog-ng holds about 15 different fd's to > /dev/log and epoll's them. > Any particular reason for such a drastic difference? > > Thanks > Alex > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Thu Apr 7 07:21:06 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 7 Apr 2011 07:21:06 +0200 Subject: [rsyslog] high cpu spkies by rsyslog which barely does anything In-Reply-To: <4D9CD8D1.3080805@cs.bgu.ac.il> References: <4D8B51B8.7050908@cs.bgu.ac.il> <4D9CD8D1.3080805@cs.bgu.ac.il> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF6D@GRFEXC.intern.adiscon.com> I don't see anything that looks particular bad -- and I've never before heard of such a problem (which obviously doesn't mean a lot ;)). Maybe you should enable debug mode so that we can see if some unusual processing happens. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Piavlo > Sent: Wednesday, April 06, 2011 11:19 PM > To: rsyslog-users > Subject: Re: [rsyslog] high cpu spkies by rsyslog which barely does > anything > > > > Anyone has an idea? > Might it be due to some rsyslog compilation option? > > Is my question inappropriate for this list? > > Thanks > Alex > > Piavlo wrote: > > > > Hi, > > > > I'd like to use rsyslog as central log collection solution. > > Manwhile I've compiled and installed as rsyslog 5.6.4 with a stock > > configuration as drop in replacement to local ksyslogd > > But I see a high quick cpu usage bursts by rsyslog every few seconds > > while it barely does anything. > > Most of the cpu time is spent in user mode. > > This happends on ec2 large instance with Centos-5.5. > > > > Below are the configs and sample strace/lsof of the rsyslogd process. > > > > Any idea why this happens? > > > > [root at www1a ~]# cat /etc/rsyslog.conf | grep -v -e '^$' -e '^#' > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > $ModLoad imklog > > $ModLoad imuxsock > > *.info;mail.none;authpriv.none;cron.none > /var/log/messages > > authpriv.* > /var/log/secure > > mail.* - > /var/log/maillog > > cron.* /var/log/cron > > *.emerg * > > uucp,news.crit > /var/log/spooler > > local7.* > /var/log/boot.log > > [root at www1a ~]# > > [root at www1a ~]# lsof -p $(pgrep rsyslogd) > > COMMAND PID USER FD TYPE DEVICE SIZE > NODE NAME > > rsyslogd 29994 root cwd DIR 8,1 4096 > 2 / > > rsyslogd 29994 root rtd DIR 8,1 4096 > 2 / > > rsyslogd 29994 root txt REG 8,1 1961412 > 2378118 > > /usr/local/rsyslog/5.6.4/sbin/rsyslogd > > rsyslogd 29994 root mem REG 8,1 137256 > 1720349 > > /lib64/ld-2.5.so > > rsyslogd 29994 root mem REG 8,1 83344 > 1543492 > > /usr/lib64/libz.so.1.2.3 > > rsyslogd 29994 root mem REG 8,1 142696 > 1720364 > > /lib64/libpthread-2.5.so > > rsyslogd 29994 root mem REG 8,1 20424 > 1720336 > > /lib64/libdl-2.5.so > > rsyslogd 29994 root mem REG 8,1 50288 > 1720370 > > /lib64/librt-2.5.so > > rsyslogd 29994 root mem REG 8,1 1712536 > 1720326 > > /lib64/libc-2.5.so > > rsyslogd 29994 root mem REG 8,1 85956 > 2378093 > > /usr/local/rsyslog/5.6.4/lib/rsyslog/lmnet.so > > rsyslogd 29994 root mem REG 8,1 90745 > 2378128 > > /usr/local/rsyslog/5.6.4/lib/rsyslog/imklog.so > > rsyslogd 29994 root mem REG 8,1 53471 > 2378110 > > /usr/local/rsyslog/5.6.4/lib/rsyslog/imuxsock.so > > rsyslogd 29994 root 0r REG 0,3 0 > 4026531849 > > /proc/kmsg > > rsyslogd 29994 root 1w REG 8,1 5019 > 1048938 > > /var/log/messages > > rsyslogd 29994 root 2w REG 8,1 18336275 > 1048939 > > /var/log/secure > > rsyslogd 29994 root 3u unix 0xffff8800241a2a80 > 80601917 > > /dev/log > > rsyslogd 29994 root 4w REG 8,1 214769 > 1048943 > > /var/log/cron > > [root at www1a ~]# > > [root at www1a ~]# strace -c -r -T -q -f -p $(pgrep rsyslogd) > > % time seconds usecs/call calls errors syscall > > ------ ----------- ----------- --------- --------- ---------------- > > nan 0.000000 0 4 write > > nan 0.000000 0 4 select > > nan 0.000000 0 4 recvfrom > > nan 0.000000 0 4 gettimeofday > > nan 0.000000 0 20 4 futex > > ------ ----------- ----------- --------- --------- ---------------- > > 100.00 0.000000 36 4 total > > [root at www1a ~]# > > [root at www1a ~]# strace -v -r -T -q -f -p $(pgrep rsyslogd) > > [pid 29995] 0.000000 futex(0x6a5d5c, FUTEX_WAIT, 1167, NULL > > > > [pid 29994] 0.005107 select(1, NULL, NULL, NULL, {85286, 180000} > > > > [pid 29997] 0.000049 read(0, > > [pid 29998] 0.000021 select(4, [3], NULL, NULL, NULL) = 1 (in > [3]) > > <4.329724> > > [pid 29998] 4.329868 recvfrom(3, "<86>Mar 24 14:01:07 > sshd[4487]: > > "..., 2048, 0, NULL, NULL) = 92 <0.000015> > > [pid 29998] 0.000097 gettimeofday({1300975267, 753128}, NULL) = > 0 > > <0.000010> > > [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > 0x6a5d58, > > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > > [pid 29998] 0.000052 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.000021 <... futex resumed> ) = 0 <4.335124> > > [pid 29995] 0.000077 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > <0.000057> > > [pid 29995] 0.000203 write(2, "Mar 24 14:01:07 www1a > > sshd[4487]"..., 95 > > [pid 29998] 0.024438 <... select resumed> ) = 1 (in [3]) > <0.024706> > > [pid 29995] 0.000024 <... write resumed> ) = 95 <0.024442> > > [pid 29998] 0.000247 recvfrom(3, "<86>Mar 24 14:01:07 > sshd[4487]: > > "..., 2048, 0, NULL, NULL) = 95 <0.000014> > > [pid 29998] 0.000066 gettimeofday({1300975267, 778334}, NULL) = > 0 > > <0.000010> > > [pid 29998] 0.000078 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.005155 write(2, "Mar 24 14:01:07 www1a > > sshd[4487]"..., 98) = 98 <0.000345> > > [pid 29995] 0.000489 futex(0x6a5d5c, FUTEX_WAIT, 1169, NULL > > > > [pid 29998] 0.107333 <... select resumed> ) = 1 (in [3]) > <0.112941> > > [pid 29998] 0.000053 recvfrom(3, "<86>Mar 24 14:01:07 > sshd[4487]: > > "..., 2048, 0, NULL, NULL) = 84 <0.000016> > > [pid 29998] 0.000068 gettimeofday({1300975267, 891512}, NULL) = > 0 > > <0.000010> > > [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > 0x6a5d58, > > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > > [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.001586 <... futex resumed> ) = 0 <0.109150> > > [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > <0.000046> > > [pid 29995] 0.000169 write(2, "Mar 24 14:01:07 www1a > > sshd[4487]"..., 87) = 87 <0.000065> > > [pid 29995] 0.000194 futex(0x6a5d5c, FUTEX_WAIT, 1171, NULL > > > > [pid 29998] 1.420908 <... select resumed> ) = 1 (in [3]) > <1.422893> > > [pid 29998] 0.000049 recvfrom(3, "<86>Mar 24 14:01:09 > sshd[4517]: > > "..., 2048, 0, NULL, NULL) = 64 <0.000014> > > [pid 29998] 0.000503 gettimeofday({1300975269, 315123}, NULL) = > 0 > > <0.000042> > > [pid 29998] 0.000121 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > 0x6a5d58, > > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} > > [pid 29995] 0.000490 <... futex resumed> ) = 0 <1.422056> > > [pid 29998] 0.000010 <... futex resumed> ) = 1 <0.000487> > > [pid 29995] 0.000122 futex(0x6a1b30, FUTEX_WAKE, 1 ...> > > [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.000067 <... futex resumed> ) = 0 <0.000072> > > [pid 29995] 0.000127 write(2, "Mar 24 14:01:09 www1a > > sshd[4517]"..., 67) = 67 <0.000105> > > [pid 29995] 0.001215 futex(0x6a5d5c, FUTEX_WAIT, 1173, NULL > > > > [pid 29998] 4.916991 <... select resumed> ) = 1 (in [3]) > <4.918369> > > [pid 29998] 0.000050 recvfrom(3, "<86>Mar 24 14:01:14 > sshd[4518]: > > "..., 2048, 0, NULL, NULL) = 92 <0.000013> > > [pid 29998] 0.000060 gettimeofday({1300975274, 234392}, NULL) = > 0 > > <0.000009> > > [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > 0x6a5d58, > > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > > [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.002861 <... futex resumed> ) = 0 <4.920060> > > [pid 29995] 0.000177 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > <0.000059> > > [pid 29995] 0.000218 write(2, "Mar 24 14:01:14 www1a > > sshd[4518]"..., 95) = 95 <0.000076> > > [pid 29995] 0.000378 futex(0x6a5d5c, FUTEX_WAIT, 1175, NULL > > > > [pid 29998] 0.067782 <... select resumed> ) = 1 (in [3]) > <0.071383> > > [pid 29998] 0.000054 recvfrom(3, "<86>Mar 24 14:01:14 > sshd[4518]: > > "..., 2048, 0, NULL, NULL) = 95 <0.000014> > > [pid 29998] 0.000069 gettimeofday({1300975274, 306060}, NULL) = > 0 > > <0.000011> > > [pid 29998] 0.000080 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > 0x6a5d58, > > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > > [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.001890 <... futex resumed> ) = 0 <0.069908> > > [pid 29995] 0.000070 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > <0.000044> > > [pid 29995] 0.000170 write(2, "Mar 24 14:01:14 www1a > > sshd[4518]"..., 98) = 98 <0.000069> > > [pid 29995] 0.000285 futex(0x6a5d5c, FUTEX_WAIT, 1177, NULL > > > > [pid 29998] 0.098664 <... select resumed> ) = 1 (in [3]) > <0.101048> > > [pid 29998] 0.001766 recvfrom(3, "<86>Mar 24 14:01:14 > sshd[4518]: > > "..., 2048, 0, NULL, NULL) = 84 <0.000047> > > [pid 29998] 0.000125 gettimeofday({1300975274, 409161}, NULL) = > 0 > > <0.000039> > > [pid 29998] 0.000119 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > 0x6a5d58, > > {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} > > [pid 29995] 0.000062 <... futex resumed> ) = 0 <0.100721> > > [pid 29998] 0.000009 <... futex resumed> ) = 1 <0.000058> > > [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1 ...> > > [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL ...> > > [pid 29995] 0.000065 <... futex resumed> ) = 0 <0.000070> > > [pid 29995] 0.000118 write(2, "Mar 24 14:01:14 www1a > > sshd[4518]"..., 87) = 87 <0.000069> > > [pid 29995] 0.000200 futex(0x6a5d5c, FUTEX_WAIT, 1179, NULL > > > > [root at www1a ~]# > > > > > > PS. One difference I noticed between stock rsyslog and syslog-ng > configs > > is that rsyslog holds just one fd open to /dev/log > > and select's it, while syslog-ng holds about 15 different fd's to > > /dev/log and epoll's them. > > Any particular reason for such a drastic difference? > > > > Thanks > > Alex > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From roland.kamke at auconet.com Thu Apr 7 08:23:08 2011 From: roland.kamke at auconet.com (Roland Kamke) Date: Thu, 7 Apr 2011 08:23:08 +0200 Subject: [rsyslog] Missing log entries from PostgreSQL backend Message-ID: <69DFC544026A844790C013C0FB16B9E0B6214C@eta.firma.lan> Rainer, I switched to 5.7.10 - and the problem was gone. No loss of log data any longer. Thanks for your hint. Another observation: The CPU load has decreased in 5.7.10 compared to 5.6.2. I just read Piavlo's posting about "high cpu spkies by rsyslog which barely does anything", who is on CentOS 5.5, too. May be it's an option for him to check 5.7.10? Regards, Roland > -----Urspr?ngliche Nachricht----- > Von: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] Im Auftrag von Rainer Gerhards > Gesendet: Dienstag, 5. April 2011 15:51 > An: rsyslog-users > Betreff: Re: [rsyslog] Missing log entries from PostgreSQL backend > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Roland Kamke > > Sent: Tuesday, April 05, 2011 3:34 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] Missing log entries from PostgreSQL backend > > > > A newbie question: > > > > With a simple C program I create 8000 syslog messages to rsyslog and > wait > > then a minute before another 8000 messages will be created. This is the > code > > to create the messages, counter is 8000: > > ------- snip ------- > > for (iy=0; iy > syslog(LOG_LOCAL3 | LOG_INFO, "C test program %d%s, %d", > > getpid(), count % 3 == 0 ? " (magic string)" : "", count++); > > ------- snap ------- > > > > This is a small snippet from the configuration: > > ------- snip ------- > > *.info;mail.none;authpriv.none;cron.none > > /var/log/messages > > > > if $msg contains 'C test program' then > > :ompgsql:localhost,secret,secret,secret; > > ------- snap ------- > > After the first 8000 messages are created I would expect to find 8000 > lines > in > > the log file and 8000 entries in the PostgreSQL database for these > messages. > > > > But: I clean up file system and data base, start rsyslogd and my test > program. > > After a short while I can find all messages in the log file, but only > 7984 > in the > > data base. This behaviour can be reproduced at will, in any attempt 16 > lines > > were missing. > > That's interesting. I unfortunately have no postgres test environment at > the > moment, so I cannot test this myself. > > > > > Tracing the database I saw that there were only 7984 all inserts into > the > log > > table. > > > > I ran rsyslogd with -dn option set. The log file created contains these > > lines: > > ------- snip ------- > > 6009.255950000:414d6940: action 9 queue: entry added, size now log 60, > phys > > 60 entries > > 6009.255964000:414d6940: action 9 queue: entry added, size now log 61, > phys > > 61 entries > > 6009.255978000:414d6940: action 9 queue: entry added, size now log 62, > phys > > 62 entries > > 6009.255992000:414d6940: action 9 queue: entry added, size now log 63, > phys > > 63 entries > > 6009.256012000:414d6940: action 9 queue: entry added, size now log 64, > phys > > 64 entries > > 6009.643753000:414d6940: action 9 queue: entry added, size now log 49, > phys > > 65 entries > > 6009.643770000:414d6940: action 9 queue: entry added, size now log 50, > phys > > 66 entries > > 6009.643784000:414d6940: action 9 queue: entry added, size now log 51, > phys > > 67 entries > > 6009.643797000:414d6940: action 9 queue: entry added, size now log 52, > phys > > 68 entries > > 6009.643811000:414d6940: action 9 queue: entry added, size now log 53, > phys > > 69 entries > > 6009.643825000:414d6940: action 9 queue: entry added, size now log 54, > phys > > 70 entries > > 6009.643838000:414d6940: action 9 queue: entry added, size now log 55, > phys > > 71 entries > > 6009.643852000:414d6940: action 9 queue: entry added, size now log 56, > phys > > 72 entries > > 6009.643865000:414d6940: action 9 queue: entry added, size now log 57, > phys > > 73 entries > > 6009.643879000:414d6940: action 9 queue: entry added, size now log 58, > phys > > 74 entries > > 6009.643892000:414d6940: action 9 queue: entry added, size now log 59, > phys > > 75 entries > > ------- snap ------- > > Ooops... phys increases nicely (64 -> 65), but there is a drop in the > log > value > > (64 -> 49), with the difference between 65 and 49 being 16... > > That means a batch of messages has been processed. In that case, the > logical > (log) queue size is reduced, but the messages remain inside the physical > queue as some cleanup condition is not yet met. > > > > > The messages with the numbers 17..24 and 26..32 (iy in the code) are > > missing. > > > > I built rsyslog from the source rpm archive rsyslog-5.6.2-1.ep.src.rpm > using > > these steps: > > > > ./configure --prefix=/opt/rsyslog --enable-silent-rules > --enable-dependency- > > tracking --enable-shared --enable-regexp --enable-zlib --enable-gssapi- > krb5 > - > > -enable-pthreads --enable-klog --enable-inet --enable-debug --enable- > rtinst > > --enable-memcheck --enable-diagtools --enable-pgsql --enable-snmp -- > > enable-gnutls --enable-mail --enable-imfile --enable-imptcp --enable- > > omprog --enable-omudpspoof --enable-omstdout --enable-omruleset make > > make install > > > > Is there a way to get all messages in the database (as they appear in > the > log > > file)? > > There have been a couple of fixes recently. Even though I am not sure they > help with the issue you see, it would be best if you could re-try with > 5.7.10. > > If that doesn't help, a complete debug log would be useful. Also, can you > reproduce the problem if you only submit, say, 100 messages? If not, when > does it occur? > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Apr 7 09:35:36 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 7 Apr 2011 09:35:36 +0200 Subject: [rsyslog] Missing log entries from PostgreSQL backend In-Reply-To: <69DFC544026A844790C013C0FB16B9E0B6214C@eta.firma.lan> References: <69DFC544026A844790C013C0FB16B9E0B6214C@eta.firma.lan> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF71@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Roland Kamke > Sent: Thursday, April 07, 2011 8:23 AM > To: rsyslog-users > Subject: Re: [rsyslog] Missing log entries from PostgreSQL backend > > Rainer, I switched to 5.7.10 - and the problem was gone. No loss of log > data any longer. Thanks for your hint. Excellent! > Another observation: The CPU load has decreased in 5.7.10 compared to > 5.6.2. I just read Piavlo's posting about "high cpu spkies by rsyslog > which barely does anything", who is on CentOS 5.5, too. May be it's an > option for him to check 5.7.10? That's a good idea. In general, newer version are faster than previous ones in the current series, because I have a focus on performance (even as a side-focus when the primary one is different). Rainer > > Regards, > Roland > > > -----Urspr?ngliche Nachricht----- > > Von: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] Im Auftrag von Rainer Gerhards > > Gesendet: Dienstag, 5. April 2011 15:51 > > An: rsyslog-users > > Betreff: Re: [rsyslog] Missing log entries from PostgreSQL backend > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Roland Kamke > > > Sent: Tuesday, April 05, 2011 3:34 PM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] Missing log entries from PostgreSQL backend > > > > > > A newbie question: > > > > > > With a simple C program I create 8000 syslog messages to rsyslog > and > > wait > > > then a minute before another 8000 messages will be created. This is > the > > code > > > to create the messages, counter is 8000: > > > ------- snip ------- > > > for (iy=0; iy > > syslog(LOG_LOCAL3 | LOG_INFO, "C test program %d%s, %d", > > > getpid(), count % 3 == 0 ? " (magic string)" : "", > count++); > > > ------- snap ------- > > > > > > This is a small snippet from the configuration: > > > ------- snip ------- > > > *.info;mail.none;authpriv.none;cron.none > > > /var/log/messages > > > > > > if $msg contains 'C test program' then > > > :ompgsql:localhost,secret,secret,secret; > > > ------- snap ------- > > > After the first 8000 messages are created I would expect to find > 8000 > > lines > > in > > > the log file and 8000 entries in the PostgreSQL database for these > > messages. > > > > > > But: I clean up file system and data base, start rsyslogd and my > test > > program. > > > After a short while I can find all messages in the log file, but > only > > 7984 > > in the > > > data base. This behaviour can be reproduced at will, in any attempt > 16 > > lines > > > were missing. > > > > That's interesting. I unfortunately have no postgres test environment > at > > the > > moment, so I cannot test this myself. > > > > > > > > Tracing the database I saw that there were only 7984 all inserts > into > > the > > log > > > table. > > > > > > I ran rsyslogd with -dn option set. The log file created contains > these > > > lines: > > > ------- snip ------- > > > 6009.255950000:414d6940: action 9 queue: entry added, size now log > 60, > > phys > > > 60 entries > > > 6009.255964000:414d6940: action 9 queue: entry added, size now log > 61, > > phys > > > 61 entries > > > 6009.255978000:414d6940: action 9 queue: entry added, size now log > 62, > > phys > > > 62 entries > > > 6009.255992000:414d6940: action 9 queue: entry added, size now log > 63, > > phys > > > 63 entries > > > 6009.256012000:414d6940: action 9 queue: entry added, size now log > 64, > > phys > > > 64 entries > > > 6009.643753000:414d6940: action 9 queue: entry added, size now log > 49, > > phys > > > 65 entries > > > 6009.643770000:414d6940: action 9 queue: entry added, size now log > 50, > > phys > > > 66 entries > > > 6009.643784000:414d6940: action 9 queue: entry added, size now log > 51, > > phys > > > 67 entries > > > 6009.643797000:414d6940: action 9 queue: entry added, size now log > 52, > > phys > > > 68 entries > > > 6009.643811000:414d6940: action 9 queue: entry added, size now log > 53, > > phys > > > 69 entries > > > 6009.643825000:414d6940: action 9 queue: entry added, size now log > 54, > > phys > > > 70 entries > > > 6009.643838000:414d6940: action 9 queue: entry added, size now log > 55, > > phys > > > 71 entries > > > 6009.643852000:414d6940: action 9 queue: entry added, size now log > 56, > > phys > > > 72 entries > > > 6009.643865000:414d6940: action 9 queue: entry added, size now log > 57, > > phys > > > 73 entries > > > 6009.643879000:414d6940: action 9 queue: entry added, size now log > 58, > > phys > > > 74 entries > > > 6009.643892000:414d6940: action 9 queue: entry added, size now log > 59, > > phys > > > 75 entries > > > ------- snap ------- > > > Ooops... phys increases nicely (64 -> 65), but there is a drop in > the > > log > > value > > > (64 -> 49), with the difference between 65 and 49 being 16... > > > > That means a batch of messages has been processed. In that case, the > > logical > > (log) queue size is reduced, but the messages remain inside the > physical > > queue as some cleanup condition is not yet met. > > > > > > > > The messages with the numbers 17..24 and 26..32 (iy in the code) > are > > > missing. > > > > > > I built rsyslog from the source rpm archive rsyslog-5.6.2- > 1.ep.src.rpm > > using > > > these steps: > > > > > > ./configure --prefix=/opt/rsyslog --enable-silent-rules > > --enable-dependency- > > > tracking --enable-shared --enable-regexp --enable-zlib --enable- > gssapi- > > krb5 > > - > > > -enable-pthreads --enable-klog --enable-inet --enable-debug -- > enable- > > rtinst > > > --enable-memcheck --enable-diagtools --enable-pgsql --enable-snmp - > - > > > enable-gnutls --enable-mail --enable-imfile --enable-imptcp -- > enable- > > > omprog --enable-omudpspoof --enable-omstdout --enable-omruleset > make > > > make install > > > > > > Is there a way to get all messages in the database (as they appear > in > > the > > log > > > file)? > > > > There have been a couple of fixes recently. Even though I am not sure > they > > help with the issue you see, it would be best if you could re-try > with > > 5.7.10. > > > > If that doesn't help, a complete debug log would be useful. Also, can > you > > reproduce the problem if you only submit, say, 100 messages? If not, > when > > does it occur? > > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From branob at gmail.com Fri Apr 8 04:38:47 2011 From: branob at gmail.com (Brano) Date: Thu, 07 Apr 2011 22:38:47 -0400 Subject: [rsyslog] Hostname missing from logs Message-ID: <4D9E7537.1030604@gmail.com> I've recently switched from syslogd to rsyslogd on CentOS 5.5 due to need of granular network logging. However, I've noticed one issue with my remote log. I'm logging from remote host called usg200 (defined in local hostfile). It is ZyXel USG200 router. With syslogd the messages in log were like this one Apr 3 21:27:43 *usg200* domain.com src="76.10.x.x:500" dst="76.10.x.x:500" msg="The cookie pair is : 0x6f28d9b0e98a895a / 0x3bfec fd059520966" note="IKE_LOG" user="unknown" devID="0019cb7273a4" cat="IKE" With rsyslog the messages are like this Apr 6 14:21:04 domain.com src="76.10.x.x: 500" dst="76.10.x.x:500" msg="Recv:[HASH][NOTFY:R_U_THERE]" note="IKE_LOG" user="unknown" devID="0019cb7273a4" cat="IKE" Notice the usg200 hostname in rsyslog just after date is missing. I need to get it back. Any advice highly appreciated. From rgerhards at hq.adiscon.com Fri Apr 8 07:34:29 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 8 Apr 2011 07:34:29 +0200 Subject: [rsyslog] Hostname missing from logs In-Reply-To: <4D9E7537.1030604@gmail.com> References: <4D9E7537.1030604@gmail.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF83@GRFEXC.intern.adiscon.com> I think the incoming message is illformed. For background, read this: http://www.rsyslog.com/doc/syslog_parsing.html You can create a file via *.* /path/to/file;RSYSLOG_DebugFormat And post the rawmsg output. Then we can probably suggest a solution. But, as said in the paper, the proper solutions are to configure the device to emit correct messages or create a custom parser if that's not possible. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Brano > Sent: Friday, April 08, 2011 4:39 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Hostname missing from logs > > I've recently switched from syslogd to rsyslogd on CentOS 5.5 due to > need of granular network logging. > However, I've noticed one issue with my remote log. > > I'm logging from remote host called usg200 (defined in local hostfile). > It is ZyXel USG200 router. > > With syslogd the messages in log were like this one > Apr 3 21:27:43 *usg200* domain.com src="76.10.x.x:500" > dst="76.10.x.x:500" msg="The cookie pair is : 0x6f28d9b0e98a895a / > 0x3bfec > fd059520966" note="IKE_LOG" user="unknown" devID="0019cb7273a4" > cat="IKE" > > With rsyslog the messages are like this > Apr 6 14:21:04 domain.com src="76.10.x.x: 500" dst="76.10.x.x:500" > msg="Recv:[HASH][NOTFY:R_U_THERE]" note="IKE_LOG" user="unknown" > devID="0019cb7273a4" cat="IKE" > > Notice the usg200 hostname in rsyslog just after date is missing. I > need > to get it back. Any advice highly appreciated. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Apr 8 09:11:32 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 8 Apr 2011 09:11:32 +0200 Subject: [rsyslog] rsyslog, fifos and SIGHUP In-Reply-To: <20110408062441.GA23792@colossus> References: <20110407101048.GI2144@colossus> <9B6E2A8877C38245BFB15CC491A11DA71DDF7B@GRFEXC.intern.adiscon.com> <20110408062441.GA23792@colossus> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF86@GRFEXC.intern.adiscon.com> I guess that the files are pre-created with the wrong user permissions (who is the owner)? Usually, this happens if some external script creates them. Rainer > -----Original Message----- > From: Petri Lehtinen [mailto:petri.lehtinen at inoi.fi] > Sent: Friday, April 08, 2011 8:25 AM > To: rsyslog at lists.adiscon.com > Cc: Rainer Gerhards > Subject: Re: rsyslog, fifos and SIGHUP > > Rainer Gerhards wrote: > > > -----Original Message----- > > > From: Petri Lehtinen [mailto:petri.lehtinen at inoi.fi] > > > > > > I'm using rsyslog 4.2.0 on Ubuntu Lucid and I'm having problems with > > > a fifo. It seems that rsyslog stops writing to the fifo when it > > > receives the SIGHUP signal. This is problematic, as, in Ubuntu's > > > default configuration, logrotate HUPs rsyslog every day. > > > > > > Is this a known issue, has it been fixed in later versions? Can I do > > > anything to help fix this issue? > > > > Two things: a) the version is pretty old, b) Ubuntu drops privileges, > > but their config is often not really up to that. Keep running as root > > for a test and check if the problem re-occurs. I guess not. > > I commented out the "$PrivDropToUser syslog" and "$PrivDropToGroup > syslog" lines from /etc/rsyslog.conf, and it started working. > > Any ideas on how to make the config "up to" running as non-root, so that the > HUP+fifo combination works? My configuration (the Ubuntu > default) is available at: > > https://gist.github.com/909386 From rgerhards at hq.adiscon.com Fri Apr 8 09:31:07 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 8 Apr 2011 09:31:07 +0200 Subject: [rsyslog] rsyslog, fifos and SIGHUP In-Reply-To: <20110408072234.GB23792@colossus> References: <20110407101048.GI2144@colossus> <9B6E2A8877C38245BFB15CC491A11DA71DDF7B@GRFEXC.intern.adiscon.com> <20110408062441.GA23792@colossus> <9B6E2A8877C38245BFB15CC491A11DA71DDF86@GRFEXC.intern.adiscon.com> <20110408072234.GB23792@colossus> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDF88@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: Petri Lehtinen [mailto:petri.lehtinen at inoi.fi] > Sent: Friday, April 08, 2011 9:23 AM > To: Rainer Gerhards > Cc: rsyslog at lists.adiscon.com > Subject: Re: rsyslog, fifos and SIGHUP > > Rainer Gerhards wrote: > > I guess that the files are pre-created with the wrong user permissions > > (who is the owner)? Usually, this happens if some external script creates > them. > > Ah true, the file owner was wrong. But did rsyslog open the fifo before > dropping privileges then, as it was able to write to it until it got HUPed? Yes! The privilege drop code still needs a much better design. See here: http://wiki.rsyslog.com/index.php/Security#Dropping_Privileges I am not too happy with the way Ubuntu uses the capability. As they rely heavily on it, it would have been more appropriate from them to contribute a couple of patches to enhance that support. Rainer From branob at gmail.com Fri Apr 8 18:31:20 2011 From: branob at gmail.com (Brano) Date: Fri, 08 Apr 2011 12:31:20 -0400 Subject: [rsyslog] Hostname missing from logs In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF83@GRFEXC.intern.adiscon.com> References: <4D9E7537.1030604@gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71DDF83@GRFEXC.intern.adiscon.com> Message-ID: <4D9F3858.3020006@gmail.com> Here is one debug line. Is there a way to log FROMHOST? that's what I'm missing. Debug line with all properties: FROMHOST: 'usg200', fromhost-ip: '192.168.10.1', HOSTNAME: 'domain.com', PRI: 181, syslogtag 'src="74.208.229.54:', programname: 'src="74.208.229.54', APP-NAME: 'src="74.208.229.54', PROCID: '-', MSGID: '-', TIMESTAMP: 'Apr 8 12:22:32', STRUCTURED-DATA: '-', msg: '0" dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access Control" ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 proto="PING"' escaped msg: '0" dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access Control" ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 proto="PING"' rawmsg: '<181>Apr 8 12:22:32 domain.com src="74.208.229.54:0" dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access Control" ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 proto="PING"' On 04/08/2011 01:34 AM, Rainer Gerhards wrote: > I think the incoming message is illformed. For background, read this: > > http://www.rsyslog.com/doc/syslog_parsing.html > > You can create a file via > > *.* /path/to/file;RSYSLOG_DebugFormat > > And post the rawmsg output. Then we can probably suggest a solution. But, as > said in the paper, the proper solutions are to configure the device to emit > correct messages or create a custom parser if that's not possible. > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Brano >> Sent: Friday, April 08, 2011 4:39 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] Hostname missing from logs >> >> I've recently switched from syslogd to rsyslogd on CentOS 5.5 due to >> need of granular network logging. >> However, I've noticed one issue with my remote log. >> >> I'm logging from remote host called usg200 (defined in local hostfile). >> It is ZyXel USG200 router. >> >> With syslogd the messages in log were like this one >> Apr 3 21:27:43 *usg200* domain.com src="76.10.x.x:500" >> dst="76.10.x.x:500" msg="The cookie pair is : 0x6f28d9b0e98a895a / >> 0x3bfec >> fd059520966" note="IKE_LOG" user="unknown" devID="0019cb7273a4" >> cat="IKE" >> >> With rsyslog the messages are like this >> Apr 6 14:21:04 domain.com src="76.10.x.x: 500" dst="76.10.x.x:500" >> msg="Recv:[HASH][NOTFY:R_U_THERE]" note="IKE_LOG" user="unknown" >> devID="0019cb7273a4" cat="IKE" >> >> Notice the usg200 hostname in rsyslog just after date is missing. I >> need >> to get it back. Any advice highly appreciated. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From branob at gmail.com Fri Apr 8 18:47:35 2011 From: branob at gmail.com (Brano) Date: Fri, 08 Apr 2011 12:47:35 -0400 Subject: [rsyslog] Hostname missing from logs In-Reply-To: <4D9F3858.3020006@gmail.com> References: <4D9E7537.1030604@gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71DDF83@GRFEXC.intern.adiscon.com> <4D9F3858.3020006@gmail.com> Message-ID: <4D9F3C27.5080301@gmail.com> Now when looking at this closer, let me ask this. How can always log for every remote host FROMHOST or FROMHOST-IP? as second field in the log entry (right aftger time/date)? ...I'm not willing to trust remote hosts that they provide correct host/shource IP. Thank you, Brano On 04/08/2011 12:31 PM, Brano wrote: > Here is one debug line. > > Is there a way to log FROMHOST? that's what I'm missing. > > Debug line with all properties: > FROMHOST: 'usg200', fromhost-ip: '192.168.10.1', HOSTNAME: > 'domain.com', PRI: 181, > syslogtag 'src="74.208.229.54:', programname: 'src="74.208.229.54', > APP-NAME: 'src="74.208.229.54', PROCID: '-', MSGID: '-', > TIMESTAMP: 'Apr 8 12:22:32', STRUCTURED-DATA: '-', > msg: '0" dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP > Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" > user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access > Control" ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 > proto="PING"' > escaped msg: '0" dst="76.10.x.x:0" msg="priority:12, from WAN to > ZyWALL, ICMP Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS > FORWARD" user="unknown" devID="0019cb7273a4" cat="Firewall" > class="Access Control" ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" > protoID=1 proto="PING"' > rawmsg: '<181>Apr 8 12:22:32 domain.com src="74.208.229.54:0" > dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP Type:8, > service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" > user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access > Control" ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 > proto="PING"' > > > On 04/08/2011 01:34 AM, Rainer Gerhards wrote: >> I think the incoming message is illformed. For background, read this: >> >> http://www.rsyslog.com/doc/syslog_parsing.html >> >> You can create a file via >> >> *.* /path/to/file;RSYSLOG_DebugFormat >> >> And post the rawmsg output. Then we can probably suggest a solution. >> But, as >> said in the paper, the proper solutions are to configure the device >> to emit >> correct messages or create a custom parser if that's not possible. >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Brano >>> Sent: Friday, April 08, 2011 4:39 AM >>> To: rsyslog at lists.adiscon.com >>> Subject: [rsyslog] Hostname missing from logs >>> >>> I've recently switched from syslogd to rsyslogd on CentOS 5.5 due to >>> need of granular network logging. >>> However, I've noticed one issue with my remote log. >>> >>> I'm logging from remote host called usg200 (defined in local hostfile). >>> It is ZyXel USG200 router. >>> >>> With syslogd the messages in log were like this one >>> Apr 3 21:27:43 *usg200* domain.com src="76.10.x.x:500" >>> dst="76.10.x.x:500" msg="The cookie pair is : 0x6f28d9b0e98a895a / >>> 0x3bfec >>> fd059520966" note="IKE_LOG" user="unknown" devID="0019cb7273a4" >>> cat="IKE" >>> >>> With rsyslog the messages are like this >>> Apr 6 14:21:04 domain.com src="76.10.x.x: 500" dst="76.10.x.x:500" >>> msg="Recv:[HASH][NOTFY:R_U_THERE]" note="IKE_LOG" user="unknown" >>> devID="0019cb7273a4" cat="IKE" >>> >>> Notice the usg200 hostname in rsyslog just after date is missing. I >>> need >>> to get it back. Any advice highly appreciated. >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > From david at lang.hm Fri Apr 8 23:11:42 2011 From: david at lang.hm (david at lang.hm) Date: Fri, 8 Apr 2011 14:11:42 -0700 (PDT) Subject: [rsyslog] Hostname missing from logs In-Reply-To: <4D9F3C27.5080301@gmail.com> References: <4D9E7537.1030604@gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71DDF83@GRFEXC.intern.adiscon.com> <4D9F3858.3020006@gmail.com> <4D9F3C27.5080301@gmail.com> Message-ID: On Fri, 8 Apr 2011, Brano wrote: > Now when looking at this closer, let me ask this. > > How can always log for every remote host FROMHOST or FROMHOST-IP? as second > field in the log entry (right aftger time/date)? > ...I'm not willing to trust remote hosts that they provide correct > host/shource IP. fromhost is a name lookup of fromhost-IP, so it's a relativly expensive thing to get (if you use DNS and can't find a name it can cripple your logging for example) if you want to change the log format from the default, just define a new template and use that template in the rsyslog.conf (similar to how you used RSYSLOG_DebugFormat for testing) the built-in formats are a bit faster (you get ~10% better peak performance from them than with an equivalent format defined in the config file), but unless you are pushing the limit, you should not have a problem note that if you are relaying messages, you only want the first machine to use fromhost-ip, everything after that should use hostname or they will show the relay box IPs instead of the source box IPs) David Lang > Thank you, > Brano > > On 04/08/2011 12:31 PM, Brano wrote: >> Here is one debug line. >> >> Is there a way to log FROMHOST? that's what I'm missing. >> >> Debug line with all properties: >> FROMHOST: 'usg200', fromhost-ip: '192.168.10.1', HOSTNAME: 'domain.com', >> PRI: 181, >> syslogtag 'src="74.208.229.54:', programname: 'src="74.208.229.54', >> APP-NAME: 'src="74.208.229.54', PROCID: '-', MSGID: '-', >> TIMESTAMP: 'Apr 8 12:22:32', STRUCTURED-DATA: '-', >> msg: '0" dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP >> Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" >> user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access Control" >> ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 proto="PING"' >> escaped msg: '0" dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, >> ICMP Type:8, service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" >> user="unknown" devID="0019cb7273a4" cat="Firewall" class="Access Control" >> ob="0" ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 proto="PING"' >> rawmsg: '<181>Apr 8 12:22:32 domain.com src="74.208.229.54:0" >> dst="76.10.x.x:0" msg="priority:12, from WAN to ZyWALL, ICMP Type:8, >> service PING, ICMP Type:8, ACCEPT" note="ACCESS FORWARD" user="unknown" >> devID="0019cb7273a4" cat="Firewall" class="Access Control" ob="0" >> ob_mac="000000000000" dir="WAN:ZyWALL" protoID=1 proto="PING"' >> >> >> On 04/08/2011 01:34 AM, Rainer Gerhards wrote: >>> I think the incoming message is illformed. For background, read this: >>> >>> http://www.rsyslog.com/doc/syslog_parsing.html >>> >>> You can create a file via >>> >>> *.* /path/to/file;RSYSLOG_DebugFormat >>> >>> And post the rawmsg output. Then we can probably suggest a solution. But, >>> as >>> said in the paper, the proper solutions are to configure the device to >>> emit >>> correct messages or create a custom parser if that's not possible. >>> >>> Rainer >>> >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of Brano >>>> Sent: Friday, April 08, 2011 4:39 AM >>>> To: rsyslog at lists.adiscon.com >>>> Subject: [rsyslog] Hostname missing from logs >>>> >>>> I've recently switched from syslogd to rsyslogd on CentOS 5.5 due to >>>> need of granular network logging. >>>> However, I've noticed one issue with my remote log. >>>> >>>> I'm logging from remote host called usg200 (defined in local hostfile). >>>> It is ZyXel USG200 router. >>>> >>>> With syslogd the messages in log were like this one >>>> Apr 3 21:27:43 *usg200* domain.com src="76.10.x.x:500" >>>> dst="76.10.x.x:500" msg="The cookie pair is : 0x6f28d9b0e98a895a / >>>> 0x3bfec >>>> fd059520966" note="IKE_LOG" user="unknown" devID="0019cb7273a4" >>>> cat="IKE" >>>> >>>> With rsyslog the messages are like this >>>> Apr 6 14:21:04 domain.com src="76.10.x.x: 500" dst="76.10.x.x:500" >>>> msg="Recv:[HASH][NOTFY:R_U_THERE]" note="IKE_LOG" user="unknown" >>>> devID="0019cb7273a4" cat="IKE" >>>> >>>> Notice the usg200 hostname in rsyslog just after date is missing. I >>>> need >>>> to get it back. Any advice highly appreciated. >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From lanas at securenet.net Sun Apr 10 19:21:27 2011 From: lanas at securenet.net (lanas) Date: Sun, 10 Apr 2011 13:21:27 -0400 Subject: [rsyslog] Remote logging and dummy IP Message-ID: <20110410132127.58fa677e@mistral.stie> Hello, Is it possible to specify a dummy IP for all outgoing rsyslog packets destined to a remote logging application ? Thanks. From david at lang.hm Mon Apr 11 01:13:01 2011 From: david at lang.hm (david at lang.hm) Date: Sun, 10 Apr 2011 16:13:01 -0700 (PDT) Subject: [rsyslog] Remote logging and dummy IP In-Reply-To: <20110410132127.58fa677e@mistral.stie> References: <20110410132127.58fa677e@mistral.stie> Message-ID: On Sun, 10 Apr 2011, lanas wrote: > Hello, > Is it possible to specify a dummy IP for all outgoing rsyslog packets > destined to a remote logging application ? it may be possible using the hooks that were put in to let you forward messages via UDP and have the source IP be the original source. what is it that you are trying to do? David Lang From lanas at securenet.net Mon Apr 11 11:35:03 2011 From: lanas at securenet.net (lanas) Date: Mon, 11 Apr 2011 05:35:03 -0400 Subject: [rsyslog] Remote logging and dummy IP In-Reply-To: References: <20110410132127.58fa677e@mistral.stie> Message-ID: <20110411053503.1169db53@mistral.stie> On Sun, 10 Apr 2011 16:13:01 -0700 (PDT), david at lang.hm wrote : > On Sun, 10 Apr 2011, lanas wrote: > > > Is it possible to specify a dummy IP for all outgoing rsyslog > > packets destined to a remote logging application ? > > it may be possible using the hooks that were put in to let you > forward messages via UDP and have the source IP be the original > source. > > what is it that you are trying to do? I sounds like it ;-) Would you have an example configuration ? Thanks. From rgerhards at hq.adiscon.com Mon Apr 11 11:36:42 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 11 Apr 2011 11:36:42 +0200 Subject: [rsyslog] Remote logging and dummy IP In-Reply-To: <20110411053503.1169db53@mistral.stie> References: <20110410132127.58fa677e@mistral.stie> <20110411053503.1169db53@mistral.stie> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFA5@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of lanas > Sent: Monday, April 11, 2011 11:35 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Remote logging and dummy IP > > On Sun, 10 Apr 2011 16:13:01 -0700 (PDT), david at lang.hm wrote : > > > On Sun, 10 Apr 2011, lanas wrote: > > > > > Is it possible to specify a dummy IP for all outgoing rsyslog > > > packets destined to a remote logging application ? > > > > it may be possible using the hooks that were put in to let you forward > > messages via UDP and have the source IP be the original source. > > > > what is it that you are trying to do? > > I sounds like it ;-) Would you have an example configuration ? I do not fully understand your question, but the first sample in the doc seems to do what I guess you want: http://www.rsyslog.com/doc/omudpspoof.html Rainer From tbergfeld at hq.adiscon.com Tue Apr 12 12:03:53 2011 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Tue, 12 Apr 2011 12:03:53 +0200 Subject: [rsyslog] rsyslog 5.8.0 (v5-stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFB1@GRFEXC.intern.adiscon.com> This is the new v5-stable version. It contains all enhancements made to the 5.7.x versions. Among others, this includes support for systemd, enhancements for imfile, new custom parsers, UDP realtime reception support, Hadoop HDFS support and many other improvements. Please note that it also contains two bug fixes that are NOT present in 5.6.5. ChangeLog: http://www.rsyslog.com/changelog-for-5-8-0-v5-stable/ Download: http://www.rsyslog.com/rsyslog-5-8-0-v5-stable/ As always, feedback is appreciated. Best regards, Tom Bergfeld -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html. From pgollucci at p6m7g8.com Tue Apr 12 22:37:20 2011 From: pgollucci at p6m7g8.com (Philip M. Gollucci) Date: Tue, 12 Apr 2011 20:37:20 +0000 Subject: [rsyslog] rsyslog 5.8.0 (v5-stable) released In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDFB1@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DDFB1@GRFEXC.intern.adiscon.com> Message-ID: <4DA4B800.7020400@p6m7g8.com> Which track does this update ? Any is any branch now deprecated ? /pr/sysutils (master) grep ^PORTVERSION rsyslog*/* rsyslog3/bsd.rsyslog.mk:PORTVERSION?= 3.22.3 rsyslog4-devel/Makefile:PORTVERSION= 4.7.3 rsyslog4/Makefile:PORTVERSION= 4.6.5 rsyslog5-devel/Makefile:PORTVERSION= 5.7.10 rsyslog5/Makefile:PORTVERSION= 5.6.5 rsyslog6-devel/Makefile:PORTVERSION= 6.1.6 On 04/12/11 10:03, Tom Bergfeld wrote: > This is the new v5-stable version. It contains all enhancements made to the > 5.7.x versions. Among others, this includes support for systemd, enhancements > for imfile, new custom parsers, UDP realtime reception support, Hadoop HDFS > support and many other improvements. Please note that it also contains two > bug fixes that are NOT present in 5.6.5. > > ChangeLog: > > http://www.rsyslog.com/changelog-for-5-8-0-v5-stable/ > > Download: > > http://www.rsyslog.com/rsyslog-5-8-0-v5-stable/ > > As always, feedback is appreciated. > > Best regards, > Tom Bergfeld > > -- > > Support > ======= > Improving rsyslog is costly, but you can help! We are looking for > organizations that find rsyslog useful and wish to contribute back. You can > contribute by reporting bugs, improve the software, or donate money or > equipment. > > Commercial support contracts for rsyslog are available, and they help finance > continued maintenance. Adiscon GmbH, a privately held German company, is > currently funding rsyslog development. We are always looking for interesting > development projects. For details on how to help, please see > http://www.rsyslog.com/doc-how2help.html. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > -- ------------------------------------------------------------------------ 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354 Member, Apache Software Foundation Committer, FreeBSD Foundation Consultant, P6M7G8 Inc. Sr. System Admin, Ridecharge Inc. Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. From lanas at securenet.net Wed Apr 13 01:29:53 2011 From: lanas at securenet.net (lanas) Date: Tue, 12 Apr 2011 19:29:53 -0400 Subject: [rsyslog] Remote logging and dummy IP In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDFA5@GRFEXC.intern.adiscon.com> References: <20110410132127.58fa677e@mistral.stie> <20110411053503.1169db53@mistral.stie> <9B6E2A8877C38245BFB15CC491A11DA71DDFA5@GRFEXC.intern.adiscon.com> Message-ID: <20110412192953.6c9ab9eb@mistral.stie> On Mon, 11 Apr 2011 11:36:42 +0200, "Rainer Gerhards" wrote : > > > On Sun, 10 Apr 2011, lanas wrote: > > > > Is it possible to specify a dummy IP for all outgoing rsyslog > > > > packets destined to a remote logging application ? > I do not fully understand your question, but the first sample in the > doc seems to do what I guess you want: > http://www.rsyslog.com/doc/omudpspoof.html There is a growing number of setups that have to specify a dummy address for certain services (not only remote sys-logging) as the source IP of the packet (and not the IP of the interface by which it is sent) in order to observe some security requieremnts. The omudpspoof configuration seems to be doing that: I will try it. Is the $ActionUDPSpoofTargetHost a mandatory parameter to have this working ? I'd have thought that setting the source IP of the packet would be independent of any target. Thanks ! From david at lang.hm Wed Apr 13 04:39:58 2011 From: david at lang.hm (david at lang.hm) Date: Tue, 12 Apr 2011 19:39:58 -0700 (PDT) Subject: [rsyslog] Remote logging and dummy IP In-Reply-To: <20110412192953.6c9ab9eb@mistral.stie> References: <20110410132127.58fa677e@mistral.stie> <20110411053503.1169db53@mistral.stie> <9B6E2A8877C38245BFB15CC491A11DA71DDFA5@GRFEXC.intern.adiscon.com> <20110412192953.6c9ab9eb@mistral.stie> Message-ID: On Tue, 12 Apr 2011, lanas wrote: > On Mon, 11 Apr 2011 11:36:42 +0200, > "Rainer Gerhards" wrote : > >>>> On Sun, 10 Apr 2011, lanas wrote: >>>>> Is it possible to specify a dummy IP for all outgoing rsyslog >>>>> packets destined to a remote logging application ? > >> I do not fully understand your question, but the first sample in the >> doc seems to do what I guess you want: > >> http://www.rsyslog.com/doc/omudpspoof.html > > There is a growing number of setups that have to specify a dummy > address for certain services (not only remote sys-logging) as the source > IP of the packet (and not the IP of the interface by which it is sent) > in order to observe some security requieremnts. I'd be interested in hearing about these security requirements (since they eliminate the possibility for reliable delivery) > The omudpspoof configuration seems to be doing that: I will try it. Is > the $ActionUDPSpoofTargetHost a mandatory parameter to have this > working ? I'd have thought that setting the source IP of the packet > would be independent of any target. the source is independent of the target, but there is no other way to specify the target. If you look at the examples on the page, you will see that you don't specify the destination on the action line. David Lang From david at lang.hm Wed Apr 13 04:50:47 2011 From: david at lang.hm (david at lang.hm) Date: Tue, 12 Apr 2011 19:50:47 -0700 (PDT) Subject: [rsyslog] rsyslog 5.8.0 (v5-stable) released In-Reply-To: <4DA4B800.7020400@p6m7g8.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DDFB1@GRFEXC.intern.adiscon.com> <4DA4B800.7020400@p6m7g8.com> Message-ID: this should replace 5.6.5 and 5.7.10 David Lang On Tue, 12 Apr 2011, Philip M. Gollucci wrote: > Which track does this update ? Any is any branch now deprecated ? > > /pr/sysutils (master) > grep ^PORTVERSION rsyslog*/* > rsyslog3/bsd.rsyslog.mk:PORTVERSION?= 3.22.3 > rsyslog4-devel/Makefile:PORTVERSION= 4.7.3 > rsyslog4/Makefile:PORTVERSION= 4.6.5 > rsyslog5-devel/Makefile:PORTVERSION= 5.7.10 > rsyslog5/Makefile:PORTVERSION= 5.6.5 > rsyslog6-devel/Makefile:PORTVERSION= 6.1.6 > > > > > > On 04/12/11 10:03, Tom Bergfeld wrote: >> This is the new v5-stable version. It contains all enhancements made to the >> 5.7.x versions. Among others, this includes support for systemd, enhancements >> for imfile, new custom parsers, UDP realtime reception support, Hadoop HDFS >> support and many other improvements. Please note that it also contains two >> bug fixes that are NOT present in 5.6.5. >> >> ChangeLog: >> >> http://www.rsyslog.com/changelog-for-5-8-0-v5-stable/ >> >> Download: >> >> http://www.rsyslog.com/rsyslog-5-8-0-v5-stable/ >> >> As always, feedback is appreciated. >> >> Best regards, >> Tom Bergfeld >> >> -- >> >> Support >> ======= >> Improving rsyslog is costly, but you can help! We are looking for >> organizations that find rsyslog useful and wish to contribute back. You can >> contribute by reporting bugs, improve the software, or donate money or >> equipment. >> >> Commercial support contracts for rsyslog are available, and they help finance >> continued maintenance. Adiscon GmbH, a privately held German company, is >> currently funding rsyslog development. We are always looking for interesting >> development projects. For details on how to help, please see >> http://www.rsyslog.com/doc-how2help.html. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > > > From pgollucci at p6m7g8.com Wed Apr 13 05:06:26 2011 From: pgollucci at p6m7g8.com (Philip M. Gollucci) Date: Tue, 12 Apr 2011 23:06:26 -0400 Subject: [rsyslog] rsyslog 5.8.0 (v5-stable) released In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DDFB1@GRFEXC.intern.adiscon.com> <4DA4B800.7020400@p6m7g8.com> Message-ID: <4DA51332.4080003@p6m7g8.com> On 4/12/2011 10:50 PM, david at lang.hm wrote: > this should replace 5.6.5 and 5.7.10 >> /pr/sysutils (master) >> grep ^PORTVERSION rsyslog*/* >> rsyslog3/bsd.rsyslog.mk:PORTVERSION?= 3.22.3 >> rsyslog4-devel/Makefile:PORTVERSION= 4.7.3 >> rsyslog4/Makefile:PORTVERSION= 4.6.5 >> rsyslog5-devel/Makefile:PORTVERSION= 5.7.10 So this one gets 'removed' from ports tree >> rsyslog5/Makefile:PORTVERSION= 5.6.5 This one gets updated. >> rsyslog6-devel/Makefile:PORTVERSION= 6.1.6 Thanks. I'll look into this, Wednesday ish. -- ------------------------------------------------------------------------ 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354 Member, Apache Software Foundation Committer, FreeBSD Foundation Consultant, P6M7G8 Inc. Sr. System Admin, Ridecharge Inc. Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. From rgerhards at hq.adiscon.com Wed Apr 13 07:16:13 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 13 Apr 2011 07:16:13 +0200 Subject: [rsyslog] Remote logging and dummy IP In-Reply-To: <20110412192953.6c9ab9eb@mistral.stie> References: <20110410132127.58fa677e@mistral.stie><20110411053503.1169db53@mistral.stie><9B6E2A8877C38245BFB15CC491A11DA71DDFA5@GRFEXC.intern.adiscon.com> <20110412192953.6c9ab9eb@mistral.stie> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFC2@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of lanas > Sent: Wednesday, April 13, 2011 1:30 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Remote logging and dummy IP > > On Mon, 11 Apr 2011 11:36:42 +0200, > "Rainer Gerhards" wrote : > > > > > On Sun, 10 Apr 2011, lanas wrote: > > > > > Is it possible to specify a dummy IP for all outgoing rsyslog > > > > > packets destined to a remote logging application ? > > > I do not fully understand your question, but the first sample in the > > doc seems to do what I guess you want: > > > http://www.rsyslog.com/doc/omudpspoof.html > > There is a growing number of setups that have to specify a dummy > address for certain services (not only remote sys-logging) as the > source > IP of the packet (and not the IP of the interface by which it is sent) > in order to observe some security requieremnts. > > The omudpspoof configuration seems to be doing that: I will try it. Is > the $ActionUDPSpoofTargetHost a mandatory parameter to have this > working ? I'd have thought that setting the source IP of the packet > would be independent of any target. This is a forwarding action. It needs to know where to forward to. It is told so by the target host. So, yes, this is mandatory. Rainer From andersns at basefarm.no Wed Apr 13 08:33:07 2011 From: andersns at basefarm.no (Anders Synstad) Date: Wed, 13 Apr 2011 08:33:07 +0200 Subject: [rsyslog] Problems with cloned syslogging to central loghost Message-ID: <4DA543A3.5030303@basefarm.no> This is a bit out of scope I'd think, but here we go. I'm attempting to use cloning on a loadbalancer, to clone my log traffic to a development loghost. The idea is to the same amount of log traffic on my development server, as I get on my production server. In theory nice. To get it working, I had to add some iptables rules to rewrite the destination address of the logpackages, otherwise the kernel would just drop them: Chain PREROUTING (policy ACCEPT 40 packets, 4057 bytes) pkts bytes target prot opt in out source destination 163 26207 NETMAP udp -- * * 0.0.0.0/0 10.27.23.11 10.27.23.100/32 0 0 NETMAP udp -- * * 0.0.0.0/0 10.27.23.12 10.27.23.100/32 And for most parts, this works nice. However, I just discovered the following: [..] 011-04-13T00:15:09.840275+02:00 loghost01 possible SYN flooding on port 10515. Sending cookies. 2011-04-13T00:15:10.473383+02:00 loghost01 netstream session 0xe0a7190 will be closed due to error 2011-04-13T00:15:19.610531+02:00 loghost01 netstream session 0xe2151a0 will be closed due to error 2011-04-13T00:15:19.647328+02:00 loghost01 netstream session 0xe216820 will be closed due to error 2011-04-13T00:15:19.652144+02:00 loghost01 netstream session 0xe0aa160 will be closed due to error 2011-04-13T00:15:24.585523+02:00 loghost01 netstream session 0xe054720 will be closed due to error 2011-04-13T00:15:27.772918+02:00 loghost01 netstream session 0xe055660 will be closed due to error [..] 2011-04-13T00:16:14.312158+02:00 loghost01 epoll_ctl failed on fd 1023, id 0/0xe218b20, op 1 with File exists 2011-04-13T00:16:14.312168+02:00 loghost01 epoll_ctl failed on fd 1023, id 0/0xe218b20, op 1 with File exists 2011-04-13T00:16:14.312175+02:00 loghost01 epoll_ctl failed on fd 1023, id 0/0xe218b20, op 1 with File exists 2011-04-13T00:16:14.312181+02:00 loghost01 epoll_ctl failed on fd 1023, id 0/0xe218b20, op 1 with File exists 2011-04-13T00:16:14.312189+02:00 loghost01 epoll_ctl failed on fd 1023, id 0/0xe218b20, op 1 with File exists 2011-04-13T00:16:14.312194+02:00 loghost01 epoll_ctl failed on fd 1023, id 0/0xe218b20, op 1 with File exists [..] My guess is that this configuration doesn't work great with tcp syslogging. However, the "epoll_ctl failed on fd" goes absolutely mental. So far today, it's logged 5.6G worth of just that error :) So two questions; 1) Is there a way to make rsyslog put the network interface in promiscuous mode for logging, so I don't need these rewrite rules, and it acts more like a silent observer (like SNORT and friends). 2) Is "epoll_ctl failed on fd" error something that perhaps is related to an unhandled exception in rsyslog? Or at least an exception that isn't handled very gracefully? Regards, Anders Synstad Basefarm AS From david at lang.hm Wed Apr 13 08:45:24 2011 From: david at lang.hm (david at lang.hm) Date: Tue, 12 Apr 2011 23:45:24 -0700 (PDT) Subject: [rsyslog] Problems with cloned syslogging to central loghost In-Reply-To: <4DA543A3.5030303@basefarm.no> References: <4DA543A3.5030303@basefarm.no> Message-ID: On Wed, 13 Apr 2011, Anders Synstad wrote: > > My guess is that this configuration doesn't work great with tcp syslogging. > However, the "epoll_ctl failed on fd" goes absolutely mental. So far today, > it's logged 5.6G worth of just that error :) > > So two questions; > > 1) Is there a way to make rsyslog put the network interface in promiscuous > mode for logging, so I don't need these rewrite rules, and it acts more like > a silent observer (like SNORT and friends). > > > 2) Is "epoll_ctl failed on fd" error something that perhaps is related to an > unhandled exception in rsyslog? Or at least an exception that isn't handled > very gracefully? I believe that the fundamental problem is that TCP requires full two-way communication with the sender, so you can't just clone the packets and have things work (the receiver sends information back to the sender, and unless the sender gets it and adjusts future packets appropriately the receiver will decide something is very wrong) if you were to use UDP you won't have this limitation. one other trick you can pull with UDP (assuming your production and preprod servers don't need to talk directly to each other) is that you can put the IP address of the production server on the loopback interface of your preprod server. If you do this and have something in the middle clone the packets (but change their MAC address) then you should receive them normally. two possible problems with this. 1. you may have to fix the checksums as well as the MAC address (I don't think so, but I'm only 90% or so sure) 2. if you are on the same VLAN (broadcast domain) as the production server, you may have to configure arptables (a variation of iptables) to make sure that your preprod server does not respond to any ARP requests intended for the production server. for ARP purposes, Linux considers the IP addresses assigned to the machine, not any particular interface, and will respond on any interface queried if it has the IP address on any other interface another option you may want to consider is multicast MAC with UDP. This allows you to have multiple machines with the same IP address on one network. you can even use CLUSTERIP (part of iptables) to do load balancing of inbound traffic across multiple machines. To have multiple machines get the same traffic, configure CLUSTERIP to make each machine think it is the only box with that IP address. David Lang From andersns at basefarm.no Wed Apr 13 08:54:58 2011 From: andersns at basefarm.no (Anders Synstad) Date: Wed, 13 Apr 2011 08:54:58 +0200 Subject: [rsyslog] Problems with cloned syslogging to central loghost In-Reply-To: References: <4DA543A3.5030303@basefarm.no> Message-ID: <4DA548C2.4090005@basefarm.no> On 04/13/2011 08:45 AM, david at lang.hm wrote: > On Wed, 13 Apr 2011, Anders Synstad wrote: > >> >> My guess is that this configuration doesn't work great with tcp >> syslogging. However, the "epoll_ctl failed on fd" goes absolutely >> mental. So far today, it's logged 5.6G worth of just that error :) >> >> So two questions; >> >> 1) Is there a way to make rsyslog put the network interface in >> promiscuous mode for logging, so I don't need these rewrite rules, and >> it acts more like a silent observer (like SNORT and friends). >> >> >> 2) Is "epoll_ctl failed on fd" error something that perhaps is related >> to an unhandled exception in rsyslog? Or at least an exception that >> isn't handled very gracefully? > > I believe that the fundamental problem is that TCP requires full two-way > communication with the sender, so you can't just clone the packets and > have things work (the receiver sends information back to the sender, and > unless the sender gets it and adjusts future packets appropriately the > receiver will decide something is very wrong) > > if you were to use UDP you won't have this limitation. > 99% of my traffic is UDP logging. It's just some devices that are logging tcp, but it's enough to make it misbehave. I can just remove the cloning on the tcp logging, and I still get more than enough traffic. > one other trick you can pull with UDP (assuming your production and > preprod servers don't need to talk directly to each other) is that you > can put the IP address of the production server on the loopback > interface of your preprod server. If you do this and have something in > the middle clone the packets (but change their MAC address) then you > should receive them normally. > > two possible problems with this. > > 1. you may have to fix the checksums as well as the MAC address (I don't > think so, but I'm only 90% or so sure) > > 2. if you are on the same VLAN (broadcast domain) as the production > server, you may have to configure arptables (a variation of iptables) to > make sure that your preprod server does not respond to any ARP requests > intended for the production server. for ARP purposes, Linux considers > the IP addresses assigned to the machine, not any particular interface, > and will respond on any interface queried if it has the IP address on > any other interface > > > > another option you may want to consider is multicast MAC with UDP. This > allows you to have multiple machines with the same IP address on one > network. you can even use CLUSTERIP (part of iptables) to do load > balancing of inbound traffic across multiple machines. To have multiple > machines get the same traffic, configure CLUSTERIP to make each machine > think it is the only box with that IP address. > In other words, this requires more work than I'm willing to put in for my development server. It's not that important, but more of a curiosity :) Thank you for you answer. Regards, Anders Synstad From david at lang.hm Wed Apr 13 08:58:29 2011 From: david at lang.hm (david at lang.hm) Date: Tue, 12 Apr 2011 23:58:29 -0700 (PDT) Subject: [rsyslog] Problems with cloned syslogging to central loghost In-Reply-To: <4DA548C2.4090005@basefarm.no> References: <4DA543A3.5030303@basefarm.no> <4DA548C2.4090005@basefarm.no> Message-ID: On Wed, 13 Apr 2011, Anders Synstad wrote: > On 04/13/2011 08:45 AM, david at lang.hm wrote: >> >> another option you may want to consider is multicast MAC with UDP. This >> allows you to have multiple machines with the same IP address on one >> network. you can even use CLUSTERIP (part of iptables) to do load >> balancing of inbound traffic across multiple machines. To have multiple >> machines get the same traffic, configure CLUSTERIP to make each machine >> think it is the only box with that IP address. >> > > In other words, this requires more work than I'm willing to put in for my > development server. It's not that important, but more of a curiosity :) this is actually surprisingly easy. I'll try and remember to do a cut-n-paste of the configuration to do this manually and send it to you tomorrow. It's an _extremely_ powerful tool to have available. David Lang From igor.sverkos at googlemail.com Wed Apr 13 11:26:14 2011 From: igor.sverkos at googlemail.com (Igor Sverkos) Date: Wed, 13 Apr 2011 11:26:14 +0200 Subject: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? In-Reply-To: References: Message-ID: <4DA56C36.1060405@googlemail.com> Hi, is nobody able to help me to understand what's going on in Debian's rsyslog? -- Regards, Igor I wrote on 2011-03-28 02:26: > Hi, > > I am new to rsyslog. Currently I don't understand, why mail.* messages > aren't included in the /var/log/syslog file. This is the default > Debian rsyslog.conf: > >> [...] >> ############### >> #### RULES #### >> ############### >> >> # >> # First some standard log files. Log by facility. >> # >> auth,authpriv.* /var/log/auth.log >> *.*;auth,authpriv.none -/var/log/syslog >> #cron.* /var/log/cron.log >> daemon.* -/var/log/daemon.log >> kern.* -/var/log/kern.log >> lpr.* -/var/log/lpr.log >> mail.* -/var/log/mail.log >> user.* -/var/log/user.log >> >> # >> # Logging for the mail system. Split it up so that >> # it is easy to write scripts to parse these files. >> # >> mail.info -/var/log/mail.info >> mail.warn -/var/log/mail.warn >> mail.err /var/log/mail.err >> >> [...] > > In my understanding, a message with the facility "mail" and severity > "info" comes in, the following should happen: > > 1. Rule: > Does *not* match facility "auth" or "authpriv", so the message won't > get into "/var/log/auth.log". > The next rule should get processed. > > 2. Rule: > *Does* match facility *and* severity "*.*", so it's included. > It doesn't match "auth" or "authpriv.*", so it isn't excluded. > > => The message *should* get into /var/log/syslog... > > But it doesn't?! > > It will be written to /var/log/mail.log (rule "mail.*") and also into > /var/log/mail.info (rule "mail.info")... but why won't it also be > logged in /var/log/syslog? > > From rgerhards at hq.adiscon.com Wed Apr 13 11:27:33 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 13 Apr 2011 11:27:33 +0200 Subject: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? In-Reply-To: <4DA56C36.1060405@googlemail.com> References: <4DA56C36.1060405@googlemail.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> Can you create me a debug log of a mail message being processed? > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Igor Sverkos > Sent: Wednesday, April 13, 2011 11:26 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? > > Hi, > > is nobody able to help me to understand what's going on in Debian's rsyslog? > > > -- > Regards, > Igor > > > I wrote on 2011-03-28 02:26: > > Hi, > > > > I am new to rsyslog. Currently I don't understand, why mail.* messages > > aren't included in the /var/log/syslog file. This is the default > > Debian rsyslog.conf: > > > >> [...] > >> ############### > >> #### RULES #### > >> ############### > >> > >> # > >> # First some standard log files. Log by facility. > >> # > >> auth,authpriv.* /var/log/auth.log > >> *.*;auth,authpriv.none -/var/log/syslog > >> #cron.* /var/log/cron.log > >> daemon.* -/var/log/daemon.log > >> kern.* -/var/log/kern.log > >> lpr.* -/var/log/lpr.log > >> mail.* -/var/log/mail.log > >> user.* -/var/log/user.log > >> > >> # > >> # Logging for the mail system. Split it up so that > >> # it is easy to write scripts to parse these files. > >> # > >> mail.info -/var/log/mail.info > >> mail.warn -/var/log/mail.warn > >> mail.err /var/log/mail.err > >> > >> [...] > > > > In my understanding, a message with the facility "mail" and severity > > "info" comes in, the following should happen: > > > > 1. Rule: > > Does *not* match facility "auth" or "authpriv", so the message won't > > get into "/var/log/auth.log". > > The next rule should get processed. > > > > 2. Rule: > > *Does* match facility *and* severity "*.*", so it's included. > > It doesn't match "auth" or "authpriv.*", so it isn't excluded. > > > > => The message *should* get into /var/log/syslog... > > > > But it doesn't?! > > > > It will be written to /var/log/mail.log (rule "mail.*") and also into > > /var/log/mail.info (rule "mail.info")... but why won't it also be > > logged in /var/log/syslog? > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From igor.sverkos at googlemail.com Wed Apr 13 15:08:54 2011 From: igor.sverkos at googlemail.com (Igor Sverkos) Date: Wed, 13 Apr 2011 15:08:54 +0200 Subject: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> References: <4DA56C36.1060405@googlemail.com> <9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> Message-ID: <4DA5A066.1000600@googlemail.com> Hi, Rainer Gerhards wrote: > Can you create me a debug log of a mail message being processed? Thanks for your reply. I am so sorry. While creating the debug log and checking the result, everything was as expected. Than I checked the logs from 28.03 when I sent the first mail and they also contained everything as expected. Don't know if calling rsyslogd with "-f $oldcfg" or the new 5.7.10 version makes the different. Maybe it helped that I cleaned my glasses ;) So there is currently no problem, my first mail seems to be bogus. Sorry for wasting your time. -- Regards, Igor From rgerhards at hq.adiscon.com Wed Apr 13 15:10:51 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 13 Apr 2011 15:10:51 +0200 Subject: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? In-Reply-To: <4DA5A066.1000600@googlemail.com> References: <4DA56C36.1060405@googlemail.com><9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> <4DA5A066.1000600@googlemail.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFD0@GRFEXC.intern.adiscon.com> Could very well be 5.7.10. Note the 5.8.0 contains a not so unimportant patch, so I suggest to update (details in announcements on site). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Igor Sverkos > Sent: Wednesday, April 13, 2011 3:09 PM > To: rsyslog-users > Subject: Re: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? > > Hi, > > Rainer Gerhards wrote: > > Can you create me a debug log of a mail message being processed? > > Thanks for your reply. > > I am so sorry. While creating the debug log and checking the result, > everything was as expected. Than I checked the logs from 28.03 when I sent > the first mail and they also contained everything as expected. > > Don't know if calling rsyslogd with "-f $oldcfg" or the new 5.7.10 version > makes the different. Maybe it helped that I cleaned my glasses ;) > > So there is currently no problem, my first mail seems to be bogus. Sorry for > wasting your time. > > > -- > Regards, > Igor > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From igor.sverkos at googlemail.com Wed Apr 13 15:21:32 2011 From: igor.sverkos at googlemail.com (Igor Sverkos) Date: Wed, 13 Apr 2011 15:21:32 +0200 Subject: [rsyslog] Why doesn't /var/log/syslog include mail.* messages? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDFD0@GRFEXC.intern.adiscon.com> References: <4DA56C36.1060405@googlemail.com><9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> <4DA5A066.1000600@googlemail.com> <9B6E2A8877C38245BFB15CC491A11DA71DDFD0@GRFEXC.intern.adiscon.com> Message-ID: <4DA5A35C.5040704@googlemail.com> Hi, Rainer Gerhards wrote: > Could very well be 5.7.10. Note the 5.8.0 contains a not so unimportant > patch, so I suggest to update (details in announcements on site). I read that annoucement. But currently, we are using Debian packages, to keep system maintenance as easy as possible. 5.8.x is already in SID, so it should be in testing next week (the main system is squeeze, but we are using rsyslog from testing, if someone is asking). -- Regards, Igor From piavka at cs.bgu.ac.il Thu Apr 14 00:57:00 2011 From: piavka at cs.bgu.ac.il (Piavlo) Date: Thu, 14 Apr 2011 01:57:00 +0300 Subject: [rsyslog] high cpu spkies by rsyslog which barely does anything In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF6D@GRFEXC.intern.adiscon.com> References: <4D8B51B8.7050908@cs.bgu.ac.il> <4D9CD8D1.3080805@cs.bgu.ac.il> <9B6E2A8877C38245BFB15CC491A11DA71DDF6D@GRFEXC.intern.adiscon.com> Message-ID: <4DA62A3C.6040304@cs.bgu.ac.il> I've recompiled rsyslog with the following options which were previously enabled -disable-debug --disable-rtinst --disable-valgrind --disable-memcheck and the problem seems to be gone. Which of the following options are safe to enable in production env - without loosing in performance? Thanks Alex Rainer Gerhards wrote: > I don't see anything that looks particular bad -- and I've never before heard > of such a problem (which obviously doesn't mean a lot ;)). Maybe you should > enable debug mode so that we can see if some unusual processing happens. > > Rainer > > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Piavlo >> Sent: Wednesday, April 06, 2011 11:19 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] high cpu spkies by rsyslog which barely does >> anything >> >> >> >> Anyone has an idea? >> Might it be due to some rsyslog compilation option? >> >> Is my question inappropriate for this list? >> >> Thanks >> Alex >> >> Piavlo wrote: >> >>> Hi, >>> >>> I'd like to use rsyslog as central log collection solution. >>> Manwhile I've compiled and installed as rsyslog 5.6.4 with a stock >>> configuration as drop in replacement to local ksyslogd >>> But I see a high quick cpu usage bursts by rsyslog every few seconds >>> while it barely does anything. >>> Most of the cpu time is spent in user mode. >>> This happends on ec2 large instance with Centos-5.5. >>> >>> Below are the configs and sample strace/lsof of the rsyslogd process. >>> >>> Any idea why this happens? >>> >>> [root at www1a ~]# cat /etc/rsyslog.conf | grep -v -e '^$' -e '^#' >>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >>> $ModLoad imklog >>> $ModLoad imuxsock >>> *.info;mail.none;authpriv.none;cron.none >>> >> /var/log/messages >> >>> authpriv.* >>> >> /var/log/secure >> >>> mail.* - >>> >> /var/log/maillog >> >>> cron.* /var/log/cron >>> *.emerg * >>> uucp,news.crit >>> >> /var/log/spooler >> >>> local7.* >>> >> /var/log/boot.log >> >>> [root at www1a ~]# >>> [root at www1a ~]# lsof -p $(pgrep rsyslogd) >>> COMMAND PID USER FD TYPE DEVICE SIZE >>> >> NODE NAME >> >>> rsyslogd 29994 root cwd DIR 8,1 4096 >>> >> 2 / >> >>> rsyslogd 29994 root rtd DIR 8,1 4096 >>> >> 2 / >> >>> rsyslogd 29994 root txt REG 8,1 1961412 >>> >> 2378118 >> >>> /usr/local/rsyslog/5.6.4/sbin/rsyslogd >>> rsyslogd 29994 root mem REG 8,1 137256 >>> >> 1720349 >> >>> /lib64/ld-2.5.so >>> rsyslogd 29994 root mem REG 8,1 83344 >>> >> 1543492 >> >>> /usr/lib64/libz.so.1.2.3 >>> rsyslogd 29994 root mem REG 8,1 142696 >>> >> 1720364 >> >>> /lib64/libpthread-2.5.so >>> rsyslogd 29994 root mem REG 8,1 20424 >>> >> 1720336 >> >>> /lib64/libdl-2.5.so >>> rsyslogd 29994 root mem REG 8,1 50288 >>> >> 1720370 >> >>> /lib64/librt-2.5.so >>> rsyslogd 29994 root mem REG 8,1 1712536 >>> >> 1720326 >> >>> /lib64/libc-2.5.so >>> rsyslogd 29994 root mem REG 8,1 85956 >>> >> 2378093 >> >>> /usr/local/rsyslog/5.6.4/lib/rsyslog/lmnet.so >>> rsyslogd 29994 root mem REG 8,1 90745 >>> >> 2378128 >> >>> /usr/local/rsyslog/5.6.4/lib/rsyslog/imklog.so >>> rsyslogd 29994 root mem REG 8,1 53471 >>> >> 2378110 >> >>> /usr/local/rsyslog/5.6.4/lib/rsyslog/imuxsock.so >>> rsyslogd 29994 root 0r REG 0,3 0 >>> >> 4026531849 >> >>> /proc/kmsg >>> rsyslogd 29994 root 1w REG 8,1 5019 >>> >> 1048938 >> >>> /var/log/messages >>> rsyslogd 29994 root 2w REG 8,1 18336275 >>> >> 1048939 >> >>> /var/log/secure >>> rsyslogd 29994 root 3u unix 0xffff8800241a2a80 >>> >> 80601917 >> >>> /dev/log >>> rsyslogd 29994 root 4w REG 8,1 214769 >>> >> 1048943 >> >>> /var/log/cron >>> [root at www1a ~]# >>> [root at www1a ~]# strace -c -r -T -q -f -p $(pgrep rsyslogd) >>> % time seconds usecs/call calls errors syscall >>> ------ ----------- ----------- --------- --------- ---------------- >>> nan 0.000000 0 4 write >>> nan 0.000000 0 4 select >>> nan 0.000000 0 4 recvfrom >>> nan 0.000000 0 4 gettimeofday >>> nan 0.000000 0 20 4 futex >>> ------ ----------- ----------- --------- --------- ---------------- >>> 100.00 0.000000 36 4 total >>> [root at www1a ~]# >>> [root at www1a ~]# strace -v -r -T -q -f -p $(pgrep rsyslogd) >>> [pid 29995] 0.000000 futex(0x6a5d5c, FUTEX_WAIT, 1167, NULL >>> >>> [pid 29994] 0.005107 select(1, NULL, NULL, NULL, {85286, 180000} >>> >>> [pid 29997] 0.000049 read(0, >>> [pid 29998] 0.000021 select(4, [3], NULL, NULL, NULL) = 1 (in >>> >> [3]) >> >>> <4.329724> >>> [pid 29998] 4.329868 recvfrom(3, "<86>Mar 24 14:01:07 >>> >> sshd[4487]: >> >>> "..., 2048, 0, NULL, NULL) = 92 <0.000015> >>> [pid 29998] 0.000097 gettimeofday({1300975267, 753128}, NULL) = >>> >> 0 >> >>> <0.000010> >>> [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, >>> >> 0x6a5d58, >> >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> >>> [pid 29998] 0.000052 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.000021 <... futex resumed> ) = 0 <4.335124> >>> [pid 29995] 0.000077 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 >>> >> <0.000057> >> >>> [pid 29995] 0.000203 write(2, "Mar 24 14:01:07 www1a >>> sshd[4487]"..., 95 >>> [pid 29998] 0.024438 <... select resumed> ) = 1 (in [3]) >>> >> <0.024706> >> >>> [pid 29995] 0.000024 <... write resumed> ) = 95 <0.024442> >>> [pid 29998] 0.000247 recvfrom(3, "<86>Mar 24 14:01:07 >>> >> sshd[4487]: >> >>> "..., 2048, 0, NULL, NULL) = 95 <0.000014> >>> [pid 29998] 0.000066 gettimeofday({1300975267, 778334}, NULL) = >>> >> 0 >> >>> <0.000010> >>> [pid 29998] 0.000078 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.005155 write(2, "Mar 24 14:01:07 www1a >>> sshd[4487]"..., 98) = 98 <0.000345> >>> [pid 29995] 0.000489 futex(0x6a5d5c, FUTEX_WAIT, 1169, NULL >>> >>> [pid 29998] 0.107333 <... select resumed> ) = 1 (in [3]) >>> >> <0.112941> >> >>> [pid 29998] 0.000053 recvfrom(3, "<86>Mar 24 14:01:07 >>> >> sshd[4487]: >> >>> "..., 2048, 0, NULL, NULL) = 84 <0.000016> >>> [pid 29998] 0.000068 gettimeofday({1300975267, 891512}, NULL) = >>> >> 0 >> >>> <0.000010> >>> [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, >>> >> 0x6a5d58, >> >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> >>> [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.001586 <... futex resumed> ) = 0 <0.109150> >>> [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 >>> >> <0.000046> >> >>> [pid 29995] 0.000169 write(2, "Mar 24 14:01:07 www1a >>> sshd[4487]"..., 87) = 87 <0.000065> >>> [pid 29995] 0.000194 futex(0x6a5d5c, FUTEX_WAIT, 1171, NULL >>> >>> [pid 29998] 1.420908 <... select resumed> ) = 1 (in [3]) >>> >> <1.422893> >> >>> [pid 29998] 0.000049 recvfrom(3, "<86>Mar 24 14:01:09 >>> >> sshd[4517]: >> >>> "..., 2048, 0, NULL, NULL) = 64 <0.000014> >>> [pid 29998] 0.000503 gettimeofday({1300975269, 315123}, NULL) = >>> >> 0 >> >>> <0.000042> >>> [pid 29998] 0.000121 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, >>> >> 0x6a5d58, >> >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} >>> [pid 29995] 0.000490 <... futex resumed> ) = 0 <1.422056> >>> [pid 29998] 0.000010 <... futex resumed> ) = 1 <0.000487> >>> [pid 29995] 0.000122 futex(0x6a1b30, FUTEX_WAKE, 1 >> >> ...> >> >>> [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.000067 <... futex resumed> ) = 0 <0.000072> >>> [pid 29995] 0.000127 write(2, "Mar 24 14:01:09 www1a >>> sshd[4517]"..., 67) = 67 <0.000105> >>> [pid 29995] 0.001215 futex(0x6a5d5c, FUTEX_WAIT, 1173, NULL >>> >>> [pid 29998] 4.916991 <... select resumed> ) = 1 (in [3]) >>> >> <4.918369> >> >>> [pid 29998] 0.000050 recvfrom(3, "<86>Mar 24 14:01:14 >>> >> sshd[4518]: >> >>> "..., 2048, 0, NULL, NULL) = 92 <0.000013> >>> [pid 29998] 0.000060 gettimeofday({1300975274, 234392}, NULL) = >>> >> 0 >> >>> <0.000009> >>> [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, >>> >> 0x6a5d58, >> >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> >>> [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.002861 <... futex resumed> ) = 0 <4.920060> >>> [pid 29995] 0.000177 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 >>> >> <0.000059> >> >>> [pid 29995] 0.000218 write(2, "Mar 24 14:01:14 www1a >>> sshd[4518]"..., 95) = 95 <0.000076> >>> [pid 29995] 0.000378 futex(0x6a5d5c, FUTEX_WAIT, 1175, NULL >>> >>> [pid 29998] 0.067782 <... select resumed> ) = 1 (in [3]) >>> >> <0.071383> >> >>> [pid 29998] 0.000054 recvfrom(3, "<86>Mar 24 14:01:14 >>> >> sshd[4518]: >> >>> "..., 2048, 0, NULL, NULL) = 95 <0.000014> >>> [pid 29998] 0.000069 gettimeofday({1300975274, 306060}, NULL) = >>> >> 0 >> >>> <0.000011> >>> [pid 29998] 0.000080 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, >>> >> 0x6a5d58, >> >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> >>> [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.001890 <... futex resumed> ) = 0 <0.069908> >>> [pid 29995] 0.000070 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 >>> >> <0.000044> >> >>> [pid 29995] 0.000170 write(2, "Mar 24 14:01:14 www1a >>> sshd[4518]"..., 98) = 98 <0.000069> >>> [pid 29995] 0.000285 futex(0x6a5d5c, FUTEX_WAIT, 1177, NULL >>> >>> [pid 29998] 0.098664 <... select resumed> ) = 1 (in [3]) >>> >> <0.101048> >> >>> [pid 29998] 0.001766 recvfrom(3, "<86>Mar 24 14:01:14 >>> >> sshd[4518]: >> >>> "..., 2048, 0, NULL, NULL) = 84 <0.000047> >>> [pid 29998] 0.000125 gettimeofday({1300975274, 409161}, NULL) = >>> >> 0 >> >>> <0.000039> >>> [pid 29998] 0.000119 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, >>> >> 0x6a5d58, >> >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} >>> [pid 29995] 0.000062 <... futex resumed> ) = 0 <0.100721> >>> [pid 29998] 0.000009 <... futex resumed> ) = 1 <0.000058> >>> [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1 >> >> ...> >> >>> [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL >> >> ...> >> >>> [pid 29995] 0.000065 <... futex resumed> ) = 0 <0.000070> >>> [pid 29995] 0.000118 write(2, "Mar 24 14:01:14 www1a >>> sshd[4518]"..., 87) = 87 <0.000069> >>> [pid 29995] 0.000200 futex(0x6a5d5c, FUTEX_WAIT, 1179, NULL >>> >>> [root at www1a ~]# >>> >>> >>> PS. One difference I noticed between stock rsyslog and syslog-ng >>> >> configs >> >>> is that rsyslog holds just one fd open to /dev/log >>> and select's it, while syslog-ng holds about 15 different fd's to >>> /dev/log and epoll's them. >>> Any particular reason for such a drastic difference? >>> >>> Thanks >>> Alex >>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Thu Apr 14 02:06:07 2011 From: david at lang.hm (david at lang.hm) Date: Wed, 13 Apr 2011 17:06:07 -0700 (PDT) Subject: [rsyslog] Problems with cloned syslogging to central loghost In-Reply-To: References: <4DA543A3.5030303@basefarm.no> <4DA548C2.4090005@basefarm.no> Message-ID: On Tue, 12 Apr 2011, david at lang.hm wrote: > On Wed, 13 Apr 2011, Anders Synstad wrote: > >> On 04/13/2011 08:45 AM, david at lang.hm wrote: >>> >>> another option you may want to consider is multicast MAC with UDP. This >>> allows you to have multiple machines with the same IP address on one >>> network. you can even use CLUSTERIP (part of iptables) to do load >>> balancing of inbound traffic across multiple machines. To have multiple >>> machines get the same traffic, configure CLUSTERIP to make each machine >>> think it is the only box with that IP address. >>> >> >> In other words, this requires more work than I'm willing to put in for my >> development server. It's not that important, but more of a curiosity :) > > this is actually surprisingly easy. I'll try and remember to do a cut-n-paste > of the configuration to do this manually and send it to you tomorrow. It's an > _extremely_ powerful tool to have available. to do this, run the following iptables command /sbin/iptables -I INPUT -d 192.168.1.8 -i eth0 -j CLUSTERIP --new --clustermac 01:02:03:04:05:06 --total-nodes 1 --local-node 1 --hashmode sourceip and then configure an interface for 192.168.1.8 on debian I have this in my /etc/network/interfaces auto eth0:0 iface eth0:0 inet static address 192.168.1.8 netmask 255.255.255.0 do this on two machines on the same network and then send UDP traffic to 192.168.1.8 and both machines will get a copy if you need to change one of the machines to a load balanced cluster, insted of saying machine 1 of 1 in the iptables line, make one 1 of 2 and the other 2 of 2 and each machine will get approximatly half the connections (since this shifts on connections, not individual packets, this is why rsyslog has the rebind options to have UDP and TCP senders periodically reopen a connection, so the new connection will be load balanced independantly of the old one. if you do this rebind a few times a second, you will get pretty even distribution between a couple of machines) you can user heartbeat (www.linux-ha.org) to manage the clusterIP configuration so that when one box fails, the other box(s) in the cluster reconfigure themselves so that you always have _some_ box responding to all traffic CLUSTERIP itself isn't limited to UDP, you can use it to load balance TCP connections across multiple machines, but since TCP requires two-way communications you can't use it with multiple machines recieving the same traffic (you would get exactly the same problem you ran into with the packet replication) David Lang From rgerhards at hq.adiscon.com Thu Apr 14 07:13:09 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 14 Apr 2011 07:13:09 +0200 Subject: [rsyslog] high cpu spkies by rsyslog which barely does anything In-Reply-To: <4DA62A3C.6040304@cs.bgu.ac.il> References: <4D8B51B8.7050908@cs.bgu.ac.il> <4D9CD8D1.3080805@cs.bgu.ac.il><9B6E2A8877C38245BFB15CC491A11DA71DDF6D@GRFEXC.intern.adiscon.com> <4DA62A3C.6040304@cs.bgu.ac.il> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFD4@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Piavlo > Sent: Thursday, April 14, 2011 12:57 AM > To: rsyslog-users > Subject: Re: [rsyslog] high cpu spkies by rsyslog which barely does > anything > > I've recompiled rsyslog with the following options which were > previously enabled > -disable-debug --disable-rtinst --disable-valgrind --disable-memcheck > and the problem seems to be gone. > > Which of the following options are safe to enable in production env - > without loosing in performance? Probably --enable-rtinst. --enable-valgrind is really only useful if you intend to run under valgrind --enable-debug causes heavy performance los (around 10 times). --enable-memcheck I have not measured. Rainer > > Thanks > Alex > > Rainer Gerhards wrote: > > I don't see anything that looks particular bad -- and I've never > before heard > > of such a problem (which obviously doesn't mean a lot ;)). Maybe you > should > > enable debug mode so that we can see if some unusual processing > happens. > > > > Rainer > > > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Piavlo > >> Sent: Wednesday, April 06, 2011 11:19 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] high cpu spkies by rsyslog which barely does > >> anything > >> > >> > >> > >> Anyone has an idea? > >> Might it be due to some rsyslog compilation option? > >> > >> Is my question inappropriate for this list? > >> > >> Thanks > >> Alex > >> > >> Piavlo wrote: > >> > >>> Hi, > >>> > >>> I'd like to use rsyslog as central log collection solution. > >>> Manwhile I've compiled and installed as rsyslog 5.6.4 with a stock > >>> configuration as drop in replacement to local ksyslogd > >>> But I see a high quick cpu usage bursts by rsyslog every few > seconds > >>> while it barely does anything. > >>> Most of the cpu time is spent in user mode. > >>> This happends on ec2 large instance with Centos-5.5. > >>> > >>> Below are the configs and sample strace/lsof of the rsyslogd > process. > >>> > >>> Any idea why this happens? > >>> > >>> [root at www1a ~]# cat /etc/rsyslog.conf | grep -v -e '^$' -e '^#' > >>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > >>> $ModLoad imklog > >>> $ModLoad imuxsock > >>> *.info;mail.none;authpriv.none;cron.none > >>> > >> /var/log/messages > >> > >>> authpriv.* > >>> > >> /var/log/secure > >> > >>> mail.* - > >>> > >> /var/log/maillog > >> > >>> cron.* > /var/log/cron > >>> *.emerg * > >>> uucp,news.crit > >>> > >> /var/log/spooler > >> > >>> local7.* > >>> > >> /var/log/boot.log > >> > >>> [root at www1a ~]# > >>> [root at www1a ~]# lsof -p $(pgrep rsyslogd) > >>> COMMAND PID USER FD TYPE DEVICE SIZE > >>> > >> NODE NAME > >> > >>> rsyslogd 29994 root cwd DIR 8,1 4096 > >>> > >> 2 / > >> > >>> rsyslogd 29994 root rtd DIR 8,1 4096 > >>> > >> 2 / > >> > >>> rsyslogd 29994 root txt REG 8,1 1961412 > >>> > >> 2378118 > >> > >>> /usr/local/rsyslog/5.6.4/sbin/rsyslogd > >>> rsyslogd 29994 root mem REG 8,1 137256 > >>> > >> 1720349 > >> > >>> /lib64/ld-2.5.so > >>> rsyslogd 29994 root mem REG 8,1 83344 > >>> > >> 1543492 > >> > >>> /usr/lib64/libz.so.1.2.3 > >>> rsyslogd 29994 root mem REG 8,1 142696 > >>> > >> 1720364 > >> > >>> /lib64/libpthread-2.5.so > >>> rsyslogd 29994 root mem REG 8,1 20424 > >>> > >> 1720336 > >> > >>> /lib64/libdl-2.5.so > >>> rsyslogd 29994 root mem REG 8,1 50288 > >>> > >> 1720370 > >> > >>> /lib64/librt-2.5.so > >>> rsyslogd 29994 root mem REG 8,1 1712536 > >>> > >> 1720326 > >> > >>> /lib64/libc-2.5.so > >>> rsyslogd 29994 root mem REG 8,1 85956 > >>> > >> 2378093 > >> > >>> /usr/local/rsyslog/5.6.4/lib/rsyslog/lmnet.so > >>> rsyslogd 29994 root mem REG 8,1 90745 > >>> > >> 2378128 > >> > >>> /usr/local/rsyslog/5.6.4/lib/rsyslog/imklog.so > >>> rsyslogd 29994 root mem REG 8,1 53471 > >>> > >> 2378110 > >> > >>> /usr/local/rsyslog/5.6.4/lib/rsyslog/imuxsock.so > >>> rsyslogd 29994 root 0r REG 0,3 0 > >>> > >> 4026531849 > >> > >>> /proc/kmsg > >>> rsyslogd 29994 root 1w REG 8,1 5019 > >>> > >> 1048938 > >> > >>> /var/log/messages > >>> rsyslogd 29994 root 2w REG 8,1 18336275 > >>> > >> 1048939 > >> > >>> /var/log/secure > >>> rsyslogd 29994 root 3u unix 0xffff8800241a2a80 > >>> > >> 80601917 > >> > >>> /dev/log > >>> rsyslogd 29994 root 4w REG 8,1 214769 > >>> > >> 1048943 > >> > >>> /var/log/cron > >>> [root at www1a ~]# > >>> [root at www1a ~]# strace -c -r -T -q -f -p $(pgrep rsyslogd) > >>> % time seconds usecs/call calls errors syscall > >>> ------ ----------- ----------- --------- --------- ---------------- > >>> nan 0.000000 0 4 write > >>> nan 0.000000 0 4 select > >>> nan 0.000000 0 4 recvfrom > >>> nan 0.000000 0 4 gettimeofday > >>> nan 0.000000 0 20 4 futex > >>> ------ ----------- ----------- --------- --------- ---------------- > >>> 100.00 0.000000 36 4 total > >>> [root at www1a ~]# > >>> [root at www1a ~]# strace -v -r -T -q -f -p $(pgrep rsyslogd) > >>> [pid 29995] 0.000000 futex(0x6a5d5c, FUTEX_WAIT, 1167, NULL > >>> > >>> [pid 29994] 0.005107 select(1, NULL, NULL, NULL, {85286, > 180000} > >>> > >>> [pid 29997] 0.000049 read(0, > >>> [pid 29998] 0.000021 select(4, [3], NULL, NULL, NULL) = 1 (in > >>> > >> [3]) > >> > >>> <4.329724> > >>> [pid 29998] 4.329868 recvfrom(3, "<86>Mar 24 14:01:07 > >>> > >> sshd[4487]: > >> > >>> "..., 2048, 0, NULL, NULL) = 92 <0.000015> > >>> [pid 29998] 0.000097 gettimeofday({1300975267, 753128}, NULL) > = > >>> > >> 0 > >> > >>> <0.000010> > >>> [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > >>> > >> 0x6a5d58, > >> > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > >>> [pid 29998] 0.000052 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.000021 <... futex resumed> ) = 0 <4.335124> > >>> [pid 29995] 0.000077 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > >>> > >> <0.000057> > >> > >>> [pid 29995] 0.000203 write(2, "Mar 24 14:01:07 www1a > >>> sshd[4487]"..., 95 > >>> [pid 29998] 0.024438 <... select resumed> ) = 1 (in [3]) > >>> > >> <0.024706> > >> > >>> [pid 29995] 0.000024 <... write resumed> ) = 95 <0.024442> > >>> [pid 29998] 0.000247 recvfrom(3, "<86>Mar 24 14:01:07 > >>> > >> sshd[4487]: > >> > >>> "..., 2048, 0, NULL, NULL) = 95 <0.000014> > >>> [pid 29998] 0.000066 gettimeofday({1300975267, 778334}, NULL) > = > >>> > >> 0 > >> > >>> <0.000010> > >>> [pid 29998] 0.000078 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.005155 write(2, "Mar 24 14:01:07 www1a > >>> sshd[4487]"..., 98) = 98 <0.000345> > >>> [pid 29995] 0.000489 futex(0x6a5d5c, FUTEX_WAIT, 1169, NULL > >>> > >>> [pid 29998] 0.107333 <... select resumed> ) = 1 (in [3]) > >>> > >> <0.112941> > >> > >>> [pid 29998] 0.000053 recvfrom(3, "<86>Mar 24 14:01:07 > >>> > >> sshd[4487]: > >> > >>> "..., 2048, 0, NULL, NULL) = 84 <0.000016> > >>> [pid 29998] 0.000068 gettimeofday({1300975267, 891512}, NULL) > = > >>> > >> 0 > >> > >>> <0.000010> > >>> [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > >>> > >> 0x6a5d58, > >> > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > >>> [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.001586 <... futex resumed> ) = 0 <0.109150> > >>> [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > >>> > >> <0.000046> > >> > >>> [pid 29995] 0.000169 write(2, "Mar 24 14:01:07 www1a > >>> sshd[4487]"..., 87) = 87 <0.000065> > >>> [pid 29995] 0.000194 futex(0x6a5d5c, FUTEX_WAIT, 1171, NULL > >>> > >>> [pid 29998] 1.420908 <... select resumed> ) = 1 (in [3]) > >>> > >> <1.422893> > >> > >>> [pid 29998] 0.000049 recvfrom(3, "<86>Mar 24 14:01:09 > >>> > >> sshd[4517]: > >> > >>> "..., 2048, 0, NULL, NULL) = 64 <0.000014> > >>> [pid 29998] 0.000503 gettimeofday({1300975269, 315123}, NULL) > = > >>> > >> 0 > >> > >>> <0.000042> > >>> [pid 29998] 0.000121 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > >>> > >> 0x6a5d58, > >> > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} > >>> [pid 29995] 0.000490 <... futex resumed> ) = 0 <1.422056> > >>> [pid 29998] 0.000010 <... futex resumed> ) = 1 <0.000487> > >>> [pid 29995] 0.000122 futex(0x6a1b30, FUTEX_WAKE, 1 >>> > >> ...> > >> > >>> [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.000067 <... futex resumed> ) = 0 <0.000072> > >>> [pid 29995] 0.000127 write(2, "Mar 24 14:01:09 www1a > >>> sshd[4517]"..., 67) = 67 <0.000105> > >>> [pid 29995] 0.001215 futex(0x6a5d5c, FUTEX_WAIT, 1173, NULL > >>> > >>> [pid 29998] 4.916991 <... select resumed> ) = 1 (in [3]) > >>> > >> <4.918369> > >> > >>> [pid 29998] 0.000050 recvfrom(3, "<86>Mar 24 14:01:14 > >>> > >> sshd[4518]: > >> > >>> "..., 2048, 0, NULL, NULL) = 92 <0.000013> > >>> [pid 29998] 0.000060 gettimeofday({1300975274, 234392}, NULL) > = > >>> > >> 0 > >> > >>> <0.000009> > >>> [pid 29998] 0.000079 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > >>> > >> 0x6a5d58, > >> > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > >>> [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.002861 <... futex resumed> ) = 0 <4.920060> > >>> [pid 29995] 0.000177 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > >>> > >> <0.000059> > >> > >>> [pid 29995] 0.000218 write(2, "Mar 24 14:01:14 www1a > >>> sshd[4518]"..., 95) = 95 <0.000076> > >>> [pid 29995] 0.000378 futex(0x6a5d5c, FUTEX_WAIT, 1175, NULL > >>> > >>> [pid 29998] 0.067782 <... select resumed> ) = 1 (in [3]) > >>> > >> <0.071383> > >> > >>> [pid 29998] 0.000054 recvfrom(3, "<86>Mar 24 14:01:14 > >>> > >> sshd[4518]: > >> > >>> "..., 2048, 0, NULL, NULL) = 95 <0.000014> > >>> [pid 29998] 0.000069 gettimeofday({1300975274, 306060}, NULL) > = > >>> > >> 0 > >> > >>> <0.000011> > >>> [pid 29998] 0.000080 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > >>> > >> 0x6a5d58, > >> > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 <0.000014> > >>> [pid 29998] 0.000050 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.001890 <... futex resumed> ) = 0 <0.069908> > >>> [pid 29995] 0.000070 futex(0x6a1b30, FUTEX_WAKE, 1) = 0 > >>> > >> <0.000044> > >> > >>> [pid 29995] 0.000170 write(2, "Mar 24 14:01:14 www1a > >>> sshd[4518]"..., 98) = 98 <0.000069> > >>> [pid 29995] 0.000285 futex(0x6a5d5c, FUTEX_WAIT, 1177, NULL > >>> > >>> [pid 29998] 0.098664 <... select resumed> ) = 1 (in [3]) > >>> > >> <0.101048> > >> > >>> [pid 29998] 0.001766 recvfrom(3, "<86>Mar 24 14:01:14 > >>> > >> sshd[4518]: > >> > >>> "..., 2048, 0, NULL, NULL) = 84 <0.000047> > >>> [pid 29998] 0.000125 gettimeofday({1300975274, 409161}, NULL) > = > >>> > >> 0 > >> > >>> <0.000039> > >>> [pid 29998] 0.000119 futex(0x6a5d5c, FUTEX_WAKE_OP, 1, 1, > >>> > >> 0x6a5d58, > >> > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} > >>> [pid 29995] 0.000062 <... futex resumed> ) = 0 <0.100721> > >>> [pid 29998] 0.000009 <... futex resumed> ) = 1 <0.000058> > >>> [pid 29995] 0.000071 futex(0x6a1b30, FUTEX_WAKE, 1 >>> > >> ...> > >> > >>> [pid 29998] 0.000019 select(4, [3], NULL, NULL, NULL > >>> > >> ...> > >> > >>> [pid 29995] 0.000065 <... futex resumed> ) = 0 <0.000070> > >>> [pid 29995] 0.000118 write(2, "Mar 24 14:01:14 www1a > >>> sshd[4518]"..., 87) = 87 <0.000069> > >>> [pid 29995] 0.000200 futex(0x6a5d5c, FUTEX_WAIT, 1179, NULL > >>> > >>> [root at www1a ~]# > >>> > >>> > >>> PS. One difference I noticed between stock rsyslog and syslog-ng > >>> > >> configs > >> > >>> is that rsyslog holds just one fd open to /dev/log > >>> and select's it, while syslog-ng holds about 15 different fd's to > >>> /dev/log and epoll's them. > >>> Any particular reason for such a drastic difference? > >>> > >>> Thanks > >>> Alex > >>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From sandeep.sukhija at mithi.com Thu Apr 14 13:46:49 2011 From: sandeep.sukhija at mithi.com (Sandeep Sukhija) Date: Thu, 14 Apr 2011 17:16:49 +0530 (IST) Subject: [rsyslog] Need help in extracting and transforming the message to some other format Message-ID: <25026108.1844.1302781609473.JavaMail.root@127.0.0.1> Hello, My rsyslog version is 5.6.5. I am working on parsing the string and transforming the string to the database query. The setup and integration of postgresql database module is working fine and has been tested. I wish to transform a string of sort say "stagename=antivirus,tid=12345,status=success" and i have to create a query of type "insert into rsyslog(stage,tid,status) values('antivirus','12345','success')". I achieved this by adding a template $template tpl_rsyslog,"insert into rsyslog(stage,tid,status) values('%msg:R,ERE,3,FIELD:(([[:space:]]*[a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+)[[:space:]]*)--end%','%msg:R,ERE,6,FIELD:(([[:space:]]*[a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+)[[:space:]]*)--end%','%msg:R,ERE,9,FIELD:(([[:space:]]*[a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+)[[:space:]]*)--end%')",SQL which works fine in this case where, input : stagename=antivirus,tid=12345,status=success output : insert into syslog(stage,tid,status) values('antivirus','12345','success') but, the regular expression provides the support for getting only the fields from range 0 to 9 in %msg:R,ERE,3,FIELD:%. If the field is at the 12th position, i am unable to get that string. Is there a workaround to get this thing done or i have missed something? Also i would like to know, whether we can pipe the output of 1 property replacer to another? Example, if that was possible, then i would have first used the field separator to get the first token stagename=antivirus and then using the regular expression or field separator again would have got the value of the stagename. Can u pls provide the suggestion regarding the method i am trying to apply or a workaround to get this. Thankyou, Sandeep Sukhija, Mithi Software Technologies Pvt. Ltd., Pune. From rgerhards at hq.adiscon.com Thu Apr 14 14:16:17 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 14 Apr 2011 14:16:17 +0200 Subject: [rsyslog] Need help in extracting and transforming the message to some other format In-Reply-To: <25026108.1844.1302781609473.JavaMail.root@127.0.0.1> References: <25026108.1844.1302781609473.JavaMail.root@127.0.0.1> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDFE0@GRFEXC.intern.adiscon.com> This sounds very much like something that is easier to achieve with the new rsyslog v6 message normalization functionality. You may want to have a look at the mmnormalize document as well as at www.liblognorm.com . We are also working on a tutorial on how to use it right now, will probably become available next week. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Sandeep Sukhija > Sent: Thursday, April 14, 2011 1:47 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Need help in extracting and transforming the message > to some other format > > Hello, > > My rsyslog version is 5.6.5. I am working on parsing the string and > transforming the string to the database query. The setup and > integration of postgresql database module is working fine and has been > tested. > > > I wish to transform a string of sort say > "stagename=antivirus,tid=12345,status=success" and i have to create a > query of type "insert into rsyslog(stage,tid,status) > values('antivirus','12345','success')". > > > I achieved this by adding a template > > > $template tpl_rsyslog,"insert into rsyslog(stage,tid,status) > values('%msg:R,ERE,3,FIELD:(([[:space:]]*[a-z0-9]*)=([a-z0- > 9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0- > 9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+)[[:space:]]*)-- > end%','%msg:R,ERE,6,FIELD:(([[:space:]]*[a-z0-9]*)=([a-z0- > 9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0- > 9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+)[[:space:]]*)-- > end%','%msg:R,ERE,9,FIELD:(([[:space:]]*[a-z0-9]*)=([a-z0- > 9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0- > 9]*)=([a-z0-9]+),[[:space:]]*)(([a-z0-9]*)=([a-z0-9]+)[[:space:]]*)-- > end%')",SQL > > > which works fine in this case where, > > input : stagename=antivirus,tid=12345,status=success > > output : insert into syslog(stage,tid,status) > values('antivirus','12345','success') > > > but, the regular expression provides the support for getting only the > fields from range 0 to 9 in %msg:R,ERE,3,FIELD:%. If the > field is at the 12th position, i am unable to get that string. > > > Is there a workaround to get this thing done or i have missed > something? > > > Also i would like to know, whether we can pipe the output of 1 property > replacer to another? > > > Example, if that was possible, then i would have first used the field > separator to get the first token stagename=antivirus and then using the > regular expression or field separator again would have got the value of > the stagename. > > > Can u pls provide the suggestion regarding the method i am trying to > apply or a workaround to get this. > > > Thankyou, > > Sandeep Sukhija, > > Mithi Software Technologies Pvt. Ltd., Pune. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From pgollucci at p6m7g8.com Fri Apr 15 00:20:06 2011 From: pgollucci at p6m7g8.com (Philip M. Gollucci) Date: Thu, 14 Apr 2011 22:20:06 +0000 Subject: [rsyslog] Fwd: cvs commit: ports/sysutils/rsyslog5 Makefile distinfo pkg-plist Message-ID: <4DA77316.3040401@p6m7g8.com> HTH -------------- next part -------------- An embedded message was scrubbed... From: "Philip M. Gollucci" Subject: cvs commit: ports/sysutils/rsyslog5 Makefile distinfo pkg-plist Date: Thu, 14 Apr 2011 22:02:25 +0000 (UTC) Size: 4428 URL: -------------- next part -------------- An embedded message was scrubbed... From: "Philip M. Gollucci" Subject: cvs commit: ports MOVED ports/sysutils Makefile ports/sysutils/rsyslog5-devel Makefile distinfo pkg-descr pkg-plist ports/sysutils/rsyslog5-devel/files pkg-message.in rsyslogd.in ports/sysutils/rsyslog5-devel-dbi Makefile ... Date: Thu, 14 Apr 2011 22:10:02 +0000 (UTC) Size: 9583 URL: From pgollucci at p6m7g8.com Fri Apr 15 00:20:28 2011 From: pgollucci at p6m7g8.com (Philip M. Gollucci) Date: Thu, 14 Apr 2011 22:20:28 +0000 Subject: [rsyslog] rsyslog5 core dumps a-plenty In-Reply-To: <4D4A518B.8020202@p6m7g8.com> References: <4D4A518B.8020202@p6m7g8.com> Message-ID: <4DA7732C.6070309@p6m7g8.com> Will be testing 5.8.0 friday or next week, and will report back. On 02/03/11 06:56, Philip M. Gollucci wrote: > Rainer (All), > > At first I thought it was something in rsyslog 5.5.6+ that changed b/c I > couldn't get the core dumps happen with 5.5.5; however, now I'm pretty > sure it has nothing to do with that milestone. Unfortunately, from the > looks of it, I'm betting its a FreeBSD specific threading issue. Below > is the useful info, gdb back trace, rsyslog.conf, and the build transcript. > > I've also uploaded the files here for easier viewing: > http://people.freebsd.org/~pgollucci/rsyslog-core > http://people.apache.org/~pgollucci/rsyslog-core > > Move info is available on request. > > > > $ uname -a > FreeBSD hulk 8.1-RELEASE FreeBSD 8.1-RELEASE #0 r212058: Wed Sep 1 > 23:12:18 UTC 2010 root at elektra > :/usr/obj/usr/src/sys/GENERIC amd64 > > $ rsyslog --version > rsyslogd 5.6.2, compiled with: > FEATURE_REGEXP: Yes > FEATURE_LARGEFILE: No > FEATURE_NETZIP (message compression): Yes > GSSAPI Kerberos 5 support: No > FEATURE_DEBUG (debug build, slow code): Yes > Atomic operations supported: Yes > Runtime Instrumentation (slow code): Yes > > $ sudo /usr/local/etc/rc.d/rsyslogd restart > rsyslogd not running? (check /var/run/syslog.pid). > Starting rsyslogd. > > $ ps -auxwww |grep rsyslogd > root 9135 6.1 0.0 25524 4388 1 S 6:37AM 0:01.29 > /usr/local/sbin/rsyslogd -c4 -i /var > /run/syslog.pid -f /usr/local/etc/rsyslog.conf > > $ cd / ; ls core.rsyslogd.8* > -rw------- 1 root wheel 10354688 Feb 3 06:25 core.rsyslogd.8742 > -rw------- 1 root wheel 8257536 Feb 3 06:26 core.rsyslogd.8782 > -rw------- 1 root wheel 8257536 Feb 3 06:27 core.rsyslogd.8826 > -rw------- 1 root wheel 8257536 Feb 3 06:28 core.rsyslogd.8844 > -rw------- 1 root wheel 10354688 Feb 3 06:29 core.rsyslogd.8875 > -rw------- 1 root wheel 8257536 Feb 3 06:29 core.rsyslogd.8900 > > $ gdb /usr/local/sbin/rsyslogd core.rsyslogd.81742 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > Core was generated by `rsyslogd'. > Program terminated with signal 11, Segmentation fault. > Reading symbols from /lib/libz.so.5...done. > Loaded symbols for /lib/libz.so.5 > Reading symbols from /lib/libthr.so.3...done. > Loaded symbols for /lib/libthr.so.3 > Reading symbols from /lib/libc.so.7...done. > Loaded symbols for /lib/libc.so.7 > Reading symbols from /usr/local/lib/rsyslog/lmnet.so...done. > Loaded symbols for /usr/local/lib/rsyslog/lmnet.so > Reading symbols from /usr/local/lib/rsyslog/immark.so...done. > Loaded symbols for /usr/local/lib/rsyslog/immark.so > Reading symbols from /usr/local/lib/rsyslog/imudp.so...done. > Loaded symbols for /usr/local/lib/rsyslog/imudp.so > Reading symbols from /usr/local/lib/rsyslog/imtcp.so...done. > Loaded symbols for /usr/local/lib/rsyslog/imtcp.so > Reading symbols from /usr/local/lib/rsyslog/lmnetstrms.so...done. > Loaded symbols for /usr/local/lib/rsyslog/lmnetstrms.so > Reading symbols from /usr/local/lib/rsyslog/lmtcpsrv.so...done. > Loaded symbols for /usr/local/lib/rsyslog/lmtcpsrv.so > Reading symbols from /usr/local/lib/rsyslog/imuxsock.so...done. > Loaded symbols for /usr/local/lib/rsyslog/imuxsock.so > Reading symbols from /usr/local/lib/rsyslog/imklog.so...done. > Loaded symbols for /usr/local/lib/rsyslog/imklog.so > Reading symbols from /usr/local/lib/rsyslog/imrelp.so...done. > Loaded symbols for /usr/local/lib/rsyslog/imrelp.so > Reading symbols from /usr/local/lib/librelp.so.0...done. > Loaded symbols for /usr/local/lib/librelp.so.0 > Reading symbols from /libexec/ld-elf.so.1...done. > Loaded symbols for /libexec/ld-elf.so.1 > #0 0x00000008007e3876 in pthread_mutex_destroy () from /lib/libthr.so.3 > [New Thread 801a3ac80 (LWP 100568)] > [New Thread 800c87fc0 (LWP 100567)] > [New Thread 800c88180 (LWP 100541)] > [New Thread 800c88340 (LWP 100540)] > [New Thread 800c88500 (LWP 100539)] > [New Thread 800c886c0 (LWP 100538)] > [New Thread 800c88880 (LWP 100379)] > [New Thread 800c041c0 (LWP 100548)] > (gdb) bt full > #0 0x00000008007e3876 in pthread_mutex_destroy () from /lib/libthr.so.3 > No symbol table info available. > #1 0x00000000004210cf in MsgDeleteMutexLockingCase (pThis=0x801aaea00) > at msg.c:624 > __func__ = "MsgDeleteMutexLockingCase" > #2 0x00000000004218d1 in msgDestruct (ppThis=0x7ffffeff8d70) at msg.c:839 > dbgCALLStaCK_POP_POINT = 7 > iRet = RS_RET_OK > iCancelStateSave = 1 > pThis = (msg_t *) 0x801aaea00 > currRefCount = 0 > pdbgFuncDB = (dbgFuncDB_t *) 0x801a12500 > __func__ = "msgDestruct" > #3 0x000000000044fbe1 in DeleteProcessedBatch (pThis=0x800c04380, > pBatch=0x800c2fe70) at queue.c:1404 > i = 0 > pUsr = (void *) 0x801aaea00 > nEnqueued = 0 > localRet = 8 > dbgCALLStaCK_POP_POINT = 6 > iRet = RS_RET_OK > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13500 > __func__ = "DeleteProcessedBatch" > #4 0x000000000044f828 in DequeueConsumableElements (pThis=0x800c04380, > pWti=0x800c2fe40, piRemainingQueueSize=0x7ffffeff8e14) at queue.c:1441 > nDequeued = 8 > nDiscarded = 4678208 > nDeleted = 2 > iQueueSize = 4673262 > pUsr = (void *) 0x40042df73 > localRet = RS_RET_OK > dbgCALLStaCK_POP_POINT = 5 > iRet = RS_RET_OK > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13600 > __func__ = "DequeueConsumableElements" > #5 0x000000000044f711 in DequeueConsumable (pThis=0x800c04380, > pWti=0x800c2fe40) at queue.c:1489 > dbgCALLStaCK_POP_POINT = 4 > iRet = RS_RET_OK > iQueueSize = 0 > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13700 > __func__ = "DequeueConsumable" > #6 0x0000000000450fca in DequeueForConsumer (pThis=0x800c04380, > pWti=0x800c2fe40) at queue.c:1626 > dbgCALLStaCK_POP_POINT = 3 > iRet = RS_RET_OK > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13800 > __func__ = "DequeueForConsumer" > #7 0x0000000000450c6b in ConsumerReg (pThis=0x800c04380, > pWti=0x800c2fe40) at queue.c:1679 > iCancelStateSave = 8 > dbgCALLStaCK_POP_POINT = 2 > iRet = RS_RET_OK > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13900 > __func__ = "ConsumerReg" > #8 0x0000000000443b21 in wtiWorker (pThis=0x800c2fe40) at wti.c:315 > __cleanup_info__ = {pthread_cleanup_pad = {140737471549248, > 4470256, 34372517440, 0, 4667374, 5824688, 0, 34368002717}} > pWtp = (wtp_t *) 0x800c1c380 > bInactivityTOOccured = 0 > localRet = RS_RET_OK > terminateRet = RS_RET_OK > iCancelStateSave = 0 > dbgCALLStaCK_POP_POINT = 1 > iRet = RS_RET_OK > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13d00 > __func__ = "wtiWorker" > #9 0x0000000000442019 in wtpWorker (arg=0x800c2fe40) at wtp.c:381 > __cleanup_info__ = {pthread_cleanup_pad = {0, 4462880, > 34372517440, 0, 0, 0, 0, 0}} > pWti = (wti_t *) 0x800c2fe40 > pThis = (wtp_t *) 0x800c1c380 > sigSet = {__bits = {1048576, 0, 0, 0}} > dbgCALLStaCK_POP_POINT = 0 > pdbgFuncDB = (dbgFuncDB_t *) 0x801a13e00 > __func__ = "wtpWorker" > #10 0x00000008007de511 in pthread_getprio () from /lib/libthr.so.3 > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > Cannot access memory at address 0x7ffffeff9000 > (gdb) frame 1 > #1 0x00000000004210cf in MsgDeleteMutexLockingCase (pThis=0x801aaea00) > at msg.c:624 > 624 msg.c: No such file or directory. > in msg.c > (gdb) info locals > __func__ = "MsgDeleteMutexLockingCase" > (gdb) p *pThis > $1 = {objData = {pObjInfo = 0x800c16400, iObjCooCKiE = 195948526, > pszName = 0x0}, flowCtlType = eFLOWCTL_NO_DELAY, mut = > 0xffffffffffffffff, iRefCount = 0, bDoLock = 0 '\0', iSeverity = 3, > iFacility = 22, of > fAfterPRI = 0, offMSG = 91, iProtocolVersion = 0, msgFlags = 32, > iLenRawMsg = 121, iLenMSG = 30, iLenTAG = 31, iLenHOSTNAME = 21, > pszRawMsg = 0x801ab5600 "<179>2011-02-03T06:20:37.681910+00:00 phoenix > app001[99856]: last message repeated 5 times", pszHOSTNAME = 0x801aae > b75 "phoenix", > pszRcvdAt3164 = 0x0, pszRcvdAt3339 = 0x0, pszRcvdAt_MySQL = 0x0, > pszRcvdAt_PgSQL = 0x0, pszTIMESTAMP3164 = 0x0, pszTIMESTAMP3339 = > 0x801aaebc8 "2011-02-03T06:25:05.776392+00:00", pszTIMESTAMP_MySQL = 0x0, > pszTIMESTAMP_PgSQL = 0x0, pCSProgName = 0x0, pCSStrucData = 0x0, > pCSAPPNAME = 0x0, pCSPROCID = 0x0, pCSMSGID = 0x0, pInputName = 0x0, > pRcvFromIP = 0x0, rcvFrom = {pRcvFrom = 0x0, pfrominet = 0x0}, pRuleset > = 0x0, ttGenTime = 1296714305, tRcvdAt = {timeType = 2 '\002', m > onth = 2 '\002', day = 3 '\003', hour = 6 '\006', > minute = 25 '\031', second = 5 '\005', secfracPrecision = 6 '\006', > OffsetMinute = 0 '\0', OffsetHour = 0 '\0', OffsetMode = 43 '+', year = > 2011, secfrac = 776392}, tTIMESTAMP = {timeType = 2 '\002', mon > th = 2 '\002', day = 3 '\003', hour = 6 '\006', minute = 25 '\031', > second = 5 '\005', secfracPrecision = 6 '\006', OffsetMinute = 0 > '\0', OffsetHour = 0 '\0', OffsetMode = 43 '+', year = 2011, secfrac = > 776392}, szRawMsg = '\xff' , szHOSTNAME = "phoen > ix\000\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff", TAG = { > pszTAG = 0x722e313030707061
bounds>, szBuf = "app001[99856]:\xff"}, pszTimestamp3164 = "\000", > '\xff' , pszTimestamp3339 = "2011-02-03T06:25:05.776392 > +00:00", pszTIMESTAMP_SecFrac = "\000\xff\xff\xff\xff\xff\xff", > pszRcvdAt_SecFrac = "\000\xff\xff\xff\xff\xff\xff"} > (gdb) quit > > rsyslog.conf: > ------------- > #### Modules > ## Input > $ModLoad immark > $ModLoad imudp > $ModLoad imtcp > $ModLoad imuxsock > $ModLoad imklog > $ModLoad imrelp > > ## Output > #$ModLoad omsnmp.so # Send SNMP traps > > #### Globals > $umask 0000 > $DirCreateMode 0640 > $FileCreateMode 0640 > > $RepeatedMsgReduction on > #$RepeatedMsgContainsOrigionalMsg on > > $DefaultNetstreamDriver ptcp > > # location for work (spool) file > $WorkDirectory /var/spool/rsyslog > > # Queue Types > $MainMsgQueueType LinkedList > $ActionQueueType LinkedList > > # Become DA Queues > $MainMsgQueueFileName mainq > $ActionQueueFileNAme actionq > > # Disable Rate Throttling > $MainMsgQueueTimeoutEnqueue 0 > $ActionQueueTimeoutEnqueue 0 > > # Discard any message at all if there double the queue we allow in memory > # 10000 in ram + 10000 on disk > $MainMsgQueueDiscardMark 20000 > $ActionQueueDiscardMark 20000 > > 0 Emergency: system is unusable > # 1 Alert: action must be taken immediately > # 2 Critical: critical conditions > # 3 Error: error conditions > # 4 Warning: warning conditions > # 5 Notice: normal but significant condition > # 6 Informational: informational messages > # 7 Debug: debug-level messages > > # When discarding kicks in (above) any msg > # with priority >= this is out the door > $MainMsgQueueDiscardSeverity 0 > $ActionQueueDiscardSeverity 0 > > # default templates > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > $UDPServerRun 514 > $InputRELPServerRun 2514 > > # Templates > $template logsaggd,"%TIMESTAMP:::date-rfc3339% %syslogfacility-text% > %syslogpriority-text% %HOSTNAME% %syslogtag% %msg:::drop-last-lf%" > > #### Logging Facilities > *.* @localhost:18712;logsaggd > > +hulk > *.err;kern.warning;auth.notice;mail.crit > /var/log/console.log > *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err > /var/log/messages > security.* /var/log/security > auth.info;authpriv.info /var/log/auth.log > mail.info /var/log/maillog > lpr.info /var/log/lpd-errs > ftp.info /var/log/xferlog > cron.* /var/log/cron > *.=debug /var/log/debug.log > *.emerg * > -- ------------------------------------------------------------------------ 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354 Member, Apache Software Foundation Committer, FreeBSD Foundation Consultant, P6M7G8 Inc. Sr. System Admin, Ridecharge Inc. Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. From tbergfeld at hq.adiscon.com Fri Apr 15 15:26:23 2011 From: tbergfeld at hq.adiscon.com (Tom Bergfeld) Date: Fri, 15 Apr 2011 15:26:23 +0200 Subject: [rsyslog] rsyslog 6.1.7 (v6-devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE001@GRFEXC.intern.adiscon.com> Version 6.1.7 introduces syslog message classification via easy to create normalization rulebases. This functionality is build on top of liblognorm and libee and so rulebases can be shared with other tools that support liblognorm (like sagan). Also, we considerably speeded up TLS syslog message forwarding. This release also contains a number of important bug fixes. Documentation on using log normalization and classification in rsyslog is currently sparse and being extended. We expect to have guides for important concepts available on www.rsyslog.com during the next week. ChangeLog: http://www.rsyslog.com/changelog-for-6-1-7-v6-devel/ Download: http://www.rsyslog.com/rsyslog-6-1-7-v6-devel/ As always, feedback is appreciated. Best regards, Tom Bergfeld -- Support ======= Improving rsyslog is costly, but you can help! We are looking for organizations that find rsyslog useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for rsyslog are available, and they help finance continued maintenance. Adiscon GmbH, a privately held German company, is currently funding rsyslog development. We are always looking for interesting development projects. For details on how to help, please see http://www.rsyslog.com/doc-how2help.html. From trefalgar at realitybytes.net Sat Apr 16 04:53:29 2011 From: trefalgar at realitybytes.net (Jacob Steinberger) Date: Fri, 15 Apr 2011 21:53:29 -0500 Subject: [rsyslog] Configuration - disk cache on remote action failure In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> References: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com> <9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> Message-ID: <4DA904A9.5000209@realitybytes.net> On 4/4/2011 2:12 PM, Rainer Gerhards wrote: >> Is there a way for an Action Queue to buffer to disk, if none of the >> remote destinations are available? I don't mean a standard& >> /var/log/moo, I mean something that rsyslog will reprocess if any of >> the remote connections are reestablished. > > http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html > >> >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >> *.* @@loadbalancedhostname >> $ActionExecOnlyWhenPreviousIsSuspended on >> & @@hostname1 >> & @@hostname2 >> $ActionExecOnlyWhenPreviousIsSuspended off With the linked config, we were able to get disk caching, which was excellent, but we couldn't find a config to work with it, that would also failover/failback. Suggestions? Will the above ActionExecOnlyWhenPreviousIsSuspended work with disk assist? Jacob From rgerhards at hq.adiscon.com Sat Apr 16 12:02:01 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 16 Apr 2011 12:02:01 +0200 Subject: [rsyslog] Configuration - disk cache on remote action failure In-Reply-To: <4DA904A9.5000209@realitybytes.net> References: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com><9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> <4DA904A9.5000209@realitybytes.net> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE005@GRFEXC.intern.adiscon.com> I have never thought about such a case. So in which case you want the failover to happen? When the disk queue is full? In any case, I don't see a way how this could be achieved with current code. Maybe it can be added, but I need to understand the use case behind it. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jacob Steinberger > Sent: Saturday, April 16, 2011 4:53 AM > To: rsyslog-users > Subject: Re: [rsyslog] Configuration - disk cache on remote action > failure > > > > On 4/4/2011 2:12 PM, Rainer Gerhards wrote: > >> Is there a way for an Action Queue to buffer to disk, if none of the > >> remote destinations are available? I don't mean a standard& > >> /var/log/moo, I mean something that rsyslog will reprocess if any of > >> the remote connections are reestablished. > > > > http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html > > > >> > >> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > >> *.* @@loadbalancedhostname > >> $ActionExecOnlyWhenPreviousIsSuspended on > >> & @@hostname1 > >> & @@hostname2 > >> $ActionExecOnlyWhenPreviousIsSuspended off > > With the linked config, we were able to get disk caching, which was > excellent, but we couldn't find a config to work with it, that would > also failover/failback. > > Suggestions? Will the above ActionExecOnlyWhenPreviousIsSuspended work > with disk assist? > > Jacob > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From trefalgar at realitybytes.net Sun Apr 17 16:45:19 2011 From: trefalgar at realitybytes.net (Jacob Steinberger) Date: Sun, 17 Apr 2011 09:45:19 -0500 Subject: [rsyslog] Configuration - disk cache on remote action failure In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE005@GRFEXC.intern.adiscon.com> References: <20110404132007.74515fk2sw4oef2v@vincent.asmallorange.com><9B6E2A8877C38245BFB15CC491A11DA71DDF34@GRFEXC.intern.adiscon.com> <4DA904A9.5000209@realitybytes.net> <9B6E2A8877C38245BFB15CC491A11DA71DE005@GRFEXC.intern.adiscon.com> Message-ID: <4DAAFCFF.6040205@realitybytes.net> Maybe I'm going about the config the wrong way, who knows ;) The problem: Clients->TCP->ServerA If ServerA fails, we have it configured to fail over to ServerB. The case is if ServerA and ServerB are unavailable (network outage, nic card failure, spantree insanity, etc), we want to still see those messages on ServerA/ServerB when the connection is restored. At the same time, we want the ability to have a large portion of the disk available so we don't overrun memory or cause other problems we can create with a poorly configured rsyslog client. We're not looking for just reliable transport (TCP/RELP) but in the case of a failure, to have a good sized buffer to store those messages and process them when the failure is resolved. Does that make sense? Jacob On 4/16/2011 5:02 AM, Rainer Gerhards wrote: > I have never thought about such a case. So in which case you want the > failover to happen? When the disk queue is full? > > In any case, I don't see a way how this could be achieved with current code. > Maybe it can be added, but I need to understand the use case behind it. > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Jacob Steinberger >> Sent: Saturday, April 16, 2011 4:53 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Configuration - disk cache on remote action >> failure >> >> >> >> On 4/4/2011 2:12 PM, Rainer Gerhards wrote: >>>> Is there a way for an Action Queue to buffer to disk, if none of the >>>> remote destinations are available? I don't mean a standard& >>>> /var/log/moo, I mean something that rsyslog will reprocess if any of >>>> the remote connections are reestablished. >>> >>> http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html >>> >>>> >>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional >>>> *.* @@loadbalancedhostname >>>> $ActionExecOnlyWhenPreviousIsSuspended on >>>> & @@hostname1 >>>> & @@hostname2 >>>> $ActionExecOnlyWhenPreviousIsSuspended off >> >> With the linked config, we were able to get disk caching, which was >> excellent, but we couldn't find a config to work with it, that would >> also failover/failback. >> >> Suggestions? Will the above ActionExecOnlyWhenPreviousIsSuspended work >> with disk assist? >> >> Jacob >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From piavka at cs.bgu.ac.il Mon Apr 18 01:13:10 2011 From: piavka at cs.bgu.ac.il (Piavlo) Date: Mon, 18 Apr 2011 02:13:10 +0300 Subject: [rsyslog] failover issue Message-ID: <4DAB7406.70105@cs.bgu.ac.il> According to http://wiki.rsyslog.com/index.php/FailoverSyslogServer config *.* @@primary-syslog.example.com $ActionExecOnlyWhenPreviousIsSuspended on & @@secondary-1-syslog.example.com & @@secondary-2-syslog.example.com & /var/log/localbuffer $ActionExecOnlyWhenPreviousIsSuspended off if connection to @@primary-syslog.example.com fails rsyslog will try to send to @@secondary-1-syslog.example.com if that fails too then to @@secondary-2-syslog.example.com and if that fails too then to /var/log/localbuffer BUT with my testing with 5.8.0 what really happens is: if connection to @@primary-syslog.example.com it will send the same messages to ALL the remaining & @@secondary-1-syslog.example.com & @@secondary-2-syslog.example.com & /var/log/localbuffer destinations Any idea why ? And how to make it behave like described in http://wiki.rsyslog.com/index.php/FailoverSyslogServer? i.e send it to only ONE failover server Thanks Alex From rgerhards at hq.adiscon.com Mon Apr 18 07:15:25 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 18 Apr 2011 07:15:25 +0200 Subject: [rsyslog] failover issue In-Reply-To: <4DAB7406.70105@cs.bgu.ac.il> References: <4DAB7406.70105@cs.bgu.ac.il> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE009@GRFEXC.intern.adiscon.com> This sounds like a bug. I'll try to address it asap. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Piavlo > Sent: Monday, April 18, 2011 1:13 AM > To: rsyslog-users > Subject: [rsyslog] failover issue > > According to http://wiki.rsyslog.com/index.php/FailoverSyslogServer > config > > *.* @@primary-syslog.example.com > $ActionExecOnlyWhenPreviousIsSuspended on > & @@secondary-1-syslog.example.com > & @@secondary-2-syslog.example.com > & /var/log/localbuffer > $ActionExecOnlyWhenPreviousIsSuspended off > > if connection to @@primary-syslog.example.com fails > rsyslog will try to send to @@secondary-1-syslog.example.com > if that fails too then to @@secondary-2-syslog.example.com > and if that fails too then to /var/log/localbuffer > > BUT with my testing with 5.8.0 what really happens is: > if connection to @@primary-syslog.example.com > it will send the same messages to ALL the remaining > & @@secondary-1-syslog.example.com > & @@secondary-2-syslog.example.com > & /var/log/localbuffer > destinations > > Any idea why ? And how to make it behave like described in > http://wiki.rsyslog.com/index.php/FailoverSyslogServer? i.e send it to > only ONE failover server > > Thanks > Alex > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Apr 18 07:59:12 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 18 Apr 2011 07:59:12 +0200 Subject: [rsyslog] failover issue In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE009@GRFEXC.intern.adiscon.com> References: <4DAB7406.70105@cs.bgu.ac.il> <9B6E2A8877C38245BFB15CC491A11DA71DE009@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE00F@GRFEXC.intern.adiscon.com> Oh, one thing: could you please open a bug tracker at http://bugzilla.adiscon.com This ensures I'll have the problem on my mind as soon as I have time to look at it. Thx, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, April 18, 2011 7:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] failover issue > > This sounds like a bug. I'll try to address it asap. > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Piavlo > > Sent: Monday, April 18, 2011 1:13 AM > > To: rsyslog-users > > Subject: [rsyslog] failover issue > > > > According to http://wiki.rsyslog.com/index.php/FailoverSyslogServer > > config > > > > *.* @@primary-syslog.example.com > > $ActionExecOnlyWhenPreviousIsSuspended on > > & @@secondary-1-syslog.example.com > > & @@secondary-2-syslog.example.com > > & /var/log/localbuffer > > $ActionExecOnlyWhenPreviousIsSuspended off > > > > if connection to @@primary-syslog.example.com fails > > rsyslog will try to send to @@secondary-1-syslog.example.com > > if that fails too then to @@secondary-2-syslog.example.com > > and if that fails too then to /var/log/localbuffer > > > > BUT with my testing with 5.8.0 what really happens is: > > if connection to @@primary-syslog.example.com > > it will send the same messages to ALL the remaining > > & @@secondary-1-syslog.example.com > > & @@secondary-2-syslog.example.com > > & /var/log/localbuffer > > destinations > > > > Any idea why ? And how to make it behave like described in > > http://wiki.rsyslog.com/index.php/FailoverSyslogServer? i.e send it > to > > only ONE failover server > > > > Thanks > > Alex > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From stefan.kleindl at rlb-stmk.raiffeisen.at Mon Apr 18 14:22:23 2011 From: stefan.kleindl at rlb-stmk.raiffeisen.at (stefan.kleindl at rlb-stmk.raiffeisen.at) Date: Mon, 18 Apr 2011 14:22:23 +0200 Subject: [rsyslog] Duplicate messages Message-ID: Hey, I?ve been getting a rather strange behaviour over the past few days and I?m not sure where it originates from. I?m running rsyslog 5.6.4 with MySQL and Eventlog to Syslog on my Windows test server. The problem: I?m filtering out messages to get the ones with Windows Event 540 - modifying them depending on what their contents are. At random though, some messages get into the logs and DB twice - with 2 different filters applied. I?m filtering with $template Win540,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, EventLogType, EventSource, EventId, EventUser, ProcessID) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%programname%', 'Successful Network Logon', '%msg:F,32:50%', '540', '%msg:F,32:9%', '%msg:F,32:14%')",sql $template WinNT540,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, EventLogType, EventSource, EventId, EventUser, ProcessID) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%programname%', 'Successful Network Logon', '%msg:F,32:51%', '540', '%msg:F,32:10%', '%msg:F,32:15%')",sql $template WinNTano540,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, EventLogType, EventSource, EventId, EventUser, ProcessID) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%programname%', 'Successful Network Logon', '%msg:F,32:51%', '540', 'ANONYMOUS', '%msg:F,32:14%')",sql :msg, startswith, " 540: NT AUTHORITY\\SYSTEM" >localhost,Syslog,rsyslog,password;WinNT540 & ~ :msg, startswith, " 540: NT AUTHORITY\\ANONYMOUS LOGON" >localhost,Syslog,rsyslog,password;WinNTano540 & ~ :msg, startswith, " 540:" >localhost,Syslog,rsyslog,password;Win540 & ~ And way way later in the config I have a default template: $template Evtsys,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, EventLogType, EventId) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%programname%', 'Generic Event', '%msg:F,58:1%')",sql :syslogtag, contains, "Security" >localhost,Syslog,rsyslog,password;Evtsys & ~ (All my Windows messages come with "Security" or "Security-Auditing". As said, this works fine for 99.x% of the messages, but at random I get things like 540: DOMAIN\USER: Successful Network Logon: User Name: USER Domain: DOMAIN Logon ID: (0x0,0x1111111) Logon Type: 3 Logon Process: (?..) Name: Logon GUID: {xxxxxxxxxxxxx-xxxxxx-xxxxxx-xxx} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: Source Port: 0 into the DB twice - one formatted with Win540 and one with Evtsys. They are 100% the same text and content wise.(and ye, there is a space ahead of the 540) also. Any ideas on why exactly I would see doubles? Regards ---------------------------------------- Raiffeisen-Landesbank Steiermark AG, Graz, FN 264700 s, Landesgericht f?r Zivilrechtssachen Graz, DVR 0040495 Der Austausch von Nachrichten mit oa. Absender via E-Mail dient ausschliesslich Informationszwecken. Rechtsgeschaeftliche Erklaerungen duerfen ueber dieses Medium nicht ausgetauscht werden. Correspondence with a.m. sender via e-mail is only for information purposes. This medium is not to be used for the exchange of legally-binding communications. ---------------------------------------- From friedl at hq.adiscon.com Mon Apr 18 15:22:03 2011 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 18 Apr 2011 15:22:03 +0200 Subject: [rsyslog] liblognorm 0.3.1 and libee 0.3.1 released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE015@GRFEXC.intern.adiscon.com> We have just released liblognorm 0.3.1 and libee 0.3.1. These releases include new features and bugfixes. Changes: Liblognorm version 0.3.1 (rgerhards), 2011-04-18 - added -t option to normalizer so that only messages with a specified tag will be output - bugfix: abort if a tag was assigned to a message without any fields parsed out (uncommon scenario) - bugfix: mem leak on parse tree destruct -- associated tags were not deleted - bugfix: potential abort in normalizer due to misadressing in debug message generation Download: http://www.liblognorm.com/files/download/liblognorm-0.3.1.tar.gz Libee version 0.3.1 (rgerhards), 2011-04-18 - API extensions - brought tag handling a bit inline with upcoming 0.6 draft CEE spec Download: http://www.libee.org/files/download/libee-0.3.1.tar.gz As always, feedback is appreciated. Best regards, Florian Riedl From pgollucci at p6m7g8.com Mon Apr 18 15:32:57 2011 From: pgollucci at p6m7g8.com (Philip M. Gollucci) Date: Mon, 18 Apr 2011 13:32:57 +0000 Subject: [rsyslog] rsyslog5 core dumps a-plenty In-Reply-To: <4DA7732C.6070309@p6m7g8.com> References: <4D4A518B.8020202@p6m7g8.com> <4DA7732C.6070309@p6m7g8.com> Message-ID: <4DAC3D89.8000905@p6m7g8.com> On 04/14/11 22:20, Philip M. Gollucci wrote: > Will be testing 5.8.0 friday or next week, and will report back. Whatever this bug was 5.8.0 seems to have fixed it. Its been up since ~Saturday on both the central, and a subset of the clients. Thanks! > > > On 02/03/11 06:56, Philip M. Gollucci wrote: >> Rainer (All), >> >> At first I thought it was something in rsyslog 5.5.6+ that changed b/c I >> couldn't get the core dumps happen with 5.5.5; however, now I'm pretty >> sure it has nothing to do with that milestone. Unfortunately, from the >> looks of it, I'm betting its a FreeBSD specific threading issue. Below >> is the useful info, gdb back trace, rsyslog.conf, and the build transcript. >> >> I've also uploaded the files here for easier viewing: >> http://people.freebsd.org/~pgollucci/rsyslog-core >> http://people.apache.org/~pgollucci/rsyslog-core >> >> Move info is available on request. >> >> >> >> $ uname -a >> FreeBSD hulk 8.1-RELEASE FreeBSD 8.1-RELEASE #0 r212058: Wed Sep 1 >> 23:12:18 UTC 2010 root at elektra >> :/usr/obj/usr/src/sys/GENERIC amd64 >> >> $ rsyslog --version >> rsyslogd 5.6.2, compiled with: >> FEATURE_REGEXP: Yes >> FEATURE_LARGEFILE: No >> FEATURE_NETZIP (message compression): Yes >> GSSAPI Kerberos 5 support: No >> FEATURE_DEBUG (debug build, slow code): Yes >> Atomic operations supported: Yes >> Runtime Instrumentation (slow code): Yes >> >> $ sudo /usr/local/etc/rc.d/rsyslogd restart >> rsyslogd not running? (check /var/run/syslog.pid). >> Starting rsyslogd. >> >> $ ps -auxwww |grep rsyslogd >> root 9135 6.1 0.0 25524 4388 1 S 6:37AM 0:01.29 >> /usr/local/sbin/rsyslogd -c4 -i /var >> /run/syslog.pid -f /usr/local/etc/rsyslog.conf >> >> $ cd / ; ls core.rsyslogd.8* >> -rw------- 1 root wheel 10354688 Feb 3 06:25 core.rsyslogd.8742 >> -rw------- 1 root wheel 8257536 Feb 3 06:26 core.rsyslogd.8782 >> -rw------- 1 root wheel 8257536 Feb 3 06:27 core.rsyslogd.8826 >> -rw------- 1 root wheel 8257536 Feb 3 06:28 core.rsyslogd.8844 >> -rw------- 1 root wheel 10354688 Feb 3 06:29 core.rsyslogd.8875 >> -rw------- 1 root wheel 8257536 Feb 3 06:29 core.rsyslogd.8900 >> >> $ gdb /usr/local/sbin/rsyslogd core.rsyslogd.81742 >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for details. >> This GDB was configured as "amd64-marcel-freebsd"... >> Core was generated by `rsyslogd'. >> Program terminated with signal 11, Segmentation fault. >> Reading symbols from /lib/libz.so.5...done. >> Loaded symbols for /lib/libz.so.5 >> Reading symbols from /lib/libthr.so.3...done. >> Loaded symbols for /lib/libthr.so.3 >> Reading symbols from /lib/libc.so.7...done. >> Loaded symbols for /lib/libc.so.7 >> Reading symbols from /usr/local/lib/rsyslog/lmnet.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/lmnet.so >> Reading symbols from /usr/local/lib/rsyslog/immark.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/immark.so >> Reading symbols from /usr/local/lib/rsyslog/imudp.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/imudp.so >> Reading symbols from /usr/local/lib/rsyslog/imtcp.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/imtcp.so >> Reading symbols from /usr/local/lib/rsyslog/lmnetstrms.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/lmnetstrms.so >> Reading symbols from /usr/local/lib/rsyslog/lmtcpsrv.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/lmtcpsrv.so >> Reading symbols from /usr/local/lib/rsyslog/imuxsock.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/imuxsock.so >> Reading symbols from /usr/local/lib/rsyslog/imklog.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/imklog.so >> Reading symbols from /usr/local/lib/rsyslog/imrelp.so...done. >> Loaded symbols for /usr/local/lib/rsyslog/imrelp.so >> Reading symbols from /usr/local/lib/librelp.so.0...done. >> Loaded symbols for /usr/local/lib/librelp.so.0 >> Reading symbols from /libexec/ld-elf.so.1...done. >> Loaded symbols for /libexec/ld-elf.so.1 >> #0 0x00000008007e3876 in pthread_mutex_destroy () from /lib/libthr.so.3 >> [New Thread 801a3ac80 (LWP 100568)] >> [New Thread 800c87fc0 (LWP 100567)] >> [New Thread 800c88180 (LWP 100541)] >> [New Thread 800c88340 (LWP 100540)] >> [New Thread 800c88500 (LWP 100539)] >> [New Thread 800c886c0 (LWP 100538)] >> [New Thread 800c88880 (LWP 100379)] >> [New Thread 800c041c0 (LWP 100548)] >> (gdb) bt full >> #0 0x00000008007e3876 in pthread_mutex_destroy () from /lib/libthr.so.3 >> No symbol table info available. >> #1 0x00000000004210cf in MsgDeleteMutexLockingCase (pThis=0x801aaea00) >> at msg.c:624 >> __func__ = "MsgDeleteMutexLockingCase" >> #2 0x00000000004218d1 in msgDestruct (ppThis=0x7ffffeff8d70) at msg.c:839 >> dbgCALLStaCK_POP_POINT = 7 >> iRet = RS_RET_OK >> iCancelStateSave = 1 >> pThis = (msg_t *) 0x801aaea00 >> currRefCount = 0 >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a12500 >> __func__ = "msgDestruct" >> #3 0x000000000044fbe1 in DeleteProcessedBatch (pThis=0x800c04380, >> pBatch=0x800c2fe70) at queue.c:1404 >> i = 0 >> pUsr = (void *) 0x801aaea00 >> nEnqueued = 0 >> localRet = 8 >> dbgCALLStaCK_POP_POINT = 6 >> iRet = RS_RET_OK >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13500 >> __func__ = "DeleteProcessedBatch" >> #4 0x000000000044f828 in DequeueConsumableElements (pThis=0x800c04380, >> pWti=0x800c2fe40, piRemainingQueueSize=0x7ffffeff8e14) at queue.c:1441 >> nDequeued = 8 >> nDiscarded = 4678208 >> nDeleted = 2 >> iQueueSize = 4673262 >> pUsr = (void *) 0x40042df73 >> localRet = RS_RET_OK >> dbgCALLStaCK_POP_POINT = 5 >> iRet = RS_RET_OK >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13600 >> __func__ = "DequeueConsumableElements" >> #5 0x000000000044f711 in DequeueConsumable (pThis=0x800c04380, >> pWti=0x800c2fe40) at queue.c:1489 >> dbgCALLStaCK_POP_POINT = 4 >> iRet = RS_RET_OK >> iQueueSize = 0 >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13700 >> __func__ = "DequeueConsumable" >> #6 0x0000000000450fca in DequeueForConsumer (pThis=0x800c04380, >> pWti=0x800c2fe40) at queue.c:1626 >> dbgCALLStaCK_POP_POINT = 3 >> iRet = RS_RET_OK >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13800 >> __func__ = "DequeueForConsumer" >> #7 0x0000000000450c6b in ConsumerReg (pThis=0x800c04380, >> pWti=0x800c2fe40) at queue.c:1679 >> iCancelStateSave = 8 >> dbgCALLStaCK_POP_POINT = 2 >> iRet = RS_RET_OK >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13900 >> __func__ = "ConsumerReg" >> #8 0x0000000000443b21 in wtiWorker (pThis=0x800c2fe40) at wti.c:315 >> __cleanup_info__ = {pthread_cleanup_pad = {140737471549248, >> 4470256, 34372517440, 0, 4667374, 5824688, 0, 34368002717}} >> pWtp = (wtp_t *) 0x800c1c380 >> bInactivityTOOccured = 0 >> localRet = RS_RET_OK >> terminateRet = RS_RET_OK >> iCancelStateSave = 0 >> dbgCALLStaCK_POP_POINT = 1 >> iRet = RS_RET_OK >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13d00 >> __func__ = "wtiWorker" >> #9 0x0000000000442019 in wtpWorker (arg=0x800c2fe40) at wtp.c:381 >> __cleanup_info__ = {pthread_cleanup_pad = {0, 4462880, >> 34372517440, 0, 0, 0, 0, 0}} >> pWti = (wti_t *) 0x800c2fe40 >> pThis = (wtp_t *) 0x800c1c380 >> sigSet = {__bits = {1048576, 0, 0, 0}} >> dbgCALLStaCK_POP_POINT = 0 >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13e00 >> __func__ = "wtpWorker" >> #10 0x00000008007de511 in pthread_getprio () from /lib/libthr.so.3 >> No symbol table info available. >> #11 0x0000000000000000 in ?? () >> No symbol table info available. >> Cannot access memory at address 0x7ffffeff9000 >> (gdb) frame 1 >> #1 0x00000000004210cf in MsgDeleteMutexLockingCase (pThis=0x801aaea00) >> at msg.c:624 >> 624 msg.c: No such file or directory. >> in msg.c >> (gdb) info locals >> __func__ = "MsgDeleteMutexLockingCase" >> (gdb) p *pThis >> $1 = {objData = {pObjInfo = 0x800c16400, iObjCooCKiE = 195948526, >> pszName = 0x0}, flowCtlType = eFLOWCTL_NO_DELAY, mut = >> 0xffffffffffffffff, iRefCount = 0, bDoLock = 0 '\0', iSeverity = 3, >> iFacility = 22, of >> fAfterPRI = 0, offMSG = 91, iProtocolVersion = 0, msgFlags = 32, >> iLenRawMsg = 121, iLenMSG = 30, iLenTAG = 31, iLenHOSTNAME = 21, >> pszRawMsg = 0x801ab5600 "<179>2011-02-03T06:20:37.681910+00:00 phoenix >> app001[99856]: last message repeated 5 times", pszHOSTNAME = 0x801aae >> b75 "phoenix", >> pszRcvdAt3164 = 0x0, pszRcvdAt3339 = 0x0, pszRcvdAt_MySQL = 0x0, >> pszRcvdAt_PgSQL = 0x0, pszTIMESTAMP3164 = 0x0, pszTIMESTAMP3339 = >> 0x801aaebc8 "2011-02-03T06:25:05.776392+00:00", pszTIMESTAMP_MySQL = 0x0, >> pszTIMESTAMP_PgSQL = 0x0, pCSProgName = 0x0, pCSStrucData = 0x0, >> pCSAPPNAME = 0x0, pCSPROCID = 0x0, pCSMSGID = 0x0, pInputName = 0x0, >> pRcvFromIP = 0x0, rcvFrom = {pRcvFrom = 0x0, pfrominet = 0x0}, pRuleset >> = 0x0, ttGenTime = 1296714305, tRcvdAt = {timeType = 2 '\002', m >> onth = 2 '\002', day = 3 '\003', hour = 6 '\006', >> minute = 25 '\031', second = 5 '\005', secfracPrecision = 6 '\006', >> OffsetMinute = 0 '\0', OffsetHour = 0 '\0', OffsetMode = 43 '+', year = >> 2011, secfrac = 776392}, tTIMESTAMP = {timeType = 2 '\002', mon >> th = 2 '\002', day = 3 '\003', hour = 6 '\006', minute = 25 '\031', >> second = 5 '\005', secfracPrecision = 6 '\006', OffsetMinute = 0 >> '\0', OffsetHour = 0 '\0', OffsetMode = 43 '+', year = 2011, secfrac = >> 776392}, szRawMsg = '\xff' , szHOSTNAME = "phoen >> ix\000\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff", TAG = { >> pszTAG = 0x722e313030707061
> bounds>, szBuf = "app001[99856]:\xff"}, pszTimestamp3164 = "\000", >> '\xff' , pszTimestamp3339 = "2011-02-03T06:25:05.776392 >> +00:00", pszTIMESTAMP_SecFrac = "\000\xff\xff\xff\xff\xff\xff", >> pszRcvdAt_SecFrac = "\000\xff\xff\xff\xff\xff\xff"} >> (gdb) quit >> >> rsyslog.conf: >> ------------- >> #### Modules >> ## Input >> $ModLoad immark >> $ModLoad imudp >> $ModLoad imtcp >> $ModLoad imuxsock >> $ModLoad imklog >> $ModLoad imrelp >> >> ## Output >> #$ModLoad omsnmp.so # Send SNMP traps >> >> #### Globals >> $umask 0000 >> $DirCreateMode 0640 >> $FileCreateMode 0640 >> >> $RepeatedMsgReduction on >> #$RepeatedMsgContainsOrigionalMsg on >> >> $DefaultNetstreamDriver ptcp >> >> # location for work (spool) file >> $WorkDirectory /var/spool/rsyslog >> >> # Queue Types >> $MainMsgQueueType LinkedList >> $ActionQueueType LinkedList >> >> # Become DA Queues >> $MainMsgQueueFileName mainq >> $ActionQueueFileNAme actionq >> >> # Disable Rate Throttling >> $MainMsgQueueTimeoutEnqueue 0 >> $ActionQueueTimeoutEnqueue 0 >> >> # Discard any message at all if there double the queue we allow in memory >> # 10000 in ram + 10000 on disk >> $MainMsgQueueDiscardMark 20000 >> $ActionQueueDiscardMark 20000 >> >> 0 Emergency: system is unusable >> # 1 Alert: action must be taken immediately >> # 2 Critical: critical conditions >> # 3 Error: error conditions >> # 4 Warning: warning conditions >> # 5 Notice: normal but significant condition >> # 6 Informational: informational messages >> # 7 Debug: debug-level messages >> >> # When discarding kicks in (above) any msg >> # with priority >= this is out the door >> $MainMsgQueueDiscardSeverity 0 >> $ActionQueueDiscardSeverity 0 >> >> # default templates >> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >> >> $UDPServerRun 514 >> $InputRELPServerRun 2514 >> >> # Templates >> $template logsaggd,"%TIMESTAMP:::date-rfc3339% %syslogfacility-text% >> %syslogpriority-text% %HOSTNAME% %syslogtag% %msg:::drop-last-lf%" >> >> #### Logging Facilities >> *.* @localhost:18712;logsaggd >> >> +hulk >> *.err;kern.warning;auth.notice;mail.crit >> /var/log/console.log >> *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err >> /var/log/messages >> security.* /var/log/security >> auth.info;authpriv.info /var/log/auth.log >> mail.info /var/log/maillog >> lpr.info /var/log/lpd-errs >> ftp.info /var/log/xferlog >> cron.* /var/log/cron >> *.=debug /var/log/debug.log >> *.emerg * >> > > -- ------------------------------------------------------------------------ 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354 Member, Apache Software Foundation Committer, FreeBSD Foundation Consultant, P6M7G8 Inc. Sr. System Admin, Ridecharge Inc. Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. From rgerhards at hq.adiscon.com Mon Apr 18 15:35:11 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 18 Apr 2011 15:35:11 +0200 Subject: [rsyslog] rsyslog5 core dumps a-plenty In-Reply-To: <4DAC3D89.8000905@p6m7g8.com> References: <4D4A518B.8020202@p6m7g8.com> <4DA7732C.6070309@p6m7g8.com> <4DAC3D89.8000905@p6m7g8.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE016@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci > Sent: Monday, April 18, 2011 3:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog5 core dumps a-plenty > > On 04/14/11 22:20, Philip M. Gollucci wrote: > > Will be testing 5.8.0 friday or next week, and will report back. > Whatever this bug was 5.8.0 seems to have fixed it. Its been up since > ~Saturday on both the central, and a subset of the clients. Thanks, I guess it was the one related to name resolution. Rainer > > Thanks! > > > > > > > > > > > On 02/03/11 06:56, Philip M. Gollucci wrote: > >> Rainer (All), > >> > >> At first I thought it was something in rsyslog 5.5.6+ that changed > >> b/c I couldn't get the core dumps happen with 5.5.5; however, now I'm > >> pretty sure it has nothing to do with that milestone. Unfortunately, > >> from the looks of it, I'm betting its a FreeBSD specific threading > >> issue. Below is the useful info, gdb back trace, rsyslog.conf, and the build > transcript. > >> > >> I've also uploaded the files here for easier viewing: > >> http://people.freebsd.org/~pgollucci/rsyslog-core > >> http://people.apache.org/~pgollucci/rsyslog-core > >> > >> Move info is available on request. > >> > >> > >> > >> $ uname -a > >> FreeBSD hulk 8.1-RELEASE FreeBSD 8.1-RELEASE #0 r212058: Wed Sep 1 > >> 23:12:18 UTC 2010 root at elektra > >> :/usr/obj/usr/src/sys/GENERIC amd64 > >> > >> $ rsyslog --version > >> rsyslogd 5.6.2, compiled with: > >> FEATURE_REGEXP: Yes > >> FEATURE_LARGEFILE: No > >> FEATURE_NETZIP (message compression): Yes > >> GSSAPI Kerberos 5 support: No > >> FEATURE_DEBUG (debug build, slow code): Yes > >> Atomic operations supported: Yes > >> Runtime Instrumentation (slow code): Yes > >> > >> $ sudo /usr/local/etc/rc.d/rsyslogd restart rsyslogd not running? > >> (check /var/run/syslog.pid). > >> Starting rsyslogd. > >> > >> $ ps -auxwww |grep rsyslogd > >> root 9135 6.1 0.0 25524 4388 1 S 6:37AM 0:01.29 > >> /usr/local/sbin/rsyslogd -c4 -i /var > >> /run/syslog.pid -f /usr/local/etc/rsyslog.conf > >> > >> $ cd / ; ls core.rsyslogd.8* > >> -rw------- 1 root wheel 10354688 Feb 3 06:25 core.rsyslogd.8742 > >> -rw------- 1 root wheel 8257536 Feb 3 06:26 core.rsyslogd.8782 > >> -rw------- 1 root wheel 8257536 Feb 3 06:27 core.rsyslogd.8826 > >> -rw------- 1 root wheel 8257536 Feb 3 06:28 core.rsyslogd.8844 > >> -rw------- 1 root wheel 10354688 Feb 3 06:29 core.rsyslogd.8875 > >> -rw------- 1 root wheel 8257536 Feb 3 06:29 core.rsyslogd.8900 > >> > >> $ gdb /usr/local/sbin/rsyslogd core.rsyslogd.81742 GNU gdb 6.1.1 > >> [FreeBSD] Copyright 2004 Free Software Foundation, Inc. > >> GDB is free software, covered by the GNU General Public License, and > >> you are welcome to change it and/or distribute copies of it under > >> certain conditions. > >> Type "show copying" to see the conditions. > >> There is absolutely no warranty for GDB. Type "show warranty" for > details. > >> This GDB was configured as "amd64-marcel-freebsd"... > >> Core was generated by `rsyslogd'. > >> Program terminated with signal 11, Segmentation fault. > >> Reading symbols from /lib/libz.so.5...done. > >> Loaded symbols for /lib/libz.so.5 > >> Reading symbols from /lib/libthr.so.3...done. > >> Loaded symbols for /lib/libthr.so.3 > >> Reading symbols from /lib/libc.so.7...done. > >> Loaded symbols for /lib/libc.so.7 > >> Reading symbols from /usr/local/lib/rsyslog/lmnet.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/lmnet.so Reading symbols > >> from /usr/local/lib/rsyslog/immark.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/immark.so Reading symbols > >> from /usr/local/lib/rsyslog/imudp.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/imudp.so Reading symbols > >> from /usr/local/lib/rsyslog/imtcp.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/imtcp.so Reading symbols > >> from /usr/local/lib/rsyslog/lmnetstrms.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/lmnetstrms.so > >> Reading symbols from /usr/local/lib/rsyslog/lmtcpsrv.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/lmtcpsrv.so > >> Reading symbols from /usr/local/lib/rsyslog/imuxsock.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/imuxsock.so > >> Reading symbols from /usr/local/lib/rsyslog/imklog.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/imklog.so Reading symbols > >> from /usr/local/lib/rsyslog/imrelp.so...done. > >> Loaded symbols for /usr/local/lib/rsyslog/imrelp.so Reading symbols > >> from /usr/local/lib/librelp.so.0...done. > >> Loaded symbols for /usr/local/lib/librelp.so.0 Reading symbols from > >> /libexec/ld-elf.so.1...done. > >> Loaded symbols for /libexec/ld-elf.so.1 > >> #0 0x00000008007e3876 in pthread_mutex_destroy () from > >> /lib/libthr.so.3 [New Thread 801a3ac80 (LWP 100568)] [New Thread > >> 800c87fc0 (LWP 100567)] [New Thread 800c88180 (LWP 100541)] [New > >> Thread 800c88340 (LWP 100540)] [New Thread 800c88500 (LWP 100539)] > >> [New Thread 800c886c0 (LWP 100538)] [New Thread 800c88880 (LWP > >> 100379)] [New Thread 800c041c0 (LWP 100548)] > >> (gdb) bt full > >> #0 0x00000008007e3876 in pthread_mutex_destroy () from > >> /lib/libthr.so.3 No symbol table info available. > >> #1 0x00000000004210cf in MsgDeleteMutexLockingCase > >> (pThis=0x801aaea00) at msg.c:624 > >> __func__ = "MsgDeleteMutexLockingCase" > >> #2 0x00000000004218d1 in msgDestruct (ppThis=0x7ffffeff8d70) at > msg.c:839 > >> dbgCALLStaCK_POP_POINT = 7 > >> iRet = RS_RET_OK > >> iCancelStateSave = 1 > >> pThis = (msg_t *) 0x801aaea00 > >> currRefCount = 0 > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a12500 > >> __func__ = "msgDestruct" > >> #3 0x000000000044fbe1 in DeleteProcessedBatch (pThis=0x800c04380, > >> pBatch=0x800c2fe70) at queue.c:1404 > >> i = 0 > >> pUsr = (void *) 0x801aaea00 > >> nEnqueued = 0 > >> localRet = 8 > >> dbgCALLStaCK_POP_POINT = 6 > >> iRet = RS_RET_OK > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13500 > >> __func__ = "DeleteProcessedBatch" > >> #4 0x000000000044f828 in DequeueConsumableElements > >> (pThis=0x800c04380, pWti=0x800c2fe40, > piRemainingQueueSize=0x7ffffeff8e14) at queue.c:1441 > >> nDequeued = 8 > >> nDiscarded = 4678208 > >> nDeleted = 2 > >> iQueueSize = 4673262 > >> pUsr = (void *) 0x40042df73 > >> localRet = RS_RET_OK > >> dbgCALLStaCK_POP_POINT = 5 > >> iRet = RS_RET_OK > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13600 > >> __func__ = "DequeueConsumableElements" > >> #5 0x000000000044f711 in DequeueConsumable (pThis=0x800c04380, > >> pWti=0x800c2fe40) at queue.c:1489 > >> dbgCALLStaCK_POP_POINT = 4 > >> iRet = RS_RET_OK > >> iQueueSize = 0 > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13700 > >> __func__ = "DequeueConsumable" > >> #6 0x0000000000450fca in DequeueForConsumer (pThis=0x800c04380, > >> pWti=0x800c2fe40) at queue.c:1626 > >> dbgCALLStaCK_POP_POINT = 3 > >> iRet = RS_RET_OK > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13800 > >> __func__ = "DequeueForConsumer" > >> #7 0x0000000000450c6b in ConsumerReg (pThis=0x800c04380, > >> pWti=0x800c2fe40) at queue.c:1679 > >> iCancelStateSave = 8 > >> dbgCALLStaCK_POP_POINT = 2 > >> iRet = RS_RET_OK > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13900 > >> __func__ = "ConsumerReg" > >> #8 0x0000000000443b21 in wtiWorker (pThis=0x800c2fe40) at wti.c:315 > >> __cleanup_info__ = {pthread_cleanup_pad = {140737471549248, > >> 4470256, 34372517440, 0, 4667374, 5824688, 0, 34368002717}} > >> pWtp = (wtp_t *) 0x800c1c380 > >> bInactivityTOOccured = 0 > >> localRet = RS_RET_OK > >> terminateRet = RS_RET_OK > >> iCancelStateSave = 0 > >> dbgCALLStaCK_POP_POINT = 1 > >> iRet = RS_RET_OK > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13d00 > >> __func__ = "wtiWorker" > >> #9 0x0000000000442019 in wtpWorker (arg=0x800c2fe40) at wtp.c:381 > >> __cleanup_info__ = {pthread_cleanup_pad = {0, 4462880, > >> 34372517440, 0, 0, 0, 0, 0}} > >> pWti = (wti_t *) 0x800c2fe40 > >> pThis = (wtp_t *) 0x800c1c380 > >> sigSet = {__bits = {1048576, 0, 0, 0}} > >> dbgCALLStaCK_POP_POINT = 0 > >> pdbgFuncDB = (dbgFuncDB_t *) 0x801a13e00 > >> __func__ = "wtpWorker" > >> #10 0x00000008007de511 in pthread_getprio () from /lib/libthr.so.3 No > >> symbol table info available. > >> #11 0x0000000000000000 in ?? () > >> No symbol table info available. > >> Cannot access memory at address 0x7ffffeff9000 > >> (gdb) frame 1 > >> #1 0x00000000004210cf in MsgDeleteMutexLockingCase > >> (pThis=0x801aaea00) at msg.c:624 > >> 624 msg.c: No such file or directory. > >> in msg.c > >> (gdb) info locals > >> __func__ = "MsgDeleteMutexLockingCase" > >> (gdb) p *pThis > >> $1 = {objData = {pObjInfo = 0x800c16400, iObjCooCKiE = 195948526, > >> pszName = 0x0}, flowCtlType = eFLOWCTL_NO_DELAY, mut = > >> 0xffffffffffffffff, iRefCount = 0, bDoLock = 0 '\0', iSeverity = 3, > >> iFacility = 22, of fAfterPRI = 0, offMSG = 91, iProtocolVersion = 0, > >> msgFlags = 32, > >> iLenRawMsg = 121, iLenMSG = 30, iLenTAG = 31, iLenHOSTNAME = 21, > >> pszRawMsg = 0x801ab5600 "<179>2011-02-03T06:20:37.681910+00:00 > >> phoenix > >> app001[99856]: last message repeated 5 times", pszHOSTNAME = > 0x801aae > >> b75 "phoenix", > >> pszRcvdAt3164 = 0x0, pszRcvdAt3339 = 0x0, pszRcvdAt_MySQL = 0x0, > >> pszRcvdAt_PgSQL = 0x0, pszTIMESTAMP3164 = 0x0, pszTIMESTAMP3339 = > >> 0x801aaebc8 "2011-02-03T06:25:05.776392+00:00", > pszTIMESTAMP_MySQL = > >> 0x0, pszTIMESTAMP_PgSQL = 0x0, pCSProgName = 0x0, pCSStrucData = > 0x0, > >> pCSAPPNAME = 0x0, pCSPROCID = 0x0, pCSMSGID = 0x0, pInputName = > >> 0x0, pRcvFromIP = 0x0, rcvFrom = {pRcvFrom = 0x0, pfrominet = 0x0}, > >> pRuleset = 0x0, ttGenTime = 1296714305, tRcvdAt = {timeType = 2 > >> '\002', m onth = 2 '\002', day = 3 '\003', hour = 6 '\006', > >> minute = 25 '\031', second = 5 '\005', secfracPrecision = 6 > >> '\006', OffsetMinute = 0 '\0', OffsetHour = 0 '\0', OffsetMode = 43 > >> '+', year = 2011, secfrac = 776392}, tTIMESTAMP = {timeType = 2 > >> '\002', mon th = 2 '\002', day = 3 '\003', hour = 6 '\006', minute = 25 '\031', > >> second = 5 '\005', secfracPrecision = 6 '\006', OffsetMinute = 0 > >> '\0', OffsetHour = 0 '\0', OffsetMode = 43 '+', year = 2011, secfrac > >> = 776392}, szRawMsg = '\xff' , szHOSTNAME = "phoen > >> ix\000\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff", TAG = { > >> pszTAG = 0x722e313030707061
>> bounds>, szBuf = "app001[99856]:\xff"}, pszTimestamp3164 = "\000", > >> '\xff' , pszTimestamp3339 = > >> "2011-02-03T06:25:05.776392 > >> +00:00", pszTIMESTAMP_SecFrac = "\000\xff\xff\xff\xff\xff\xff", > >> pszRcvdAt_SecFrac = "\000\xff\xff\xff\xff\xff\xff"} > >> (gdb) quit > >> > >> rsyslog.conf: > >> ------------- > >> #### Modules > >> ## Input > >> $ModLoad immark > >> $ModLoad imudp > >> $ModLoad imtcp > >> $ModLoad imuxsock > >> $ModLoad imklog > >> $ModLoad imrelp > >> > >> ## Output > >> #$ModLoad omsnmp.so # Send SNMP traps > >> > >> #### Globals > >> $umask 0000 > >> $DirCreateMode 0640 > >> $FileCreateMode 0640 > >> > >> $RepeatedMsgReduction on > >> #$RepeatedMsgContainsOrigionalMsg on > >> > >> $DefaultNetstreamDriver ptcp > >> > >> # location for work (spool) file > >> $WorkDirectory /var/spool/rsyslog > >> > >> # Queue Types > >> $MainMsgQueueType LinkedList > >> $ActionQueueType LinkedList > >> > >> # Become DA Queues > >> $MainMsgQueueFileName mainq > >> $ActionQueueFileNAme actionq > >> > >> # Disable Rate Throttling > >> $MainMsgQueueTimeoutEnqueue 0 > >> $ActionQueueTimeoutEnqueue 0 > >> > >> # Discard any message at all if there double the queue we allow in > >> memory # 10000 in ram + 10000 on disk $MainMsgQueueDiscardMark > 20000 > >> $ActionQueueDiscardMark 20000 > >> > >> 0 Emergency: system is unusable > >> # 1 Alert: action must be taken immediately > >> # 2 Critical: critical conditions > >> # 3 Error: error conditions > >> # 4 Warning: warning conditions > >> # 5 Notice: normal but significant condition > >> # 6 Informational: informational messages > >> # 7 Debug: debug-level messages > >> > >> # When discarding kicks in (above) any msg # with priority >= this is > >> out the door $MainMsgQueueDiscardSeverity 0 > >> $ActionQueueDiscardSeverity 0 > >> > >> # default templates > >> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > >> > >> $UDPServerRun 514 > >> $InputRELPServerRun 2514 > >> > >> # Templates > >> $template logsaggd,"%TIMESTAMP:::date-rfc3339% %syslogfacility-text% > >> %syslogpriority-text% %HOSTNAME% %syslogtag% %msg:::drop-last-lf%" > >> > >> #### Logging Facilities > >> *.* @localhost:18712;logsaggd > >> > >> +hulk > >> *.err;kern.warning;auth.notice;mail.crit > >> /var/log/console.log > >> *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err > >> /var/log/messages > >> security.* /var/log/security > >> auth.info;authpriv.info /var/log/auth.log > >> mail.info /var/log/maillog > >> lpr.info /var/log/lpd-errs > >> ftp.info /var/log/xferlog > >> cron.* /var/log/cron > >> *.=debug /var/log/debug.log > >> *.emerg * > >> > > > > > > > -- > ------------------------------------------------------------------------ > 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C > Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354 > Member, Apache Software Foundation > Committer, FreeBSD Foundation > Consultant, P6M7G8 Inc. > Sr. System Admin, Ridecharge Inc. > > Work like you don't need the money, > love like you'll never get hurt, > and dance like nobody's watching. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From bhava_cse at yahoo.co.in Tue Apr 19 08:09:24 2011 From: bhava_cse at yahoo.co.in (BHAVANI KUMAR KUPPA) Date: Tue, 19 Apr 2011 11:39:24 +0530 (IST) Subject: [rsyslog] Regading ommail module in rsyslog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> References: <4DA56C36.1060405@googlemail.com> <9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> Message-ID: <161209.78907.qm@web95603.mail.in.yahoo.com> Hi, ?I have started using rsyslog recently for one of my projects and have been struggling to accomplish the following task: Task: A mail is to be sent every 15 minutes with the body of the mail being the collection of all the logs written in those span of 15 minutes .. For this I have been trying to configure ommail in several ways but couldn't find a way yet. Though the " $ActionExecOnlyOnceEveryInterval " directive ensures sending a mail at regular intervals, I am unable to have a control on the body of the mail wherein I need to collect all the logs being written to the log file during that interval. Is there a way we can do this ?? Or do we need to make use of a separate program (Shell script, etc.. ) to do this job?? Also can we have the logs written to a file wherein the files are created as per the current date i.e. say the logs being written on 19th April 2011 have to be stored in a file log_04_19_11 and that of 20th April 2011 in log_04_20_11. Is there a way we can accomplish this using rsyslog ?? Thanks and Regards, K.V.M. Bhavani Kumar From rgerhards at hq.adiscon.com Tue Apr 19 08:11:53 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 19 Apr 2011 08:11:53 +0200 Subject: [rsyslog] Regading ommail module in rsyslog In-Reply-To: <161209.78907.qm@web95603.mail.in.yahoo.com> References: <4DA56C36.1060405@googlemail.com><9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> <161209.78907.qm@web95603.mail.in.yahoo.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE021@GRFEXC.intern.adiscon.com> You need to write a shell script for this. But are you sure you *really* want to have log files mailed every 15 minutes? Think about the mailbox clutter... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of BHAVANI KUMAR KUPPA > Sent: Tuesday, April 19, 2011 8:09 AM > To: rsyslog-users > Subject: [rsyslog] Regading ommail module in rsyslog > > Hi, > > ?I have started using rsyslog recently for one of my projects and have been > struggling to accomplish the following task: > > Task: A mail is to be sent every 15 minutes with the body of the mail being > the collection of all the logs written in those span of 15 minutes .. > > For this I have been trying to configure ommail in several ways but couldn't > find a way yet. Though the " $ActionExecOnlyOnceEveryInterval " directive > ensures sending a mail at regular intervals, I am unable to have a control on > the body of the mail wherein I need to collect all the logs being written to the > log file during that interval. Is there a way we can do this ?? Or do we need to > make use of a separate program (Shell script, etc.. ) to do this job?? > > Also can we have the logs written to a file wherein the files are created as > per the current date i.e. say the logs being written on 19th April 2011 have to > be stored in a file log_04_19_11 and that of 20th April 2011 in log_04_20_11. > Is there a way we can accomplish this using rsyslog ?? > > > > Thanks and Regards, > K.V.M. Bhavani Kumar > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From bhava_cse at yahoo.co.in Tue Apr 19 08:55:23 2011 From: bhava_cse at yahoo.co.in (BHAVANI KUMAR KUPPA) Date: Tue, 19 Apr 2011 12:25:23 +0530 (IST) Subject: [rsyslog] Regarding ommail module in rsyslog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE021@GRFEXC.intern.adiscon.com> References: <4DA56C36.1060405@googlemail.com><9B6E2A8877C38245BFB15CC491A11DA71DDFCC@GRFEXC.intern.adiscon.com> <161209.78907.qm@web95603.mail.in.yahoo.com> <9B6E2A8877C38245BFB15CC491A11DA71DE021@GRFEXC.intern.adiscon.com> Message-ID: <598169.29722.qm@web95609.mail.in.yahoo.com> Thanks for the reply. Actually we are expecting around 100 logs being recorded in a span of 15 minutes and hence a mail for each would in fact spam the inbox. So we thought of mailing a collection of them at fixed intervals. Any way as you suggested we will rethink about this. Is it possible to have the log file created date wise using rsyslog?? (Just to ease the process of book-keeping).. Thanks and Regards,K.V.M. Bhavani Kumar ________________________________ From: Rainer Gerhards To: BHAVANI KUMAR KUPPA ; rsyslog-users Sent: Tuesday, 19 April 2011 11:41 AM Subject: RE: [rsyslog] Regading ommail module in rsyslog You need to write a shell script for this. But are you sure you *really* want to have log files mailed every 15 minutes? Think about the mailbox clutter... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of BHAVANI KUMAR KUPPA > Sent: Tuesday, April 19, 2011 8:09 AM > To: rsyslog-users > Subject: [rsyslog] Regading ommail module in rsyslog > > Hi, > > ?I have started using rsyslog recently for one of my projects and have been > struggling to accomplish the following task: > > Task: A mail is to be sent every 15 minutes with the body of the mail being > the collection of all the logs written in those span of 15 minutes .. > > For this I have been trying to configure ommail in several ways but couldn't > find a way yet. Though the " $ActionExecOnlyOnceEveryInterval " directive > ensures sending a mail at regular intervals, I am unable to have a control on > the body of the mail wherein I need to collect all the logs being written to the > log file during that interval. Is there a way we can do this ?? Or do we need to > make use of a separate program (Shell script, etc.. ) to do this job?? > > Also can we have the logs written to a file wherein the files are created as > per the current date i.e. say the logs being written on 19th April 2011 have to > be stored in a file log_04_19_11 and that of 20th April 2011 in log_04_20_11. > Is there a way we can accomplish this using rsyslog ?? > > > > Thanks and Regards, > K.V.M. Bhavani Kumar > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From David.Berard at univ-savoie.fr Thu Apr 21 15:32:21 2011 From: David.Berard at univ-savoie.fr (David Berard) Date: Thu, 21 Apr 2011 15:32:21 +0200 (CEST) Subject: [rsyslog] RHEL 5.5 rsyslog package and functionnality Message-ID: Hi, We try to centralize logs of approximately twenty servers to one host with rsyslogd. The rsyslog server run under RedHat RHEL 5.5 with the rsyslog package of RedHat (3.22.1-3el5_5.1). For the ease of maintenance, we prefer to keep the redhat package if possible. We want to distibute logs in differents files according to the hostname of the sender, the facility or the program name. We try to use filtering by properties and by expression with no luck... I think this is because our rsyslog version is too old. But can someone confirm this ? I try things like this : :hostname, startswith, "XXXX";programname, isequal, "nginx" -?DynFileZMProxyNginx & ~ :hostname, startswith, "XXXX";:programname, isequal, "nginx" -?DynFileZMProxyNginx & ~ if $hostname startswith "XXXX" and $programname isequal "nginx" then -?DynFileZMProxyNginx & ~ if $hostname startswith "XXXX" and $programname == "nginx" then -?DynFileZMProxyNginx & ~ I always obtain a 'CONFIG ERROR' from rsyslog. rsyslogd is running with option "-c3" Is my rsyslog version too old, or do I misunderstand filters usages ? Is there an other solution for doing the filtering ? Best regards David From david at lang.hm Fri Apr 22 20:07:54 2011 From: david at lang.hm (david at lang.hm) Date: Fri, 22 Apr 2011 11:07:54 -0700 (PDT) Subject: [rsyslog] RHEL 5.5 rsyslog package and functionnality In-Reply-To: References: Message-ID: On Thu, 21 Apr 2011, David Berard wrote: > Hi, > > We try to centralize logs of approximately twenty servers to > one host with rsyslogd. > > The rsyslog server run under RedHat RHEL 5.5 with the rsyslog package of > RedHat (3.22.1-3el5_5.1). > For the ease of maintenance, we prefer to keep the redhat package if > possible. > > We want to distibute logs in differents files according to the > hostname of the sender, the facility or the program name. > > We try to use filtering by properties and by expression with no luck... > I think this is because our rsyslog version is too old. But can someone > confirm this ? > > I try things like this : > :hostname, startswith, "XXXX";programname, isequal, "nginx" > -?DynFileZMProxyNginx > & ~ this format of filers does not allow you to use multiple conditions on one line, you would have to go to the slower if ( ) type of conditions. David Lang > :hostname, startswith, "XXXX";:programname, isequal, "nginx" > -?DynFileZMProxyNginx > & ~ > > if $hostname startswith "XXXX" and $programname isequal "nginx" then > -?DynFileZMProxyNginx > & ~ > > if $hostname startswith "XXXX" and $programname == "nginx" then > -?DynFileZMProxyNginx > & ~ > > I always obtain a 'CONFIG ERROR' from rsyslog. > rsyslogd is running with option "-c3" > > Is my rsyslog version too old, or do I misunderstand filters usages ? > > Is there an other solution for doing the filtering ? > > > Best regards > David > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From hoyang at redhat.com Mon Apr 25 10:16:08 2011 From: hoyang at redhat.com (Hongqing Yang) Date: Mon, 25 Apr 2011 04:16:08 -0400 (EDT) Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <1255862468.90810.1303719167880.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <1552407899.90832.1303719368543.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Hi, All I try to turn off the rate limit by adding $SystemLogRateLimitInterval 0 after $ModLoad imuxsock.so, but it does not work, now I cannot get all the logs. Do you have any idea how to fix it? Thanks. Hongqing From rgerhards at hq.adiscon.com Mon Apr 25 10:57:43 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 25 Apr 2011 10:57:43 +0200 Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <1552407899.90832.1303719368543.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> References: <1255862468.90810.1303719167880.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> <1552407899.90832.1303719368543.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE06F@GRFEXC.intern.adiscon.com> Can you please send a debug log. Probably it is sufficient to see the startup, but we may need a full session. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 10:16 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] cannot turn off the rate limit > > Hi, All > > I try to turn off the rate limit by adding $SystemLogRateLimitInterval > 0 after $ModLoad imuxsock.so, > but it does not work, now I cannot get all the logs. Do you have any > idea how to fix it? Thanks. > > > Hongqing > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From hoyang at redhat.com Mon Apr 25 11:24:29 2011 From: hoyang at redhat.com (Hongqing Yang) Date: Mon, 25 Apr 2011 05:24:29 -0400 (EDT) Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE06F@GRFEXC.intern.adiscon.com> Message-ID: <1098892893.91067.1303723469894.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Hi, Rainer, Thanks for your reply, Now I try to transfer guest installation logs from KVM to host by virtio. But I find the logs are different, after discussed with the anaconda team, I know I have to turn ratelimit off. The configuration of rsyslog is below: #### MODULES #### $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $SystemLogRateLimitInterval 0 $ModLoad imklog.so # provides kernel logging support (previously done by rklogd) #$ModLoad immark.so # provides --MARK-- message capability the logs on the guest can be found in attachment. Thanks. ----- Original Message ----- From: "Rainer Gerhards" To: "rsyslog-users" Sent: Monday, April 25, 2011 4:57:43 PM Subject: Re: [rsyslog] cannot turn off the rate limit Can you please send a debug log. Probably it is sufficient to see the startup, but we may need a full session. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 10:16 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] cannot turn off the rate limit > > Hi, All > > I try to turn off the rate limit by adding $SystemLogRateLimitInterval > 0 after $ModLoad imuxsock.so, > but it does not work, now I cannot get all the logs. Do you have any > idea how to fix it? Thanks. > > > Hongqing > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From hoyang at redhat.com Mon Apr 25 11:27:23 2011 From: hoyang at redhat.com (Hongqing Yang) Date: Mon, 25 Apr 2011 05:27:23 -0400 (EDT) Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <1098892893.91067.1303723469894.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <705205113.91077.1303723643418.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Sorry, failed to attach the file. Now it is fine. ----- Original Message ----- From: "Hongqing Yang" To: "rsyslog-users" Sent: Monday, April 25, 2011 5:24:29 PM Subject: Re: [rsyslog] cannot turn off the rate limit Hi, Rainer, Thanks for your reply, Now I try to transfer guest installation logs from KVM to host by virtio. But I find the logs are different, after discussed with the anaconda team, I know I have to turn ratelimit off. The configuration of rsyslog is below: #### MODULES #### $ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command) $SystemLogRateLimitInterval 0 $ModLoad imklog.so # provides kernel logging support (previously done by rklogd) #$ModLoad immark.so # provides --MARK-- message capability the logs on the guest can be found in attachment. Thanks. ----- Original Message ----- From: "Rainer Gerhards" To: "rsyslog-users" Sent: Monday, April 25, 2011 4:57:43 PM Subject: Re: [rsyslog] cannot turn off the rate limit Can you please send a debug log. Probably it is sufficient to see the startup, but we may need a full session. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 10:16 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] cannot turn off the rate limit > > Hi, All > > I try to turn off the rate limit by adding $SystemLogRateLimitInterval > 0 after $ModLoad imuxsock.so, > but it does not work, now I cannot get all the logs. Do you have any > idea how to fix it? Thanks. > > > Hongqing > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com -------------- next part -------------- A non-text attachment was scrubbed... Name: anaconda.syslog Type: application/octet-stream Size: 56925 bytes Desc: not available URL: From rgerhards at hq.adiscon.com Mon Apr 25 11:55:56 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 25 Apr 2011 11:55:56 +0200 Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <705205113.91077.1303723643418.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> References: <1098892893.91067.1303723469894.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> <705205113.91077.1303723643418.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE071@GRFEXC.intern.adiscon.com> Sorry, I'Ve not been precise enough. I need a rsyslog debug log, not the logfile. See this page: http://www.rsyslog.com/doc/troubleshoot.html Note that we have a public holiday over here and I am meeting with family. So I have no lab at hand. But with the debug log, I may be able to tell something. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 11:27 AM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Sorry, failed to attach the file. Now it is fine. > > ----- Original Message ----- > From: "Hongqing Yang" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 5:24:29 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > Hi, Rainer, > > Thanks for your reply, Now I try to transfer guest installation logs > from KVM to host by virtio. > But I find the logs are different, after discussed with the anaconda > team, I know I have > to turn ratelimit off. > > The configuration of rsyslog is below: > > #### MODULES #### > > $ModLoad imuxsock.so # provides support for local system logging > (e.g. via logger command) > $SystemLogRateLimitInterval 0 > $ModLoad imklog.so # provides kernel logging support (previously > done by rklogd) > #$ModLoad immark.so # provides --MARK-- message capability > > the logs on the guest can be found in attachment. > > Thanks. > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 4:57:43 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > Can you please send a debug log. Probably it is sufficient to see the > startup, but we may need a full session. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Monday, April 25, 2011 10:16 AM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] cannot turn off the rate limit > > > > Hi, All > > > > I try to turn off the rate limit by adding > $SystemLogRateLimitInterval > > 0 after $ModLoad imuxsock.so, > > but it does not work, now I cannot get all the logs. Do you have any > > idea how to fix it? Thanks. > > > > > > Hongqing > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From hoyang at redhat.com Mon Apr 25 12:36:25 2011 From: hoyang at redhat.com (Hongqing Yang) Date: Mon, 25 Apr 2011 06:36:25 -0400 (EDT) Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE071@GRFEXC.intern.adiscon.com> Message-ID: <252464480.91397.1303727785532.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Hi, Rainer, Please find the rsyslog.conf and rsyslog log file in attachments. Thanks. ----- Original Message ----- From: "Rainer Gerhards" To: "rsyslog-users" Sent: Monday, April 25, 2011 5:55:56 PM Subject: Re: [rsyslog] cannot turn off the rate limit Sorry, I'Ve not been precise enough. I need a rsyslog debug log, not the logfile. See this page: http://www.rsyslog.com/doc/troubleshoot.html Note that we have a public holiday over here and I am meeting with family. So I have no lab at hand. But with the debug log, I may be able to tell something. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 11:27 AM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Sorry, failed to attach the file. Now it is fine. > > ----- Original Message ----- > From: "Hongqing Yang" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 5:24:29 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > Hi, Rainer, > > Thanks for your reply, Now I try to transfer guest installation logs > from KVM to host by virtio. > But I find the logs are different, after discussed with the anaconda > team, I know I have > to turn ratelimit off. > > The configuration of rsyslog is below: > > #### MODULES #### > > $ModLoad imuxsock.so # provides support for local system logging > (e.g. via logger command) > $SystemLogRateLimitInterval 0 > $ModLoad imklog.so # provides kernel logging support (previously > done by rklogd) > #$ModLoad immark.so # provides --MARK-- message capability > > the logs on the guest can be found in attachment. > > Thanks. > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 4:57:43 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > Can you please send a debug log. Probably it is sufficient to see the > startup, but we may need a full session. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Monday, April 25, 2011 10:16 AM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] cannot turn off the rate limit > > > > Hi, All > > > > I try to turn off the rate limit by adding > $SystemLogRateLimitInterval > > 0 after $ModLoad imuxsock.so, > > but it does not work, now I cannot get all the logs. Do you have any > > idea how to fix it? Thanks. > > > > > > Hongqing > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com -------------- next part -------------- A non-text attachment was scrubbed... Name: rsyslog.conf Type: application/octet-stream Size: 2771 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: rsyslog Type: application/octet-stream Size: 81034 bytes Desc: not available URL: From rgerhards at hq.adiscon.com Mon Apr 25 14:58:54 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 25 Apr 2011 14:58:54 +0200 Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <252464480.91397.1303727785532.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE071@GRFEXC.intern.adiscon.com> <252464480.91397.1303727785532.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE072@GRFEXC.intern.adiscon.com> You should have a look at the syslog error log ;) 6950.906989271:7f1afcdeb720: cfline: '$SystemLogRateLimitInterval 0' 6950.907004432:7f1afcdeb720: Called LogError, msg: invalid or yet-unknown config file command - have you forgotten to load a module? 6950.907033424:7f1afcdeb720: config line NOT successfully processed The error message is emitted, but probably thrown away ;) Anyhow, you use 4.6.3, which does not support the new type of rate limiting at all. So I guess what comes into your way is repated message reduction, which you can turn off via a separate statement (Which I don't know out of my head, but if you search for repated message reduction you should find it relatively easily :). HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 12:36 PM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Hi, Rainer, > > Please find the rsyslog.conf and rsyslog log file in attachments. > Thanks. > > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 5:55:56 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > Sorry, I'Ve not been precise enough. I need a rsyslog debug log, not > the > logfile. See this page: > > http://www.rsyslog.com/doc/troubleshoot.html > > Note that we have a public holiday over here and I am meeting with > family. So > I have no lab at hand. But with the debug log, I may be able to tell > something. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Monday, April 25, 2011 11:27 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Sorry, failed to attach the file. Now it is fine. > > > > ----- Original Message ----- > > From: "Hongqing Yang" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 5:24:29 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Hi, Rainer, > > > > Thanks for your reply, Now I try to transfer guest installation logs > > from KVM to host by virtio. > > But I find the logs are different, after discussed with the anaconda > > team, I know I have > > to turn ratelimit off. > > > > The configuration of rsyslog is below: > > > > #### MODULES #### > > > > $ModLoad imuxsock.so # provides support for local system logging > > (e.g. via logger command) > > $SystemLogRateLimitInterval 0 > > $ModLoad imklog.so # provides kernel logging support (previously > > done by rklogd) > > #$ModLoad immark.so # provides --MARK-- message capability > > > > the logs on the guest can be found in attachment. > > > > Thanks. > > > > ----- Original Message ----- > > From: "Rainer Gerhards" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 4:57:43 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Can you please send a debug log. Probably it is sufficient to see the > > startup, but we may need a full session. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > Sent: Monday, April 25, 2011 10:16 AM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] cannot turn off the rate limit > > > > > > Hi, All > > > > > > I try to turn off the rate limit by adding > > $SystemLogRateLimitInterval > > > 0 after $ModLoad imuxsock.so, > > > but it does not work, now I cannot get all the logs. Do you have > any > > > idea how to fix it? Thanks. > > > > > > > > > Hongqing > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From hoyang at redhat.com Tue Apr 26 08:11:25 2011 From: hoyang at redhat.com (Hongqing Yang) Date: Tue, 26 Apr 2011 02:11:25 -0400 (EDT) Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE072@GRFEXC.intern.adiscon.com> Message-ID: <229393435.110184.1303798285165.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Rainer, Thanks for your reply. I think the commend you mentioned is RepeatedMsgReduction off, am I right? but the messages are still dropped. <46>2011-04-26T05:36:20.002095+00:00 localhost rsyslogd-2177: imuxsock begins to drop messages from pid 358 due to rate-limiting <46>2011-04-26T05:36:23.297653+00:00 localhost rsyslogd-2177: imuxsock lost 559 messages from pid 358 due to rate-limiting There is no errors in the debug logs. By the way, in my situation, the dropped messages are not repeated messages. Thanks. ----- Original Message ----- From: "Rainer Gerhards" To: "rsyslog-users" Sent: Monday, April 25, 2011 8:58:54 PM Subject: Re: [rsyslog] cannot turn off the rate limit You should have a look at the syslog error log ;) 6950.906989271:7f1afcdeb720: cfline: '$SystemLogRateLimitInterval 0' 6950.907004432:7f1afcdeb720: Called LogError, msg: invalid or yet-unknown config file command - have you forgotten to load a module? 6950.907033424:7f1afcdeb720: config line NOT successfully processed The error message is emitted, but probably thrown away ;) Anyhow, you use 4.6.3, which does not support the new type of rate limiting at all. So I guess what comes into your way is repated message reduction, which you can turn off via a separate statement (Which I don't know out of my head, but if you search for repated message reduction you should find it relatively easily :). HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Monday, April 25, 2011 12:36 PM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Hi, Rainer, > > Please find the rsyslog.conf and rsyslog log file in attachments. > Thanks. > > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 5:55:56 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > Sorry, I'Ve not been precise enough. I need a rsyslog debug log, not > the > logfile. See this page: > > http://www.rsyslog.com/doc/troubleshoot.html > > Note that we have a public holiday over here and I am meeting with > family. So > I have no lab at hand. But with the debug log, I may be able to tell > something. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Monday, April 25, 2011 11:27 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Sorry, failed to attach the file. Now it is fine. > > > > ----- Original Message ----- > > From: "Hongqing Yang" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 5:24:29 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Hi, Rainer, > > > > Thanks for your reply, Now I try to transfer guest installation logs > > from KVM to host by virtio. > > But I find the logs are different, after discussed with the anaconda > > team, I know I have > > to turn ratelimit off. > > > > The configuration of rsyslog is below: > > > > #### MODULES #### > > > > $ModLoad imuxsock.so # provides support for local system logging > > (e.g. via logger command) > > $SystemLogRateLimitInterval 0 > > $ModLoad imklog.so # provides kernel logging support (previously > > done by rklogd) > > #$ModLoad immark.so # provides --MARK-- message capability > > > > the logs on the guest can be found in attachment. > > > > Thanks. > > > > ----- Original Message ----- > > From: "Rainer Gerhards" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 4:57:43 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Can you please send a debug log. Probably it is sufficient to see the > > startup, but we may need a full session. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > Sent: Monday, April 25, 2011 10:16 AM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] cannot turn off the rate limit > > > > > > Hi, All > > > > > > I try to turn off the rate limit by adding > > $SystemLogRateLimitInterval > > > 0 after $ModLoad imuxsock.so, > > > but it does not work, now I cannot get all the logs. Do you have > any > > > idea how to fix it? Thanks. > > > > > > > > > Hongqing > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com -------------- next part -------------- A non-text attachment was scrubbed... Name: logfile Type: application/octet-stream Size: 84731 bytes Desc: not available URL: From rgerhards at hq.adiscon.com Tue Apr 26 15:13:19 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 26 Apr 2011 15:13:19 +0200 Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <229393435.110184.1303798285165.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE072@GRFEXC.intern.adiscon.com> <229393435.110184.1303798285165.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE07F@GRFEXC.intern.adiscon.com> There is some mismatch in your rsyslog versions... Because the messages you cite from your log ("imuxsock begins to drop messages from pid 358 due to rate-limiting") are NOT even present in 4.6.3. It looks like you are running either a newer version as service and use an old one for the debug log OR rsyslog versions and modules are mixed up on your system. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Tuesday, April 26, 2011 8:11 AM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Rainer, > > Thanks for your reply. I think the commend you mentioned is > RepeatedMsgReduction off, am I right? > but the messages are still dropped. > > <46>2011-04-26T05:36:20.002095+00:00 localhost rsyslogd-2177: imuxsock > begins to drop messages from pid 358 due to rate-limiting > <46>2011-04-26T05:36:23.297653+00:00 localhost rsyslogd-2177: imuxsock > lost 559 messages from pid 358 due to rate-limiting > > There is no errors in the debug logs. By the way, in my situation, the > dropped messages are not repeated messages. > > Thanks. > > > > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 8:58:54 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > You should have a look at the syslog error log ;) > > 6950.906989271:7f1afcdeb720: cfline: '$SystemLogRateLimitInterval 0' > 6950.907004432:7f1afcdeb720: Called LogError, msg: invalid or yet- > unknown > config > file command - have you forgotten to load a module? > 6950.907033424:7f1afcdeb720: config line NOT successfully processed > > The error message is emitted, but probably thrown away ;) > > Anyhow, you use 4.6.3, which does not support the new type of rate > limiting > at all. So I guess what comes into your way is repated message > reduction, > which you can turn off via a separate statement (Which I don't know out > of my > head, but if you search for repated message reduction you should find > it > relatively easily :). > > HTH > > Rainer > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Monday, April 25, 2011 12:36 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Hi, Rainer, > > > > Please find the rsyslog.conf and rsyslog log file in attachments. > > Thanks. > > > > > > ----- Original Message ----- > > From: "Rainer Gerhards" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 5:55:56 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Sorry, I'Ve not been precise enough. I need a rsyslog debug log, not > > the > > logfile. See this page: > > > > http://www.rsyslog.com/doc/troubleshoot.html > > > > Note that we have a public holiday over here and I am meeting with > > family. So > > I have no lab at hand. But with the debug log, I may be able to tell > > something. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > Sent: Monday, April 25, 2011 11:27 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Sorry, failed to attach the file. Now it is fine. > > > > > > ----- Original Message ----- > > > From: "Hongqing Yang" > > > To: "rsyslog-users" > > > Sent: Monday, April 25, 2011 5:24:29 PM > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Hi, Rainer, > > > > > > Thanks for your reply, Now I try to transfer guest installation > logs > > > from KVM to host by virtio. > > > But I find the logs are different, after discussed with the > anaconda > > > team, I know I have > > > to turn ratelimit off. > > > > > > The configuration of rsyslog is below: > > > > > > #### MODULES #### > > > > > > $ModLoad imuxsock.so # provides support for local system logging > > > (e.g. via logger command) > > > $SystemLogRateLimitInterval 0 > > > $ModLoad imklog.so # provides kernel logging support > (previously > > > done by rklogd) > > > #$ModLoad immark.so # provides --MARK-- message capability > > > > > > the logs on the guest can be found in attachment. > > > > > > Thanks. > > > > > > ----- Original Message ----- > > > From: "Rainer Gerhards" > > > To: "rsyslog-users" > > > Sent: Monday, April 25, 2011 4:57:43 PM > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Can you please send a debug log. Probably it is sufficient to see > the > > > startup, but we may need a full session. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > > Sent: Monday, April 25, 2011 10:16 AM > > > > To: rsyslog at lists.adiscon.com > > > > Subject: [rsyslog] cannot turn off the rate limit > > > > > > > > Hi, All > > > > > > > > I try to turn off the rate limit by adding > > > $SystemLogRateLimitInterval > > > > 0 after $ModLoad imuxsock.so, > > > > but it does not work, now I cannot get all the logs. Do you have > > any > > > > idea how to fix it? Thanks. > > > > > > > > > > > > Hongqing > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jmergy at lwhs.org Tue Apr 26 22:32:35 2011 From: jmergy at lwhs.org (Jonathan Mergy) Date: Tue, 26 Apr 2011 13:32:35 -0700 Subject: [rsyslog] Newbie - time range directive? Message-ID: Hiya, Looking through config samples and trying to dump logs from couple of hosts but have a scenario where I am making the logs available to an external vendor for analysis and only want to give them a few hours a day (typically at peak usage.) I have the filters working and dumping to a log file and I will rotate that daily or something but really need to limit the range rsyslog uses to something like 9am to 2pm. Rsyslog will continue to get data from the hosts, but right now is writing all data it gets. Is there a decent way to limit the timeframe of the logging via a directive? Thanks. Jonathan ................................ Jonathan Mergy Director Of Technology Lick-Wilmerding High School 755 Ocean Ave, SF CA 94112 P:415.585.1725 x365 http://www.lwhs.org From jmergy at lwhs.org Wed Apr 27 06:09:48 2011 From: jmergy at lwhs.org (Jonathan Mergy) Date: Tue, 26 Apr 2011 21:09:48 -0700 Subject: [rsyslog] Newbie - time range directive? In-Reply-To: Message-ID: I am just going with stopping and starting rsyslog for now. Looked all over and unable to get something to filter to file from host during a time range. Thanks Jonathan ................................ Jonathan Mergy Director Of Technology Lick-Wilmerding High School 755 Ocean Ave, SF CA 94112 P:415.585.1725 x365 http://www.lwhs.org > From: Jonathan Mergy > Reply-To: rsyslog-users > Date: Tue, 26 Apr 2011 13:32:35 -0700 > To: > Subject: [rsyslog] Newbie - time range directive? > > Hiya, > > Looking through config samples and trying to dump logs from couple of hosts > but have a scenario where I am making the logs available to an external > vendor for analysis and only want to give them a few hours a day (typically > at peak usage.) I have the filters working and dumping to a log file and I > will rotate that daily or something but really need to limit the range > rsyslog uses to something like 9am to 2pm. Rsyslog will continue to get data > from the hosts, but right now is writing all data it gets. > > Is there a decent way to limit the timeframe of the logging via a directive? > > Thanks. > > Jonathan > ................................ > Jonathan Mergy > Director Of Technology > Lick-Wilmerding High School > 755 Ocean Ave, SF CA 94112 > P:415.585.1725 x365 > http://www.lwhs.org > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Apr 27 08:27:15 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 08:27:15 +0200 Subject: [rsyslog] Newbie - time range directive? In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE082@GRFEXC.intern.adiscon.com> I think there is no such directive, though I barely remember that once I created such a beast (but that may have been in our Windows toolset). I'd suggest to check the doc if there is a directive that sounds like if could disable an action for a given period. With filters, you could probably create a big if where you check the hour part of the time and execute the action only for hours "09", "10", ..., "20" (using 24hr clock). HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Jonathan Mergy > Sent: Wednesday, April 27, 2011 6:10 AM > To: rsyslog-users > Subject: Re: [rsyslog] Newbie - time range directive? > > I am just going with stopping and starting rsyslog for now. Looked all > over > and unable to get something to filter to file from host during a time > range. > > Thanks > > Jonathan > ................................ > Jonathan Mergy > Director Of Technology > Lick-Wilmerding High School > 755 Ocean Ave, SF CA 94112 > P:415.585.1725 x365 > http://www.lwhs.org > > > > From: Jonathan Mergy > > Reply-To: rsyslog-users > > Date: Tue, 26 Apr 2011 13:32:35 -0700 > > To: > > Subject: [rsyslog] Newbie - time range directive? > > > > Hiya, > > > > Looking through config samples and trying to dump logs from couple of > hosts > > but have a scenario where I am making the logs available to an > external > > vendor for analysis and only want to give them a few hours a day > (typically > > at peak usage.) I have the filters working and dumping to a log file > and I > > will rotate that daily or something but really need to limit the > range > > rsyslog uses to something like 9am to 2pm. Rsyslog will continue to > get data > > from the hosts, but right now is writing all data it gets. > > > > Is there a decent way to limit the timeframe of the logging via a > directive? > > > > Thanks. > > > > Jonathan > > ................................ > > Jonathan Mergy > > Director Of Technology > > Lick-Wilmerding High School > > 755 Ocean Ave, SF CA 94112 > > P:415.585.1725 x365 > > http://www.lwhs.org > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From hoyang at redhat.com Wed Apr 27 08:31:51 2011 From: hoyang at redhat.com (Hongqing Yang) Date: Wed, 27 Apr 2011 02:31:51 -0400 (EDT) Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE07F@GRFEXC.intern.adiscon.com> Message-ID: <188803413.132289.1303885911909.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Rainer, I have tested rsyslog 5.7.9 with $SystemLogRateLimitInterval 0 and $RepeatedMsgReduction off as below #### MODULES #### $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $SystemLogRateLimitInterval 0 $RepeatedMsgReduction off $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability The rsyslog template is used, but imuxsock still drops messages, please find the logs in attachments. Thanks, Hongqing ----- Original Message ----- From: "Rainer Gerhards" To: "rsyslog-users" Sent: Tuesday, April 26, 2011 9:13:19 PM Subject: Re: [rsyslog] cannot turn off the rate limit There is some mismatch in your rsyslog versions... Because the messages you cite from your log ("imuxsock begins to drop messages from pid 358 due to rate-limiting") are NOT even present in 4.6.3. It looks like you are running either a newer version as service and use an old one for the debug log OR rsyslog versions and modules are mixed up on your system. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Tuesday, April 26, 2011 8:11 AM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Rainer, > > Thanks for your reply. I think the commend you mentioned is > RepeatedMsgReduction off, am I right? > but the messages are still dropped. > > <46>2011-04-26T05:36:20.002095+00:00 localhost rsyslogd-2177: imuxsock > begins to drop messages from pid 358 due to rate-limiting > <46>2011-04-26T05:36:23.297653+00:00 localhost rsyslogd-2177: imuxsock > lost 559 messages from pid 358 due to rate-limiting > > There is no errors in the debug logs. By the way, in my situation, the > dropped messages are not repeated messages. > > Thanks. > > > > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Monday, April 25, 2011 8:58:54 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > You should have a look at the syslog error log ;) > > 6950.906989271:7f1afcdeb720: cfline: '$SystemLogRateLimitInterval 0' > 6950.907004432:7f1afcdeb720: Called LogError, msg: invalid or yet- > unknown > config > file command - have you forgotten to load a module? > 6950.907033424:7f1afcdeb720: config line NOT successfully processed > > The error message is emitted, but probably thrown away ;) > > Anyhow, you use 4.6.3, which does not support the new type of rate > limiting > at all. So I guess what comes into your way is repated message > reduction, > which you can turn off via a separate statement (Which I don't know out > of my > head, but if you search for repated message reduction you should find > it > relatively easily :). > > HTH > > Rainer > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Monday, April 25, 2011 12:36 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Hi, Rainer, > > > > Please find the rsyslog.conf and rsyslog log file in attachments. > > Thanks. > > > > > > ----- Original Message ----- > > From: "Rainer Gerhards" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 5:55:56 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Sorry, I'Ve not been precise enough. I need a rsyslog debug log, not > > the > > logfile. See this page: > > > > http://www.rsyslog.com/doc/troubleshoot.html > > > > Note that we have a public holiday over here and I am meeting with > > family. So > > I have no lab at hand. But with the debug log, I may be able to tell > > something. > > > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > Sent: Monday, April 25, 2011 11:27 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Sorry, failed to attach the file. Now it is fine. > > > > > > ----- Original Message ----- > > > From: "Hongqing Yang" > > > To: "rsyslog-users" > > > Sent: Monday, April 25, 2011 5:24:29 PM > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Hi, Rainer, > > > > > > Thanks for your reply, Now I try to transfer guest installation > logs > > > from KVM to host by virtio. > > > But I find the logs are different, after discussed with the > anaconda > > > team, I know I have > > > to turn ratelimit off. > > > > > > The configuration of rsyslog is below: > > > > > > #### MODULES #### > > > > > > $ModLoad imuxsock.so # provides support for local system logging > > > (e.g. via logger command) > > > $SystemLogRateLimitInterval 0 > > > $ModLoad imklog.so # provides kernel logging support > (previously > > > done by rklogd) > > > #$ModLoad immark.so # provides --MARK-- message capability > > > > > > the logs on the guest can be found in attachment. > > > > > > Thanks. > > > > > > ----- Original Message ----- > > > From: "Rainer Gerhards" > > > To: "rsyslog-users" > > > Sent: Monday, April 25, 2011 4:57:43 PM > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Can you please send a debug log. Probably it is sufficient to see > the > > > startup, but we may need a full session. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > > Sent: Monday, April 25, 2011 10:16 AM > > > > To: rsyslog at lists.adiscon.com > > > > Subject: [rsyslog] cannot turn off the rate limit > > > > > > > > Hi, All > > > > > > > > I try to turn off the rate limit by adding > > > $SystemLogRateLimitInterval > > > > 0 after $ModLoad imuxsock.so, > > > > but it does not work, now I cannot get all the logs. Do you have > > any > > > > idea how to fix it? Thanks. > > > > > > > > > > > > Hongqing > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com -------------- next part -------------- A non-text attachment was scrubbed... Name: analog Type: text/x-python Size: 8131 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: debug_unknown_source.log Type: text/x-log Size: 251 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: logfile Type: application/octet-stream Size: 210394 bytes Desc: not available URL: From rgerhards at hq.adiscon.com Wed Apr 27 10:12:22 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 10:12:22 +0200 Subject: [rsyslog] cannot turn off the rate limit In-Reply-To: <188803413.132289.1303885911909.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE07F@GRFEXC.intern.adiscon.com> <188803413.132289.1303885911909.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE086@GRFEXC.intern.adiscon.com> Can you please give this fix a try (details in description): http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=24b62834801043bf2d9560b0 cca3967e5762bea1 This will be released as part of 5.8.1. Please provide feedback. Thanks, Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > Sent: Wednesday, April 27, 2011 8:32 AM > To: rsyslog-users > Subject: Re: [rsyslog] cannot turn off the rate limit > > Rainer, > > I have tested rsyslog 5.7.9 with $SystemLogRateLimitInterval 0 and > $RepeatedMsgReduction off as below > > #### MODULES #### > > $ModLoad imuxsock # provides support for local system logging (e.g. via > logger command) > $SystemLogRateLimitInterval 0 > $RepeatedMsgReduction off > $ModLoad imklog # provides kernel logging support (previously done by > rklogd) > #$ModLoad immark # provides --MARK-- message capability > > The rsyslog template is used, but imuxsock still drops messages, please > find the logs in attachments. > > Thanks, > Hongqing > > > > > > ----- Original Message ----- > From: "Rainer Gerhards" > To: "rsyslog-users" > Sent: Tuesday, April 26, 2011 9:13:19 PM > Subject: Re: [rsyslog] cannot turn off the rate limit > > There is some mismatch in your rsyslog versions... Because the messages > you > cite from your log ("imuxsock begins to drop messages from pid 358 due > to > rate-limiting") are NOT even present in 4.6.3. It looks like you are > running > either a newer version as service and use an old one for the debug log > OR > rsyslog versions and modules are mixed up on your system. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > Sent: Tuesday, April 26, 2011 8:11 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > Rainer, > > > > Thanks for your reply. I think the commend you mentioned is > > RepeatedMsgReduction off, am I right? > > but the messages are still dropped. > > > > <46>2011-04-26T05:36:20.002095+00:00 localhost rsyslogd-2177: > imuxsock > > begins to drop messages from pid 358 due to rate-limiting > > <46>2011-04-26T05:36:23.297653+00:00 localhost rsyslogd-2177: > imuxsock > > lost 559 messages from pid 358 due to rate-limiting > > > > There is no errors in the debug logs. By the way, in my situation, > the > > dropped messages are not repeated messages. > > > > Thanks. > > > > > > > > > > ----- Original Message ----- > > From: "Rainer Gerhards" > > To: "rsyslog-users" > > Sent: Monday, April 25, 2011 8:58:54 PM > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > You should have a look at the syslog error log ;) > > > > 6950.906989271:7f1afcdeb720: cfline: '$SystemLogRateLimitInterval 0' > > 6950.907004432:7f1afcdeb720: Called LogError, msg: invalid or yet- > > unknown > > config > > file command - have you forgotten to load a module? > > 6950.907033424:7f1afcdeb720: config line NOT successfully processed > > > > The error message is emitted, but probably thrown away ;) > > > > Anyhow, you use 4.6.3, which does not support the new type of rate > > limiting > > at all. So I guess what comes into your way is repated message > > reduction, > > which you can turn off via a separate statement (Which I don't know > out > > of my > > head, but if you search for repated message reduction you should find > > it > > relatively easily :). > > > > HTH > > > > Rainer > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > Sent: Monday, April 25, 2011 12:36 PM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Hi, Rainer, > > > > > > Please find the rsyslog.conf and rsyslog log file in attachments. > > > Thanks. > > > > > > > > > ----- Original Message ----- > > > From: "Rainer Gerhards" > > > To: "rsyslog-users" > > > Sent: Monday, April 25, 2011 5:55:56 PM > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > Sorry, I'Ve not been precise enough. I need a rsyslog debug log, > not > > > the > > > logfile. See this page: > > > > > > http://www.rsyslog.com/doc/troubleshoot.html > > > > > > Note that we have a public holiday over here and I am meeting with > > > family. So > > > I have no lab at hand. But with the debug log, I may be able to > tell > > > something. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > > Sent: Monday, April 25, 2011 11:27 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > > > Sorry, failed to attach the file. Now it is fine. > > > > > > > > ----- Original Message ----- > > > > From: "Hongqing Yang" > > > > To: "rsyslog-users" > > > > Sent: Monday, April 25, 2011 5:24:29 PM > > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > > > Hi, Rainer, > > > > > > > > Thanks for your reply, Now I try to transfer guest installation > > logs > > > > from KVM to host by virtio. > > > > But I find the logs are different, after discussed with the > > anaconda > > > > team, I know I have > > > > to turn ratelimit off. > > > > > > > > The configuration of rsyslog is below: > > > > > > > > #### MODULES #### > > > > > > > > $ModLoad imuxsock.so # provides support for local system > logging > > > > (e.g. via logger command) > > > > $SystemLogRateLimitInterval 0 > > > > $ModLoad imklog.so # provides kernel logging support > > (previously > > > > done by rklogd) > > > > #$ModLoad immark.so # provides --MARK-- message capability > > > > > > > > the logs on the guest can be found in attachment. > > > > > > > > Thanks. > > > > > > > > ----- Original Message ----- > > > > From: "Rainer Gerhards" > > > > To: "rsyslog-users" > > > > Sent: Monday, April 25, 2011 4:57:43 PM > > > > Subject: Re: [rsyslog] cannot turn off the rate limit > > > > > > > > Can you please send a debug log. Probably it is sufficient to see > > the > > > > startup, but we may need a full session. > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Hongqing Yang > > > > > Sent: Monday, April 25, 2011 10:16 AM > > > > > To: rsyslog at lists.adiscon.com > > > > > Subject: [rsyslog] cannot turn off the rate limit > > > > > > > > > > Hi, All > > > > > > > > > > I try to turn off the rate limit by adding > > > > $SystemLogRateLimitInterval > > > > > 0 after $ModLoad imuxsock.so, > > > > > but it does not work, now I cannot get all the logs. Do you > have > > > any > > > > > idea how to fix it? Thanks. > > > > > > > > > > > > > > > Hongqing > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Apr 27 12:16:48 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 12:16:48 +0200 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> Hi all, without any pressing need, I'd like to share with you some thoughts of a potential future rsyslog multi-config system: http://blog.gerhards.net/2011/04/rsyslog-config-reload-random-thoughts.html This is NOT something I intend to fully implement soon, but I am working a bit on paving the way. Rainer From rgerhards at hq.adiscon.com Wed Apr 27 13:40:46 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 13:40:46 +0200 Subject: [rsyslog] Feedback requested: legacy options support Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE090@GRFEXC.intern.adiscon.com> Hi all, via the compatibility mode option (-c), rsyslog still supports legacy options (like -t to start a tcp listener). This code complicates a couple of things quite a bit, especially in regard to the config system. I am very tempted to drop support for legacy options in v6. That could lead to smaller and simpler code. Also, I think it is acceptable that someone running v6 finally moves away from the sysklogd/rsyslog v1 style of configuration via command line options. I also noticed that the average user seems to have problems identifying where each distro places the actual call to rsyslogd, so users seem to prefer configuring all options inside the main configuration file (what I tend to think to be more useful as well). Does anyone has a good argument why to retain the legacy support in v6? If so, please make yourself heard, because otherwise I'll probably drop that support. Rainer From david at lang.hm Wed Apr 27 20:44:07 2011 From: david at lang.hm (david at lang.hm) Date: Wed, 27 Apr 2011 11:44:07 -0700 (PDT) Subject: [rsyslog] Feedback requested: legacy options support In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE090@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE090@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 27 Apr 2011, Rainer Gerhards wrote: > Hi all, > > via the compatibility mode option (-c), rsyslog still supports legacy options > (like -t to start a tcp listener). This code complicates a couple of things > quite a bit, especially in regard to the config system. > > I am very tempted to drop support for legacy options in v6. That could lead > to smaller and simpler code. Also, I think it is acceptable that someone > running v6 finally moves away from the sysklogd/rsyslog v1 style of > configuration via command line options. > > I also noticed that the average user seems to have problems identifying where > each distro places the actual call to rsyslogd, so users seem to prefer > configuring all options inside the main configuration file (what I tend to > think to be more useful as well). > > Does anyone has a good argument why to retain the legacy support in v6? If > so, please make yourself heard, because otherwise I'll probably drop that > support. Distros tend to make it hard to fiddle with command-line parameters to daemons, so I don't think there's a lot of reason to keep the old options. there are some options that are relavent to use that do not have a config equivalent (-x -4 to name just a couple), you may want to try and make sure that there is a config version of all options (except for specifying where the config file is ;-) and then consider dropping all of them eventually. there will need to be some time for migration, so these extra config options will need to get backported to v5 if you want to eliminate the command-line options in v6 David Lang From david at lang.hm Wed Apr 27 21:04:58 2011 From: david at lang.hm (david at lang.hm) Date: Wed, 27 Apr 2011 12:04:58 -0700 (PDT) Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 27 Apr 2011, Rainer Gerhards wrote: > Hi all, > > without any pressing need, I'd like to share with you some thoughts of a > potential future rsyslog multi-config system: > > http://blog.gerhards.net/2011/04/rsyslog-config-reload-random-thoughts.html > > This is NOT something I intend to fully implement soon, but I am working a > bit on paving the way. my first thought (while still reading the post) is that you really shouldn't need to load the modules to parse the config file into a memory structure. you will need to load the module to _validate_ the config (to see if all the config items are valid and contain valid data types) but the config language should be such that the structure of the config and the process of parsing the config options into a memory structure should be a separate step before doing the validation. I think this should significantly simplify your task compared to doing the validation and parsing at the same time. among the other benifits, this lets you have one validation engine, even if you support multiple config languages (old + new for example) David Lang From rgerhards at hq.adiscon.com Wed Apr 27 21:19:32 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 21:19:32 +0200 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> Hi David, thanks for the feedback. But I don't think I can avoid loading modules if I will support the current config format. And this is something I definitely want to do. Other than that, you are of course right that I can use name-value pairs to avoid loading modules in the first step. However, validation is required. Unforutnately I did not explicitely spell out the validation step. But I think it is a necessary step to be done right after loading a config and before it turns into a real candidate config. So even in this PoV, loading modules is necessary for validation and thus for the config load process. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, April 27, 2011 9:05 PM > To: rsyslog-users > Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > > On Wed, 27 Apr 2011, Rainer Gerhards wrote: > > > Hi all, > > > > without any pressing need, I'd like to share with you some thoughts > of a > > potential future rsyslog multi-config system: > > > > http://blog.gerhards.net/2011/04/rsyslog-config-reload-random- > thoughts.html > > > > This is NOT something I intend to fully implement soon, but I am > working a > > bit on paving the way. > > my first thought (while still reading the post) is that you really > shouldn't need to load the modules to parse the config file into a > memory > structure. > > you will need to load the module to _validate_ the config (to see if > all > the config items are valid and contain valid data types) > > but the config language should be such that the structure of the config > and the process of parsing the config options into a memory structure > should be a separate step before doing the validation. > > I think this should significantly simplify your task compared to doing > the > validation and parsing at the same time. > > among the other benifits, this lets you have one validation engine, > even > if you support multiple config languages (old + new for example) > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Apr 27 21:29:52 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 21:29:52 +0200 Subject: [rsyslog] Feedback requested: legacy options support In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE090@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE09C@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, April 27, 2011 8:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] Feedback requested: legacy options support > > On Wed, 27 Apr 2011, Rainer Gerhards wrote: > > > Hi all, > > > > via the compatibility mode option (-c), rsyslog still supports legacy > options > > (like -t to start a tcp listener). This code complicates a couple of > things > > quite a bit, especially in regard to the config system. > > > > I am very tempted to drop support for legacy options in v6. That > could lead > > to smaller and simpler code. Also, I think it is acceptable that > someone > > running v6 finally moves away from the sysklogd/rsyslog v1 style of > > configuration via command line options. > > > > I also noticed that the average user seems to have problems > identifying where > > each distro places the actual call to rsyslogd, so users seem to > prefer > > configuring all options inside the main configuration file (what I > tend to > > think to be more useful as well). > > > > Does anyone has a good argument why to retain the legacy support in > v6? If > > so, please make yourself heard, because otherwise I'll probably drop > that > > support. > > Distros tend to make it hard to fiddle with command-line parameters to > daemons, so I don't think there's a lot of reason to keep the old > options. > > there are some options that are relavent to use that do not have a > config > equivalent (-x -4 to name just a couple), you may want to try and make > sure that there is a config version of all options (except for > specifying > where the config file is ;-) and then consider dropping all of them > eventually. there will need to be some time for migration, so these > extra > config options will need to get backported to v5 if you want to > eliminate > the command-line options in v6 This makes a lot of sense and sounds like a good plan-- thanks :) Rainer > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Wed Apr 27 21:32:18 2011 From: david at lang.hm (david at lang.hm) Date: Wed, 27 Apr 2011 12:32:18 -0700 (PDT) Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 27 Apr 2011, Rainer Gerhards wrote: > Hi David, > > thanks for the feedback. But I don't think I can avoid loading modules if I > will support the current config format. And this is something I definitely > want to do. > > Other than that, you are of course right that I can use name-value pairs to > avoid loading modules in the first step. However, validation is required. > Unforutnately I did not explicitely spell out the validation step. But I > think it is a necessary step to be done right after loading a config and > before it turns into a real candidate config. So even in this PoV, loading > modules is necessary for validation and thus for the config load process. Ok, I was thinking that the validation could be delayed until very late in the process, basically at the point where you are considering switching to the config. you are spending a lot of effort in worrying about loading and unloading modules. what is the harm in leaving additional modules loaded? yes, they will eat a little memory (but generally not a lot), but will they actually have any impact on the running system? If not, I wouldn't worry about trying to unload them (and all the tracking that requires) As for the problem of multiple module paths, I would tend to say that if you are changing things to that extent, you really should restart rsyslog. that's like changing the rsyslog binary and expecting the system to cope with it in flight. one other headache that you don't mention is the problem of changing queue types while there is data in the queue. David Lang > Rainer >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, April 27, 2011 9:05 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system >> >> On Wed, 27 Apr 2011, Rainer Gerhards wrote: >> >>> Hi all, >>> >>> without any pressing need, I'd like to share with you some thoughts >> of a >>> potential future rsyslog multi-config system: >>> >>> http://blog.gerhards.net/2011/04/rsyslog-config-reload-random- >> thoughts.html >>> >>> This is NOT something I intend to fully implement soon, but I am >> working a >>> bit on paving the way. >> >> my first thought (while still reading the post) is that you really >> shouldn't need to load the modules to parse the config file into a >> memory >> structure. >> >> you will need to load the module to _validate_ the config (to see if >> all >> the config items are valid and contain valid data types) >> >> but the config language should be such that the structure of the config >> and the process of parsing the config options into a memory structure >> should be a separate step before doing the validation. >> >> I think this should significantly simplify your task compared to doing >> the >> validation and parsing at the same time. >> >> among the other benifits, this lets you have one validation engine, >> even >> if you support multiple config languages (old + new for example) >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Apr 27 21:45:39 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 27 Apr 2011 21:45:39 +0200 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, April 27, 2011 9:32 PM > To: rsyslog-users > Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > > On Wed, 27 Apr 2011, Rainer Gerhards wrote: > > > Hi David, > > > > thanks for the feedback. But I don't think I can avoid loading > modules if I > > will support the current config format. And this is something I > definitely > > want to do. > > > > Other than that, you are of course right that I can use name-value > pairs to > > avoid loading modules in the first step. However, validation is > required. > > Unforutnately I did not explicitely spell out the validation step. > But I > > think it is a necessary step to be done right after loading a config > and > > before it turns into a real candidate config. So even in this PoV, > loading > > modules is necessary for validation and thus for the config load > process. > > Ok, I was thinking that the validation could be delayed until very late > in > the process, basically at the point where you are considering switching > to > the config. That's of course debatable, but I think it is useful to do the validation as part of loading (so errors appear close to what caused them). > > you are spending a lot of effort in worrying about loading and > unloading > modules. > > what is the harm in leaving additional modules loaded? yes, they will > eat > a little memory (but generally not a lot), but will they actually have > any > impact on the running system? If not, I wouldn't worry about trying to > unload them (and all the tracking that requires) Actually, loading and unloading is not that bad -- the current system already handles that. I may have been a bit too specific on these topics. With that writing, I really wanted to make my mind clear on the issues that can occur. If you have a look at git, I am currently working on the config system. My goal currently is not necessarily to create a new system at that stage, but I would like to clean up the system so that it becomes simpler to switch to a new config language AND have a more structured config load process (which in the long term can pave the way to a delta-based config reactivation). I am sorry when this was not clear enough from the post. Actually, I started with a very, very blurry picture of how to handle moduls, but things considerably cleared up during writing. I have even been able to implement some of the thoughts during the afternoon. Oh, yes: one goal of the rewrite is finally to do a clean privilege drop. That is actually a goal that I want to achive. The bad thing is that to reach it, a lot of code needs to be moved between places and rewritten. > > As for the problem of multiple module paths, I would tend to say that > if > you are changing things to that extent, you really should restart > rsyslog. > that's like changing the rsyslog binary and expecting the system to > cope > with it in flight. > This is a very useful comment: I begin to think that there are some things that simply cannot be changed on the fly. One thing where this ultimately is the case is if privileges are dropped, which cannot be undone by design. The module path proably is another. > > > one other headache that you don't mention is the problem of changing > queue > types while there is data in the queue. > Yes, I did not elaborate on the delta-change process. There is much more to consider. Just think about identifying what actually has changed. That's far from trivial (think about a new rule added in between, and targeting the same file like the previous rule at this place but with a different filter). However, the delta-change process is probably the longest term goal. I would be really happy if I could use a better config language some time this year. Thanks again for the good feedback, much appreciated. It often requires discussion to gain good solutions ;) Rainer > David Lang > > > Rainer > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >> Sent: Wednesday, April 27, 2011 9:05 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > >> > >> On Wed, 27 Apr 2011, Rainer Gerhards wrote: > >> > >>> Hi all, > >>> > >>> without any pressing need, I'd like to share with you some thoughts > >> of a > >>> potential future rsyslog multi-config system: > >>> > >>> http://blog.gerhards.net/2011/04/rsyslog-config-reload-random- > >> thoughts.html > >>> > >>> This is NOT something I intend to fully implement soon, but I am > >> working a > >>> bit on paving the way. > >> > >> my first thought (while still reading the post) is that you really > >> shouldn't need to load the modules to parse the config file into a > >> memory > >> structure. > >> > >> you will need to load the module to _validate_ the config (to see if > >> all > >> the config items are valid and contain valid data types) > >> > >> but the config language should be such that the structure of the > config > >> and the process of parsing the config options into a memory > structure > >> should be a separate step before doing the validation. > >> > >> I think this should significantly simplify your task compared to > doing > >> the > >> validation and parsing at the same time. > >> > >> among the other benifits, this lets you have one validation engine, > >> even > >> if you support multiple config languages (old + new for example) > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From piavka at cs.bgu.ac.il Thu Apr 28 01:26:43 2011 From: piavka at cs.bgu.ac.il (Piavlo) Date: Thu, 28 Apr 2011 02:26:43 +0300 Subject: [rsyslog] problems with using omhdfs module Message-ID: <4DB8A633.3000101@cs.bgu.ac.il> Hi, I'm tryin to send logs from rsyslog server to remote hdfs cluster The rsyslog server is compiled with omhdfs support with following CDH3 rpms installed root at dev1 ~]# rpm -qa | grep hadoop hadoop-0.20-libhdfs-0.20.2+923.21-1 hadoop-0.20-source-0.20.2+923.21-1 hadoop-0.20-native-0.20.2+923.21-1 hadoop-0.20-0.20.2+923.21-1 [root at dev1 ~]# trying the following ---------------- $ModLoad omhdfs $OMHDFSHost hadoop-master1.internal $OMHDFSPort 8020 $OMHDFSFileName /var/log/logfile local3.info :omhdfs: ---------------- i get the following error 2011-04-27T23:19:06.335954+00:00 dev1 rsyslogd-2179: omhdfs: failed to open /var/log/logfile - retrying later [try http://www.rsyslog.com/e/2179 ] I fail to understand the meaning of $OMHDFSFileName directive does this refers to file which will/should be created withing the hdfs filesystem? In core-site.xml I have fs.default.name hdfs://hadoop-master1a.internal:8020 so I've set the following $OMHDFSHost hadoop-master1.internal $OMHDFSPort 8020 What am I doing wrong? Thanks Alex From rgerhards at hq.adiscon.com Thu Apr 28 13:02:59 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 28 Apr 2011 13:02:59 +0200 Subject: [rsyslog] problems with using omhdfs module In-Reply-To: <4DB8A633.3000101@cs.bgu.ac.il> References: <4DB8A633.3000101@cs.bgu.ac.il> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE0A3@GRFEXC.intern.adiscon.com> My HDFS knowledge is unfortunately rather limited. I just know what was required to write the module. Most importantly, I do not know the exact setup and config files for HDFS. But... [inline] > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Piavlo > Sent: Thursday, April 28, 2011 1:27 AM > To: rsyslog-users > Subject: [rsyslog] problems with using omhdfs module > > Hi, > > I'm tryin to send logs from rsyslog server to remote hdfs cluster > The rsyslog server is compiled with omhdfs support with following CDH3 > rpms installed > > root at dev1 ~]# rpm -qa | grep hadoop > hadoop-0.20-libhdfs-0.20.2+923.21-1 > hadoop-0.20-source-0.20.2+923.21-1 > hadoop-0.20-native-0.20.2+923.21-1 > hadoop-0.20-0.20.2+923.21-1 > [root at dev1 ~]# > > trying the following > ---------------- > $ModLoad omhdfs > > $OMHDFSHost hadoop-master1.internal > $OMHDFSPort 8020 > $OMHDFSFileName /var/log/logfile > local3.info :omhdfs: > ---------------- > > i get the following error > > 2011-04-27T23:19:06.335954+00:00 dev1 rsyslogd-2179: omhdfs: failed to > open /var/log/logfile - retrying later [try > http://www.rsyslog.com/e/2179 ] > > I fail to understand the meaning of $OMHDFSFileName directive > does this refers to file which will/should be created withing the hdfs > filesystem? ... this I can say for sure: yes, it is. There needs to be a way to specify the actual file name to write to. I *guess* that the open failure may be related to a permission problem. > > In core-site.xml I have > > fs.default.name > hdfs://hadoop-master1a.internal:8020 > > so I've set the following > $OMHDFSHost hadoop-master1.internal > $OMHDFSPort 8020 Not konwing exactly what the Hadoop config does, this at least looks plausible. Does that help? Rainer > > What am I doing wrong? > > Thanks > Alex > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Thu Apr 28 18:11:19 2011 From: david at lang.hm (david at lang.hm) Date: Thu, 28 Apr 2011 09:11:19 -0700 (PDT) Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> Message-ID: a couple of additional thoughts 1. privilage drop will also cause problems with binding to listening ports (you can't listen to port 514 if you aren't root) 2. a wierd, out of the box thought (in the 90% as good for 20% effort category) rather than tring to modify the running config, how about forking the main thread, having the old main thread close the inputs, and the new main thread 'shutdown' all it's other threads, then the new main thread can treat this as something very close to a normal startup (basically modulo privilage drop issue) you could either let the old process/threads run until they finish delivering all their messages, or you could have them deliver the messages to the new main thread (the former is probably the easiest, the question is are there more corner cases with both sets of threads outputting to the same destinations, or with potentially mishandling messages that were already processed in the old copy that would have been processed differently in the new copy) David Lang On Wed, 27 Apr 2011, Rainer Gerhards wrote: > Date: Wed, 27 Apr 2011 21:45:39 +0200 > From: Rainer Gerhards > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, April 27, 2011 9:32 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system >> >> On Wed, 27 Apr 2011, Rainer Gerhards wrote: >> >>> Hi David, >>> >>> thanks for the feedback. But I don't think I can avoid loading >> modules if I >>> will support the current config format. And this is something I >> definitely >>> want to do. >>> >>> Other than that, you are of course right that I can use name-value >> pairs to >>> avoid loading modules in the first step. However, validation is >> required. >>> Unforutnately I did not explicitely spell out the validation step. >> But I >>> think it is a necessary step to be done right after loading a config >> and >>> before it turns into a real candidate config. So even in this PoV, >> loading >>> modules is necessary for validation and thus for the config load >> process. >> >> Ok, I was thinking that the validation could be delayed until very late >> in >> the process, basically at the point where you are considering switching >> to >> the config. > > That's of course debatable, but I think it is useful to do the validation as > part of loading (so errors appear close to what caused them). > >> >> you are spending a lot of effort in worrying about loading and >> unloading >> modules. >> >> what is the harm in leaving additional modules loaded? yes, they will >> eat >> a little memory (but generally not a lot), but will they actually have >> any >> impact on the running system? If not, I wouldn't worry about trying to >> unload them (and all the tracking that requires) > > Actually, loading and unloading is not that bad -- the current system already > handles that. I may have been a bit too specific on these topics. With that > writing, I really wanted to make my mind clear on the issues that can occur. > > If you have a look at git, I am currently working on the config system. My > goal currently is not necessarily to create a new system at that stage, but I > would like to clean up the system so that it becomes simpler to switch to a > new config language AND have a more structured config load process (which in > the long term can pave the way to a delta-based config reactivation). I am > sorry when this was not clear enough from the post. Actually, I started with > a very, very blurry picture of how to handle moduls, but things considerably > cleared up during writing. I have even been able to implement some of the > thoughts during the afternoon. > > Oh, yes: one goal of the rewrite is finally to do a clean privilege drop. > That is actually a goal that I want to achive. The bad thing is that to reach > it, a lot of code needs to be moved between places and rewritten. >> >> As for the problem of multiple module paths, I would tend to say that >> if >> you are changing things to that extent, you really should restart >> rsyslog. >> that's like changing the rsyslog binary and expecting the system to >> cope >> with it in flight. >> > This is a very useful comment: I begin to think that there are some things > that simply cannot be changed on the fly. One thing where this ultimately is > the case is if privileges are dropped, which cannot be undone by design. The > module path proably is another. >> >> >> one other headache that you don't mention is the problem of changing >> queue >> types while there is data in the queue. >> > Yes, I did not elaborate on the delta-change process. There is much more to > consider. Just think about identifying what actually has changed. That's far > from trivial (think about a new rule added in between, and targeting the same > file like the previous rule at this place but with a different filter). > However, the delta-change process is probably the longest term goal. I would > be really happy if I could use a better config language some time this year. > > Thanks again for the good feedback, much appreciated. It often requires > discussion to gain good solutions ;) > > Rainer >> David Lang >> >>> Rainer >>>> -----Original Message----- >>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>>> Sent: Wednesday, April 27, 2011 9:05 PM >>>> To: rsyslog-users >>>> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system >>>> >>>> On Wed, 27 Apr 2011, Rainer Gerhards wrote: >>>> >>>>> Hi all, >>>>> >>>>> without any pressing need, I'd like to share with you some thoughts >>>> of a >>>>> potential future rsyslog multi-config system: >>>>> >>>>> http://blog.gerhards.net/2011/04/rsyslog-config-reload-random- >>>> thoughts.html >>>>> >>>>> This is NOT something I intend to fully implement soon, but I am >>>> working a >>>>> bit on paving the way. >>>> >>>> my first thought (while still reading the post) is that you really >>>> shouldn't need to load the modules to parse the config file into a >>>> memory >>>> structure. >>>> >>>> you will need to load the module to _validate_ the config (to see if >>>> all >>>> the config items are valid and contain valid data types) >>>> >>>> but the config language should be such that the structure of the >> config >>>> and the process of parsing the config options into a memory >> structure >>>> should be a separate step before doing the validation. >>>> >>>> I think this should significantly simplify your task compared to >> doing >>>> the >>>> validation and parsing at the same time. >>>> >>>> among the other benifits, this lets you have one validation engine, >>>> even >>>> if you support multiple config languages (old + new for example) >>>> >>>> David Lang >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From epiphani at gmail.com Thu Apr 28 18:28:13 2011 From: epiphani at gmail.com (Aaron Wiebe) Date: Thu, 28 Apr 2011 12:28:13 -0400 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> Message-ID: On Thu, Apr 28, 2011 at 12:11 PM, wrote: > > 2. a wierd, out of the box thought (in the 90% as good for 20% effort > category) > > rather than tring to modify the running config, how about forking the main > thread, having the old main thread close the inputs, and the new main thread > 'shutdown' all it's other threads, then the new main thread can treat this > as something very close to a normal startup (basically modulo privilage drop > issue) > > you could either let the old process/threads run until they finish > delivering all their messages, or you could have them deliver the messages > to the new main thread (the former is probably the easiest, the question is > are there more corner cases with both sets of threads outputting to the same > destinations, or with potentially mishandling messages that were already > processed in the old copy that would have been processed differently in the > new copy) I've tried to do this with other software - it's a PAIN. Effectively you have to transfer the memory state between threads/processes. It's not easy. Honestly, config reload is less error prone. -Aaron From david at lang.hm Thu Apr 28 20:16:36 2011 From: david at lang.hm (david at lang.hm) Date: Thu, 28 Apr 2011 11:16:36 -0700 (PDT) Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> Message-ID: On Thu, 28 Apr 2011, Aaron Wiebe wrote: > On Thu, Apr 28, 2011 at 12:11 PM, wrote: >> >> 2. a wierd, out of the box thought (in the 90% as good for 20% effort >> category) >> >> rather than tring to modify the running config, how about forking the main >> thread, having the old main thread close the inputs, and the new main thread >> 'shutdown' all it's other threads, then the new main thread can treat this >> as something very close to a normal startup (basically modulo privilage drop >> issue) >> >> you could either let the old process/threads run until they finish >> delivering all their messages, or you could have them deliver the messages >> to the new main thread (the former is probably the easiest, the question is >> are there more corner cases with both sets of threads outputting to the same >> destinations, or with potentially mishandling messages that were already >> processed in the old copy that would have been processed differently in the >> new copy) > > I've tried to do this with other software - it's a PAIN. Effectively > you have to transfer the memory state between threads/processes. It's > not easy. > > Honestly, config reload is less error prone. my point is that insead of figuring out what effect all the loaded modules have, how to apply all the modifiers in order, etc. just throw out all of the existing config and start the new config clean from scratch, but do it in a separate process so that the existing one can keep processing the logs it's already received. David Lang From rgerhards at hq.adiscon.com Thu Apr 28 20:20:33 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 28 Apr 2011 20:20:33 +0200 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Thursday, April 28, 2011 8:17 PM > To: rsyslog-users > Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > > On Thu, 28 Apr 2011, Aaron Wiebe wrote: > > > On Thu, Apr 28, 2011 at 12:11 PM, wrote: > >> > >> 2. a wierd, out of the box thought (in the 90% as good for 20% > effort > >> category) > >> > >> rather than tring to modify the running config, how about forking > the main > >> thread, having the old main thread close the inputs, and the new > main thread > >> 'shutdown' all it's other threads, then the new main thread can > treat this > >> as something very close to a normal startup (basically modulo > privilage drop > >> issue) > >> > >> you could either let the old process/threads run until they finish > >> delivering all their messages, or you could have them deliver the > messages > >> to the new main thread (the former is probably the easiest, the > question is > >> are there more corner cases with both sets of threads outputting to > the same > >> destinations, or with potentially mishandling messages that were > already > >> processed in the old copy that would have been processed differently > in the > >> new copy) > > > > I've tried to do this with other software - it's a PAIN. Effectively > > you have to transfer the memory state between threads/processes. > It's > > not easy. > > > > Honestly, config reload is less error prone. > > my point is that insead of figuring out what effect all the loaded > modules > have, how to apply all the modifiers in order, etc. just throw out all > of > the existing config and start the new config clean from scratch, but do > it > in a separate process so that the existing one can keep processing the > logs it's already received. That bad thing is that this doesn't work for existing TCP connections. Of course you can start processing new connections with the "new" instance, but that means that new data coming in via old connections - even hours later - would still be processed by the old config. Rainer From david at lang.hm Thu Apr 28 20:30:50 2011 From: david at lang.hm (david at lang.hm) Date: Thu, 28 Apr 2011 11:30:50 -0700 (PDT) Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> Message-ID: On Thu, 28 Apr 2011, Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> >> my point is that insead of figuring out what effect all the loaded >> modules have, how to apply all the modifiers in order, etc. just throw >> out all of the existing config and start the new config clean from >> scratch, but do it in a separate process so that the existing one can >> keep processing the logs it's already received. > > That bad thing is that this doesn't work for existing TCP connections. Of > course you can start processing new connections with the "new" instance, but > that means that new data coming in via old connections - even hours later - > would still be processed by the old config. wouldn't a fork duplicate all the file descripters? then the old copy would close it's copy of the file descripters and the new copy would start using them (you would have to time this transition for when the old copy has finished reading a message so you don't make the transition in the middle of a message) David Lang From epiphani at gmail.com Thu Apr 28 20:32:15 2011 From: epiphani at gmail.com (Aaron Wiebe) Date: Thu, 28 Apr 2011 14:32:15 -0400 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> Message-ID: This is the process I've used in the past: 1. Read the file into memory. 2. First pass parse and check for config validity. 3. Repeat 1 & 2 for included/referenced files. 4. Replace or merge new config after all config checks are complete 5. Clean up old config options that have not been validated/merged from new config -Aaron On Thu, Apr 28, 2011 at 2:20 PM, Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Thursday, April 28, 2011 8:17 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system >> >> On Thu, 28 Apr 2011, Aaron Wiebe wrote: >> >> > On Thu, Apr 28, 2011 at 12:11 PM, ? wrote: >> >> >> >> 2. a wierd, out of the box thought (in the 90% as good for 20% >> effort >> >> category) >> >> >> >> rather than tring to modify the running config, how about forking >> the main >> >> thread, having the old main thread close the inputs, and the new >> main thread >> >> 'shutdown' all it's other threads, then the new main thread can >> treat this >> >> as something very close to a normal startup (basically modulo >> privilage drop >> >> issue) >> >> >> >> you could either let the old process/threads run until they finish >> >> delivering all their messages, or you could have them deliver the >> messages >> >> to the new main thread (the former is probably the easiest, the >> question is >> >> are there more corner cases with both sets of threads outputting to >> the same >> >> destinations, or with potentially mishandling messages that were >> already >> >> processed in the old copy that would have been processed differently >> in the >> >> new copy) >> > >> > I've tried to do this with other software - it's a PAIN. ?Effectively >> > you have to transfer the memory state between threads/processes. >> It's >> > not easy. >> > >> > Honestly, config reload is less error prone. >> >> my point is that insead of figuring out what effect all the loaded >> modules >> have, how to apply all the modifiers in order, etc. just throw out all >> of >> the existing config and start the new config clean from scratch, but do >> it >> in a separate process so that the existing one can keep processing the >> logs it's already received. > > That bad thing is that this doesn't work for existing TCP connections. Of > course you can start processing new connections with the "new" instance, but > that means that new data coming in via old connections - even hours later - > would still be processed by the old config. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Thu Apr 28 20:40:31 2011 From: david at lang.hm (david at lang.hm) Date: Thu, 28 Apr 2011 11:40:31 -0700 (PDT) Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> Message-ID: the headache is that currently on rsyslog, it's not a matter of reading the config and then implementing it. it's a matter of read a line of the config, implement that line, read the next line, implement that line, etc each line can load a module, change the userid, set a config option, etc (and can have side effects depending on the module involved) in an ideal world your approach would be great, but I question if the benifits are worth the large effort right now to re-write the configuration parsing/implementation (including changing how all the modules add config options) David Lang On Thu, 28 Apr 2011, Aaron Wiebe wrote: > This is the process I've used in the past: > > 1. Read the file into memory. > 2. First pass parse and check for config validity. > 3. Repeat 1 & 2 for included/referenced files. > 4. Replace or merge new config after all config checks are complete > 5. Clean up old config options that have not been validated/merged > from new config > > -Aaron > > On Thu, Apr 28, 2011 at 2:20 PM, Rainer Gerhards > wrote: >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Thursday, April 28, 2011 8:17 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system >>> >>> On Thu, 28 Apr 2011, Aaron Wiebe wrote: >>> >>>> On Thu, Apr 28, 2011 at 12:11 PM, ? wrote: >>>>> >>>>> 2. a wierd, out of the box thought (in the 90% as good for 20% >>> effort >>>>> category) >>>>> >>>>> rather than tring to modify the running config, how about forking >>> the main >>>>> thread, having the old main thread close the inputs, and the new >>> main thread >>>>> 'shutdown' all it's other threads, then the new main thread can >>> treat this >>>>> as something very close to a normal startup (basically modulo >>> privilage drop >>>>> issue) >>>>> >>>>> you could either let the old process/threads run until they finish >>>>> delivering all their messages, or you could have them deliver the >>> messages >>>>> to the new main thread (the former is probably the easiest, the >>> question is >>>>> are there more corner cases with both sets of threads outputting to >>> the same >>>>> destinations, or with potentially mishandling messages that were >>> already >>>>> processed in the old copy that would have been processed differently >>> in the >>>>> new copy) >>>> >>>> I've tried to do this with other software - it's a PAIN. ?Effectively >>>> you have to transfer the memory state between threads/processes. >>> It's >>>> not easy. >>>> >>>> Honestly, config reload is less error prone. >>> >>> my point is that insead of figuring out what effect all the loaded >>> modules >>> have, how to apply all the modifiers in order, etc. just throw out all >>> of >>> the existing config and start the new config clean from scratch, but do >>> it >>> in a separate process so that the existing one can keep processing the >>> logs it's already received. >> >> That bad thing is that this doesn't work for existing TCP connections. Of >> course you can start processing new connections with the "new" instance, but >> that means that new data coming in via old connections - even hours later - >> would still be processed by the old config. >> >> Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Thu Apr 28 21:09:37 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 28 Apr 2011 21:09:37 +0200 Subject: [rsyslog] thoughts on rsyslog's (multi-)config system In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71DE08D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09A@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE09D@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71DE0B4@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE0B5@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Thursday, April 28, 2011 8:41 PM > To: rsyslog-users > Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > > the headache is that currently on rsyslog, it's not a matter of reading > the config and then implementing it. it's a matter of read a line of > the config, implement that line, read the next line, implement that > line, etc > > each line can load a module, change the userid, set a config option, > etc (and can have side effects depending on the module involved) > > in an ideal world your approach would be great, but I question if the > benifits are worth the large effort right now to re-write the > configuration parsing/implementation (including changing how all the > modules add config options) Well, actually this is what I am currently working on -- restructure the config system to a more decent one. It's considerable work, and there will be no immediate benefits (other than that privilege dropping will work fully correct). I don't even know if I can complete that work in a single iteration or may need more than one. I think the long-term benefits will justify this "waste of time", as it first will seem to be. The current config system really is a blocker for some very useful things (e.g. the one we just discuss or a new config language). Rainer > > David Lang > > > On Thu, 28 Apr 2011, Aaron Wiebe wrote: > > > This is the process I've used in the past: > > > > 1. Read the file into memory. > > 2. First pass parse and check for config validity. > > 3. Repeat 1 & 2 for included/referenced files. > > 4. Replace or merge new config after all config checks are complete > > 5. Clean up old config options that have not been validated/merged > > from new config > > > > -Aaron > > > > On Thu, Apr 28, 2011 at 2:20 PM, Rainer Gerhards > > wrote: > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Thursday, April 28, 2011 8:17 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] thoughts on rsyslog's (multi-)config system > >>> > >>> On Thu, 28 Apr 2011, Aaron Wiebe wrote: > >>> > >>>> On Thu, Apr 28, 2011 at 12:11 PM, ? wrote: > >>>>> > >>>>> 2. a wierd, out of the box thought (in the 90% as good for 20% > >>> effort > >>>>> category) > >>>>> > >>>>> rather than tring to modify the running config, how about forking > >>> the main > >>>>> thread, having the old main thread close the inputs, and the new > >>> main thread > >>>>> 'shutdown' all it's other threads, then the new main thread can > >>> treat this > >>>>> as something very close to a normal startup (basically modulo > >>> privilage drop > >>>>> issue) > >>>>> > >>>>> you could either let the old process/threads run until they > finish > >>>>> delivering all their messages, or you could have them deliver the > >>> messages > >>>>> to the new main thread (the former is probably the easiest, the > >>> question is > >>>>> are there more corner cases with both sets of threads outputting > >>>>> to > >>> the same > >>>>> destinations, or with potentially mishandling messages that were > >>> already > >>>>> processed in the old copy that would have been processed > >>>>> differently > >>> in the > >>>>> new copy) > >>>> > >>>> I've tried to do this with other software - it's a PAIN. > >>>> Effectively you have to transfer the memory state between > threads/processes. > >>> It's > >>>> not easy. > >>>> > >>>> Honestly, config reload is less error prone. > >>> > >>> my point is that insead of figuring out what effect all the loaded > >>> modules have, how to apply all the modifiers in order, etc. just > >>> throw out all of the existing config and start the new config clean > >>> from scratch, but do it in a separate process so that the existing > >>> one can keep processing the logs it's already received. > >> > >> That bad thing is that this doesn't work for existing TCP > >> connections. Of course you can start processing new connections with > >> the "new" instance, but that means that new data coming in via old > >> connections - even hours later - would still be processed by the old > config. > >> > >> Rainer > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > From ismail at namtrac.org Fri Apr 29 14:04:36 2011 From: ismail at namtrac.org (=?UTF-8?B?xLBzbWFpbCBEw7ZubWV6?=) Date: Fri, 29 Apr 2011 14:04:36 +0200 Subject: [rsyslog] [PATCH] Fix memcpy overflow in runtime/net.c Message-ID: Hi; runtime/net.c around line 720 has: ????????????????????????memcpy(&(SIN(allowIP.addr.NetAddr)->sin_addr.s_addr), ????????????????????????????&(SIN6(res->ai_addr)->sin6_addr.s6_addr32[3]), ??????????????????????????? sizeof (struct sockaddr_in)); This is wrong because &(SIN(allowIP.addr.NetAddr)->sin_addr.s_addr is type of struct in_addr, gcc 4.6 correctly warns that this will overflow. Attached is a patch that will fix this problem. Regards, ismail -------------- next part -------------- A non-text attachment was scrubbed... Name: rsyslog-5.8.0-memcpy.patch Type: text/x-patch Size: 461 bytes Desc: not available URL: From mhalligan at pagerduty.com Sat Apr 30 00:19:34 2011 From: mhalligan at pagerduty.com (Michael Halligan) Date: Fri, 29 Apr 2011 15:19:34 -0700 Subject: [rsyslog] Rsyslog-gnutls + ubuntu fails for peer-checking tls connections with Called LogError, msg: could not load module '/usr/lib/rsyslog/lmnsd_gtls.so' Message-ID: Greetings! I'm trying to setup rsyslog to log to Loggly with peer certificate checking as desribed in http://wiki.loggly.com/rsysloginstallation. I'm running into a problem which doesn't appear to be a configuration problem, but from googling, appears to be fairly common. I've been able to reproduce this on physical hardware running Ubuntu 10.04.2 (32-bit, rsyslog-4.2.0-2 and rsyslog-gnutls packages installed), and on two different AMIs, both with the rsyslog-gnutls-5.6.3 packages, (one AMI is ubuntu 10.04.1 LTS) with the other AMI is Ubuntu 10.10. The error I see generated is rsyslogd-2068: could not load module '/usr/lib/rsyslog/lmnsd_gtls.so', rsyslog error -2078. The file exists, this isn't a filesystem issue. My debug log is at http://halligan.org/logfile. Michael From rgerhards at hq.adiscon.com Sat Apr 30 10:20:02 2011 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 30 Apr 2011 10:20:02 +0200 Subject: [rsyslog] Rsyslog-gnutls + ubuntu fails for peer-checking tls connections with Called LogError, msg: could not load module '/usr/lib/rsyslog/lmnsd_gtls.so' In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DE0CE@GRFEXC.intern.adiscon.com> There seems to be something wrong with the actual error message. The debug log shows this as a cause: 4682.996819547:b7502b70: source file nsd_gtls.c requested reference for module 'lmnsd_ptcp', reference count now 1 4682.998211411:b7502b70: GTLS CA file: '/etc/rsyslog/loggly_full.crt' 4682.998255141:b7502b70: unexpected GnuTLS error -64 in nsd_gtls.c:578: Error while reading file. 4682.998276984:b7502b70: Called LogError, msg: could not load module '/usr/lib/rsyslog/lmnsd_gtls.so', rsyslog error -2078 Unfortunately, GnuTLS isn't very specific in its error messages either. The -64 GnuTLS error means that there is some file format error for the certificate. But there is no way (I know) to obtain specifics about what is wrong... :( Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Halligan > Sent: Saturday, April 30, 2011 12:20 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Rsyslog-gnutls + ubuntu fails for peer-checking tls > connections with Called LogError, msg: could not load module > '/usr/lib/rsyslog/lmnsd_gtls.so' > > Greetings! > > I'm trying to setup rsyslog to log to Loggly with peer certificate > checking > as desribed in http://wiki.loggly.com/rsysloginstallation. I'm running > into > a problem which doesn't appear to be a configuration problem, but from > googling, appears to be fairly common. > > I've been able to reproduce this on physical hardware running Ubuntu > 10.04.2 > (32-bit, rsyslog-4.2.0-2 and rsyslog-gnutls packages installed), and on > two > different AMIs, both with the rsyslog-gnutls-5.6.3 packages, (one AMI > is > ubuntu 10.04.1 LTS) with the other AMI is Ubuntu 10.10. The error I see > generated is rsyslogd-2068: could not load module > '/usr/lib/rsyslog/lmnsd_gtls.so', rsyslog error -2078. The file exists, > this > isn't a filesystem issue. > > My debug log is at http://halligan.org/logfile. > > Michael > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mhalligan at pagerduty.com Sat Apr 30 00:15:12 2011 From: mhalligan at pagerduty.com (Michael Halligan) Date: Fri, 29 Apr 2011 15:15:12 -0700 Subject: [rsyslog] Rsyslog-gnutls + ubuntu fails for peer-checking tls connections with Called LogError, msg: could not load module '/usr/lib/rsyslog/lmnsd_gtls.so', Message-ID: Greetings! I'm trying to setup rsyslog to log to Loggly with peer certificate checking as desribed in http://wiki.loggly.com/rsysloginstallation. I'm running into a problem which doesn't appear to be a configuration problem, but from googling, appears to be fairly common. I've been able to reproduce this on physical hardware running Ubuntu 10.04.2 (32-bit, rsyslog-4.2.0-2 and rsyslog-gnutls packages installed), and on two different AMIs, both with the rsyslog-gnutls-5.6.3 packages, (one AMI is ubuntu 10.04.1 LTS) with the other AMI is Ubuntu 10.10. The error I see generated is rsyslogd-2068: could not load module '/usr/lib/rsyslog/lmnsd_gtls.so', rsyslog error -2078. The file exists, this isn't a filesystem issue. I've attached a debug log. Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: logfile Type: application/octet-stream Size: 641604 bytes Desc: not available URL: