From rgerhards at hq.adiscon.com Mon Mar 2 08:06:51 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 2 Mar 2009 08:06:51 +0100 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to always use fqdn of sending devices? References: <49993125.2060603@ecker-software.de><4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com><4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com><4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> Hi all, I have (obviously) no strong position in this. I do not object putting distro-specific files into a "contrib" directory and make them available with the tarball *as long as it is clear that I do not support them*. I concur to David that this may be useful and I also concur to Michael that it may cause some confusion. To me, the important point is that I can not support distro-specific things (at least outside of the core code) and that I will not want to create and release dependencies. So if we put some package files into the tarball, that means I will update them if I receivea patch or am asked to pull the, but I will neither verify them nor will I hold releases. So, in short, they will be unmaintained and often not matching the rest of the tarball. HOWEVER, I can see that there are cases where it would be useful to hae those files available. On the other hand, at least for Debian, I think it is possible to obtain the package files from Debian directly (but, granted, it may not have the newest ones, e.g. v4). I have a pragmatic suggestion: if you have package specific files, you can send them to me. I will create a subdirectory for them. There will be a README telling people that this stuff is (from my POV) unmaintained, probably outdated and to be used with care. If a maintainer (like Michael) later decides it was a bad idea to put the files into the tarball, I'll also happily delete them. Does this sound like a workable compromise? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Saturday, February 28, 2009 3:16 AM > To: rsyslog-users > Subject: Re: [rsyslog] Get rsyslog to always use fqdn of > sending devices? > > On Sat, 28 Feb 2009, Michael Biebl wrote: > > >>> > >>> If the fedora bits are kept in an entirely separate > upstream packaging > >>> branch, then I don't really care. > >>> But I wouldn't like to see them (or any debian related > files) shipped > >>> in a release tarball. > >> > >> so how am I (a debian user) supposed to create debian > compatible packages > >> for versions that you don't yet deal with? > >> > >> why couldn't you push the debian related files upstream > and maintain them > >> there? (submitting patches, or git pull requests for updates) > > > > Pretty simple: It's less work for me and Rainer and more flexible. > > Say I (for Debian) start adding the files upstream, so does > Fedora, BSD, etc... > > Now when Rainer wants to make a new release to not have any stale > > packaging files, he would have to ping all package > maintainer first to > > update the build files and push those changes. That simply doesn't > > scale. > > Packaging and upstream software releases should be decoupled. > > > > If you are really interested in the Debian Packaging, you > can grab the > > git repository from [1] and either work from there or at it as a > > "remote" to the rsyslog git repo and merge the debian specific bits. > > it's not that I'm interested in debian packaging, it's that I need to > install the stuff that you haven't decided to ship in debian > yet on my > debian system in such a way that I keep the package manager > happy (and > don't have it overwriting what I've compiled with an update > of an obsolete > version) > > it's not that the upstream version of the files need to be > perfect, but > they should be good enough to avoid the need for users to > have to fight > the packaging system and duplicate your efforts. > > I hate to have to pull in some stuff from your tree and > combine it with > stuff from the upstream tree because I don't know enough to > be sure that > I'm both pulling everything I need and not pulling something > that will > cause grief. > > you've made your decision, count this as one voice > disagreeing with that > decision. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Mon Mar 2 08:56:33 2009 From: david at lang.hm (david at lang.hm) Date: Sun, 1 Mar 2009 23:56:33 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 27 Feb 2009, david at lang.hm wrote: > On Thu, 26 Feb 2009, david at lang.hm wrote: > >> >> this works for reopening the socket each time, but if I uncomment the bind >> the sendto fails (error 22, invalid input) >> >> I haven't yet figured out what I'm missing on the bind that's causing this > > a little more testing and I find that the bind succeeds, but no traffic goes > out unless the source IP exists somewhere on the box (it can be bound to > lo:0, but it needs to exist) > > so the non-local-bind approach may not work :-( > > it's just hit midnight here, so I'm going to call it a night and try again > tomorrow. I abandoned this approach and spent the weekend learning how to do raw sockets. I found a library that makes it not that bad to do (at least for the IPv4 that I've done so far, IPv6 adds some wrinkles) the one thing thats not clear to me at this point is how to find the original source IP of the message. Is that available in a variable inside UDPSend, or is it something that I will have to get earlier in the process and then pass explicitly to UDPSend? David Lang From david at lang.hm Mon Mar 2 10:04:58 2009 From: david at lang.hm (david at lang.hm) Date: Mon, 2 Mar 2009 01:04:58 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: <1235670387.28865.2.camel@rf10up.intern.adiscon.com> References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: On Thu, 26 Feb 2009, Rainer Gerhards wrote: > On Sun, 2009-03-01 at 23:56 -0800, david at lang.hm wrote: >> On Fri, 27 Feb 2009, david at lang.hm wrote: >> >>> On Thu, 26 Feb 2009, david at lang.hm wrote: >>> >>>> >>>> this works for reopening the socket each time, but if I uncomment the bind >>>> the sendto fails (error 22, invalid input) >>>> >>>> I haven't yet figured out what I'm missing on the bind that's causing this >>> >>> a little more testing and I find that the bind succeeds, but no traffic goes >>> out unless the source IP exists somewhere on the box (it can be bound to >>> lo:0, but it needs to exist) >>> >>> so the non-local-bind approach may not work :-( >>> >>> it's just hit midnight here, so I'm going to call it a night and try again >>> tomorrow. >> >> I abandoned this approach and spent the weekend learning how to do raw >> sockets. I found a library that makes it not that bad to do (at least for >> the IPv4 that I've done so far, IPv6 adds some wrinkles) >> >> the one thing thats not clear to me at this point is how to find the >> original source IP of the message. Is that available in a variable inside >> UDPSend, or is it something that I will have to get earlier in the process >> and then pass explicitly to UDPSend? > > Actually, output modules do not receive access to the full message > object. This was originally done for security reasons (do not pass more > than needed). All they can receive is the strings that are passed to > them. So the module would need to be modified so that a second string > (like ommail) is passed and that string needs to be defined as the > to-be-spoofed IP (what also enables to rewrite the source IP). I will look into this. >> From all the discussion, it may make sense to start with a different > output plugin that may later be merged back into the original one... Ok, I won't try to have it do everything and just concentrate on doing the forging. forging on an all IPv4 network is very simple, on an all IPv6 network just a bit harder, it's not clear what to do for a mixed network (for a IPv6 destination and IPv4 source you can do a mapping, but what is the right thing to do for an IPv6 source with a IPv4 destination?) note that the other item (closing the output socket every X messages) should be pretty trivial to add into the existing module and is useful for both TCP and UDP. David Lang > Rainer >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 2 12:51:13 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 2 Mar 2009 12:51:13 +0100 Subject: [rsyslog] Weird fromhost property value References: <49A78F5C.3000400@net-m.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71EE7@GRFEXC.intern.adiscon.com> Can you retry with v4? That should be much cleaner now, maybe relp does not yet provide the resolved info (that is a protocol transport driver [lib] issue). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick Shen > Sent: Friday, February 27, 2009 8:00 AM > To: rsyslog-users > Subject: [rsyslog] Weird fromhost property value > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi All, > > I've utilized rsyslog as my company's central logging server for half a > year. > > Today I encounterd a very weird issue about value of fromhost property. > We use dynamic templates to store logs from clients. > > The template is like below: > > $template > d_hosts,"/var/rsyslog/HOSTS/%fromhost%/%$year%/%$month%/%syslogfacility > -text%_%fromhost%_%$year%_%$month%_%$day > %.log" > > You can see we group logs by fromhost value. > > Today, I did 3 times test that a client named (sobek) sent logs to > central logging server by UDP, TCP and RELP. > > The FQDN of client node is "sobek.net-m.internal", short name is > "sobek", ip address is "172.21.101.13". > > After testing, I got when sending via UDP, the fromhost value is short > name. And via TCP, the value is FQDN. Via RELP, the value is IP > address. > > So I got a very weird directory organization at "/var/rsyslog/HOSTS". > > ####################################################################### > ### > drwxr-x--- 3 root syslog 80 Feb 27 07:24 172.21.101.13 <- RELP > drwxr-x--- 3 root syslog 80 Feb 27 05:58 sobek <- UDP > drwxr-x--- 3 root syslog 80 Feb 27 06:03 sobek.net-m.internal <- TCP > ####################################################################### > ### > > We are running rsyslog 3.20.0 both on client and server. So I wanna > know > if any other has encountered this before? > > Thanks, > Patrick > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFJp49ckHhYtFevC+MRApbbAJ9Dgxtw5mf+ax9D81OZPfh5E9aJPgCdEqF/ > FlkFDJpWr4k6pVV4AQiLhRw= > =cQzr > -----END PGP SIGNATURE----- > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 14:42:45 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:42:45 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200902161141.21380.Luis.Fernando.Munoz.Mejias@cern.ch> <577465F99B41C842AAFBE9ED71E70ABA44FC08@grfint2.intern.adiscon.com> <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <1236001365.28865.44.camel@rf10up.intern.adiscon.com> On Fri, 2009-02-27 at 18:48 +0100, Luis Fernando Mu?oz Mej?as wrote: > Rainer, > > Good and bad news... > > > > That sounds really great. Before you start coding or preparing > > > anything, let me check how well our DBs perform, because it's not > > > yet clear if they'll be able to cope with the high insertion rate we > > > expect. If we don't go for the Oracle database this work doesn't > > > make sense. I bet we'll want the Oracle, anyways. > > > > Sounds fair. > > Good news: I did my tests and, for many tasks I need to do, Oracle is > our way to go. So, I'm willing to write the module, with your > guidance/advise. > so far this sounds good ;) > As I said, I need **excellent** performance. I definitely need batch > operations, the ability to prepare the statements given as arguments on > the configuration file, and not to commit entries one by one, but after > a number of entries are ready or (better) after some not so small > time. According to the advise I got from experts around here, I'll have > to use Oracle Call Interface for this module, I don't know if there are > any licensing issues. I can't comment on the licensing issue, I simply don't know what Oracle demands. On thing to do it is let the output module handle the "combination work" together. The output module is called one per message, however, it does not mean the output must directly write them to the database. It may buffer them until the batch is large enough. But this currently needs to be implemented on the output module basis. Obviously, that will not make coding simpler. If we find a sponsor for the necessary non-trivial extension of the core engine, the output module's task may become much easier. If things go well, such a sponsor may show up... > > It seems I'll have to review how rsyslog's queing modules work... I would suggest not to move into them - but, of course, if you like to... Lol, this is the non-trival task I talked about, there are numerous subtleties and, of course, they are weakly documented (but the inline doc is quite good). > > > > For this evaluation, I already have a timestamp formatter that fits > > > into Oracle, something that can be used with the property replacer, > > > like %timereported:::date-oracle%. > > > The bad news is that this timestamp formatter works perfectly on > interactive sessions (sqlplus) but not on non-interactive ones, f.i, in > Python scripts. You need to call Oracle's to_timestamp(string, format), > and by bloating your code with this ugly function the rfc-3339 formatter > is good enough. So I won't submit this one. > Sounds fair ;) Do you have a time frame for your project? (and maybe a rough overview of the "big picture" - I am always soooo curios ;)) Rainer > Cheers. From rgerhards at hq.adiscon.com Mon Mar 2 14:57:34 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:57:34 +0100 Subject: [rsyslog] Get rsyslog to always use fqdn of sending devices? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44FB9E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FBAF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FBFE@grfint2.intern.adiscon.com> <49993125.2060603@ecker-software.de> <4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com> <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> Message-ID: <1236002254.28865.46.camel@rf10up.intern.adiscon.com> Hi RB, on twitter, I was pointed to rpmforge. Does this sound like something that could be used? Rainer On Thu, 2009-02-26 at 17:49 +0100, Rainer Gerhards wrote: > Hi RB, > > thanks for all your hard work. I am absolutely willing to help make > succeed in that. Just one question before we do down to details. Are > there any other options that we can pursue? I remember, quite some time > ago, that someone posted the idea that some well-known (non-RH, not > EPEL) repositories exist. Unfortunatley, I do no longer know which these > were. > > So the question is: are there any other such repositories where RHEL > users turn to and, if so, can we work with them to achieve our joint > goals? > > Sorry for some backtracking here... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of RB > > Sent: Thursday, February 26, 2009 4:54 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Get rsyslog to always use fqdn of sending > > devices? > > > > On Tue, Feb 17, 2009 at 13:11, RB wrote: > > > Regardless, I'll take the flag and see what I can do to get a > > > readily-accessible reasonably current build available for CentOS-5. > > > > Good & bad news - the good news is the Fedora upstream is very > > responsive, the bad news is I got sidetracked after his response. > > > > I have been told that rsyslog cannot be put in EPEL since it is > > already packaged in RHEL, be that package good or bad. Tomas has > > offered to help with the SPEC should I have any problems, but it looks > > like we're on our own for the time being. > > > > RPM package distribution can be done to various depths. The simplest > > is to just provide both the SRPM and unsigned binary RPMs for a few > > chosen CPU architectures for each packaged release as an HTTP or FTP > > download. This would allow one-off installations (updates would be > > manual) and generally get the package 'out there' for use. Further > > steps would involve signing the binaries and possibly publishing a > > repo that users could subscribe to (using /etc/yum.* or equivalent) > > for automated updates. > > > > Distributing a binary package in whatever form is going to increase > > the load (however mildly) on the project - each release will involve > > compiling and distributing binaries and SRPMs, if not signing them as > > well. I can work with you [Rainer] to automate that process, but as a > > random user I should probably not be doing the compilation and signing > > myself. > > > > So, we have 4 basic questions: > > 1. What versions are desired? > > 2. Are there any rsyslog components or functionality not packaged in > > the Fedora distribution users here would like to see included? > > 3. Do we want to sign the packages? > > 4. Who will perform the compilation/signing? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 14:54:00 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:54:00 +0100 Subject: [rsyslog] Matching hostname and facility? In-Reply-To: <49A6D8CB.1010506@web-ster.com> References: <49A2E460.50604@web-ster.com> <49A5A521.8040107@web-ster.com> <49A6D8CB.1010506@web-ster.com> Message-ID: <1236002040.28865.45.camel@rf10up.intern.adiscon.com> On Thu, 2009-02-26 at 10:00 -0800, Scott Baker wrote: > On 02/25/2009 03:38 PM, (private) HKS wrote: > >> Does this syntax work on rsyslog 2.0.x, that's what my server has on it. > >> I've tried this syntax, but it's not logging. > >> > >> - Scott > > > > > > No, this will require 3+ - which you really should upgrade to anyway. > > That's what I figured... this is my CORE syslog server, so I'll need to > play a good upgrade proceedure. > > Is their documentation on configuration file changes going from 2.x to 3.x? There is a compatiblity guide, I guess this is what you are looking for: http://www.rsyslog.com/doc-v3compatibility.html Rainer > > - Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 3 08:54:16 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 08:54:16 +0100 Subject: [rsyslog] Three questions! Message-ID: Hello! I have a few questions. 1. The man page on the website is really outdated. Is it possible for it to be updated automatically? 2. Is it possible for the man page for rsyslog.conf to be up there too? 3. Can rsyslog handle importing existing log files? e.g. sending the latest entries from /var/log/httpd/somename.acc across rsyslog to a logging server? Thanks! From jackmarrow2 at gmail.com Tue Mar 3 09:05:51 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 09:05:51 +0100 Subject: [rsyslog] rsyslog changelog Message-ID: Hello, Is there a changelog for rsyslog, particularly showing the differences between the current version (3.x) and the 2.x version found in RHEL? Thanks, Jack From rgerhards at hq.adiscon.com Tue Mar 3 09:05:47 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 09:05:47 +0100 Subject: [rsyslog] Three questions! References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F03@GRFEXC.intern.adiscon.com> Hi, you asked just in time. See note on doc here: http://blog.gerhards.net/2009/03/rsyslog-doc-state-of-art.html For the file import, you can do this with imfile: http://www.rsyslog.com/doc-imfile.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 03, 2009 8:54 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Three questions! > > Hello! > > I have a few questions. > > 1. The man page on the website is really outdated. Is it possible for > it to be updated automatically? > 2. Is it possible for the man page for rsyslog.conf to be up there too? > 3. Can rsyslog handle importing existing log files? e.g. sending the > latest entries from /var/log/httpd/somename.acc across rsyslog to a > logging server? > > Thanks! > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Tue Mar 3 09:08:23 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 00:08:23 -0800 (PST) Subject: [rsyslog] rsyslog changelog In-Reply-To: References: Message-ID: On Tue, 3 Mar 2009, jack marrow wrote: > Hello, > > Is there a changelog for rsyslog, particularly showing the differences > between the current version (3.x) and the 2.x version found in RHEL? the best way to see the differences would be through git, however the differences between 2.x and 3.x are going to be so massive that it's going to be hard to see anything useful. what are you looking for? David Lang From rgerhards at hq.adiscon.com Tue Mar 3 09:09:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 09:09:17 +0100 Subject: [rsyslog] rsyslog changelog References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Well, you can see all change log entries by following the "change log" menu item in the menu to the left ;) But it may even be more convenient in that case that you get it directly from git as a single text file: http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=ba2a6c13e22b7 f67401c7edb15ea17d31162bde4;hb=HEAD Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 03, 2009 9:06 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog changelog > > Hello, > > Is there a changelog for rsyslog, particularly showing the differences > between the current version (3.x) and the 2.x version found in RHEL? > > Thanks, > > Jack > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 3 09:16:08 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 09:16:08 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: References: Message-ID: 2009/3/3 : > On Tue, 3 Mar 2009, jack marrow wrote: > >> Hello, >> >> Is there a changelog for rsyslog, particularly showing the differences >> between the current version (3.x) and the 2.x version found in RHEL? > > the best way to see the differences would be through git, however the > differences between 2.x and 3.x are going to be so massive that it's going > to be hard to see anything useful. > > what are you looking for? I need to know which features are in the RHEL 5 version (2.x) and which are in the upstream stable version (3.x). Is there a matrix somewhere? It would be good if there was. I am looking for imfile support, regular expressions (are these perl regular expressions or posix?). Plus the general major differences. Also are actions are supported? Thanks > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Tue Mar 3 13:01:29 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 04:01:29 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: On Mon, 2 Mar 2009, david at lang.hm wrote: > On Thu, 26 Feb 2009, Rainer Gerhards wrote: > >> Actually, output modules do not receive access to the full message >> object. This was originally done for security reasons (do not pass more >> than needed). All they can receive is the strings that are passed to >> them. So the module would need to be modified so that a second string >> (like ommail) is passed and that string needs to be defined as the >> to-be-spoofed IP (what also enables to rewrite the source IP). > > I will look into this. I haven't had time to figure this out yet. >>> From all the discussion, it may make sense to start with a different >> output plugin that may later be merged back into the original one... > > Ok, I won't try to have it do everything and just concentrate on doing the > forging. attached is a diff that turns the UDP forwarding into forging, currently with a fixed from address of 1.1.1.1 port 2 I also needed to modify the makefile to add LIBS = /usr/lib/libnet.a for it to compile in my research, I learned that syslog-ng uses this same library for their forging. so far I have avoided looking at the syslog-ng code (I wanted to understand what was happening on my own, and I also avoid any potential license issues until I can check on them) David Lang From Luis.Fernando.Munoz.Mejias at cern.ch Tue Mar 3 15:28:58 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Tue, 3 Mar 2009 15:28:58 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <1236001365.28865.44.camel@rf10up.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> <1236001365.28865.44.camel@rf10up.intern.adiscon.com> Message-ID: <200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> Hi there. > > As I said, I need **excellent** performance. I definitely need batch > > operations, the ability to prepare the statements given as arguments > > on the configuration file, and not to commit entries one by one, but > > after a number of entries are ready or (better) after some not so > > small time. According to the advise I got from experts around here, > > I'll have to use Oracle Call Interface for this module, I don't know > > if there are any licensing issues. > > I can't comment on the licensing issue, I simply don't know what > Oracle demands. I'm not sure how GPL-compatible it is to link to already existing proprietary code. Anyways, first I code, then we test, then we (you, actually) decide the legal aspects. > On thing to do it is let the output module handle the "combination > work" together. The output module is called one per message, however, > it does not mean the output must directly write them to the > database. It may buffer them until the batch is large enough. But this > currently needs to be implemented on the output module basis. > Obviously, that will not make coding simpler. That's what I expected, indeed. > > It seems I'll have to review how rsyslog's queing modules work... > > I would suggest not to move into them - but, of course, if you like > to... Lol, this is the non-trival task I talked about, there are > numerous subtleties and, of course, they are weakly documented (but > the inline doc is quite good). OK. I'll just have a buffer of entries to be committed. > Do you have a time frame for your project? (and maybe a rough overview > of the "big picture" - I am always soooo curios ;)) Not a full timescale. Let's say that as soon as you can provide me with the documentation/skeleton module most (say 70%) of my work will be developing this output module. Then, when I understand what a bad nightmare OCI is I'll be able to give a full timescale. After looking at ompgsql, it looks like writing output modules is easy if you know what you're doing. ;) Then, I'll be able to provide support for this module (fixing bugs and so on) for a couple of years, so it won't be shoot and forget. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Tue Mar 3 15:26:26 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 15:26:26 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch><1236001365.28865.44.camel@rf10up.intern.adiscon.com> <200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> Just one quick note, more following: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Tuesday, March 03, 2009 3:29 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > Hi there. > > > > As I said, I need **excellent** performance. I definitely need > batch > > > operations, the ability to prepare the statements given as > arguments > > > on the configuration file, and not to commit entries one by one, > but > > > after a number of entries are ready or (better) after some not so > > > small time. According to the advise I got from experts around here, > > > I'll have to use Oracle Call Interface for this module, I don't > know > > > if there are any licensing issues. > > > > I can't comment on the licensing issue, I simply don't know what > > Oracle demands. > > I'm not sure how GPL-compatible it is to link to already existing > proprietary code. Anyways, first I code, then we test, then we (you, > actually) decide the legal aspects. Actually, not me ;) I leave this risk to the user. If someone pays the legal counselor, I'll add his POV to the project doc. Rainer From aoz.syn at gmail.com Tue Mar 3 16:15:10 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 3 Mar 2009 08:15:10 -0700 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to always use fqdn of sending devices? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> References: <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903030715x55403ed9k59253789295ffcba@mail.gmail.com> On Mon, Mar 2, 2009 at 00:06, Rainer Gerhards wrote: > I have a pragmatic suggestion: if you have package specific files, you > can send them to me. I will create a subdirectory for them. There will > be a README telling people that this stuff is (from my POV) > unmaintained, probably outdated and to be used with care. If a > maintainer (like Michael) later decides it was a bad idea to put the > files into the tarball, I'll also happily delete them. > > Does this sound like a workable compromise? It does, but I'm not sure how it will mesh with wanting to provide packages for other distros that aren't so responsive as Debian or up-to-date as Fedora. I'll be happy to provide an RPM specfile for -stable and -dev (since Fedora already does a -beta package) but that may not be sufficient for the general clicky-package group. From aoz.syn at gmail.com Tue Mar 3 16:18:12 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 3 Mar 2009 08:18:12 -0700 Subject: [rsyslog] Get rsyslog to always use fqdn of sending devices? In-Reply-To: <1236002254.28865.46.camel@rf10up.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44FBFE@grfint2.intern.adiscon.com> <49993125.2060603@ecker-software.de> <4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com> <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> <1236002254.28865.46.camel@rf10up.intern.adiscon.com> Message-ID: <4255c2570903030718t73f55871n26d83867c3a3e621@mail.gmail.com> On Mon, Mar 2, 2009 at 06:57, Rainer Gerhards wrote: > on twitter, I was pointed to rpmforge. Does this sound like something > that could be used? That definitely looks viable, I'll submit a request and see how it goes. From danson at rackspace.com Tue Mar 3 23:57:10 2009 From: danson at rackspace.com (Daniel Anson) Date: Tue, 3 Mar 2009 16:57:10 -0600 Subject: [rsyslog] Double quotes Problem Message-ID: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> Does anyone know of a quick and easy template to remove the double quote character from a %msg% before it is inserted into the database (MySQL in my case). I have a %msg% that looks like this: user pid=21214 uid=0 auid=4294967295 msg='PAM setcred: user="oracle" exe="/bin/su" (hostname=?, addr=?, terminal=? result=Success)' I am reading the %msg% from the MySQL database and returning it in JSON formatting. When it encounters a double-quote character, it causes issues. I can always fix the program that returns it in JSON, but I think rsyslog can pre-fix the %msg%. Daniel M. Anson Linux Systems Engineer Rackspace danson at rackspace.com Office: (210)312-5114 Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse at rackspace.com, and delete the original message. Your cooperation is appreciated. From david at lang.hm Wed Mar 4 00:54:14 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 15:54:14 -0800 (PST) Subject: [rsyslog] filtering by message size Message-ID: is it possible to filter by message size? I'm looking at a situation where I would like to send the message via UDP if it's below a given size and by TCP if it's larger. David Lang From david at lang.hm Wed Mar 4 01:42:20 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 16:42:20 -0800 (PST) Subject: [rsyslog] Double quotes Problem In-Reply-To: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> References: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> Message-ID: On Tue, 3 Mar 2009, Daniel Anson wrote: > Does anyone know of a quick and easy template to remove the double quote > character from a %msg% before it is inserted into the database (MySQL in > my case). I have a %msg% that looks like this: > > user pid=21214 uid=0 auid=4294967295 msg='PAM setcred: user="oracle" > exe="/bin/su" (hostname=?, addr=?, terminal=? result=Success)' > > I am reading the %msg% from the MySQL database and returning it in JSON > formatting. When it encounters a double-quote character, it causes > issues. I can always fix the program that returns it in JSON, but I > think rsyslog can pre-fix the %msg%. you will need to change the mySQL template in rsyslog I think you have two options. 1. you can put any valid SQL in the rsyslog config that does the insert, so write SQL that eliminates the quote 2. I think you can change the template to remove the quotes before sending it to MySQL (but this may end up removing quotes needed for MySQL to work) David Lang From rgerhards at hq.adiscon.com Wed Mar 4 07:13:27 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 07:13:27 +0100 Subject: [rsyslog] filtering by message size References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Oh, that's an interesting use case. It is not yet possible. I think we can implement (fairly simple) the size for a field (via the property replacer). However, that does not help you with the resulting size of a template string. I probably also need to check the supporting infrastructure for "greater than" comparisons... Would that help? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 12:54 AM > To: rsyslog-users > Subject: [rsyslog] filtering by message size > > is it possible to filter by message size? > > I'm looking at a situation where I would like to send the > message via UDP > if it's below a given size and by TCP if it's larger. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Wed Mar 4 08:06:03 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:06:03 -0800 (PST) Subject: [rsyslog] filtering by message size In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 4 Mar 2009, Rainer Gerhards wrote: > Oh, that's an interesting use case. It is not yet possible. I think we > can implement (fairly simple) the size for a field (via the property > replacer). However, that does not help you with the resulting size of a > template string. I probably also need to check the supporting > infrastructure for "greater than" comparisons... > > Would that help? yes, I can set the value to something conservative to account for the variable-length fields. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, March 04, 2009 12:54 AM >> To: rsyslog-users >> Subject: [rsyslog] filtering by message size >> >> is it possible to filter by message size? >> >> I'm looking at a situation where I would like to send the >> message via UDP >> if it's below a given size and by TCP if it's larger. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 4 08:10:56 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 08:10:56 +0100 Subject: [rsyslog] filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F30@GRFEXC.intern.adiscon.com> Let me see what I can do - it looks so trivial that I tend to think I have overlooked some subtlety ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:06 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > Oh, that's an interesting use case. It is not yet possible. > I think we > > can implement (fairly simple) the size for a field (via the property > > replacer). However, that does not help you with the > resulting size of a > > template string. I probably also need to check the supporting > > infrastructure for "greater than" comparisons... > > > > Would that help? > > yes, I can set the value to something conservative to account for the > variable-length fields. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >> Sent: Wednesday, March 04, 2009 12:54 AM > >> To: rsyslog-users > >> Subject: [rsyslog] filtering by message size > >> > >> is it possible to filter by message size? > >> > >> I'm looking at a situation where I would like to send the > >> message via UDP > >> if it's below a given size and by TCP if it's larger. > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Wed Mar 4 08:16:59 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:16:59 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: Ok, here is a diff that works. it cycles the source IP address from 32000-42000 (since we are just sending, and not creating a normal socket this should not matter) it needs LIBS = /usr/lib/libnet.a in the Makefile in tools to use it create a template that puts the hostname-ip ahead of what you want to send, similar to $template TraditionalFwdFormat,"%fromhost-ip% <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" *.* @10.0.0.100;TraditionalFwdFormat the one problem right now is that any logs sent from the local box will go out with a source IP of 127.0.0.1 I wasted a bit of time trying to setup filters to use a different template if $myhostname == $fromhost, but apparently the filtering doesn't allow comparing two properties, and then I realized that you have a very high-performance name cache now, so you could easily replace my trivial inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); line with a call to the name lookup and then the %fromhost-ip% could be replaced by %fromhost% in the template and everything would work sanely (assuming forward and reverse name resolution are sane ;-) I haven't tried to do IPv6 yet, I know that it requires more effort to set the IP layer options, but I don't know exactly what yet. I wanted to float this first to see what you think before spending much more time on it. David Lang From rgerhards at hq.adiscon.com Wed Mar 4 08:14:00 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 08:14:00 +0100 Subject: [rsyslog] UDP source forging. References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com><1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> David, Just a quick info: I'll initially create a separate branch for these changes, as I can not go through them in details right now. I'll keep that branch updated and the goal is to move it into the master branch as soon as possible. Thanks for all your hard work! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] UDP source forging. > > Ok, here is a diff that works. > > it cycles the source IP address from 32000-42000 (since we are just > sending, and not creating a normal socket this should not matter) > > it needs LIBS = /usr/lib/libnet.a in the Makefile in tools > > to use it create a template that puts the hostname-ip ahead > of what you > want to send, similar to > > $template TraditionalFwdFormat,"%fromhost-ip% > <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" > > *.* @10.0.0.100;TraditionalFwdFormat > > the one problem right now is that any logs sent from the > local box will go > out with a source IP of 127.0.0.1 > > I wasted a bit of time trying to setup filters to use a > different template > if $myhostname == $fromhost, but apparently the filtering > doesn't allow > comparing two properties, and then I realized that you have a very > high-performance name cache now, so you could easily replace > my trivial > inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); > line with a call to the name lookup and then the > %fromhost-ip% could be > replaced by %fromhost% in the template and everything would > work sanely > (assuming forward and reverse name resolution are sane ;-) > > I haven't tried to do IPv6 yet, I know that it requires more > effort to set > the IP layer options, but I don't know exactly what yet. > > I wanted to float this first to see what you think before > spending much > more time on it. > > David Lang > From david at lang.hm Wed Mar 4 08:32:16 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:32:16 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com><1235670387.28865.2.camel@rf10up.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 4 Mar 2009, Rainer Gerhards wrote: > David, > > Just a quick info: I'll initially create a separate branch for these > changes, as I can not go through them in details right now. I'll keep > that branch updated and the goal is to move it into the master branch as > soon as possible. Thanks for all your hard work! no problem, once you can comment on it I'll work on adding IPv6. one problem I will have at that point is that I don't have any systems that use it (and most of my systems don't even have it compiled into the kernel) one thing that would be very useful for people looking to create additional modules would be if there was a simple example module that did something, but didn't use all the callbacks and helper functions that you have created. trying to untangle those to figure out what's happening is pretty hard. the current imtemplate is close to what's needed, but it is just a little bit too trivial. it's not clear from that exactly where you would do things like opening sockets, initializing global variables, etc. I'm thinking that probably the most trivial example would be a stripped-down version of imudp and omfwd that just did the minimum needed to get the packets in and out. (possibly with one config option, just to show how it is done, but everything else hard-coded) Rainer doesn't need to be the person to do this, if there is someone else who understands the modules and has a little time it would sure help the rest of us. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, March 04, 2009 8:17 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] UDP source forging. >> >> Ok, here is a diff that works. >> >> it cycles the source IP address from 32000-42000 (since we are just >> sending, and not creating a normal socket this should not matter) >> >> it needs LIBS = /usr/lib/libnet.a in the Makefile in tools >> >> to use it create a template that puts the hostname-ip ahead >> of what you >> want to send, similar to >> >> $template TraditionalFwdFormat,"%fromhost-ip% >> <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" >> >> *.* @10.0.0.100;TraditionalFwdFormat >> >> the one problem right now is that any logs sent from the >> local box will go >> out with a source IP of 127.0.0.1 >> >> I wasted a bit of time trying to setup filters to use a >> different template >> if $myhostname == $fromhost, but apparently the filtering >> doesn't allow >> comparing two properties, and then I realized that you have a very >> high-performance name cache now, so you could easily replace >> my trivial >> inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); >> line with a call to the name lookup and then the >> %fromhost-ip% could be >> replaced by %fromhost% in the template and everything would >> work sanely >> (assuming forward and reverse name resolution are sane ;-) >> >> I haven't tried to do IPv6 yet, I know that it requires more >> effort to set >> the IP layer options, but I don't know exactly what yet. >> >> I wanted to float this first to see what you think before >> spending much >> more time on it. >> >> David Lang >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 4 09:49:48 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 09:49:48 +0100 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to alwaysuse fqdn of sending devices? References: <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com><4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> <4255c2570903030715x55403ed9k59253789295ffcba@mail.gmail.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F32@GRFEXC.intern.adiscon.com> RB, Not addressing all the meat of your message (I can't...), I'd like to spell out that if you have something that should go into the tarball, just mail me and I'll see it gets in. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: Tuesday, March 03, 2009 4:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] Packages in tarball - was: RE: Get rsyslog to > alwaysuse fqdn of sending devices? > > On Mon, Mar 2, 2009 at 00:06, Rainer Gerhards > wrote: > > I have a pragmatic suggestion: if you have package specific files, > you > > can send them to me. I will create a subdirectory for them. There > will > > be a README telling people that this stuff is (from my POV) > > unmaintained, probably outdated and to be used with care. If a > > maintainer (like Michael) later decides it was a bad idea to put the > > files into the tarball, I'll also happily delete them. > > > > Does this sound like a workable compromise? > > It does, but I'm not sure how it will mesh with wanting to provide > packages for other distros that aren't so responsive as Debian or > up-to-date as Fedora. I'll be happy to provide an RPM specfile for > -stable and -dev (since Fedora already does a -beta package) but that > may not be sufficient for the general clicky-package group. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From pieter.thysebaert at intec.ugent.be Wed Mar 4 10:06:48 2009 From: pieter.thysebaert at intec.ugent.be (pieter.thysebaert at intec.ugent.be) Date: Wed, 4 Mar 2009 10:06:48 +0100 (CET) Subject: [rsyslog] (no subject) Message-ID: Hello Rsyslog users, I have been trying to get Rsyslogd up and running on HP-UX 11.31 ia64. For what it's worth: my preliminary results can be found on http://wiki.rsyslog.com/index.php/HP-UX Best regards, Pieter From rgerhards at hq.adiscon.com Wed Mar 4 11:53:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 11:53:40 +0100 Subject: [rsyslog] rsyslog on HP-UX References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F33@GRFEXC.intern.adiscon.com> Hi Pieter, thanks for your effort. Some time ago, I did an initial port on HP-UX via their web offering. As far as I remember, it compiled well at that time. However, I do not know what has changed in the mean time and how it "feels" now on that platform. I'll see that I integrate your patch ASAP (but that may take a while). The important thing is that I cannot integrate it as-is but need to make sure it does not break the other platforms. I would appreciate if you could check out interim versions when I have them available. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of > pieter.thysebaert at intec.ugent.be > Sent: Wednesday, March 04, 2009 10:07 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] (no subject) > > Hello Rsyslog users, > > I have been trying to get Rsyslogd up and running on HP-UX 11.31 ia64. > > For what it's worth: my preliminary results can be found on > http://wiki.rsyslog.com/index.php/HP-UX > > Best regards, > Pieter > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From janisg at latnetdc.lv Wed Mar 4 16:20:21 2009 From: janisg at latnetdc.lv (Janis) Date: Wed, 04 Mar 2009 17:20:21 +0200 Subject: [rsyslog] Right regex format for property based filters Message-ID: <49AE9C35.4050605@latnetdc.lv> Hello list. I have a question regarding to rsyslog configuration. What is the correct syntax of property based filter with regex. I'm using this configuration right now, and would like to create date based logfiles for each host - hostA, hostB, hostC. But it doesn't work this way. $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile And when running rsyslog with -d, I got only false matches on this regex. I seems that it tries to match all the text inside quotes instead of regexp. As I have red in man page, and html docs, then regexp should be in POSIX RE format (tryed also everything enclosed in braces). For example, if I change regex like this: :HOSTNAME, regex, "host" -?TplFile Then it works and matches all the hosts (A,B,C), and creates the files for each (well it's the same as using contains). But that doesn't solve the problem, when there isn't equal start prefixes for all hosts. For example if I want to match hosts - dog,cat,cow. Best regards --janis From rgerhards at hq.adiscon.com Wed Mar 4 16:35:59 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 16:35:59 +0100 Subject: [rsyslog] Right regex format for property based filters References: <49AE9C35.4050605@latnetdc.lv> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F35@GRFEXC.intern.adiscon.com> Hi Janis, the regex is Posix BRE, nor ERE. I think the syntax you use is not supported in BRE (as a side-note, this reminds me that I wanted to check what it takes to upgrade them to use ERE, too). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Janis > Sent: Wednesday, March 04, 2009 4:20 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Right regex format for property based filters > > Hello list. > > I have a question regarding to rsyslog configuration. What is the > correct syntax of property based > filter with regex. > > I'm using this configuration right now, and would like to create date > based logfiles for each host - hostA, hostB, hostC. > But it doesn't work this way. > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%- > %$DAY%.log" > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > And when running rsyslog with -d, I got only false matches on this > regex. I seems that it tries to match all the text > inside quotes instead of regexp. As I have red in man page, and html > docs, then regexp should be in POSIX RE format > (tryed also everything enclosed in braces). For example, if I change > regex like this: > > :HOSTNAME, regex, "host" -?TplFile > > Then it works and matches all the hosts (A,B,C), and creates the files > for each (well it's the same as using contains). > But that doesn't solve the problem, when there isn't equal start > prefixes for all hosts. > For example if I want to match hosts - dog,cat,cow. > > Best regards > --janis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:03:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 18:03:17 +0100 Subject: [rsyslog] Right regex format for property based filters References: <49AE9C35.4050605@latnetdc.lv> <9B6E2A8877C38245BFB15CC491A11DA71F35@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F37@GRFEXC.intern.adiscon.com> ERE looks trivial - just seeing if I get it in... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, March 04, 2009 4:36 PM > To: janisg at latnetdc.lv; rsyslog-users > Subject: Re: [rsyslog] Right regex format for property based filters > > Hi Janis, > > the regex is Posix BRE, nor ERE. I think the syntax you use is not > supported in BRE (as a side-note, this reminds me that I wanted to > check > what it takes to upgrade them to use ERE, too). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Janis > > Sent: Wednesday, March 04, 2009 4:20 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] Right regex format for property based filters > > > > Hello list. > > > > I have a question regarding to rsyslog configuration. What is the > > correct syntax of property based > > filter with regex. > > > > I'm using this configuration right now, and would like to create date > > based logfiles for each host - hostA, hostB, hostC. > > But it doesn't work this way. > > > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%- > > %$DAY%.log" > > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > > > And when running rsyslog with -d, I got only false matches on this > > regex. I seems that it tries to match all the text > > inside quotes instead of regexp. As I have red in man page, and html > > docs, then regexp should be in POSIX RE format > > (tryed also everything enclosed in braces). For example, if I change > > regex like this: > > > > :HOSTNAME, regex, "host" -?TplFile > > > > Then it works and matches all the hosts (A,B,C), and creates the > files > > for each (well it's the same as using contains). > > But that doesn't solve the problem, when there isn't equal start > > prefixes for all hosts. > > For example if I want to match hosts - dog,cat,cow. > > > > Best regards > > --janis > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:38:44 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 04 Mar 2009 18:38:44 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <49AE9C35.4050605@latnetdc.lv> References: <49AE9C35.4050605@latnetdc.lv> Message-ID: <1236188324.27835.2.camel@rf10up.intern.adiscon.com> Janis, I have added ERE filter support to the devel branch and your use case described below now works - you just need to use "ereregexp" instead of "regexp". No release tarball yet, the patch is here: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952 I did some quick checks, but would appreciate if some others try it out. Rainer On Wed, 2009-03-04 at 17:20 +0200, Janis wrote: > Hello list. > > I have a question regarding to rsyslog configuration. What is the > correct syntax of property based > filter with regex. > > I'm using this configuration right now, and would like to create date > based logfiles for each host - hostA, hostB, hostC. > But it doesn't work this way. > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > And when running rsyslog with -d, I got only false matches on this > regex. I seems that it tries to match all the text > inside quotes instead of regexp. As I have red in man page, and html > docs, then regexp should be in POSIX RE format > (tryed also everything enclosed in braces). For example, if I change > regex like this: > > :HOSTNAME, regex, "host" -?TplFile > > Then it works and matches all the hosts (A,B,C), and creates the files > for each (well it's the same as using contains). > But that doesn't solve the problem, when there isn't equal start > prefixes for all hosts. > For example if I want to match hosts - dog,cat,cow. > > Best regards > --janis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:56:05 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 04 Mar 2009 18:56:05 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <1236188324.27835.2.camel@rf10up.intern.adiscon.com> References: <49AE9C35.4050605@latnetdc.lv> <1236188324.27835.2.camel@rf10up.intern.adiscon.com> Message-ID: <1236189365.27835.19.camel@rf10up.intern.adiscon.com> All, I introduced a memory leak with the ERE enhancement. It is fixed now. So be sure to apply all patches after the one I mentioned. For your convenience, I created a temporary tarball based on the fixed version. It is available at http://download.rsyslog.com/rsyslog/tmp.tar.gz The tarball claims to contain 4.1.4, but you should not count on that it is equal to the released version. I will *not* care any more about this tarball. But I think it is useful to have a version right at hand. Also, this doesn't require any autotools tricks ;) Rainer On Wed, 2009-03-04 at 18:38 +0100, Rainer Gerhards wrote: > Janis, > > I have added ERE filter support to the devel branch and your use case > described below now works - you just need to use "ereregexp" instead of > "regexp". No release tarball yet, the patch is here: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952 > > I did some quick checks, but would appreciate if some others try it out. > > Rainer > > On Wed, 2009-03-04 at 17:20 +0200, Janis wrote: > > Hello list. > > > > I have a question regarding to rsyslog configuration. What is the > > correct syntax of property based > > filter with regex. > > > > I'm using this configuration right now, and would like to create date > > based logfiles for each host - hostA, hostB, hostC. > > But it doesn't work this way. > > > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" > > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > > > And when running rsyslog with -d, I got only false matches on this > > regex. I seems that it tries to match all the text > > inside quotes instead of regexp. As I have red in man page, and html > > docs, then regexp should be in POSIX RE format > > (tryed also everything enclosed in braces). For example, if I change > > regex like this: > > > > :HOSTNAME, regex, "host" -?TplFile > > > > Then it works and matches all the hosts (A,B,C), and creates the files > > for each (well it's the same as using contains). > > But that doesn't solve the problem, when there isn't equal start > > prefixes for all hosts. > > For example if I want to match hosts - dog,cat,cow. > > > > Best regards > > --janis > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Wed Mar 4 20:39:56 2009 From: mbiebl at gmail.com (Michael Biebl) Date: Wed, 4 Mar 2009 20:39:56 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <1236189365.27835.19.camel@rf10up.intern.adiscon.com> References: <49AE9C35.4050605@latnetdc.lv> <1236188324.27835.2.camel@rf10up.intern.adiscon.com> <1236189365.27835.19.camel@rf10up.intern.adiscon.com> Message-ID: 2009/3/4 Rainer Gerhards : > All, > > I introduced a memory leak with the ERE enhancement. It is fixed now. So > be sure to apply all patches after the one I mentioned. > > For your convenience, I created a temporary tarball based on the fixed > version. It is available at > > http://download.rsyslog.com/rsyslog/tmp.tar.gz > > The tarball claims to contain 4.1.4, but you should not count on that it > is equal to the released version. I will *not* care any more about this > tarball. But I think it is useful to have a version right at hand. Also, > this doesn't require any autotools tricks ;) Rainer, gitweb has the nice snapshot feature, which allows to download a tarball for a given SHA1 [1] It doesn't contain the build system, so requires a "autoreconf -vfi" run, but otherwise it should work just fine. Cheers, Michael [1] http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=42db7de5968d2db0fa855a9f029f6bccc0a30650;sf=tgz -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Thu Mar 5 18:52:34 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 5 Mar 2009 18:52:34 +0100 Subject: [rsyslog] rsyslog on Solaris Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Hi all, I have spent some time integrating the Solaris patches the past days (actually, learning [installing] Solaris took the most time). Now I have an environment and the compile process works rather well. However, there seem to be some issue with building the archives. I have to admit I am a bit clueless. After my sig is a build log of the affected part. I would appreciate if someone could provide some hints. Thanks, Rainer Making all in runtime make[2]: Entering directory `/root/rsyslog/runtime' /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o librsyslog.la librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la-linkedlist.lo librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo librsyslog_la-datetime.lo librsyslog_la-srutils.lo librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo librsyslog_la-queue.lo librsyslog_la-cfsysline.lo librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la-parse.lo librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o .libs/librsyslog_la-template.o make[2]: *** [librsyslog.la] Error 1 make[2]: Leaving directory `/root/rsyslog/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog' make: *** [all] Error 2 From epiphani at gmail.com Thu Mar 5 19:01:14 2009 From: epiphani at gmail.com (Aaron Wiebe) Date: Thu, 5 Mar 2009 13:01:14 -0500 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Message-ID: Hey Rainer, On Thu, Mar 5, 2009 at 12:52 PM, Rainer Gerhards wrote: > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > make[2]: *** [librsyslog.la] Error 1 > make[2]: Leaving directory `/root/rsyslog/runtime' First guess, what is that 'false' doing there? That would make the command return nonzero to make, hence the error code. -Aaron From david at ecker-software.de Thu Mar 5 19:33:11 2009 From: david at ecker-software.de (David Ecker) Date: Thu, 05 Mar 2009 19:33:11 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Message-ID: <49B01AE7.8080406@ecker-software.de> Hi, found the following in a another forum: the problem is resolved. the $PATH didn't include the /usr/ccs/bin, so the configure script couldn't find the ar. just a guess, see http://www.fantasticunix.com/forum/general-solaris-discussion/212026-mono-solaris-8-a.html bye David Ecker Rainer Gerhards schrieb: > Hi all, > > I have spent some time integrating the Solaris patches the past days > (actually, learning [installing] Solaris took the most time). > > Now I have an environment and the compile process works rather well. > However, there seem to be some issue with building the archives. I have > to admit I am a bit clueless. After my sig is a build log of the > affected part. > > I would appreciate if someone could provide some hints. > > Thanks, > Rainer > > Making all in runtime > make[2]: Entering directory `/root/rsyslog/runtime' > /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > -Wmissing-format-attribute -g -o librsyslog.la > librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo > librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la-linkedlist.lo > librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo > librsyslog_la-datetime.lo librsyslog_la-srutils.lo > librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo > librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo > librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo > librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo > librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo > librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo > librsyslog_la-queue.lo librsyslog_la-cfsysline.lo > librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la-parse.lo > librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o > .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o > .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o > .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o > .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o > .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o > .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o > .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o > .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o > .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o > .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o > .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o > .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o > .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o > .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o > .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o > .libs/librsyslog_la-template.o > make[2]: *** [librsyslog.la] Error 1 > make[2]: Leaving directory `/root/rsyslog/runtime' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/rsyslog' > make: *** [all] Error 2 > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From thomas.mieslinger at 1und1.de Fri Mar 6 10:17:48 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 10:17:48 +0100 Subject: [rsyslog] wrong permissons on directories Message-ID: <49B0EA3C.1060104@1und1.de> Hi *, when creating directories through dynamic templates, the directory permissons are incomplete: rsyslog.conf: $template ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%-%$MONTH%-%$DAY%.log" resulting directories: ls -al /data/log drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ ls -al /data/log/zeusmw drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ # rsyslogd -version rsyslogd 3.21.3, compiled with: FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No Runtime Instrumentation (slow code): No (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) I'd be happy to know if thats a bug. Thanks Thomas From thomas.mieslinger at 1und1.de Fri Mar 6 10:22:54 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 10:22:54 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <49B01AE7.8080406@ecker-software.de> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> <49B01AE7.8080406@ecker-software.de> Message-ID: <49B0EB6E.1050209@1und1.de> Hi, is that code modified for Solaris already available in git? Could you please send me a pointer to a checkout location? Thanks Thomas From rgerhards at hq.adiscon.com Fri Mar 6 11:41:24 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 11:41:24 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> Yes, it is part of the regular git tree: http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris $ git clone git://git.adiscon.com/git/rsyslog.git then checkout the "solaris" branch: $ git checkout --track -b solaris origin/solaris Rainer PS: commands may be wrong ;) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 10:23 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Hi, > > is that code modified for Solaris already available in git? Could you > please send me a pointer to a checkout location? > > Thanks > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 12:17:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 12:17:40 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> Hi Thomas, can it be that your default umask gets into your way? In any case, you can set the permissions explicitely with $FileCreateMode $FileGroup $FileOwner And set the umask with $umask (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) Does this help? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 10:18 AM > To: rsyslog-users > Subject: [rsyslog] wrong permissons on directories > > Hi *, > > when creating directories through dynamic templates, the directory > permissons are incomplete: > > rsyslog.conf: > $template > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%- > %$MONTH%-%$DAY%.log" > > resulting directories: > ls -al /data/log > drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ > > ls -al /data/log/zeusmw > drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ > > # rsyslogd -version > rsyslogd 3.21.3, compiled with: > FEATURE_REGEXP: Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > GSSAPI Kerberos 5 support: Yes > FEATURE_DEBUG (debug build, slow code): No > Runtime Instrumentation (slow code): No > > (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > > I'd be happy to know if thats a bug. > > Thanks > Thomas > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 14:07:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 14:07:06 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> <49B01AE7.8080406@ecker-software.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F60@GRFEXC.intern.adiscon.com> Thanks to you and Aaron, It was a combination of ar not being present plus autoconfig then using false... So that was a purely environment-base thing. Now I am one step further and the next issue is a pthreads linker error message ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of David Ecker > Sent: Thursday, March 05, 2009 7:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Hi, > > found the following in a another forum: > > > the problem is resolved. the $PATH didn't include the /usr/ccs/bin, so > the configure script couldn't find the ar. > > > just a guess, see > http://www.fantasticunix.com/forum/general-solaris-discussion/212026- > mono-solaris-8-a.html > > bye > David Ecker > > Rainer Gerhards schrieb: > > Hi all, > > > > I have spent some time integrating the Solaris patches the past days > > (actually, learning [installing] Solaris took the most time). > > > > Now I have an environment and the compile process works rather well. > > However, there seem to be some issue with building the archives. I > have > > to admit I am a bit clueless. After my sig is a build log of the > > affected part. > > > > I would appreciate if someone could provide some hints. > > > > Thanks, > > Rainer > > > > Making all in runtime > > make[2]: Entering directory `/root/rsyslog/runtime' > > /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > > -Wmissing-format-attribute -g -o librsyslog.la > > librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo > > librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la- > linkedlist.lo > > librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo > > librsyslog_la-datetime.lo librsyslog_la-srutils.lo > > librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo > > librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo > > librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo > > librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo > > librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo > > librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo > > librsyslog_la-queue.lo librsyslog_la-cfsysline.lo > > librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la- > parse.lo > > librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt > > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > > .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o > > .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o > > .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o > > .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o > > .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o > > .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o > > .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o > > .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o > > .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o > > .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o > > .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o > > .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o > > .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o > > .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o > > .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o > > .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o > > .libs/librsyslog_la-template.o > > make[2]: *** [librsyslog.la] Error 1 > > make[2]: Leaving directory `/root/rsyslog/runtime' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/root/rsyslog' > > make: *** [all] Error 2 > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > From thomas.mieslinger at 1und1.de Fri Mar 6 14:37:49 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 14:37:49 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> Message-ID: <49B1272D.4010408@1und1.de> Rainer Gerhards wrote: > Yes, it is part of the regular git tree: > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > $ git clone git://git.adiscon.com/git/rsyslog.git > then checkout the "solaris" branch: > $ git checkout --track -b solaris origin/solaris That worked. Thanks. What is the minimal required autoconf/automake Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU automake) 1.10.1 which came which opensolaris. it complains about undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? Thanks Thomas From rgerhards at hq.adiscon.com Fri Mar 6 14:39:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 14:39:06 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> <49B1272D.4010408@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F62@GRFEXC.intern.adiscon.com> I am using 2.63 on Solaris 10 x64 and I just successfully compiled. I am about to write a few notes about the state of solaris development in a few moments. My twitter feed may also be useful for you: http://twitter.com/rgerhards My environment is described on http://wiki.rsyslog.com/index.php/Solaris Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 2:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Rainer Gerhards wrote: > > Yes, it is part of the regular git tree: > > > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > > $ git clone git://git.adiscon.com/git/rsyslog.git > > then checkout the "solaris" branch: > > $ git checkout --track -b solaris origin/solaris > > That worked. Thanks. What is the minimal required autoconf/automake > Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU > automake) 1.10.1 which came which opensolaris. it complains about > undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? > > Thanks Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Fri Mar 6 15:13:55 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 15:13:55 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> Message-ID: <49B12FA3.2030202@1und1.de> Thanks for the pointer to the documentation.. it is $DirCreateMode what I asked for... and now I ask for a change of the default documentation says: Default: 0644 Reality demands 0755. I changed it in my configuration. I'd be happy to see that changed in rsyslog. Thomas Rainer Gerhards wrote: > Hi Thomas, > > can it be that your default umask gets into your way? In any case, you > can set the permissions explicitely with > > $FileCreateMode > $FileGroup > $FileOwner > > And set the umask with > > $umask > > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > > Does this help? > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> Sent: Friday, March 06, 2009 10:18 AM >> To: rsyslog-users >> Subject: [rsyslog] wrong permissons on directories >> >> Hi *, >> >> when creating directories through dynamic templates, the directory >> permissons are incomplete: >> >> rsyslog.conf: >> $template >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%- >> %$MONTH%-%$DAY%.log" >> >> resulting directories: >> ls -al /data/log >> drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ >> >> ls -al /data/log/zeusmw >> drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ >> >> # rsyslogd -version >> rsyslogd 3.21.3, compiled with: >> FEATURE_REGEXP: Yes >> FEATURE_LARGEFILE: Yes >> FEATURE_NETZIP (message compression): Yes >> GSSAPI Kerberos 5 support: Yes >> FEATURE_DEBUG (debug build, slow code): No >> Runtime Instrumentation (slow code): No >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) >> >> I'd be happy to know if thats a bug. >> >> Thanks >> Thomas >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- Thomas Mieslinger IT Infrastructure Systems Telefon: +49-721-91374-4404 E-Mail: thomas.mieslinger at 1und1.de 1&1 Internet AG Brauerstra?e 48 76135 Karlsruhe Amtsgericht Montabaur HRB 6484 Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver Mauss, Jan Oetjen Aufsichtsratsvorsitzender: Michael Scheeren From rgerhards at hq.adiscon.com Fri Mar 6 15:19:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 15:19:06 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Thomas, do I correctly understand that you propose the default be changed? If so, I am hesitant to do that - wouldn't that potentially break existing deployments? On the other hand... how could that work... Umm... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 3:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > Thanks for the pointer to the documentation.. it is $DirCreateMode what > I asked for... > > and now I ask for a change of the default > documentation says: > Default: 0644 > > Reality demands 0755. I changed it in my configuration. I'd be happy to > see that changed in rsyslog. > > Thomas > > > > Rainer Gerhards wrote: > > Hi Thomas, > > > > can it be that your default umask gets into your way? In any case, > you > > can set the permissions explicitely with > > > > $FileCreateMode > > $FileGroup > > $FileOwner > > > > And set the umask with > > > > $umask > > > > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > > > > Does this help? > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> Sent: Friday, March 06, 2009 10:18 AM > >> To: rsyslog-users > >> Subject: [rsyslog] wrong permissons on directories > >> > >> Hi *, > >> > >> when creating directories through dynamic templates, the directory > >> permissons are incomplete: > >> > >> rsyslog.conf: > >> $template > >> > > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- > %$YEAR%- > >> %$MONTH%-%$DAY%.log" > >> > >> resulting directories: > >> ls -al /data/log > >> drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ > >> > >> ls -al /data/log/zeusmw > >> drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ > >> > >> # rsyslogd -version > >> rsyslogd 3.21.3, compiled with: > >> FEATURE_REGEXP: Yes > >> FEATURE_LARGEFILE: Yes > >> FEATURE_NETZIP (message compression): Yes > >> GSSAPI Kerberos 5 support: Yes > >> FEATURE_DEBUG (debug build, slow code): No > >> Runtime Instrumentation (slow code): No > >> > >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > >> > >> I'd be happy to know if thats a bug. > >> > >> Thanks > >> Thomas > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > -- > Thomas Mieslinger > IT Infrastructure Systems > Telefon: +49-721-91374-4404 > E-Mail: thomas.mieslinger at 1und1.de > > 1&1 Internet AG > Brauerstra?e 48 > 76135 Karlsruhe > > Amtsgericht Montabaur HRB 6484 > Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas > Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver > Mauss, Jan Oetjen > Aufsichtsratsvorsitzender: Michael Scheeren > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Fri Mar 6 15:54:24 2009 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 6 Mar 2009 15:54:24 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: FWIW, the Debian default rsyslog.conf ships with $DirCreateMode 0755 2009/3/6 Rainer Gerhards : > Thomas, > > do I correctly understand that you propose the default be changed? > > If so, I am hesitant to do that - wouldn't that potentially break existing deployments? On the other hand... how could that work... Umm... > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> Sent: Friday, March 06, 2009 3:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] wrong permissons on directories >> >> Thanks for the pointer to the documentation.. it is $DirCreateMode what >> I asked for... >> >> and now I ask for a change of the default >> documentation says: >> Default: 0644 >> >> Reality demands 0755. I changed it in my configuration. I'd be happy to >> see that changed in rsyslog. >> >> Thomas >> >> >> >> Rainer Gerhards wrote: >> > Hi Thomas, >> > >> > can it be that your default umask gets into your way? In any case, >> you >> > can set the permissions explicitely with >> > >> > $FileCreateMode >> > $FileGroup >> > $FileOwner >> > >> > And set the umask with >> > >> > $umask >> > >> > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) >> > >> > Does this help? >> > >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> >> Sent: Friday, March 06, 2009 10:18 AM >> >> To: rsyslog-users >> >> Subject: [rsyslog] wrong permissons on directories >> >> >> >> Hi *, >> >> >> >> when creating directories through dynamic templates, the directory >> >> permissons are incomplete: >> >> >> >> rsyslog.conf: >> >> $template >> >> >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- >> %$YEAR%- >> >> %$MONTH%-%$DAY%.log" >> >> >> >> resulting directories: >> >> ls -al /data/log >> >> drw-r--r-- 3 root root 4096 Mar ?5 15:53 zeusmw/ >> >> >> >> ls -al /data/log/zeusmw >> >> drw-r--r-- 2 root root 4096 Mar ?6 10:11 2009-03/ >> >> >> >> # rsyslogd -version >> >> rsyslogd 3.21.3, compiled with: >> >> ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >> ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?Yes >> >> ? ?FEATURE_NETZIP (message compression): ? Yes >> >> ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >> ? ?FEATURE_DEBUG (debug build, slow code): No >> >> ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) >> >> >> >> I'd be happy to know if thats a bug. >> >> >> >> Thanks >> >> Thomas >> >> >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> >> -- >> Thomas Mieslinger >> IT Infrastructure Systems >> Telefon: +49-721-91374-4404 >> E-Mail: thomas.mieslinger at 1und1.de >> >> 1&1 Internet AG >> Brauerstra?e 48 >> 76135 Karlsruhe >> >> Amtsgericht Montabaur HRB 6484 >> Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas >> Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver >> Mauss, Jan Oetjen >> Aufsichtsratsvorsitzender: Michael Scheeren >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From thomas.mieslinger at 1und1.de Fri Mar 6 16:17:30 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 16:17:30 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: <49B13E8A.2080308@1und1.de> I guess nobody did let rsyslog make directories. Rainer Gerhards wrote: > Thomas, > > do I correctly understand that you propose the default be changed? Yepp. > If so, I am hesitant to do that - wouldn't that potentially break existing deployments? hmm Maybe I haven't seen enough yet, but I can't imagine a deployment built on directory permissions 644.... > On the other hand... how could that work... Umm... They are all working as root out there :-) I think it would be good if you just double check it yourself that the directories get created with 644 and decicde on your findings. Thomas From rgerhards at hq.adiscon.com Fri Mar 6 16:40:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 16:40:12 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com><49B1272D.4010408@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F62@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F67@GRFEXC.intern.adiscon.com> I have just finished my "current state" writeup on rsyslog and solaris: http://blog.gerhards.net/2009/03/rsyslog-and-solaris.html I guess it contains some useful information ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, March 06, 2009 2:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > I am using 2.63 on Solaris 10 x64 and I just successfully compiled. I > am > about to write a few notes about the state of solaris development in a > few moments. My twitter feed may also be useful for you: > > http://twitter.com/rgerhards > > My environment is described on > > http://wiki.rsyslog.com/index.php/Solaris > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > > Sent: Friday, March 06, 2009 2:38 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog on Solaris > > > > Rainer Gerhards wrote: > > > Yes, it is part of the regular git tree: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > > > $ git clone git://git.adiscon.com/git/rsyslog.git > > > then checkout the "solaris" branch: > > > $ git checkout --track -b solaris origin/solaris > > > > That worked. Thanks. What is the minimal required autoconf/automake > > Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU > > automake) 1.10.1 which came which opensolaris. it complains about > > undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? > > > > Thanks Thomas > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 16:40:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 16:40:12 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com><49B12FA3.2030202@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> The more I think about it, the more it smells like a real bug. Has anyone objections changing the default? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Friday, March 06, 2009 3:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > FWIW, the Debian default rsyslog.conf ships with > > $DirCreateMode 0755 > > > 2009/3/6 Rainer Gerhards : > > Thomas, > > > > do I correctly understand that you propose the default be changed? > > > > If so, I am hesitant to do that - wouldn't that potentially break > existing deployments? On the other hand... how could that work... > Umm... > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> Sent: Friday, March 06, 2009 3:14 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] wrong permissons on directories > >> > >> Thanks for the pointer to the documentation.. it is $DirCreateMode > what > >> I asked for... > >> > >> and now I ask for a change of the default > >> documentation says: > >> Default: 0644 > >> > >> Reality demands 0755. I changed it in my configuration. I'd be happy > to > >> see that changed in rsyslog. > >> > >> Thomas > >> > >> > >> > >> Rainer Gerhards wrote: > >> > Hi Thomas, > >> > > >> > can it be that your default umask gets into your way? In any case, > >> you > >> > can set the permissions explicitely with > >> > > >> > $FileCreateMode > >> > $FileGroup > >> > $FileOwner > >> > > >> > And set the umask with > >> > > >> > $umask > >> > > >> > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > >> > > >> > Does this help? > >> > > >> > Rainer > >> > > >> >> -----Original Message----- > >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> >> Sent: Friday, March 06, 2009 10:18 AM > >> >> To: rsyslog-users > >> >> Subject: [rsyslog] wrong permissons on directories > >> >> > >> >> Hi *, > >> >> > >> >> when creating directories through dynamic templates, the > directory > >> >> permissons are incomplete: > >> >> > >> >> rsyslog.conf: > >> >> $template > >> >> > >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- > >> %$YEAR%- > >> >> %$MONTH%-%$DAY%.log" > >> >> > >> >> resulting directories: > >> >> ls -al /data/log > >> >> drw-r--r-- 3 root root 4096 Mar ?5 15:53 zeusmw/ > >> >> > >> >> ls -al /data/log/zeusmw > >> >> drw-r--r-- 2 root root 4096 Mar ?6 10:11 2009-03/ > >> >> > >> >> # rsyslogd -version > >> >> rsyslogd 3.21.3, compiled with: > >> >> ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >> >> ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?Yes > >> >> ? ?FEATURE_NETZIP (message compression): ? Yes > >> >> ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >> >> ? ?FEATURE_DEBUG (debug build, slow code): No > >> >> ? ?Runtime Instrumentation (slow code): ? ?No > >> >> > >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > >> >> > >> >> I'd be happy to know if thats a bug. > >> >> > >> >> Thanks > >> >> Thomas > >> >> > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > >> -- > >> Thomas Mieslinger > >> IT Infrastructure Systems > >> Telefon: +49-721-91374-4404 > >> E-Mail: thomas.mieslinger at 1und1.de > >> > >> 1&1 Internet AG > >> Brauerstra?e 48 > >> 76135 Karlsruhe > >> > >> Amtsgericht Montabaur HRB 6484 > >> Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas > >> Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver > >> Mauss, Jan Oetjen > >> Aufsichtsratsvorsitzender: Michael Scheeren > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 17:09:05 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 17:09:05 +0100 Subject: [rsyslog] Intro presentation Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F6C@GRFEXC.intern.adiscon.com> Hi all, I think about doing an online intro presentation to rsyslog that should be useful to new users, in addition to the doc. One may claim that updating the doc makes more sense, but this is a major effort, plus someone has volunteered to help with that (plus I'd like to experiment with online tutorials). So in short, I think I'd like to try this out. Question now: what do you think would be most useful? I think about 10 to 60 minutes of presentation, something that I should be able to create over some evenings than try to deliver. What would be the best candidates to go into such material? Feedback appreciated, Rainer From jules at visionintel.com Fri Mar 6 17:16:32 2009 From: jules at visionintel.com (jules at visionintel.com) Date: Fri, 06 Mar 2009 16:16:32 +0000 Subject: [rsyslog] Intro presentation Message-ID: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Remote loggin Sent from my Nokia phone -----Original Message----- From: Rainer Gerhards Sent: 06/03/2009 16:09:05 Subject: [rsyslog] Intro presentation Hi all, I think about doing an online intro presentation to rsyslog that should be useful to new users, in addition to the doc. One may claim that updating the doc makes more sense, but this is a major effort, plus someone has volunteered to help with that (plus I'd like to experiment with online tutorials). So in short, I think I'd like to try this out. Question now: what do you think would be most useful? I think about 10 to 60 minutes of presentation, something that I should be able to create over some evenings than try to deliver. What would be the best candidates to go into such material? Feedback appreciated, Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From aoz.syn at gmail.com Fri Mar 6 17:25:24 2009 From: aoz.syn at gmail.com (RB) Date: Fri, 6 Mar 2009 09:25:24 -0700 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903060825l37364ab2w738468329e628e82@mail.gmail.com> On Fri, Mar 6, 2009 at 08:40, Rainer Gerhards wrote: > The more I think about it, the more it smells like a real bug. Has anyone objections changing the default? None. It is unrealistic (and generally unusable) to have UNIX directory permissions without the execute bit (S_IX*). The only reason to do it would be to have an 'archive' directory of sorts, in which users may see names of children, but none of their permissions or contents. As has been noted, the only reason it's worked thus far is that most people either change the default or run the daemon as root, for whom those permissions aren't really a limiting factor. From u.a.martin at gmail.com Fri Mar 6 17:38:57 2009 From: u.a.martin at gmail.com (Ben Martin) Date: Fri, 6 Mar 2009 09:38:57 -0700 Subject: [rsyslog] Intro presentation In-Reply-To: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> References: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Message-ID: <661ae2b20903060838q1aa1f5d8g91c79cff9bc606ab@mail.gmail.com> Rainer I think a video tutorial is great idea. You might even start with a very brief discussion of the importance of centralized logging, from both the security and management perspective. Discussing the basic differences between v2 and v3 would also be helpful I think, as some distros (like CentOS) are still only packaging v2, while others (Debian) are installing v3 by default. - Ben On Fri, Mar 6, 2009 at 9:16 AM, wrote: > Remote loggin > > Sent from my Nokia phone > -----Original Message----- > From: Rainer Gerhards > Sent: ?06/03/2009 16:09:05 > Subject: ?[rsyslog] Intro presentation > > Hi all, > > I think about doing an online intro presentation to rsyslog that should > be useful to new users, in addition to the doc. One may claim that > updating the doc makes more sense, but this is a major effort, plus > someone has volunteered to help with that (plus I'd like to experiment > with online tutorials). So in short, I think I'd like to try this out. > > Question now: what do you think would be most useful? I think about 10 > to 60 minutes of presentation, something that I should be able to create > over some evenings than try to deliver. What would be the best > candidates to go into such material? > > Feedback appreciated, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Mar 6 18:21:14 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 18:21:14 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> <49B13E8A.2080308@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F70@GRFEXC.intern.adiscon.com> I guess the "root issue" is more a probably cause. I know that lot's of folks use rsyslog to create dirs. Will probably change the default, but in the beta first. Thanks for bringing this up. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 4:18 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > I guess nobody did let rsyslog make directories. > > Rainer Gerhards wrote: > > Thomas, > > > > do I correctly understand that you propose the default be changed? > > Yepp. > > > If so, I am hesitant to do that - wouldn't that potentially break > existing deployments? > > hmm Maybe I haven't seen enough yet, but I can't imagine a deployment > built on directory permissions 644.... > > > On the other hand... how could that work... Umm... > > They are all working as root out there :-) > > I think it would be good if you just double check it yourself that the > directories get created with 644 and decicde on your findings. > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Fri Mar 6 19:53:14 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 10:53:14 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly Message-ID: I'm running into problems trying to do filtering. it looks as if the log parsing is not properly filling in the properties. what I've run into so far when I use the property 'programname' the content that I see is what I would expect in 'hostname' when I use the property 'hostname' the content that I see is what I would expect in 'fromhost' I haven't checked all the other properties, but my guess is that somehow rsyslog is off-by-one in filling them in. David Lang From david at lang.hm Fri Mar 6 19:54:00 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 10:54:00 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: References: Message-ID: On Fri, 6 Mar 2009, david at lang.hm wrote: > I'm running into problems trying to do filtering. it looks as if the log > parsing is not properly filling in the properties. > > what I've run into so far > > when I use the property 'programname' the content that I see is what I would > expect in 'hostname' > > when I use the property 'hostname' the content that I see is what I would > expect in 'fromhost' > > I haven't checked all the other properties, but my guess is that somehow > rsyslog is off-by-one in filling them in. having said this, date, fromhost, and from-ip appear to be filled in correctly. David Lang From rgerhards at hq.adiscon.com Fri Mar 6 19:54:11 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 19:54:11 +0100 Subject: [rsyslog] properties not getting filled in correctly References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> That's why I am after the log samples :) I just termed a new acronym this afternoon: YAMSF - yet another malformed syslog format ;) http://blog.gerhards.net/2009/02/calling-for-log-samples.html I try hard to get the fields right, but often this is impossible, resulting in the issues you see. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, March 06, 2009 7:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Fri, 6 Mar 2009, david at lang.hm wrote: > > > I'm running into problems trying to do filtering. it looks as if the > log > > parsing is not properly filling in the properties. > > > > what I've run into so far > > > > when I use the property 'programname' the content that I see is what > I would > > expect in 'hostname' > > > > when I use the property 'hostname' the content that I see is what I > would > > expect in 'fromhost' > > > > I haven't checked all the other properties, but my guess is that > somehow > > rsyslog is off-by-one in filling them in. > > having said this, date, fromhost, and from-ip appear to be filled in > correctly. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Sat Mar 7 02:25:32 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 17:25:32 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 6 Mar 2009, Rainer Gerhards wrote: > That's why I am after the log samples :) I just termed a new acronym > this afternoon: > YAMSF - yet another malformed syslog format ;) > > http://blog.gerhards.net/2009/02/calling-for-log-samples.html > > I try hard to get the fields right, but often this is impossible, > resulting in the issues you see. these logs come from several different servers, including different OSs, but all are misparsed by rsyslog. I am not seeing anything obviously wrong with them <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Friday, March 06, 2009 7:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Fri, 6 Mar 2009, david at lang.hm wrote: >> >>> I'm running into problems trying to do filtering. it looks as if the >> log >>> parsing is not properly filling in the properties. >>> >>> what I've run into so far >>> >>> when I use the property 'programname' the content that I see is what >> I would >>> expect in 'hostname' >>> >>> when I use the property 'hostname' the content that I see is what I >> would >>> expect in 'fromhost' >>> >>> I haven't checked all the other properties, but my guess is that >> somehow >>> rsyslog is off-by-one in filling them in. >> >> having said this, date, fromhost, and from-ip appear to be filled in >> correctly. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Sat Mar 7 03:55:49 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 18:55:49 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 6 Mar 2009, david at lang.hm wrote: > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >> That's why I am after the log samples :) I just termed a new acronym >> this afternoon: >> YAMSF - yet another malformed syslog format ;) >> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >> >> I try hard to get the fields right, but often this is impossible, >> resulting in the issues you see. > > these logs come from several different servers, including different OSs, > but all are misparsed by rsyslog. > > I am not seeing anything obviously wrong with them > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw doing some more digging I see some very definant problems I created the following template $template DumpAll,"msg =%msg%\nrawmsg =%rawmsg%\nuxtradmsg =%uxtradmsg%\nhostname =%hostname%\nsource =%source%\nfromhost =%fromhost%\nfromhost-ip =%fromhost-ip%\nsyslogtag =%syslogtag%\nprogramname =%programname%\npri =%pri%\npri-text =%pri-text%\niut =%iut%\nsyslogfacility =%syslogfacility%\nsyslogfacility-text =%syslogfacility-text%\nsyslogseverity =%syslogseverity%\nsyslogseverity-text =%syslogseverity-text%\nsyslogpriority =%syslogpriority%\nsyslogpriority-text =%syslogpriority-text%\ntimegenerated =%timegenerated%\ntimereported =%timereported%\ntimestamp =%timestamp%\nprotocol-version =%protocol-version%\nstructured-data =%structured-data%\napp-name =%app-name%\nprocid =%procid%\nmsgid =%msgid%\ninputname =%inputname%\n\n" which creates a nice table for each log message showing what's in each property. things that I am seeing hostname and source are fromhost rather than the name/IP that's in the record. msg includes the programname programname and appname are what hostname should be David Lang msg = %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 rawmsg =<167>Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 uxtradmsg =Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 hostname =itascan1a-p source =itascan1a-p fromhost =itascan1a-p fromhost-ip =192.168.210.6 syslogtag =172.20.245.8 programname =172.20.245.8 pri =167 pri-text =local4.debug<167> iut =1 syslogfacility =20 syslogfacility-text =local4 syslogseverity =7 syslogseverity-text =debug syslogpriority =7 syslogpriority-text =debug timegenerated =Mar 7 02:33:47 timereported =Mar 6 18:33:47 timestamp =Mar 6 18:33:47 protocol-version =0 structured-data =- app-name =172.20.245.8 procid =- msgid =- inputname =imudp msg = plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 rawmsg =<29>Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 uxtradmsg =Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 hostname =itascan1a-p source =itascan1a-p fromhost =itascan1a-p fromhost-ip =192.168.210.6 syslogtag =methane1d-b programname =methane1d-b pri =29 pri-text =daemon.notice<29> iut =1 syslogfacility =3 syslogfacility-text =daemon syslogseverity =5 syslogseverity-text =notice syslogpriority =5 syslogpriority-text =notice timegenerated =Mar 7 02:33:47 timereported =Mar 6 18:33:47 timestamp =Mar 6 18:33:47 protocol-version =0 structured-data =- app-name =methane1d-b procid =- msgid =- inputname =imudp From rgerhards at hq.adiscon.com Sat Mar 7 10:47:54 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 7 Mar 2009 10:47:54 +0100 Subject: [rsyslog] properties not getting filled in correctly Message-ID: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> The messages indeed look ok. I'll feed them into my parser and will see what happens. rainer ----- Urspr?ngliche Nachricht ----- Von: "david at lang.hm" An: "rsyslog-users" Gesendet: 07.03.09 02:20 Betreff: Re: [rsyslog] properties not getting filled in correctly On Fri, 6 Mar 2009, Rainer Gerhards wrote: > That's why I am after the log samples :) I just termed a new acronym > this afternoon: > YAMSF - yet another malformed syslog format ;) > > http://blog.gerhards.net/2009/02/calling-for-log-samples.html > > I try hard to get the fields right, but often this is impossible, > resulting in the issues you see. these logs come from several different servers, including different OSs, but all are misparsed by rsyslog. I am not seeing anything obviously wrong with them <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Friday, March 06, 2009 7:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Fri, 6 Mar 2009, david at lang.hm wrote: >> >>> I'm running into problems trying to do filtering. it looks as if the >> log >>> parsing is not properly filling in the properties. >>> >>> what I've run into so far >>> >>> when I use the property 'programname' the content that I see is what >> I would >>> expect in 'hostname' >>> >>> when I use the property 'hostname' the content that I see is what I >> would >>> expect in 'fromhost' >>> >>> I haven't checked all the other properties, but my guess is that >> somehow >>> rsyslog is off-by-one in filling them in. >> >> having said this, date, fromhost, and from-ip appear to be filled in >> correctly. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 9 07:14:49 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 9 Mar 2009 07:14:49 +0100 Subject: [rsyslog] filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F7B@GRFEXC.intern.adiscon.com> Hi David, Sorry for the late reply. Of course, the change is not as trivial as I initially thought. It is very easy to add a length modifier to the property replacer, but you can not use the property replacer in property-based filters. Of course, I can modify those filters, but there no concept of a numerical value with these filters. The proper thing would be to do this in the script engine, where it was scheduled for, but the script engine does not yet support functions. Doh... I will look where I can best hack this into. My current thinking is that I will check what it takes to make the script engine support built-in (rather than loadable) functions, so that I could implement a set of core functions. I am not sure how much effort that is, but it doesn't look too scary (plus it would be really good to have this functionality, so it would be well-spent time). It that turns out not to be an option, I'll probably hack the script engine to support a unary operation "lengthof", that should be simple enough - but it is a dirty approach. I won't be able to do anything of this today, but I hope I can do either of the two within this week. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:06 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > Oh, that's an interesting use case. It is not yet possible. > I think we > > can implement (fairly simple) the size for a field (via the property > > replacer). However, that does not help you with the > resulting size of a > > template string. I probably also need to check the supporting > > infrastructure for "greater than" comparisons... > > > > Would that help? > > yes, I can set the value to something conservative to account for the > variable-length fields. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >> Sent: Wednesday, March 04, 2009 12:54 AM > >> To: rsyslog-users > >> Subject: [rsyslog] filtering by message size > >> > >> is it possible to filter by message size? > >> > >> I'm looking at a situation where I would like to send the > >> message via UDP > >> if it's below a given size and by TCP if it's larger. > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jackmarrow2 at gmail.com Tue Mar 10 11:15:09 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:15:09 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/3 Rainer Gerhards : > Well, you can see all change log entries by following the "change log" > menu item in the menu to the left ;) But it may even be more convenient > in that case that you get it directly from git as a single text file: > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=ba2a6c13e22b7 > f67401c7edb15ea17d31162bde4;hb=HEAD > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of jack marrow >> Sent: Tuesday, March 03, 2009 9:06 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] rsyslog changelog >> >> Hello, >> >> Is there a changelog for rsyslog, particularly showing the differences >> between the current version (3.x) and the 2.x version found in RHEL? >> >> Thanks, >> >> Jack >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > Thanks for this. One last question: on the receiving server side, can I see which logs came from which log file? From rgerhards at hq.adiscon.com Tue Mar 10 11:12:32 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 11:12:32 +0100 Subject: [rsyslog] rsyslog changelog References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> > One last question: on the receiving server side, can I see which logs > came from which log file? Usually, the log line should contain the host that sent the message. Does your's not? Rainer From jackmarrow2 at gmail.com Tue Mar 10 11:21:57 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:21:57 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/10 Rainer Gerhards : >> One last question: on the receiving server side, can I see which logs >> came from which log file? > > Usually, the log line should contain the host that sent the message. > Does your's not? > If a client sends /var/log/httpd/blah and /var/log/vsftpd/blah, does the receiving side simply receive the log contents or the filename as well? Is there a way to get both? From rgerhards at hq.adiscon.com Tue Mar 10 11:17:46 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 11:17:46 +0100 Subject: [rsyslog] rsyslog changelog References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Please post configs and elaborate a bit more about what you are trying to accomplish and what you have set up. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 10, 2009 11:22 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog changelog > > 2009/3/10 Rainer Gerhards : > >> One last question: on the receiving server side, can I see which > logs > >> came from which log file? > > > > Usually, the log line should contain the host that sent the message. > > Does your's not? > > > > If a client sends /var/log/httpd/blah and /var/log/vsftpd/blah, does > the receiving side simply receive the log contents or the filename as > well? Is there a way to get both? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 10 11:28:35 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:28:35 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/10 Rainer Gerhards : > Please post configs and elaborate a bit more about what you are trying > to accomplish and what you have set up. I am evaluating rsyslog at the moment. I would like to know if I can use it for log collection on the client for writing on the server. The server must know which log file is which. From david at lang.hm Tue Mar 10 16:21:45 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 10 Mar 2009 08:21:45 -0700 (PDT) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: On Sat, 7 Mar 2009, Rainer Gerhards wrote: > The messages indeed look ok. I'll feed them into my parser and will see what happens. any idea what's happening here yet? David Lang > rainer > > ----- Urspr?ngliche Nachricht ----- > Von: "david at lang.hm" > An: "rsyslog-users" > Gesendet: 07.03.09 02:20 > Betreff: Re: [rsyslog] properties not getting filled in correctly > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >> That's why I am after the log samples :) I just termed a new acronym >> this afternoon: >> YAMSF - yet another malformed syslog format ;) >> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >> >> I try hard to get the fields right, but often this is impossible, >> resulting in the issues you see. > > these logs come from several different servers, including different OSs, > but all are misparsed by rsyslog. > > I am not seeing anything obviously wrong with them > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > David Lang > >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Friday, March 06, 2009 7:54 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] properties not getting filled in correctly >>> >>> On Fri, 6 Mar 2009, david at lang.hm wrote: >>> >>>> I'm running into problems trying to do filtering. it looks as if the >>> log >>>> parsing is not properly filling in the properties. >>>> >>>> what I've run into so far >>>> >>>> when I use the property 'programname' the content that I see is what >>> I would >>>> expect in 'hostname' >>>> >>>> when I use the property 'hostname' the content that I see is what I >>> would >>>> expect in 'fromhost' >>>> >>>> I haven't checked all the other properties, but my guess is that >>> somehow >>>> rsyslog is off-by-one in filling them in. >>> >>> having said this, date, fromhost, and from-ip appear to be filled in >>> correctly. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Tue Mar 10 16:24:31 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 16:24:31 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FB1@GRFEXC.intern.adiscon.com> Not at the moment, I am currently looking into the scripting engine (for stringlength-based evaluations) I highly suggest http://twitter.com/rgerhards to keep track of what I am looking at. You do NOT need to be subscribed to twitter to use this service. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, March 10, 2009 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Sat, 7 Mar 2009, Rainer Gerhards wrote: > > > The messages indeed look ok. I'll feed them into my parser and will > see what happens. > > any idea what's happening here yet? > > David Lang > > > rainer > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "david at lang.hm" > > An: "rsyslog-users" > > Gesendet: 07.03.09 02:20 > > Betreff: Re: [rsyslog] properties not getting filled in correctly > > > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > > > >> That's why I am after the log samples :) I just termed a new acronym > >> this afternoon: > >> YAMSF - yet another malformed syslog format ;) > >> > >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >> > >> I try hard to get the fields right, but often this is impossible, > >> resulting in the issues you see. > > > > these logs come from several different servers, including different > OSs, > > but all are misparsed by rsyslog. > > > > I am not seeing anything obviously wrong with them > > > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > /192.168.243.37 destination=179.50.100.130/60029 > > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > duration=1 > > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > /192.168.22.8 destination=192.168.104.31/5667 > > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, > pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( > <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > delivery) > > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > > > David Lang > > > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Friday, March 06, 2009 7:54 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>> > >>>> I'm running into problems trying to do filtering. it looks as if > the > >>> log > >>>> parsing is not properly filling in the properties. > >>>> > >>>> what I've run into so far > >>>> > >>>> when I use the property 'programname' the content that I see is > what > >>> I would > >>>> expect in 'hostname' > >>>> > >>>> when I use the property 'hostname' the content that I see is what > I > >>> would > >>>> expect in 'fromhost' > >>>> > >>>> I haven't checked all the other properties, but my guess is that > >>> somehow > >>>> rsyslog is off-by-one in filling them in. > >>> > >>> having said this, date, fromhost, and from-ip appear to be filled > in > >>> correctly. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Tue Mar 10 18:14:20 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 10 Mar 2009 11:14:20 -0600 Subject: [rsyslog] rsyslog changelog In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903101014n37d41ea2s23db8accc90f96ba@mail.gmail.com> On Tue, Mar 10, 2009 at 04:28, jack marrow wrote: > 2009/3/10 Rainer Gerhards : >> Please post configs and elaborate a bit more about what you are trying >> to accomplish and what you have set up. > > I am evaluating rsyslog at the moment. > > I would like to know if I can use it for log collection on the client > for writing on the server. The server must know which log file is > which. This is more a "basic understanding of logging" question than one specific to rsyslog. Generally speaking, log daemons just log what client apps tell them to - httpd says, "I'm facility 6 and is my critical message". If the local log daemon is sending logs upstream, it will basically tell the upstream server "I'm myhostname and httpd (facility 6) just said with a critical priority". If all your daemons (httpd, vsftpd, etc.) log directly to the local syslog as opposed to a flat file, things should "just work". However, if you're configuring your "client" syslog instance to follow /var/log/httpd/access and retransmit that data to an upstream server, all that metadata (application name, facility, priority, etc) is lost. Hence, you must configure your client syslog to inject that data - with rsyslog, that would be done something like this: $ModLoad imfile.so $InputFileName /var/log/httpd/access $InputFileTag http_access $InputFilePollIntervalSeconds 5 $InputFileMonitor *.* @192.168.1.1 That sets up a monitor that polls /var/log/httpd/access every 5 seconds, prepends "http_access" to every line, and sends it via UDP to 192.168.1.1. From rgerhards at hq.adiscon.com Tue Mar 10 18:24:02 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 18:24:02 +0100 Subject: [rsyslog] RainerScript functions - was: RE: filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F7B@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FB3@GRFEXC.intern.adiscon.com> David, I have extended RainerScript with the ability to call functions. The current implementation is very much focused on the immediate needs and it has a VM instruction set design issue that prevents nested function calls from working. Also, it only supports build-in functions (not loadable modules), and the only build-in function so far is strlen() ;) - but it should do what you need. So far, it resides in its own git branch "rscript-func". I will continue to work on it (at least on the VM opcode issue), but would really appreciate some early feedback. With that version you can do things like if strlen($msg) > 80 then @@tcp-host if strlen($msg) <= 80 then @udp-host Note that the function argument can be any valid expression (but NOT another function call!), so the following is also valid (and maybe useful to get to a better guess): if strlen($msg & $syslogtag & $fromhost) > 80 then @@tcp-host Note that & is the string concatenation operator. Today's commit: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=e8499c6d33d09f6d8b42df72 da1661be0ef0f088 Feedback from you and all others is appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, March 09, 2009 7:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > Hi David, > > Sorry for the late reply. Of course, the change is not as trivial as I > initially thought. It is very easy to add a length modifier to the > property replacer, but you can not use the property replacer in > property-based filters. Of course, I can modify those filters, but > there > no concept of a numerical value with these filters. The proper thing > would be to do this in the script engine, where it was scheduled for, > but the script engine does not yet support functions. Doh... > > I will look where I can best hack this into. My current thinking is > that > I will check what it takes to make the script engine support built-in > (rather than loadable) functions, so that I could implement a set of > core functions. I am not sure how much effort that is, but it doesn't > look too scary (plus it would be really good to have this > functionality, > so it would be well-spent time). It that turns out not to be an option, > I'll probably hack the script engine to support a unary operation > "lengthof", that should be simple enough - but it is a dirty approach. > I > won't be able to do anything of this today, but I hope I can do either > of the two within this week. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > > Sent: Wednesday, March 04, 2009 8:06 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] filtering by message size > > > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > > > Oh, that's an interesting use case. It is not yet possible. > > I think we > > > can implement (fairly simple) the size for a field (via the > property > > > replacer). However, that does not help you with the > > resulting size of a > > > template string. I probably also need to check the supporting > > > infrastructure for "greater than" comparisons... > > > > > > Would that help? > > > > yes, I can set the value to something conservative to account for the > > variable-length fields. > > > > David Lang > > > > > Rainer > > > > > >> -----Original Message----- > > >> From: rsyslog-bounces at lists.adiscon.com > > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > > david at lang.hm > > >> Sent: Wednesday, March 04, 2009 12:54 AM > > >> To: rsyslog-users > > >> Subject: [rsyslog] filtering by message size > > >> > > >> is it possible to filter by message size? > > >> > > >> I'm looking at a situation where I would like to send the > > >> message via UDP > > >> if it's below a given size and by TCP if it's larger. > > >> > > >> David Lang > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 11 13:49:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 13:49:08 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> David, the issue is in v4 only (and so far UDP only, too). It was introduced by the optimizations, which pass some wrong parameters to the now-decoupled parser. Need to find root cause, though. Will keep you posted. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, March 10, 2009 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Sat, 7 Mar 2009, Rainer Gerhards wrote: > > > The messages indeed look ok. I'll feed them into my parser and will > see what happens. > > any idea what's happening here yet? > > David Lang > > > rainer > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "david at lang.hm" > > An: "rsyslog-users" > > Gesendet: 07.03.09 02:20 > > Betreff: Re: [rsyslog] properties not getting filled in correctly > > > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > > > >> That's why I am after the log samples :) I just termed a new acronym > >> this afternoon: > >> YAMSF - yet another malformed syslog format ;) > >> > >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >> > >> I try hard to get the fields right, but often this is impossible, > >> resulting in the issues you see. > > > > these logs come from several different servers, including different > OSs, > > but all are misparsed by rsyslog. > > > > I am not seeing anything obviously wrong with them > > > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > /192.168.243.37 destination=179.50.100.130/60029 > > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > duration=1 > > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > /192.168.22.8 destination=192.168.104.31/5667 > > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, > pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( > <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > delivery) > > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > > > David Lang > > > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Friday, March 06, 2009 7:54 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>> > >>>> I'm running into problems trying to do filtering. it looks as if > the > >>> log > >>>> parsing is not properly filling in the properties. > >>>> > >>>> what I've run into so far > >>>> > >>>> when I use the property 'programname' the content that I see is > what > >>> I would > >>>> expect in 'hostname' > >>>> > >>>> when I use the property 'hostname' the content that I see is what > I > >>> would > >>>> expect in 'fromhost' > >>>> > >>>> I haven't checked all the other properties, but my guess is that > >>> somehow > >>>> rsyslog is off-by-one in filling them in. > >>> > >>> having said this, date, fromhost, and from-ip appear to be filled > in > >>> correctly. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Wed Mar 11 13:51:18 2009 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Mar 2009 05:51:18 -0700 (PDT) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 11 Mar 2009, Rainer Gerhards wrote: > David, > > the issue is in v4 only (and so far UDP only, too). It was introduced by the > optimizations, which pass some wrong parameters to the now-decoupled parser. > Need to find root cause, though. > > Will keep you posted. thanks. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Tuesday, March 10, 2009 4:22 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Sat, 7 Mar 2009, Rainer Gerhards wrote: >> >>> The messages indeed look ok. I'll feed them into my parser and will >> see what happens. >> >> any idea what's happening here yet? >> >> David Lang >> >>> rainer >>> >>> ----- Urspr?ngliche Nachricht ----- >>> Von: "david at lang.hm" >>> An: "rsyslog-users" >>> Gesendet: 07.03.09 02:20 >>> Betreff: Re: [rsyslog] properties not getting filled in correctly >>> >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote: >>> >>>> That's why I am after the log samples :) I just termed a new acronym >>>> this afternoon: >>>> YAMSF - yet another malformed syslog format ;) >>>> >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >>>> >>>> I try hard to get the fields right, but often this is impossible, >>>> resulting in the issues you see. >>> >>> these logs come from several different servers, including different >> OSs, >>> but all are misparsed by rsyslog. >>> >>> I am not seeing anything obviously wrong with them >>> >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601 >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= >> /192.168.243.37 destination=179.50.100.130/60029 >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 >> duration=1 >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= >> /192.168.22.8 destination=192.168.104.31/5667 >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: >> to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( >> <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for >> delivery) >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw >>> >>> David Lang >>> >>>> Rainer >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>>>> Sent: Friday, March 06, 2009 7:54 PM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly >>>>> >>>>> On Fri, 6 Mar 2009, david at lang.hm wrote: >>>>> >>>>>> I'm running into problems trying to do filtering. it looks as if >> the >>>>> log >>>>>> parsing is not properly filling in the properties. >>>>>> >>>>>> what I've run into so far >>>>>> >>>>>> when I use the property 'programname' the content that I see is >> what >>>>> I would >>>>>> expect in 'hostname' >>>>>> >>>>>> when I use the property 'hostname' the content that I see is what >> I >>>>> would >>>>>> expect in 'fromhost' >>>>>> >>>>>> I haven't checked all the other properties, but my guess is that >>>>> somehow >>>>>> rsyslog is off-by-one in filling them in. >>>>> >>>>> having said this, date, fromhost, and from-ip appear to be filled >> in >>>>> correctly. >>>>> >>>>> David Lang >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 11 14:32:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 14:32:17 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FC3@GRFEXC.intern.adiscon.com> David, there is now a patch available: http://git.adiscon.com/?p=rsyslog.git;a=commit;h=59192611db992e7357337beb8e68 ec6cee5b3fec I will release a new devel today, and it will include the patch. I expect to release another one next week, which will then have the Solaris work plus the script engine with functions (feedback on that is still appreciated). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 11, 2009 1:51 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Wed, 11 Mar 2009, Rainer Gerhards wrote: > > > David, > > > > the issue is in v4 only (and so far UDP only, too). It was introduced > by the > > optimizations, which pass some wrong parameters to the now-decoupled > parser. > > Need to find root cause, though. > > > > Will keep you posted. > > thanks. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >> Sent: Tuesday, March 10, 2009 4:22 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] properties not getting filled in correctly > >> > >> On Sat, 7 Mar 2009, Rainer Gerhards wrote: > >> > >>> The messages indeed look ok. I'll feed them into my parser and will > >> see what happens. > >> > >> any idea what's happening here yet? > >> > >> David Lang > >> > >>> rainer > >>> > >>> ----- Urspr?ngliche Nachricht ----- > >>> Von: "david at lang.hm" > >>> An: "rsyslog-users" > >>> Gesendet: 07.03.09 02:20 > >>> Betreff: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >>> > >>>> That's why I am after the log samples :) I just termed a new > acronym > >>>> this afternoon: > >>>> YAMSF - yet another malformed syslog format ;) > >>>> > >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >>>> > >>>> I try hard to get the fields right, but often this is impossible, > >>>> resulting in the issues you see. > >>> > >>> these logs come from several different servers, including different > >> OSs, > >>> but all are misparsed by rsyslog. > >>> > >>> I am not seeing anything obviously wrong with them > >>> > >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > >> /192.168.243.37 destination=179.50.100.130/60029 > >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > >> duration=1 > >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > >> /192.168.22.8 destination=192.168.104.31/5667 > >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > >> to=, delay=00:00:01, xdelay=00:00:01, > mailer=esmtp, > >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, > stat=Sent ( > >> <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > >> delivery) > >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > >>> > >>> David Lang > >>> > >>>> Rainer > >>>> > >>>>> -----Original Message----- > >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>>>> Sent: Friday, March 06, 2009 7:54 PM > >>>>> To: rsyslog-users > >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>>>> > >>>>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>>>> > >>>>>> I'm running into problems trying to do filtering. it looks as if > >> the > >>>>> log > >>>>>> parsing is not properly filling in the properties. > >>>>>> > >>>>>> what I've run into so far > >>>>>> > >>>>>> when I use the property 'programname' the content that I see is > >> what > >>>>> I would > >>>>>> expect in 'hostname' > >>>>>> > >>>>>> when I use the property 'hostname' the content that I see is > what > >> I > >>>>> would > >>>>>> expect in 'fromhost' > >>>>>> > >>>>>> I haven't checked all the other properties, but my guess is that > >>>>> somehow > >>>>>> rsyslog is off-by-one in filling them in. > >>>>> > >>>>> having said this, date, fromhost, and from-ip appear to be filled > >> in > >>>>> correctly. > >>>>> > >>>>> David Lang > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 11 15:22:51 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 15:22:51 +0100 Subject: [rsyslog] rsyslog 4.1.5 (devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FC6@GRFEXC.intern.adiscon.com> Hi all, I have just released rsyslog 4.1.5, a member of the development branch. It offers ERE support in filter conditions as well as the ability to contain part of the repeated text in a "last message repeated n times" message. Also, it fixes a bug that caused invalid parsing when receiving messages via UDP. This is a recommended update for all development branch users. Change Log: http://www.rsyslog.com/Article349.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-150.phtml I hope this release is useful. As always, feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Thu Mar 12 10:53:30 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Mar 2009 10:53:30 +0100 Subject: [rsyslog] Intro presentation References: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FDC@GRFEXC.intern.adiscon.com> Hi all, I created a first video tutorial today, please see blog for questions: http://blog.gerhards.net/2009/03/rsyslog-video-tutorials.html For this test, I have used something that I had ready at hand, thus none of the suggested topics yet touched. Feedback to the questions raised in the blog post would be most welcome. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jules at visionintel.com > Sent: Friday, March 06, 2009 5:17 PM > To: rsyslog-users > Subject: Re: [rsyslog] Intro presentation > > Remote loggin > > Sent from my Nokia phone > -----Original Message----- > From: Rainer Gerhards > Sent: 06/03/2009 16:09:05 > Subject: [rsyslog] Intro presentation > > Hi all, > > I think about doing an online intro presentation to rsyslog that should > be useful to new users, in addition to the doc. One may claim that > updating the doc makes more sense, but this is a major effort, plus > someone has volunteered to help with that (plus I'd like to experiment > with online tutorials). So in short, I think I'd like to try this out. > > Question now: what do you think would be most useful? I think about 10 > to 60 minutes of presentation, something that I should be able to > create > over some evenings than try to deliver. What would be the best > candidates to go into such material? > > Feedback appreciated, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Mar 12 18:36:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Mar 2009 18:36:08 +0100 Subject: [rsyslog] rant on software (rsyslog) stability Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FE7@GRFEXC.intern.adiscon.com> Hi all, I was recently asked a couple of times if I could quickly create a "stable version" of this and that new feature. So I have finally taken out some time today (more than expected...) to pen down my position on it. The rant also tells a lot about rsyslog's branches, so I thought it is useful to circulate it on the mailing list: http://blog.gerhards.net/2009/03/how-software-gets-stable.html As always, feedback is appreciated. Rainer From mtant621 at charter.net Fri Mar 13 19:53:19 2009 From: mtant621 at charter.net (Michael Tant) Date: Fri, 13 Mar 2009 14:53:19 -0400 Subject: [rsyslog] Please Help! IPTables dumping to Console!!! Message-ID: I am running Fedora 10 linux with rsyslogd as my active logger. Recently I have had an issue with my iptables LOG target output going to the console and not going to the /var/log/messages file, even with the --log-level 6 argument. I have halfway resolved this issue by editing the /etc/rsyslog.conf file to include: kern.warning /var/log/iptables.log and appending --log-level 4 to my LOG target rules. This caused the output to go to the aforementioned file AND the console. I wish to still have the log data going to the iptables.log file, but wish to stop the dump to the console. I have reviewed the rsyslog.conf file, and the only statement which references /dev/console is kern.* but it is commented out with #. I am tempted to remove this statement to see if it helps, but I am unsure if this is safe, and furthermore convinced it will not change the outcome as this line is nothing more than a comment. Is there something somewhere I am perhaps missing? I don't fully understand the steps that move the log target output to the file, other than rsyslogd is in the middle somewhere with the kernel. Any suggestions would be greatly appreciated! Please send suggestion to mtant621 at chater.net I thank everyone for your help... Michael Tant From rvandolson at esri.com Sat Mar 14 00:18:14 2009 From: rvandolson at esri.com (Ray Van Dolson) Date: Fri, 13 Mar 2009 16:18:14 -0700 Subject: [rsyslog] Filtering on a group of IP's Message-ID: <20090313231814.GA7833@esri.com> I'm trying to shunt a bunch of logs from a group of IP's (about 10 IP's or so) to a fifo. Is the best way to do this with a property filter like the following? $template SplunkPipe,"|/logs/splunk/splunk.fifo" :fromhost-ip, isequal, "10.1.5.3" *.* -?SplunkPipe And how would I easily specify many 10 IP's? I'm thinking it would be slick to be able to find a "netgroup" that has the member IP's I want then just have my selector match against that netgroup. Is that sort of magic possible? Unfortunately I'm using rsyslog with RHEL5 which is only v2.0.6. Examples appreciated. :) Ray From david at lang.hm Sat Mar 14 00:48:22 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Mar 2009 16:48:22 -0700 (PDT) Subject: [rsyslog] Please Help! IPTables dumping to Console!!! In-Reply-To: References: Message-ID: On Fri, 13 Mar 2009, Michael Tant wrote: > Date: Fri, 13 Mar 2009 14:53:19 -0400 > From: Michael Tant > Reply-To: rsyslog-users > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Please Help! IPTables dumping to Console!!! > > I am running Fedora 10 linux with rsyslogd as my active logger. Recently I > have had an issue with my iptables LOG target output going to the console > and not going to the /var/log/messages file, even with the --log-level 6 > argument. I have halfway resolved this issue by editing the > /etc/rsyslog.conf file to include: kern.warning /var/log/iptables.log and > appending --log-level 4 to my LOG target rules. This caused the output to > go to the aforementioned file AND the console. > > I wish to still have the log data going to the iptables.log file, but wish > to stop the dump to the console. I have reviewed the rsyslog.conf file, and > the only statement which references /dev/console is kern.* but it is > commented out with #. I am tempted to remove this statement to see if it > helps, but I am unsure if this is safe, and furthermore convinced it will > not change the outcome as this line is nothing more than a comment. > > Is there something somewhere I am perhaps missing? I don't fully understand > the steps that move the log target output to the file, other than rsyslogd > is in the middle somewhere with the kernel. Any suggestions would be > greatly appreciated! Please send suggestion to mtant621 at chater.net there are a couple of possibilities here 1. you have something in /etc/rsyslog.conf that sends output to the console (or to root) the fix for this is to just remove/change the rsyslog.conf file 2. take a look in /etc/sysctl and see what you have log levels set to. some distros think that the iptables logs are important enough to spam everyone who's logged in, no matter what syslog is configured for. David Lang > I thank everyone for your help... > > Michael Tant > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From mtant621 at charter.net Sat Mar 14 16:19:13 2009 From: mtant621 at charter.net (Michael Tant) Date: Sat, 14 Mar 2009 11:19:13 -0400 Subject: [rsyslog] Still Dumping to Console Message-ID: I am still attempting to get the logging to stop dumping to console. IPtables is the only one doing this. I am currently logging to a different file by adding kern.warning /var/log/iptables.log to rsyslog.conf and --log-level 4 argument for the LOG targets. The data is making it to the file as specified, but is also being echoed to console if one of the tty's is displayed. It does echo to console in an X environment though, even a Konsole. I have check and found no logging references in the sysctl.conf file. I have completely removed the line: #kern.* /dev/console from the rsyslog.conf file, and have looked for auxilliary logging processes running and found none. I'm not skilled enough to fully understand the sysctl -a output so that could be the next possible culprit. If someone wants to take a look at that, rather than dumping it here and flooding you with huge email, you can find this at: http://fpaste.org/paste/6106 If there is something I'm overlooking or if there's some other way to fix this and force the correct behavior please let me know. As I don't quite have your skills with linux yet, please try to include as much information as you can, to assist with the fix. Again this is under Fedora 10. uname -a gives: Linux MTFedora 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb 23 13:21:22 EST 2009 i686 i686 i386 GNU/Linux if that gives any help. Thank You so much, Michael Tant From david at lang.hm Sat Mar 14 16:39:01 2009 From: david at lang.hm (david at lang.hm) Date: Sat, 14 Mar 2009 08:39:01 -0700 (PDT) Subject: [rsyslog] Still Dumping to Console In-Reply-To: References: Message-ID: On Sat, 14 Mar 2009, Michael Tant wrote: > I am still attempting to get the logging to stop dumping to console. > IPtables is the only one doing this. I am currently logging to a different > file by adding kern.warning /var/log/iptables.log to rsyslog.conf > and --log-level 4 argument for the LOG targets. The data is making it to > the file as specified, but is also being echoed to console if one of the > tty's is displayed. It does echo to console in an X environment though, > even a Konsole. I have check and found no logging references in the > sysctl.conf file. I have completely removed the line: #kern.* > /dev/console from the rsyslog.conf file, and have looked for auxilliary > logging processes running and found none. I'm not skilled enough to fully > understand the sysctl -a output so that could be the next possible culprit. > If someone wants to take a look at that, rather than dumping it here and > flooding you with huge email, you can find this at: > http://fpaste.org/paste/6106 > > If there is something I'm overlooking or if there's some other way to fix > this and force the correct behavior please let me know. As I don't quite > have your skills with linux yet, please try to include as much information > as you can, to assist with the fix. Again this is under Fedora 10. > uname -a gives: Linux MTFedora 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb > 23 13:21:22 EST 2009 i686 i686 i386 GNU/Linux if that gives any help. my ubuntu desktop has the following in /etc/sysctl.conf # the following stops low-level messages on console kernel.printk = 4 4 1 7 From rgerhards at hq.adiscon.com Sun Mar 15 11:20:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 15 Mar 2009 11:20:08 +0100 Subject: [rsyslog] webinar: "rsyslog templates" Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72000@GRFEXC.intern.adiscon.com> Hi all, in my effort to try video tutorials (or webinars as some pointed out ;)), I have now created a first live demo version, focused on templates. I hope it is useful: http://www.rsyslog.com/Article354.phtml Rainer From julianokyap at gmail.com Mon Mar 16 05:32:14 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 18:32:14 -1000 Subject: [rsyslog] Logging all messages from a remote server Message-ID: I'm having trouble logging ALL the syslog messages received from a server. I'm not sure if it's because it's from a non-standard piece of hardware (ie. not a Linux server). Logging to another server running syslogd works fine (but syslogd doesn't allow me to log messages from a remote server to a separate file and it's not my central syslogd server). I've tried several lines but none seem to work for me: if $fromhost == 'server' then /var/log/remote/server/all if $source == 'server' then /var/log/remote/server/all :FROMHOST, isequal, "server" /var/log/remote/server/all if $fromhost == 'server.domain.com' then /var/log/remote/server/all if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all .. Running Rsyslog 3.21.10. Thanks, Julian From david at lang.hm Mon Mar 16 06:16:04 2009 From: david at lang.hm (david at lang.hm) Date: Sun, 15 Mar 2009 22:16:04 -0700 (PDT) Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: References: Message-ID: On Sun, 15 Mar 2009, Julian Yap wrote: > I'm having trouble logging ALL the syslog messages received from a > server. I'm not sure if it's because it's from a non-standard piece > of hardware (ie. not a Linux server). Logging to another server > running syslogd works fine (but syslogd doesn't allow me to log > messages from a remote server to a separate file and it's not my > central syslogd server). > > I've tried several lines but none seem to work for me: > if $fromhost == 'server' then /var/log/remote/server/all > if $source == 'server' then /var/log/remote/server/all > :FROMHOST, isequal, "server" /var/log/remote/server/all > if $fromhost == 'server.domain.com' then /var/log/remote/server/all > if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all there are a few possible reasons that this could have problems is it that you have a high volume of logs and some just get dropped? if you just write everything to a file (*.* /var/log/test) does it have all the logs from this server? or is it missing some? do the logs from this server sometimes include the host and sometimes not? what is different between the logs that you match and the ones that you miss? David Lang From julianokyap at gmail.com Mon Mar 16 09:14:56 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 22:14:56 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: References: Message-ID: OK, I narrowed the issues down. Now I've faced strange issues like this before when using the $IncludeConfig directive. This is what I have just tested with in my /etc/rsyslog.conf file (and other lines) and it worked fine: ---- $IncludeConfig /etc/rsyslog.d/ :FROMHOST, isequal, "server" /var/log/remote/server/all ---- Now if I have a file /etc/rsyslog.d/testalert_for_another_server, things turn strange and only certain messages are logged from the first server.: ---- $ModLoad ommail $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $template DYNserver2, "/var/log/remote/server2.log" $template TraditionalFormatNoHostname,"%timegenerated% %syslogtag%%msg:::drop-last-lf%\n" if $hostname == 'server2.domain.com' then ?DYNserver2;TraditionalFormatNoHostname $ActionMailFrom rsyslog at domain.com $ActionMailTo server2_alert $template mailSubjectTestAlert,"INFO: Alert detected" $template mailBodyTestAlert,"Message is..." $ActionMailSubject mailSubjectTestAlert $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 if $hostname == 'server2.domain.com' and $msg contains 'Some message' then :ommail:;mailBodyTestAlert ---- Now if I add the contents of /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and remove file /etc/rsyslog.d/testalert_for_another_server) then things work fine... Now if I remove the previous changes to /etc/rsyslog.conf and modify /etc/rsyslog.d/testalert_for_another_server and remove the following lines then things work OK again: $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 - Julian On Sun, Mar 15, 2009 at 7:16 PM, wrote: > On Sun, 15 Mar 2009, Julian Yap wrote: > >> I'm having trouble logging ALL the syslog messages received from a >> server. ?I'm not sure if it's because it's from a non-standard piece >> of hardware (ie. not a Linux server). ?Logging to another server >> running syslogd works fine (but syslogd doesn't allow me to log >> messages from a remote server to a separate file and it's not my >> central syslogd server). >> >> I've tried several lines but none seem to work for me: >> if $fromhost == 'server' then /var/log/remote/server/all >> if $source == 'server' then /var/log/remote/server/all >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > > there are a few possible reasons that this could have problems > > is it that you have a high volume of logs and some just get dropped? > > if you just write everything to a file (*.* /var/log/test) does it have > all the logs from this server? or is it missing some? > > do the logs from this server sometimes include the host and sometimes not? > > what is different between the logs that you match and the ones that you > miss? > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 09:52:54 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 09:52:54 +0100 Subject: [rsyslog] Logging all messages from a remote server References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> The issue is that these statements $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 Modify the *next* action. So you need to specify them in front of the action. If you use the $includeConfig option, and have part of the action inside the include file and other parts (the statements) outside (or vice versa), you never know which action gets configured how. So place all of them together. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 9:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > OK, I narrowed the issues down. Now I've faced strange issues like > this before when using the $IncludeConfig directive. > > This is what I have just tested with in my /etc/rsyslog.conf file (and > other lines) and it worked fine: > ---- > $IncludeConfig /etc/rsyslog.d/ > :FROMHOST, isequal, "server" /var/log/remote/server/all > ---- > > Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > things turn strange and only certain messages are logged from the > first server.: > ---- > $ModLoad ommail > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > $template DYNserver2, "/var/log/remote/server2.log" > $template TraditionalFormatNoHostname,"%timegenerated% > %syslogtag%%msg:::drop-last-lf%\n" > > if $hostname == 'server2.domain.com' then > ?DYNserver2;TraditionalFormatNoHostname > > $ActionMailFrom rsyslog at domain.com > $ActionMailTo server2_alert > $template mailSubjectTestAlert,"INFO: Alert detected" > $template mailBodyTestAlert,"Message is..." > $ActionMailSubject mailSubjectTestAlert > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > if $hostname == 'server2.domain.com' and $msg contains 'Some message' > then :ommail:;mailBodyTestAlert > ---- > > Now if I add the contents of > /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and > remove file /etc/rsyslog.d/testalert_for_another_server) then things > work fine... > > Now if I remove the previous changes to /etc/rsyslog.conf and modify > /etc/rsyslog.d/testalert_for_another_server and remove the following > lines then things work OK again: > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > > - Julian > > > On Sun, Mar 15, 2009 at 7:16 PM, wrote: > > On Sun, 15 Mar 2009, Julian Yap wrote: > > > >> I'm having trouble logging ALL the syslog messages received from a > >> server. ?I'm not sure if it's because it's from a non-standard piece > >> of hardware (ie. not a Linux server). ?Logging to another server > >> running syslogd works fine (but syslogd doesn't allow me to log > >> messages from a remote server to a separate file and it's not my > >> central syslogd server). > >> > >> I've tried several lines but none seem to work for me: > >> if $fromhost == 'server' then /var/log/remote/server/all > >> if $source == 'server' then /var/log/remote/server/all > >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all > >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > > > > there are a few possible reasons that this could have problems > > > > is it that you have a high volume of logs and some just get dropped? > > > > if you just write everything to a file (*.* /var/log/test) does it > have > > all the logs from this server? or is it missing some? > > > > do the logs from this server sometimes include the host and sometimes > not? > > > > what is different between the logs that you match and the ones that > you > > miss? > > > > David Lang > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Mon Mar 16 10:04:37 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 23:04:37 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> Message-ID: Rainer, Would you recommend against using $includeConfig? In that case, it tends to lead to more unknown config issues. - Julian On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards wrote: > The issue is that these statements > > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > Modify the *next* action. So you need to specify them in front of the action. > If you use the $includeConfig option, and have part of the action inside the > include file and other parts (the statements) outside (or vice versa), you > never know which action gets configured how. So place all of them together. > > HTH > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> Sent: Monday, March 16, 2009 9:15 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> OK, I narrowed the issues down. ?Now I've faced strange issues like >> this before when using the $IncludeConfig directive. >> >> This is what I have just tested with in my /etc/rsyslog.conf file (and >> other lines) and it worked fine: >> ---- >> $IncludeConfig /etc/rsyslog.d/ >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all >> ---- >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, >> things turn strange and only certain messages are logged from the >> first server.: >> ---- >> $ModLoad ommail >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat >> >> $template DYNserver2, "/var/log/remote/server2.log" >> $template TraditionalFormatNoHostname,"%timegenerated% >> %syslogtag%%msg:::drop-last-lf%\n" >> >> if $hostname == 'server2.domain.com' then >> ?DYNserver2;TraditionalFormatNoHostname >> >> $ActionMailFrom rsyslog at domain.com >> $ActionMailTo server2_alert >> $template mailSubjectTestAlert,"INFO: Alert detected" >> $template mailBodyTestAlert,"Message is..." >> $ActionMailSubject mailSubjectTestAlert >> $ActionExecOnlyOnceEveryInterval 300 >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> $ActionExecOnlyEveryNthTime 3 >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some message' >> then :ommail:;mailBodyTestAlert >> ---- >> >> Now if I add the contents of >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and >> remove file /etc/rsyslog.d/testalert_for_another_server) then things >> work fine... >> >> Now if I remove the previous changes to /etc/rsyslog.conf and modify >> /etc/rsyslog.d/testalert_for_another_server and remove the following >> lines then things work OK again: >> $ActionExecOnlyOnceEveryInterval 300 >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> $ActionExecOnlyEveryNthTime 3 >> >> >> - Julian >> >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: >> > On Sun, 15 Mar 2009, Julian Yap wrote: >> > >> >> I'm having trouble logging ALL the syslog messages received from a >> >> server. ?I'm not sure if it's because it's from a non-standard piece >> >> of hardware (ie. not a Linux server). ?Logging to another server >> >> running syslogd works fine (but syslogd doesn't allow me to log >> >> messages from a remote server to a separate file and it's not my >> >> central syslogd server). >> >> >> >> I've tried several lines but none seem to work for me: >> >> if $fromhost == 'server' then /var/log/remote/server/all >> >> if $source == 'server' then /var/log/remote/server/all >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all >> > >> > there are a few possible reasons that this could have problems >> > >> > is it that you have a high volume of logs and some just get dropped? >> > >> > if you just write everything to a file (*.* /var/log/test) does it >> have >> > all the logs from this server? or is it missing some? >> > >> > do the logs from this server sometimes include the host and sometimes >> not? >> > >> > what is different between the logs that you match and the ones that >> you >> > miss? >> > >> > David Lang >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 10:08:36 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 10:08:36 +0100 Subject: [rsyslog] Logging all messages from a remote server References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 10:05 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > Rainer, > > Would you recommend against using $includeConfig? In that case, it > tends to lead to more unknown config issues. No, but do not split config directives that need to go together over several places. You need to put this together # this starts the definition of a single action $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 $... *.* action #this ends it So you need to put everything together. If you rip it apart, you will get undefined results. This is - to phrase it politely - not very well documented. You need to read the fine print, most of the $Action... params modify the *next* action - NOT *all* actions. So it is vitally important where they occur. Will try to make this clear as soon as I have a bit more time. Rainer > > - Julian > > On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards > wrote: > > The issue is that these statements > > > > $ActionExecOnlyOnceEveryInterval 300 > > $ActionExecOnlyEveryNthTimeTimeout 1200 > > $ActionExecOnlyEveryNthTime 3 > > > > Modify the *next* action. So you need to specify them in front of the > action. > > If you use the $includeConfig option, and have part of the action > inside the > > include file and other parts (the statements) outside (or vice > versa), you > > never know which action gets configured how. So place all of them > together. > > > > HTH > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> Sent: Monday, March 16, 2009 9:15 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> > >> OK, I narrowed the issues down. ?Now I've faced strange issues like > >> this before when using the $IncludeConfig directive. > >> > >> This is what I have just tested with in my /etc/rsyslog.conf file > (and > >> other lines) and it worked fine: > >> ---- > >> $IncludeConfig /etc/rsyslog.d/ > >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all > >> ---- > >> > >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > >> things turn strange and only certain messages are logged from the > >> first server.: > >> ---- > >> $ModLoad ommail > >> > >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat > >> > >> $template DYNserver2, "/var/log/remote/server2.log" > >> $template TraditionalFormatNoHostname,"%timegenerated% > >> %syslogtag%%msg:::drop-last-lf%\n" > >> > >> if $hostname == 'server2.domain.com' then > >> ?DYNserver2;TraditionalFormatNoHostname > >> > >> $ActionMailFrom rsyslog at domain.com > >> $ActionMailTo server2_alert > >> $template mailSubjectTestAlert,"INFO: Alert detected" > >> $template mailBodyTestAlert,"Message is..." > >> $ActionMailSubject mailSubjectTestAlert > >> $ActionExecOnlyOnceEveryInterval 300 > >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> $ActionExecOnlyEveryNthTime 3 > >> > >> if $hostname == 'server2.domain.com' and $msg contains 'Some > message' > >> then :ommail:;mailBodyTestAlert > >> ---- > >> > >> Now if I add the contents of > >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf > (and > >> remove file /etc/rsyslog.d/testalert_for_another_server) then things > >> work fine... > >> > >> Now if I remove the previous changes to /etc/rsyslog.conf and modify > >> /etc/rsyslog.d/testalert_for_another_server and remove the following > >> lines then things work OK again: > >> $ActionExecOnlyOnceEveryInterval 300 > >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> $ActionExecOnlyEveryNthTime 3 > >> > >> > >> - Julian > >> > >> > >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: > >> > On Sun, 15 Mar 2009, Julian Yap wrote: > >> > > >> >> I'm having trouble logging ALL the syslog messages received from > a > >> >> server. ?I'm not sure if it's because it's from a non-standard > piece > >> >> of hardware (ie. not a Linux server). ?Logging to another server > >> >> running syslogd works fine (but syslogd doesn't allow me to log > >> >> messages from a remote server to a separate file and it's not my > >> >> central syslogd server). > >> >> > >> >> I've tried several lines but none seem to work for me: > >> >> if $fromhost == 'server' then /var/log/remote/server/all > >> >> if $source == 'server' then /var/log/remote/server/all > >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> >> if $fromhost == 'server.domain.com' then > /var/log/remote/server/all > >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > >> > > >> > there are a few possible reasons that this could have problems > >> > > >> > is it that you have a high volume of logs and some just get > dropped? > >> > > >> > if you just write everything to a file (*.* /var/log/test) does it > >> have > >> > all the logs from this server? or is it missing some? > >> > > >> > do the logs from this server sometimes include the host and > sometimes > >> not? > >> > > >> > what is different between the logs that you match and the ones > that > >> you > >> > miss? > >> > > >> > David Lang > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Mon Mar 16 10:18:23 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 23:18:23 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> Message-ID: Thanks all. My config is working fine now. I can take some of the blame for requesting the $ActionExecOnlyEveryNthTime* params in the first place :P. Just to shed some light, my previous understanding (or what I initially gathered from the docs) was that the $Action params needed to just be in a block and the order of params didn't matter. So: #start Action $Action... $Action... $Action... #end Action So that was just what I gathered in my head. But it's all clear now. - Julian On Sun, Mar 15, 2009 at 11:08 PM, Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> Sent: Monday, March 16, 2009 10:05 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> Rainer, >> >> Would you recommend against using $includeConfig? ?In that case, it >> tends to lead to more unknown config issues. > > No, but do not split config directives that need to go together over several > places. You need to put this together > > # this starts the definition of a single action > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > $... > *.* ?action > #this ends it > > So you need to put everything together. If you rip it apart, you will get > undefined results. > > This is - to phrase it politely - not very well documented. You need to read > the fine print, most of the $Action... params modify the *next* action - NOT > *all* actions. So it is vitally important where they occur. > > Will try to make this clear as soon as I have a bit more time. > > > Rainer >> >> - Julian >> >> On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards >> wrote: >> > The issue is that these statements >> > >> > $ActionExecOnlyOnceEveryInterval 300 >> > $ActionExecOnlyEveryNthTimeTimeout 1200 >> > $ActionExecOnlyEveryNthTime 3 >> > >> > Modify the *next* action. So you need to specify them in front of the >> action. >> > If you use the $includeConfig option, and have part of the action >> inside the >> > include file and other parts (the statements) outside (or vice >> versa), you >> > never know which action gets configured how. So place all of them >> together. >> > >> > HTH >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> >> Sent: Monday, March 16, 2009 9:15 AM >> >> To: rsyslog-users >> >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> >> >> OK, I narrowed the issues down. ?Now I've faced strange issues like >> >> this before when using the $IncludeConfig directive. >> >> >> >> This is what I have just tested with in my /etc/rsyslog.conf file >> (and >> >> other lines) and it worked fine: >> >> ---- >> >> $IncludeConfig /etc/rsyslog.d/ >> >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all >> >> ---- >> >> >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, >> >> things turn strange and only certain messages are logged from the >> >> first server.: >> >> ---- >> >> $ModLoad ommail >> >> >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat >> >> >> >> $template DYNserver2, "/var/log/remote/server2.log" >> >> $template TraditionalFormatNoHostname,"%timegenerated% >> >> %syslogtag%%msg:::drop-last-lf%\n" >> >> >> >> if $hostname == 'server2.domain.com' then >> >> ?DYNserver2;TraditionalFormatNoHostname >> >> >> >> $ActionMailFrom rsyslog at domain.com >> >> $ActionMailTo server2_alert >> >> $template mailSubjectTestAlert,"INFO: Alert detected" >> >> $template mailBodyTestAlert,"Message is..." >> >> $ActionMailSubject mailSubjectTestAlert >> >> $ActionExecOnlyOnceEveryInterval 300 >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> >> $ActionExecOnlyEveryNthTime 3 >> >> >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some >> message' >> >> then :ommail:;mailBodyTestAlert >> >> ---- >> >> >> >> Now if I add the contents of >> >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf >> (and >> >> remove file /etc/rsyslog.d/testalert_for_another_server) then things >> >> work fine... >> >> >> >> Now if I remove the previous changes to /etc/rsyslog.conf and modify >> >> /etc/rsyslog.d/testalert_for_another_server and remove the following >> >> lines then things work OK again: >> >> $ActionExecOnlyOnceEveryInterval 300 >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> >> $ActionExecOnlyEveryNthTime 3 >> >> >> >> >> >> - Julian >> >> >> >> >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: >> >> > On Sun, 15 Mar 2009, Julian Yap wrote: >> >> > >> >> >> I'm having trouble logging ALL the syslog messages received from >> a >> >> >> server. ?I'm not sure if it's because it's from a non-standard >> piece >> >> >> of hardware (ie. not a Linux server). ?Logging to another server >> >> >> running syslogd works fine (but syslogd doesn't allow me to log >> >> >> messages from a remote server to a separate file and it's not my >> >> >> central syslogd server). >> >> >> >> >> >> I've tried several lines but none seem to work for me: >> >> >> if $fromhost == 'server' then /var/log/remote/server/all >> >> >> if $source == 'server' then /var/log/remote/server/all >> >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> >> >> if $fromhost == 'server.domain.com' then >> /var/log/remote/server/all >> >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all >> >> > >> >> > there are a few possible reasons that this could have problems >> >> > >> >> > is it that you have a high volume of logs and some just get >> dropped? >> >> > >> >> > if you just write everything to a file (*.* /var/log/test) does it >> >> have >> >> > all the logs from this server? or is it missing some? >> >> > >> >> > do the logs from this server sometimes include the host and >> sometimes >> >> not? >> >> > >> >> > what is different between the logs that you match and the ones >> that >> >> you >> >> > miss? >> >> > >> >> > David Lang >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com >> >> > >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 10:22:47 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 10:22:47 +0100 Subject: [rsyslog] Logging all messages from a remote server References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72007@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 10:18 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > Thanks all. My config is working fine now. > > I can take some of the blame for requesting the > $ActionExecOnlyEveryNthTime* params in the first place :P. > > Just to shed some light, my previous understanding (or what I > initially gathered from the docs) was that the $Action params needed > to just be in a block and the order of params didn't matter. > > So: > #start Action > $Action... > $Action... > $Action... > #end Action > > So that was just what I gathered in my head. But it's all clear now. Well, the order doesn't matter BUT (!) above you do NOT define an action - because the action itself is missing! So whatever action comes next, it will receive these parameters. Rainer > > - Julian > > On Sun, Mar 15, 2009 at 11:08 PM, Rainer Gerhards > wrote: > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> Sent: Monday, March 16, 2009 10:05 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> > >> Rainer, > >> > >> Would you recommend against using $includeConfig? ?In that case, it > >> tends to lead to more unknown config issues. > > > > No, but do not split config directives that need to go together over > several > > places. You need to put this together > > > > # this starts the definition of a single action > > $ActionExecOnlyOnceEveryInterval 300 > > $ActionExecOnlyEveryNthTimeTimeout 1200 > > $ActionExecOnlyEveryNthTime 3 > > $... > > *.* ?action > > #this ends it > > > > So you need to put everything together. If you rip it apart, you will > get > > undefined results. > > > > This is - to phrase it politely - not very well documented. You need > to read > > the fine print, most of the $Action... params modify the *next* > action - NOT > > *all* actions. So it is vitally important where they occur. > > > > Will try to make this clear as soon as I have a bit more time. > > > > > > Rainer > >> > >> - Julian > >> > >> On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards > >> wrote: > >> > The issue is that these statements > >> > > >> > $ActionExecOnlyOnceEveryInterval 300 > >> > $ActionExecOnlyEveryNthTimeTimeout 1200 > >> > $ActionExecOnlyEveryNthTime 3 > >> > > >> > Modify the *next* action. So you need to specify them in front of > the > >> action. > >> > If you use the $includeConfig option, and have part of the action > >> inside the > >> > include file and other parts (the statements) outside (or vice > >> versa), you > >> > never know which action gets configured how. So place all of them > >> together. > >> > > >> > HTH > >> > Rainer > >> > > >> >> -----Original Message----- > >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> >> Sent: Monday, March 16, 2009 9:15 AM > >> >> To: rsyslog-users > >> >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> >> > >> >> OK, I narrowed the issues down. ?Now I've faced strange issues > like > >> >> this before when using the $IncludeConfig directive. > >> >> > >> >> This is what I have just tested with in my /etc/rsyslog.conf file > >> (and > >> >> other lines) and it worked fine: > >> >> ---- > >> >> $IncludeConfig /etc/rsyslog.d/ > >> >> :FROMHOST, isequal, "server" > /var/log/remote/server/all > >> >> ---- > >> >> > >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > >> >> things turn strange and only certain messages are logged from the > >> >> first server.: > >> >> ---- > >> >> $ModLoad ommail > >> >> > >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat > >> >> > >> >> $template DYNserver2, "/var/log/remote/server2.log" > >> >> $template TraditionalFormatNoHostname,"%timegenerated% > >> >> %syslogtag%%msg:::drop-last-lf%\n" > >> >> > >> >> if $hostname == 'server2.domain.com' then > >> >> ?DYNserver2;TraditionalFormatNoHostname > >> >> > >> >> $ActionMailFrom rsyslog at domain.com > >> >> $ActionMailTo server2_alert > >> >> $template mailSubjectTestAlert,"INFO: Alert detected" > >> >> $template mailBodyTestAlert,"Message is..." > >> >> $ActionMailSubject mailSubjectTestAlert > >> >> $ActionExecOnlyOnceEveryInterval 300 > >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> >> $ActionExecOnlyEveryNthTime 3 > >> >> > >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some > >> message' > >> >> then :ommail:;mailBodyTestAlert > >> >> ---- > >> >> > >> >> Now if I add the contents of > >> >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf > >> (and > >> >> remove file /etc/rsyslog.d/testalert_for_another_server) then > things > >> >> work fine... > >> >> > >> >> Now if I remove the previous changes to /etc/rsyslog.conf and > modify > >> >> /etc/rsyslog.d/testalert_for_another_server and remove the > following > >> >> lines then things work OK again: > >> >> $ActionExecOnlyOnceEveryInterval 300 > >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> >> $ActionExecOnlyEveryNthTime 3 > >> >> > >> >> > >> >> - Julian > >> >> > >> >> > >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: > >> >> > On Sun, 15 Mar 2009, Julian Yap wrote: > >> >> > > >> >> >> I'm having trouble logging ALL the syslog messages received > from > >> a > >> >> >> server. ?I'm not sure if it's because it's from a non-standard > >> piece > >> >> >> of hardware (ie. not a Linux server). ?Logging to another > server > >> >> >> running syslogd works fine (but syslogd doesn't allow me to > log > >> >> >> messages from a remote server to a separate file and it's not > my > >> >> >> central syslogd server). > >> >> >> > >> >> >> I've tried several lines but none seem to work for me: > >> >> >> if $fromhost == 'server' then /var/log/remote/server/all > >> >> >> if $source == 'server' then /var/log/remote/server/all > >> >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> >> >> if $fromhost == 'server.domain.com' then > >> /var/log/remote/server/all > >> >> >> if $fromhost-ip == '192.168.0.60' then > /var/log/remote/server/all > >> >> > > >> >> > there are a few possible reasons that this could have problems > >> >> > > >> >> > is it that you have a high volume of logs and some just get > >> dropped? > >> >> > > >> >> > if you just write everything to a file (*.* /var/log/test) does > it > >> >> have > >> >> > all the logs from this server? or is it missing some? > >> >> > > >> >> > do the logs from this server sometimes include the host and > >> sometimes > >> >> not? > >> >> > > >> >> > what is different between the logs that you match and the ones > >> that > >> >> you > >> >> > miss? > >> >> > > >> >> > David Lang > >> >> > _______________________________________________ > >> >> > rsyslog mailing list > >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > http://www.rsyslog.com > >> >> > > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Mon Mar 16 16:34:09 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Mon, 16 Mar 2009 16:34:09 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean Message-ID: <49BE7171.3090601@1und1.de> Hi, I've configured rsyslog to use relp as transport protocol. sw version: rsyslog-relp-3.21.3-4 and rsyslog-3.21.3-4. in the log I see these messages: 2009-03-16T16:12:10.769408+01:00 zeus-log01-2 rsyslogd: [origin software="rsyslogd" swVersion="3.21.3" x-pid="3239" x-info="http://www.rsyslog.com"] restart 2009-03-16T16:12:10.769447+01:00 zeus-log01-2 rsyslogd: error -2077 trying to add listener 2009-03-16T16:12:10.769458+01:00 zeus-log01-2 rsyslogd: the last error occured in /data/etc/rsyslog/rsyslog.conf, line 6 2009-03-16T16:12:10.769470+01:00 zeus-log01-2 rsyslogd: the last error occured in /data/etc/rsyslog/rsyslog.conf, line 7 The config line in question read: ------snip # Global $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imrelp.so $UDPServerRun 514 $InputTCPServerRun 514 <-- line 6 $InputRELPServerRun 2514 <-- line 7 $DirCreateMode 0755 -------snap netstat -an| grep 514 shows all configured udp and tcp ports open. So where can I find a description of error -2077? Thanks in advance Thomas From rgerhards at hq.adiscon.com Mon Mar 16 16:39:21 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 16:39:21 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean References: <49BE7171.3090601@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> There should be informtion on that error on the web, but 2077 is "could not bind to port". A short reference can be found in git in file ./runtime/rsyslog.h Not sure where it orginates from in this case... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Monday, March 16, 2009 4:34 PM > To: rsyslog-users > Subject: [rsyslog] what does error -2077 trying to add listener mean > > Hi, > > I've configured rsyslog to use relp as transport protocol. > sw version: rsyslog-relp-3.21.3-4 and rsyslog-3.21.3-4. > > in the log I see these messages: > 2009-03-16T16:12:10.769408+01:00 zeus-log01-2 rsyslogd: [origin > software="rsyslogd" swVersion="3.21.3" x-pid="3239" > x-info="http://www.rsyslog.com"] restart > 2009-03-16T16:12:10.769447+01:00 zeus-log01-2 rsyslogd: error -2077 > trying to add listener > 2009-03-16T16:12:10.769458+01:00 zeus-log01-2 rsyslogd: the last error > occured in /data/etc/rsyslog/rsyslog.conf, line 6 > 2009-03-16T16:12:10.769470+01:00 zeus-log01-2 rsyslogd: the last error > occured in /data/etc/rsyslog/rsyslog.conf, line 7 > > The config line in question read: > > ------snip > # Global > $ModLoad imudp.so > $ModLoad imtcp.so > $ModLoad imrelp.so > $UDPServerRun 514 > $InputTCPServerRun 514 <-- line 6 > $InputRELPServerRun 2514 <-- line 7 > > $DirCreateMode 0755 > > -------snap > > netstat -an| grep 514 shows all configured udp and tcp ports open. > > So where can I find a description of error -2077? > > Thanks in advance > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Mon Mar 16 16:56:31 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Mon, 16 Mar 2009 16:56:31 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> References: <49BE7171.3090601@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> Message-ID: <49BE76AF.4030608@1und1.de> Hi Rainer, there is only one place where RS_RET_COULD_NOT_BIND is returned: runtime/nsd_ptcp.c numSocks = 0; /* num of sockets counter at start of array */ for(r = res; r != NULL ; r = r->ai_next) { sock = socket(r->ai_family, r->ai_socktype, r->ai_protocol); [ lots of magic ] } if(numSocks == 0) { dbgprintf("No TCP listen sockets could successfully be initialized"); ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); } I have no idea why the OS reports the Sockets open and messages get received, maybe there is a minor problem in the code, but somehow it works... Thomas From rgerhards at hq.adiscon.com Mon Mar 16 16:59:52 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 16:59:52 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean References: <49BE7171.3090601@1und1.de><9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> <49BE76AF.4030608@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7200F@GRFEXC.intern.adiscon.com> This sounds like some quirk with IPv6... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Monday, March 16, 2009 4:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] what does error -2077 trying to add listener > mean > > Hi Rainer, > > there is only one place where RS_RET_COULD_NOT_BIND is returned: > > runtime/nsd_ptcp.c > > numSocks = 0; /* num of sockets counter at start of array */ > for(r = res; r != NULL ; r = r->ai_next) { > sock = socket(r->ai_family, r->ai_socktype, r- > >ai_protocol); > > [ lots of magic ] > > } > > if(numSocks == 0) { > dbgprintf("No TCP listen sockets could successfully be > initialized"); > ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); > } > > I have no idea why the OS reports the Sockets open and messages get > received, maybe there is a minor problem in the code, but somehow it > works... > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 16 17:53:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 17:53:40 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch><1236001365.28865.44.camel@rf10up.intern.adiscon.com><200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> Sorry for the delay, it is currently quite busy here at my end :( I have now created a very rough skeleton template output module. You need to pull from git. It is contained in the master branch. So far, it does not perform useful work. I was a bit hesitant to add much more description, because I think this can either be brief and not matching what you need - or very elaborate (bbok-like), for what I currently do not have enough time. I suggest that you have a look at the template module, and then we simply try to get this going. It would be good if you could ask questions or tell me what needs to be placed inside the module. Or I can create yet another skeleton, based on ommysql, that has a bit more logic so that you can fill in the initial Oracle functionality. That will not offer superior performance, but I think it would be a good starting point to pursue the rest of this project. Please let me know what you think. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, March 03, 2009 3:26 PM > To: rsyslog-users > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > Just one quick note, more following: > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > > Sent: Tuesday, March 03, 2009 3:29 PM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > > > Hi there. > > > > > > As I said, I need **excellent** performance. I definitely need > > batch > > > > operations, the ability to prepare the statements given as > > arguments > > > > on the configuration file, and not to commit entries one by one, > > but > > > > after a number of entries are ready or (better) after some not so > > > > small time. According to the advise I got from experts around > here, > > > > I'll have to use Oracle Call Interface for this module, I don't > > know > > > > if there are any licensing issues. > > > > > > I can't comment on the licensing issue, I simply don't know what > > > Oracle demands. > > > > I'm not sure how GPL-compatible it is to link to already existing > > proprietary code. Anyways, first I code, then we test, then we (you, > > actually) decide the legal aspects. > > Actually, not me ;) I leave this risk to the user. If someone pays the > legal counselor, I'll add his POV to the project doc. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Tue Mar 17 20:44:57 2009 From: julianokyap at gmail.com (Julian Yap) Date: Tue, 17 Mar 2009 09:44:57 -1000 Subject: [rsyslog] Dynamic remote log files Message-ID: I have the following set up to generate Dynamic remote log files. $template DYNmessages, "/var/log/remote/%HOSTNAME%/messages" *.info,mail.none,authpriv.none,cron.none ?DYNmessages Unfortunately some devices log poorly without the hostname for some syslog messages. This means I'm ending up with lots of useless directories in /var/log/remote. If I log everything from a server to a file then it works fine: if $fromhost == 'server' then /var/log/remote/server/all As you can see the difference in file sizes as syslog messages are lost: # ls -l /var/log/remote/server/ total 1724 -rw------- 1 root root 980053 Mar 17 08:57 all -rw------- 1 root root 773533 Mar 17 08:57 messages I guess, I'm looking for config suggestions on setting up more robust dynamic logging for remote hosts. - Julian From aoz.syn at gmail.com Tue Mar 17 20:57:14 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 17 Mar 2009 13:57:14 -0600 Subject: [rsyslog] Dynamic remote log files In-Reply-To: References: Message-ID: <4255c2570903171257w4801cc3co8998ca883b5ae78@mail.gmail.com> On Tue, Mar 17, 2009 at 13:44, Julian Yap wrote: > I guess, I'm looking for config suggestions on setting up more robust > dynamic logging for remote hosts. The single most robust host-based structure I've found to use is 'fromhost-ip'. It's locally "generated" by the rsyslog daemon from the receiving socket and isn't affected by any of the message content. From Luis.Fernando.Munoz.Mejias at cern.ch Wed Mar 18 10:53:34 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Wed, 18 Mar 2009 10:53:34 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> Message-ID: <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> El Lunes, 16 de Marzo de 2009 17:53, Rainer Gerhards escribi?: > Please let me know what you think. I just came back from a week of holidays, I'm reviewing the skeleton, which looks pretty comprehensive. Thanks! -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Wed Mar 18 11:04:19 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 18 Mar 2009 11:04:19 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> When you are ready, I'd actually suggest that I create an "omoracle" git branch for you and do place a copy of ommysql into it. This, together with the comments from omtemplate, would probably one way to get a (non-optimal) quick start. I would suggest that we build a very basic oracle driver first and after we see it works well, then look into the performance optimization. Let me know what you think (and when you have time). I could actually create what I have proposed with little delay once you give a go. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Wednesday, March 18, 2009 10:54 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > El Lunes, 16 de Marzo de 2009 17:53, Rainer Gerhards escribi?: > > Please let me know what you think. > > I just came back from a week of holidays, I'm reviewing the skeleton, > which looks pretty comprehensive. Thanks! > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From kenneho.ndu at gmail.com Thu Mar 19 11:51:43 2009 From: kenneho.ndu at gmail.com (Kenneth Holter) Date: Thu, 19 Mar 2009 11:51:43 +0100 Subject: [rsyslog] rsyslog TCP session closing Message-ID: Hi. My rsyslog log host keep getting these messages in syslog: *rsyslogd:TCP session 66 will be closed, error ignored * The session numer (i.e. 66 in this case) varies. Are these messages of any importance? I'm guessing the sessions are closed due to being idle, and that the session will be re-established when the next syslog message are ready to be sent from the client. Regards, Kenneth From rgerhards at hq.adiscon.com Thu Mar 19 12:13:56 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 19 Mar 2009 12:13:56 +0100 Subject: [rsyslog] omfile reliability Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD37@GRFEXC.intern.adiscon.com> Hi all, as mentioned in some past posts, omfile did not really care if log data made it into the file system. The overall reaction was "don't care, discard message". I have now improved the situation very much (at least I hope so ;)). There is currently an experimental git branch omfile-errHandler which properly suspends the action if something goes wrong. The only thing it currently does not do is truncate partially written lines. I'll save this for some later release when I revamp to module as whole. I plan to merge this change into the main development branch soon and then do a new devel release. If you would like to play with the current version, I of course would appreciate that. If so, please let me know your results. Also, I found one strange thing while testing with the cifs (SMB) handler. It does not properly return a failure state, so I currently have no clue how to detect a failure condition in that case. Below, I post some excerpt from a forum thread related to the work [1]. If you happen to have any suggestions, please let me know. ===== good news and bad news: I have found a bug inside the code, and been able to fix that (not yet committed). However, I tried with the smb redirector (don't have nfs at hand) and it acks the writes, but does not ensure data is actually put onto the remote site. So there probably is no way to make sure we really have the data. Maybe the situation is better with NFS. below some excerpts from my twitter stream: # i have lots of garbage inside the log when I reconnect the network... looks like cifs driver can not really handle this situation1 minute ago from twhirl # it is interesting to see how the smb driver continues to accept data (at a very slow rate) while the network is off....9 minutes ago from twhirl # #rsyslog: issue is more complicated than I thought - probably a bug in dynafile creation processabout 1 hour ago from twhirl # ok, think I got a bug. FD is not set to indicated "closed" after actual close call - can lead to endless loopabout 2 hours ago from twhirl # as soon as I enter a new message, the missing content *is* writtenabout 2 hours ago from twhirl # after disconnect, nothing is written...about 2 hours ago from twhirl # very interesting... I get successful returns from write() to the network file - with plugged cable, lazy write, I guess...about 2 hours ago from twhirl # #rsyslog: OS buffering plays a big role in network-file retries - on the initial tries I do not see any error code at all! (w/o cable!!)about 2 hours ago from twhirl ==== Thanks, Rainer [1] http://kb.monitorware.com/log-to-nfs-and-buffer-if-unavailable-t8963-30.html# p15732 From Jefferson.Cowart at libraries.claremont.edu Thu Mar 19 23:18:00 2009 From: Jefferson.Cowart at libraries.claremont.edu (Jefferson Cowart) Date: Thu, 19 Mar 2009 15:18:00 -0700 Subject: [rsyslog] Separating Log files based on partial IP match Message-ID: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> I'm new to rsyslog, and I'm trying to set it up to centralize logging for a number of devices on my network. I'd like for it to log anything from my network switch to a single log file, my printers to another log file, etc. I'm able to separate the devices based on their IP address (e.g. my switches are in one IP subnet and my printers in another.) I see how to do per device logging on http://www.rsyslog.com/Article60.phtml, but I don't see a way to adjust that to do it based on IP subnet or anything like that. Unfortunately it looks like both FROMHOST and HOSTNAME are names not IPs, so it's not even clear if I could filter on that. Any help would be appreciated. Thanks. -- Thank You Jefferson Cowart Network and Systems Administrator Claremont University Consortium From david at lang.hm Fri Mar 20 00:44:36 2009 From: david at lang.hm (david at lang.hm) Date: Thu, 19 Mar 2009 16:44:36 -0700 (PDT) Subject: [rsyslog] Separating Log files based on partial IP match In-Reply-To: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> References: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> Message-ID: On Thu, 19 Mar 2009, Jefferson Cowart wrote: > I'm new to rsyslog, and I'm trying to set it up to centralize logging > for a number of devices on my network. I'd like for it to log anything > from my network switch to a single log file, my printers to another log > file, etc. I'm able to separate the devices based on their IP address > (e.g. my switches are in one IP subnet and my printers in another.) I > see how to do per device logging on > http://www.rsyslog.com/Article60.phtml, but I don't see a way to adjust > that to do it based on IP subnet or anything like that. Unfortunately it > looks like both FROMHOST and HOSTNAME are names not IPs, so it's not > even clear if I could filter on that. Any help would be appreciated. > Thanks. there is fromhost-ip that will give you the last-hop IP address I don't see an easy way to do it based on subnets, but take a look at the rscript stuff that just went into the development branch in the last week or so. that may give you the hooks needed to do the subnet calculation that will let you do what you want. David Lang From rgerhards at hq.adiscon.com Fri Mar 20 07:23:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 20 Mar 2009 07:23:12 +0100 Subject: [rsyslog] Separating Log files based on partial IP match References: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD3E@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, March 20, 2009 12:45 AM > To: rsyslog-users > Subject: Re: [rsyslog] Separating Log files based on partial IP match > > On Thu, 19 Mar 2009, Jefferson Cowart wrote: > > > I'm new to rsyslog, and I'm trying to set it up to centralize logging > > for a number of devices on my network. I'd like for it to log > anything > > from my network switch to a single log file, my printers to another > log > > file, etc. I'm able to separate the devices based on their IP address > > (e.g. my switches are in one IP subnet and my printers in another.) I > > see how to do per device logging on > > http://www.rsyslog.com/Article60.phtml, but I don't see a way to > adjust > > that to do it based on IP subnet or anything like that. Unfortunately > it > > looks like both FROMHOST and HOSTNAME are names not IPs, so it's not > > even clear if I could filter on that. Any help would be appreciated. > > Thanks. > > there is fromhost-ip that will give you the last-hop IP address > > I don't see an easy way to do it based on subnets, but take a look at > the > rscript stuff that just went into the development branch in the last > week > or so. that may give you the hooks needed to do the subnet calculation > that will let you do what you want. The only function currently supported is strlen(), but this is a very interesting use case to extend function support. I think I will add a couple of functions even without a full loadable interface, just to get some basic things done. If everything turns out to go smooth, I can hopefully do this next week. In the mean time, I would see if a property-based (regex) filter can do the job. For a classical class A,B,C net that should be easy to do. Rainer From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 20 18:01:31 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 20 Mar 2009 18:01:31 +0100 Subject: [rsyslog] Starting a native Oracle output module (was Re: Documentation on writing rsyslog modules?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> Message-ID: <200903201801.32116.Luis.Fernando.Munoz.Mejias@cern.ch> El Mi?rcoles, 18 de Marzo de 2009 11:04, Rainer Gerhards escribi?: > When you are ready, I'd actually suggest that I create an "omoracle" git > branch for you and do place a copy of ommysql into it. This, together with > the comments from omtemplate, would probably one way to get a (non-optimal) > quick start. > So, I'm starting it and I already have something that compiles. Next step is to have something I can test, then have something that makes something, then something that does the same but fast. > I would suggest that we build a very basic oracle driver first and after we > see it works well, then look into the performance optimization. > That's my idea, too. I want something that: 1) Connects to the DB at createInstance() time. 2) Runs the un-prepared statement passed as template on each syslog entry. 3) Disconnects only at freeInstance() time. Prepared statements and batch operations will be added later, indeed. But first, I'd like to know what ways I have to test my module, other than recompiling it, installing and restarting rsyslog for each change. > Let me know what you think (and when you have time). I'm already on it. I hope to deliver something for review next week. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 20 18:08:55 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 20 Mar 2009 18:08:55 +0100 Subject: [rsyslog] Starting a native Oracle output module (was Re:Documentation on writing rsyslog modules?) References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> <200903201801.32116.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD51@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Luis > Fernando Mu?oz Mej?as > Sent: Friday, March 20, 2009 6:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Starting a native Oracle output module > (was Re:Documentation on writing rsyslog modules?) > > El Mi?rcoles, 18 de Marzo de 2009 11:04, Rainer Gerhards escribi?: > > When you are ready, I'd actually suggest that I create an > "omoracle" git > > branch for you and do place a copy of ommysql into it. > This, together with > > the comments from omtemplate, would probably one way to get > a (non-optimal) > > quick start. > > > So, I'm starting it and I already have something that compiles. Next > step is to have something I can test, then have something that makes > something, then something that does the same but fast. > > > I would suggest that we build a very basic oracle driver > first and after we > > see it works well, then look into the performance optimization. > > > That's my idea, too. I want something that: > > 1) Connects to the DB at createInstance() time. > 2) Runs the un-prepared statement passed as template on each syslog > entry. > 3) Disconnects only at freeInstance() time. > > Prepared statements and batch operations will be added later, indeed. > > But first, I'd like to know what ways I have to test my module, other > than recompiling it, installing and restarting rsyslog for > each change. You can run rsyslog interactively, that's the key to a useful testing environment. In my development environment, I have a couple of conf files, and a shell script that starts rsyslogd in a variety of test settings (don't forget about running valgrind on it frequently, it safes you a lot of time ;)). I am not at my devel machine right now, but the core command looks something like cp "all plugins" runtime/.libs # or so ./tools/rsyslogd -dn -c 4 -f myconf.conf -M runtime/.libs ... Maybe some more... Then you run rsyslogd for your test, and press ctl-c when you are done. My cycle is Loop edit make run-script End Loop Does this help? Oh, and I have disabled the regular rsyslogd on that devel box. If you don't do, you probably need to add some extra quirks to it. I have also begun to work on some tcl-based tests yesterday, hope to have them in git mid next week. Rainer > > > Let me know what you think (and when you have time). > > I'm already on it. I hope to deliver something for review next week. :) > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 23 18:44:27 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 23 Mar 2009 18:44:27 +0100 Subject: [rsyslog] graph of rsyslog versions and branches Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD64@GRFEXC.intern.adiscon.com> Hi all, I created a condensed graph of rsyslog versions and branches today. I have done this as an example of how a software project evolves (what I'll write about soon), but I think it is also educating for folks interested in rsyslog. If you are interested, please find the entry point at my blog: http://blog.gerhards.net/2009/03/rsyslog-family-tree.html Rainer From pieter.thysebaert at intec.ugent.be Tue Mar 24 12:02:44 2009 From: pieter.thysebaert at intec.ugent.be (pieter.thysebaert at intec.ugent.be) Date: Tue, 24 Mar 2009 12:02:44 +0100 (CET) Subject: [rsyslog] imfile module - input line transformation Message-ID: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Hello rsyslog users, We are currently running a small rsyslog setup (i.e. TCP-based remote logging) in our test environment. This setup is also used to transfer Apache access logs, using the pipe operator in the Apache config and a Bash shell script which calls the "logger" tool to log a message to local rsyslog in a loop like # read first line #... while [ $result -eq 0 ]; do # log $line to $filename logger -p local0.info -t "APACHE" "$filename?$line" read line result=$? done The problem with this approach is twofold. First, we are experiencing performance issues under increased load (all Apache workers in status "L" on the Apache server status page when stress testing). Secondly, in order to resolve the first issue, we thought about moving to the file based input module which would make (we hope) Apache performance less depending on the logging infrastructure - as it would just log to the native filesystem as usual. However, as can be seen above, we're currently transforming the log messages to include the destination filename. On the remote rsyslog server (the receiving end), the messages are logged into a file whose name is dynamically derived from the first part of the log (the part before the first question mark). So my question is: can rsyslog be configured to 1. Read new lines from Apache access log as they become available 2. prepend an arbitrary string to the message (the destination filename) 3. log this transformed message instead of the original Or is there a more "best-practices" approach to do what I want (which is : filter messages on the remote end based on the tag and write them to a dynamically generated filename using regexps) Thanks, Pieter From Luis.Fernando.Munoz.Mejias at cern.ch Tue Mar 24 12:44:27 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?iso-8859-1?q?Mu=F1oz_Mej=EDas?=) Date: Tue, 24 Mar 2009 12:44:27 +0100 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <200903241244.27879.Luis.Fernando.Munoz.Mejias@cern.ch> Hi, > We are currently running a small rsyslog setup (i.e. TCP-based remote > logging) in our test environment. > > This setup is also used to transfer Apache access logs, using the pipe > operator in the Apache config and a Bash shell script which calls the > "logger" tool to log a message to local rsyslog in a loop like > > # read first line > #... > > while [ $result -eq 0 ]; do > # log $line to $filename > logger -p local0.info -t "APACHE" "$filename?$line" > read line > result=$? > done Why not use the CustomLog Apache directive to pipe directly the logger command: ... LogFormat "%b%l%a%h %b%l%a%h ..." logger_pipe CustomLog |/usr/bin/logger -p local0.info -t "apache" logger_pipe It should spawn only one logger process for the whole Apache host, and most likely reduce the load. > Secondly, in order to resolve the first issue, we thought about moving to > the file based input module which would make (we hope) Apache performance > less depending on the logging infrastructure - as it would just log to the > native filesystem as usual. However, as can be seen above, we're currently > transforming the log messages to include the destination filename. > On the remote rsyslog server (the receiving end), the messages are logged > into a file whose name is dynamically derived from the first part of the > log (the part before the first question mark). Again, you can use the LogFormat for that, and let Apache do the work without spawning processes over and over, which is most likely the slow part. > > So my question is: can rsyslog be configured to > 1. Read new lines from Apache access log as they become available > 2. prepend an arbitrary string to the message (the destination > filename) In principle, you should use a template for that (untested): $template TemplateName,"CONSTANT_ARBITRARY_STRING?%message%" if ($programname = "apache") then destination;TemplateName (Although I cannot assure how it behaves with TCP...) Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From THe_ZiPMaN+rsyslog at zipman.it Tue Mar 24 14:24:52 2009 From: THe_ZiPMaN+rsyslog at zipman.it (THe_ZiPMaN+rsyslog at zipman.it) Date: Tue, 24 Mar 2009 14:24:52 +0100 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <49C8DF24.8010507@zipman.it> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 pieter.thysebaert at intec.ugent.be wrote: | # read first line | #... | | while [ $result -eq 0 ]; do | # log $line to $filename | logger -p local0.info -t "APACHE" "$filename?$line" | read line | result=$? | done You are spawning a logger process for each log line... brrrr.... | Or is there a more "best-practices" approach to do what I want (which is : | filter messages on the remote end based on the tag and write them to a | dynamically generated filename using regexps) Personally I do this way: On the apache side for every VirtualHost: ErrorLog "|/usr/bin/logger -p local5.err -t http_example.com" CustomLog "|/usr/bin/logger -p local5.info -t http_example.com" combined On the rsyslog side: # Let the message "untouched" without adding any information for easy # parsing by webalizer & company $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" # Define an archiving policy that allows for simpler analisys and archiving # The number 58 should be tuned for your system. Obviously everything must # be on the same line. $template ArchiveApache,"/var/log/apache/%$YEAR%/%$MONTH%/%$DAY%/%syslogtag:F,58:1%_%syslogseverity-text%.log" # Define the destinations and prevent writing on other standard logs # Put this near the beginning of the conf file :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog :syslogtag,startswith,"http" ~ - -- Flavio Visentin GPG Key: http://www.zipman.it/gpgkey.asc There are only 10 types of people in this world: those who understand binary, and those who don't. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknI3yMACgkQusUmHkh1cnrISACfQNkWSda9yPICMM/ie78SGhLe FOMAniAk8S0coDfgCSNQp/IXGqCRfZd2 =IhIf -----END PGP SIGNATURE----- From aoz.syn at gmail.com Tue Mar 24 17:17:35 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 24 Mar 2009 10:17:35 -0600 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <4255c2570903240917l27354e08jc65a525b67e7c933@mail.gmail.com> > The problem with this approach is twofold. First, we are experiencing > performance issues under increased load (all Apache workers in status "L" > on the Apache server status page when stress testing). I am somewhat surprised neither of the responders did what seems obvious to me and bypass the pipe/execution altogether. Unless someone else here has had a problem doing so, there's no reason you couldn't just use a named pipe on both ends: [shell] mkfifo /var/run/htlog-1 [apache] CustomLog "/var/run/htlog-1" [rsyslog] $ModLoad imfile $InputFileName /var/run/htlog-1 $InputFileTag apache1 $InputFileRunMonitor That puts the logs in rsyslog with no extra executions or running processes; what you do after that for filtering is up to you. The nice thing about using a named pipe is that if the reading process dies, the buffer doesn't go away and you have less chance of losing messages. From erik at readmedia.com Tue Mar 24 20:32:41 2009 From: erik at readmedia.com (Erik Morton) Date: Tue, 24 Mar 2009 15:32:41 -0400 Subject: [rsyslog] Have I made rsyslog synchronous by mistake? Message-ID: <6829D0E0-079A-448C-8766-C190249425C1@readmedia.com> Hello there. I have rsyslog configured to forward logging messages from several application servers to a central log server. It's a Ruby on Rails app and I'm using the SyslogLogger gem to talk to rsyslog. From time to time under moderate volume my application, or more accurately one or more of my application containers, begins to freeze. I haven't been able to pin down the cause, but I did notice a couple of interesting things related to rsyslog. Very soon before the application begins to experience problems the central log file (to which all app servers forward) stops updating. This has happened every time the application has had problems. On a lark I decided to disable rsyslog and instead use the native rails logging framework. Each time this change has completely cleared up all the problems on the site. Obviously this is a grossly unscientific observation but I just can't ignore the coincidence. I'm thinking that I have borked the config of my installation to, somehow, cause this failure. Is it possible that I have configured rsyslog to somehow wait for a successful write to the log file instead of firing and forgetting? Am I required to create a local spool per http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html? Many thanks in advance. This is the configuration for the host: $ModLoad imtcp $InputTCPServerRun 200 *.info;mail.none;authpriv.none;cron.none;my_app.none / var/log/messages authpriv.* /var/log/secure mail.* -/var/log/ maillog cron.* /var/log/cron *.emerg * uucp,news.crit /var/log/spooler local7.* /var/log/ boot.log $outchannel my_app_rotate,/vol/logs/my_app.log,5242880,/usr/bin/ loganalysis /vol/logs/my_app.log !my_app *.* $my_app_rotate Each host then has this in rsyslog.conf !my_app *.* @@log_host:200 And I start rsyslogd on the central log host with SYSLOGD_OPTIONS="- t200 -m 0" From rgerhards at hq.adiscon.com Wed Mar 25 09:45:48 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 25 Mar 2009 09:45:48 +0100 Subject: [rsyslog] rsyslog branches (german content) Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD89@GRFEXC.intern.adiscon.com> Hi all, those of you who understand a bit of German may find this German blog post interesting: http://www.wissenslogs.de/wblogs/blog/mehr-als-bits-und-bytes/allgemein/2009- 03-24/software-evolution It talks about "software evolution" based on rsyslog's development process. While doing so, I think it captures also a lot of the spirit in which versions are created today for rsyslog. Sorry I have no English version currently... Rainer From Luis.Fernando.Munoz.Mejias at cern.ch Thu Mar 26 15:28:30 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Thu, 26 Mar 2009 15:28:30 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 Message-ID: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> Hi, I have a funny problem. Around here we have a number of nodes using old, syslogd, which report to their headnodes, which use rsyslog v3, wich keep relaying till I get a small copy on a test box. This test box uses, since yesterday, rsyslog v4. I noticed that for rsyslog v4, the last relay is considered to be the source host, the real source host is considered to be the syslogtag and everything else is inside the %msg% property. For the default template, I get messages like these: 2009-03-26T00:00:00+01:00 relayhost sourcehost1 cvs: GSSAPI userok: cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG GSS_C_CONF_FLAG 2009-03-26T00:00:00+01:00 relayhost sourcehost2 cvs: GSSAPI userok: cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG GSS_C_CONF_FLAG And, as I used to have a single file per host, I now have a single, huge "relayhost" file. Filters based on source or program name are broken, of course. What did I screw when upgrading? Thanks. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Thu Mar 26 15:30:45 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Mar 2009 15:30:45 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> You screw nothing - that's a bug in v4. You need to pull the latest devel from git ;) A new release is due soon. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Thursday, March 26, 2009 3:29 PM > To: rsyslog-users > Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > Hi, > > I have a funny problem. Around here we have a number of nodes using > old, syslogd, which report to their headnodes, which use rsyslog v3, > wich keep relaying till I get a small copy on a test box. This test box > uses, since yesterday, rsyslog v4. > > I noticed that for rsyslog v4, the last relay is considered to be the > source host, the real source host is considered to be the syslogtag and > everything else is inside the %msg% property. For the default template, > I get messages like these: > > 2009-03-26T00:00:00+01:00 relayhost sourcehost1 cvs: GSSAPI userok: > cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG > GSS_C_CONF_FLAG > 2009-03-26T00:00:00+01:00 relayhost sourcehost2 cvs: GSSAPI userok: > cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG > GSS_C_CONF_FLAG > > And, as I used to have a single file per host, I now have a single, > huge > "relayhost" file. Filters based on source or program name are broken, > of > course. > > What did I screw when upgrading? > > Thanks. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Luis.Fernando.Munoz.Mejias at cern.ch Thu Mar 26 16:24:24 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Thu, 26 Mar 2009 16:24:24 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> Message-ID: <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> El Jueves, 26 de Marzo de 2009 15:30, Rainer Gerhards escribi?: > You screw nothing - that's a bug in v4. You need to pull the latest devel > from git ;) I just tried (if it's "master" branch, I mean), with no success. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Thu Mar 26 17:04:39 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Mar 2009 17:04:39 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> It's the master branch and I am sure I fixed this... mhhh... Need to finally complete what I am working on right now, will look after that... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Thursday, March 26, 2009 4:24 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > El Jueves, 26 de Marzo de 2009 15:30, Rainer Gerhards escribi?: > > You screw nothing - that's a bug in v4. You need to pull the latest > devel > > from git ;) > > I just tried (if it's "master" branch, I mean), with no success. > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From kenneho.ndu at gmail.com Fri Mar 27 16:21:03 2009 From: kenneho.ndu at gmail.com (Kenneth Holter) Date: Fri, 27 Mar 2009 16:21:03 +0100 Subject: [rsyslog] Client syslog messages are logged twice Message-ID: Hi I'm running rsyslog v2.0.6, and have the following setup: rsyslog clients => rsyslog relay => rsyslog master <= rsyslog clients The /etc/rsyslog.conf file at the master has these lines in it: ** *$template DynaFile,"/var/log/syslog-clients/%HOSTNAME%/%$YEAR%/%$MONTH%/system-%HOSTNAME%-%$NOW%.log" *.* -?DynaFile * At my rsyslog master I see that many (or all) the client log messages are logged in two the different places, both under its hostname (i.e. %HOSTNAME% is replaced by the hostname) and under its IP-adresss (i.e. %HOSTNAME% is replaced by the IP-adress). So in effect, all the messages are logged in twice. I figured it might have something to do with reverse DNS, so I added the necessary entries the /etc/hosts-file, but with no success. Does anyone have a clue as to why this is happening? Regards, Kenneth From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 27 18:09:42 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 27 Mar 2009 18:09:42 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> Message-ID: <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> El Jueves, 26 de Marzo de 2009 17:04, Rainer Gerhards escribi?: > It's the master branch and I am sure I fixed this... I'm sorry to say it's not. I just pulled git master branch, rebuilt, reinstalled and no changes. 5 minutes ago I downgraded to v3.20, and my new log files appeared as I expected them to, and my filters work as expected. > mhhh... Need to finally complete what I am working on right now, will > look after that... Of course. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 27 18:21:59 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 27 Mar 2009 18:21:59 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch><200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> Can you send me an on-the-wire sample of those messages (I mean that are invalidly interpreted). I have now created the parser test suite and they would make a good addition, especially as I need to troubleshoot them ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Friday, March 27, 2009 6:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > El Jueves, 26 de Marzo de 2009 17:04, Rainer Gerhards escribi?: > > It's the master branch and I am sure I fixed this... > > I'm sorry to say it's not. I just pulled git master branch, rebuilt, > reinstalled and no changes. > > 5 minutes ago I downgraded to v3.20, and my new log files appeared as I > expected them to, and my filters work as expected. > > > mhhh... Need to finally complete what I am working on right now, will > > look after that... > > Of course. > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 27 19:23:15 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 27 Mar 2009 19:23:15 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> Message-ID: <200903271923.16168.Luis.Fernando.Munoz.Mejias@cern.ch> Rainer, > Can you send me an on-the-wire sample of those messages (I mean that are > invalidly interpreted). I have now created the parser test suite and they > would make a good addition, especially as I need to troubleshoot them ;) > > Rainer Before disclosing enough data I have to ask for permission. I can tell you that the last hop in this relay chain is using rsyslog v3, and that the format I got (tcpdump dixit) for these messages is always like this: <38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) And what gets actually logged for that is: 2009-03-27T19:06:53+01:00 last_hop_server source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) Then, last_hop_server becomes %hostname% and source_server becomes %syslogtag%. This last hop server is using rsyslog v3, so it seems to me I have to instruct v4 that the input is coming in a non-default format. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 27 22:38:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 27 Mar 2009 22:38:06 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 Message-ID: <000d01c9af24$6fb6e7ff$100013ac@intern.adiscon.com> These samples are enough, no need to disclose more. Single lines are sufficient, as long as they can repro the problem :) rainer ----- Urspr?ngliche Nachricht ----- Von: "Luis Fernando Mu?oz Mej?as" An: "rsyslog-users" Gesendet: 27.03.09 19:23 Betreff: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 Rainer, > Can you send me an on-the-wire sample of those messages (I mean that are > invalidly interpreted). I have now created the parser test suite and they > would make a good addition, especially as I need to troubleshoot them ;) > > Rainer Before disclosing enough data I have to ask for permission. I can tell you that the last hop in this relay chain is using rsyslog v3, and that the format I got (tcpdump dixit) for these messages is always like this: <38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) And what gets actually logged for that is: 2009-03-27T19:06:53+01:00 last_hop_server source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) Then, last_hop_server becomes %hostname% and source_server becomes %syslogtag%. This last hop server is using rsyslog v3, so it seems to me I have to instruct v4 that the input is coming in a non-default format. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 08:06:51 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 2 Mar 2009 08:06:51 +0100 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to always use fqdn of sending devices? References: <49993125.2060603@ecker-software.de><4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com><4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com><4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> Hi all, I have (obviously) no strong position in this. I do not object putting distro-specific files into a "contrib" directory and make them available with the tarball *as long as it is clear that I do not support them*. I concur to David that this may be useful and I also concur to Michael that it may cause some confusion. To me, the important point is that I can not support distro-specific things (at least outside of the core code) and that I will not want to create and release dependencies. So if we put some package files into the tarball, that means I will update them if I receivea patch or am asked to pull the, but I will neither verify them nor will I hold releases. So, in short, they will be unmaintained and often not matching the rest of the tarball. HOWEVER, I can see that there are cases where it would be useful to hae those files available. On the other hand, at least for Debian, I think it is possible to obtain the package files from Debian directly (but, granted, it may not have the newest ones, e.g. v4). I have a pragmatic suggestion: if you have package specific files, you can send them to me. I will create a subdirectory for them. There will be a README telling people that this stuff is (from my POV) unmaintained, probably outdated and to be used with care. If a maintainer (like Michael) later decides it was a bad idea to put the files into the tarball, I'll also happily delete them. Does this sound like a workable compromise? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Saturday, February 28, 2009 3:16 AM > To: rsyslog-users > Subject: Re: [rsyslog] Get rsyslog to always use fqdn of > sending devices? > > On Sat, 28 Feb 2009, Michael Biebl wrote: > > >>> > >>> If the fedora bits are kept in an entirely separate > upstream packaging > >>> branch, then I don't really care. > >>> But I wouldn't like to see them (or any debian related > files) shipped > >>> in a release tarball. > >> > >> so how am I (a debian user) supposed to create debian > compatible packages > >> for versions that you don't yet deal with? > >> > >> why couldn't you push the debian related files upstream > and maintain them > >> there? (submitting patches, or git pull requests for updates) > > > > Pretty simple: It's less work for me and Rainer and more flexible. > > Say I (for Debian) start adding the files upstream, so does > Fedora, BSD, etc... > > Now when Rainer wants to make a new release to not have any stale > > packaging files, he would have to ping all package > maintainer first to > > update the build files and push those changes. That simply doesn't > > scale. > > Packaging and upstream software releases should be decoupled. > > > > If you are really interested in the Debian Packaging, you > can grab the > > git repository from [1] and either work from there or at it as a > > "remote" to the rsyslog git repo and merge the debian specific bits. > > it's not that I'm interested in debian packaging, it's that I need to > install the stuff that you haven't decided to ship in debian > yet on my > debian system in such a way that I keep the package manager > happy (and > don't have it overwriting what I've compiled with an update > of an obsolete > version) > > it's not that the upstream version of the files need to be > perfect, but > they should be good enough to avoid the need for users to > have to fight > the packaging system and duplicate your efforts. > > I hate to have to pull in some stuff from your tree and > combine it with > stuff from the upstream tree because I don't know enough to > be sure that > I'm both pulling everything I need and not pulling something > that will > cause grief. > > you've made your decision, count this as one voice > disagreeing with that > decision. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Mon Mar 2 08:56:33 2009 From: david at lang.hm (david at lang.hm) Date: Sun, 1 Mar 2009 23:56:33 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 27 Feb 2009, david at lang.hm wrote: > On Thu, 26 Feb 2009, david at lang.hm wrote: > >> >> this works for reopening the socket each time, but if I uncomment the bind >> the sendto fails (error 22, invalid input) >> >> I haven't yet figured out what I'm missing on the bind that's causing this > > a little more testing and I find that the bind succeeds, but no traffic goes > out unless the source IP exists somewhere on the box (it can be bound to > lo:0, but it needs to exist) > > so the non-local-bind approach may not work :-( > > it's just hit midnight here, so I'm going to call it a night and try again > tomorrow. I abandoned this approach and spent the weekend learning how to do raw sockets. I found a library that makes it not that bad to do (at least for the IPv4 that I've done so far, IPv6 adds some wrinkles) the one thing thats not clear to me at this point is how to find the original source IP of the message. Is that available in a variable inside UDPSend, or is it something that I will have to get earlier in the process and then pass explicitly to UDPSend? David Lang From david at lang.hm Mon Mar 2 10:04:58 2009 From: david at lang.hm (david at lang.hm) Date: Mon, 2 Mar 2009 01:04:58 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: <1235670387.28865.2.camel@rf10up.intern.adiscon.com> References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: On Thu, 26 Feb 2009, Rainer Gerhards wrote: > On Sun, 2009-03-01 at 23:56 -0800, david at lang.hm wrote: >> On Fri, 27 Feb 2009, david at lang.hm wrote: >> >>> On Thu, 26 Feb 2009, david at lang.hm wrote: >>> >>>> >>>> this works for reopening the socket each time, but if I uncomment the bind >>>> the sendto fails (error 22, invalid input) >>>> >>>> I haven't yet figured out what I'm missing on the bind that's causing this >>> >>> a little more testing and I find that the bind succeeds, but no traffic goes >>> out unless the source IP exists somewhere on the box (it can be bound to >>> lo:0, but it needs to exist) >>> >>> so the non-local-bind approach may not work :-( >>> >>> it's just hit midnight here, so I'm going to call it a night and try again >>> tomorrow. >> >> I abandoned this approach and spent the weekend learning how to do raw >> sockets. I found a library that makes it not that bad to do (at least for >> the IPv4 that I've done so far, IPv6 adds some wrinkles) >> >> the one thing thats not clear to me at this point is how to find the >> original source IP of the message. Is that available in a variable inside >> UDPSend, or is it something that I will have to get earlier in the process >> and then pass explicitly to UDPSend? > > Actually, output modules do not receive access to the full message > object. This was originally done for security reasons (do not pass more > than needed). All they can receive is the strings that are passed to > them. So the module would need to be modified so that a second string > (like ommail) is passed and that string needs to be defined as the > to-be-spoofed IP (what also enables to rewrite the source IP). I will look into this. >> From all the discussion, it may make sense to start with a different > output plugin that may later be merged back into the original one... Ok, I won't try to have it do everything and just concentrate on doing the forging. forging on an all IPv4 network is very simple, on an all IPv6 network just a bit harder, it's not clear what to do for a mixed network (for a IPv6 destination and IPv4 source you can do a mapping, but what is the right thing to do for an IPv6 source with a IPv4 destination?) note that the other item (closing the output socket every X messages) should be pretty trivial to add into the existing module and is useful for both TCP and UDP. David Lang > Rainer >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 2 12:51:13 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 2 Mar 2009 12:51:13 +0100 Subject: [rsyslog] Weird fromhost property value References: <49A78F5C.3000400@net-m.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71EE7@GRFEXC.intern.adiscon.com> Can you retry with v4? That should be much cleaner now, maybe relp does not yet provide the resolved info (that is a protocol transport driver [lib] issue). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick Shen > Sent: Friday, February 27, 2009 8:00 AM > To: rsyslog-users > Subject: [rsyslog] Weird fromhost property value > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi All, > > I've utilized rsyslog as my company's central logging server for half a > year. > > Today I encounterd a very weird issue about value of fromhost property. > We use dynamic templates to store logs from clients. > > The template is like below: > > $template > d_hosts,"/var/rsyslog/HOSTS/%fromhost%/%$year%/%$month%/%syslogfacility > -text%_%fromhost%_%$year%_%$month%_%$day > %.log" > > You can see we group logs by fromhost value. > > Today, I did 3 times test that a client named (sobek) sent logs to > central logging server by UDP, TCP and RELP. > > The FQDN of client node is "sobek.net-m.internal", short name is > "sobek", ip address is "172.21.101.13". > > After testing, I got when sending via UDP, the fromhost value is short > name. And via TCP, the value is FQDN. Via RELP, the value is IP > address. > > So I got a very weird directory organization at "/var/rsyslog/HOSTS". > > ####################################################################### > ### > drwxr-x--- 3 root syslog 80 Feb 27 07:24 172.21.101.13 <- RELP > drwxr-x--- 3 root syslog 80 Feb 27 05:58 sobek <- UDP > drwxr-x--- 3 root syslog 80 Feb 27 06:03 sobek.net-m.internal <- TCP > ####################################################################### > ### > > We are running rsyslog 3.20.0 both on client and server. So I wanna > know > if any other has encountered this before? > > Thanks, > Patrick > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFJp49ckHhYtFevC+MRApbbAJ9Dgxtw5mf+ax9D81OZPfh5E9aJPgCdEqF/ > FlkFDJpWr4k6pVV4AQiLhRw= > =cQzr > -----END PGP SIGNATURE----- > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 14:42:45 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:42:45 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200902161141.21380.Luis.Fernando.Munoz.Mejias@cern.ch> <577465F99B41C842AAFBE9ED71E70ABA44FC08@grfint2.intern.adiscon.com> <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <1236001365.28865.44.camel@rf10up.intern.adiscon.com> On Fri, 2009-02-27 at 18:48 +0100, Luis Fernando Mu?oz Mej?as wrote: > Rainer, > > Good and bad news... > > > > That sounds really great. Before you start coding or preparing > > > anything, let me check how well our DBs perform, because it's not > > > yet clear if they'll be able to cope with the high insertion rate we > > > expect. If we don't go for the Oracle database this work doesn't > > > make sense. I bet we'll want the Oracle, anyways. > > > > Sounds fair. > > Good news: I did my tests and, for many tasks I need to do, Oracle is > our way to go. So, I'm willing to write the module, with your > guidance/advise. > so far this sounds good ;) > As I said, I need **excellent** performance. I definitely need batch > operations, the ability to prepare the statements given as arguments on > the configuration file, and not to commit entries one by one, but after > a number of entries are ready or (better) after some not so small > time. According to the advise I got from experts around here, I'll have > to use Oracle Call Interface for this module, I don't know if there are > any licensing issues. I can't comment on the licensing issue, I simply don't know what Oracle demands. On thing to do it is let the output module handle the "combination work" together. The output module is called one per message, however, it does not mean the output must directly write them to the database. It may buffer them until the batch is large enough. But this currently needs to be implemented on the output module basis. Obviously, that will not make coding simpler. If we find a sponsor for the necessary non-trivial extension of the core engine, the output module's task may become much easier. If things go well, such a sponsor may show up... > > It seems I'll have to review how rsyslog's queing modules work... I would suggest not to move into them - but, of course, if you like to... Lol, this is the non-trival task I talked about, there are numerous subtleties and, of course, they are weakly documented (but the inline doc is quite good). > > > > For this evaluation, I already have a timestamp formatter that fits > > > into Oracle, something that can be used with the property replacer, > > > like %timereported:::date-oracle%. > > > The bad news is that this timestamp formatter works perfectly on > interactive sessions (sqlplus) but not on non-interactive ones, f.i, in > Python scripts. You need to call Oracle's to_timestamp(string, format), > and by bloating your code with this ugly function the rfc-3339 formatter > is good enough. So I won't submit this one. > Sounds fair ;) Do you have a time frame for your project? (and maybe a rough overview of the "big picture" - I am always soooo curios ;)) Rainer > Cheers. From rgerhards at hq.adiscon.com Mon Mar 2 14:57:34 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:57:34 +0100 Subject: [rsyslog] Get rsyslog to always use fqdn of sending devices? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44FB9E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FBAF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FBFE@grfint2.intern.adiscon.com> <49993125.2060603@ecker-software.de> <4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com> <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> Message-ID: <1236002254.28865.46.camel@rf10up.intern.adiscon.com> Hi RB, on twitter, I was pointed to rpmforge. Does this sound like something that could be used? Rainer On Thu, 2009-02-26 at 17:49 +0100, Rainer Gerhards wrote: > Hi RB, > > thanks for all your hard work. I am absolutely willing to help make > succeed in that. Just one question before we do down to details. Are > there any other options that we can pursue? I remember, quite some time > ago, that someone posted the idea that some well-known (non-RH, not > EPEL) repositories exist. Unfortunatley, I do no longer know which these > were. > > So the question is: are there any other such repositories where RHEL > users turn to and, if so, can we work with them to achieve our joint > goals? > > Sorry for some backtracking here... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of RB > > Sent: Thursday, February 26, 2009 4:54 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Get rsyslog to always use fqdn of sending > > devices? > > > > On Tue, Feb 17, 2009 at 13:11, RB wrote: > > > Regardless, I'll take the flag and see what I can do to get a > > > readily-accessible reasonably current build available for CentOS-5. > > > > Good & bad news - the good news is the Fedora upstream is very > > responsive, the bad news is I got sidetracked after his response. > > > > I have been told that rsyslog cannot be put in EPEL since it is > > already packaged in RHEL, be that package good or bad. Tomas has > > offered to help with the SPEC should I have any problems, but it looks > > like we're on our own for the time being. > > > > RPM package distribution can be done to various depths. The simplest > > is to just provide both the SRPM and unsigned binary RPMs for a few > > chosen CPU architectures for each packaged release as an HTTP or FTP > > download. This would allow one-off installations (updates would be > > manual) and generally get the package 'out there' for use. Further > > steps would involve signing the binaries and possibly publishing a > > repo that users could subscribe to (using /etc/yum.* or equivalent) > > for automated updates. > > > > Distributing a binary package in whatever form is going to increase > > the load (however mildly) on the project - each release will involve > > compiling and distributing binaries and SRPMs, if not signing them as > > well. I can work with you [Rainer] to automate that process, but as a > > random user I should probably not be doing the compilation and signing > > myself. > > > > So, we have 4 basic questions: > > 1. What versions are desired? > > 2. Are there any rsyslog components or functionality not packaged in > > the Fedora distribution users here would like to see included? > > 3. Do we want to sign the packages? > > 4. Who will perform the compilation/signing? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 14:54:00 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:54:00 +0100 Subject: [rsyslog] Matching hostname and facility? In-Reply-To: <49A6D8CB.1010506@web-ster.com> References: <49A2E460.50604@web-ster.com> <49A5A521.8040107@web-ster.com> <49A6D8CB.1010506@web-ster.com> Message-ID: <1236002040.28865.45.camel@rf10up.intern.adiscon.com> On Thu, 2009-02-26 at 10:00 -0800, Scott Baker wrote: > On 02/25/2009 03:38 PM, (private) HKS wrote: > >> Does this syntax work on rsyslog 2.0.x, that's what my server has on it. > >> I've tried this syntax, but it's not logging. > >> > >> - Scott > > > > > > No, this will require 3+ - which you really should upgrade to anyway. > > That's what I figured... this is my CORE syslog server, so I'll need to > play a good upgrade proceedure. > > Is their documentation on configuration file changes going from 2.x to 3.x? There is a compatiblity guide, I guess this is what you are looking for: http://www.rsyslog.com/doc-v3compatibility.html Rainer > > - Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 3 08:54:16 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 08:54:16 +0100 Subject: [rsyslog] Three questions! Message-ID: Hello! I have a few questions. 1. The man page on the website is really outdated. Is it possible for it to be updated automatically? 2. Is it possible for the man page for rsyslog.conf to be up there too? 3. Can rsyslog handle importing existing log files? e.g. sending the latest entries from /var/log/httpd/somename.acc across rsyslog to a logging server? Thanks! From jackmarrow2 at gmail.com Tue Mar 3 09:05:51 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 09:05:51 +0100 Subject: [rsyslog] rsyslog changelog Message-ID: Hello, Is there a changelog for rsyslog, particularly showing the differences between the current version (3.x) and the 2.x version found in RHEL? Thanks, Jack From rgerhards at hq.adiscon.com Tue Mar 3 09:05:47 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 09:05:47 +0100 Subject: [rsyslog] Three questions! References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F03@GRFEXC.intern.adiscon.com> Hi, you asked just in time. See note on doc here: http://blog.gerhards.net/2009/03/rsyslog-doc-state-of-art.html For the file import, you can do this with imfile: http://www.rsyslog.com/doc-imfile.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 03, 2009 8:54 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Three questions! > > Hello! > > I have a few questions. > > 1. The man page on the website is really outdated. Is it possible for > it to be updated automatically? > 2. Is it possible for the man page for rsyslog.conf to be up there too? > 3. Can rsyslog handle importing existing log files? e.g. sending the > latest entries from /var/log/httpd/somename.acc across rsyslog to a > logging server? > > Thanks! > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Tue Mar 3 09:08:23 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 00:08:23 -0800 (PST) Subject: [rsyslog] rsyslog changelog In-Reply-To: References: Message-ID: On Tue, 3 Mar 2009, jack marrow wrote: > Hello, > > Is there a changelog for rsyslog, particularly showing the differences > between the current version (3.x) and the 2.x version found in RHEL? the best way to see the differences would be through git, however the differences between 2.x and 3.x are going to be so massive that it's going to be hard to see anything useful. what are you looking for? David Lang From rgerhards at hq.adiscon.com Tue Mar 3 09:09:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 09:09:17 +0100 Subject: [rsyslog] rsyslog changelog References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Well, you can see all change log entries by following the "change log" menu item in the menu to the left ;) But it may even be more convenient in that case that you get it directly from git as a single text file: http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=ba2a6c13e22b7 f67401c7edb15ea17d31162bde4;hb=HEAD Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 03, 2009 9:06 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog changelog > > Hello, > > Is there a changelog for rsyslog, particularly showing the differences > between the current version (3.x) and the 2.x version found in RHEL? > > Thanks, > > Jack > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 3 09:16:08 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 09:16:08 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: References: Message-ID: 2009/3/3 : > On Tue, 3 Mar 2009, jack marrow wrote: > >> Hello, >> >> Is there a changelog for rsyslog, particularly showing the differences >> between the current version (3.x) and the 2.x version found in RHEL? > > the best way to see the differences would be through git, however the > differences between 2.x and 3.x are going to be so massive that it's going > to be hard to see anything useful. > > what are you looking for? I need to know which features are in the RHEL 5 version (2.x) and which are in the upstream stable version (3.x). Is there a matrix somewhere? It would be good if there was. I am looking for imfile support, regular expressions (are these perl regular expressions or posix?). Plus the general major differences. Also are actions are supported? Thanks > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Tue Mar 3 13:01:29 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 04:01:29 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: On Mon, 2 Mar 2009, david at lang.hm wrote: > On Thu, 26 Feb 2009, Rainer Gerhards wrote: > >> Actually, output modules do not receive access to the full message >> object. This was originally done for security reasons (do not pass more >> than needed). All they can receive is the strings that are passed to >> them. So the module would need to be modified so that a second string >> (like ommail) is passed and that string needs to be defined as the >> to-be-spoofed IP (what also enables to rewrite the source IP). > > I will look into this. I haven't had time to figure this out yet. >>> From all the discussion, it may make sense to start with a different >> output plugin that may later be merged back into the original one... > > Ok, I won't try to have it do everything and just concentrate on doing the > forging. attached is a diff that turns the UDP forwarding into forging, currently with a fixed from address of 1.1.1.1 port 2 I also needed to modify the makefile to add LIBS = /usr/lib/libnet.a for it to compile in my research, I learned that syslog-ng uses this same library for their forging. so far I have avoided looking at the syslog-ng code (I wanted to understand what was happening on my own, and I also avoid any potential license issues until I can check on them) David Lang From Luis.Fernando.Munoz.Mejias at cern.ch Tue Mar 3 15:28:58 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Tue, 3 Mar 2009 15:28:58 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <1236001365.28865.44.camel@rf10up.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> <1236001365.28865.44.camel@rf10up.intern.adiscon.com> Message-ID: <200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> Hi there. > > As I said, I need **excellent** performance. I definitely need batch > > operations, the ability to prepare the statements given as arguments > > on the configuration file, and not to commit entries one by one, but > > after a number of entries are ready or (better) after some not so > > small time. According to the advise I got from experts around here, > > I'll have to use Oracle Call Interface for this module, I don't know > > if there are any licensing issues. > > I can't comment on the licensing issue, I simply don't know what > Oracle demands. I'm not sure how GPL-compatible it is to link to already existing proprietary code. Anyways, first I code, then we test, then we (you, actually) decide the legal aspects. > On thing to do it is let the output module handle the "combination > work" together. The output module is called one per message, however, > it does not mean the output must directly write them to the > database. It may buffer them until the batch is large enough. But this > currently needs to be implemented on the output module basis. > Obviously, that will not make coding simpler. That's what I expected, indeed. > > It seems I'll have to review how rsyslog's queing modules work... > > I would suggest not to move into them - but, of course, if you like > to... Lol, this is the non-trival task I talked about, there are > numerous subtleties and, of course, they are weakly documented (but > the inline doc is quite good). OK. I'll just have a buffer of entries to be committed. > Do you have a time frame for your project? (and maybe a rough overview > of the "big picture" - I am always soooo curios ;)) Not a full timescale. Let's say that as soon as you can provide me with the documentation/skeleton module most (say 70%) of my work will be developing this output module. Then, when I understand what a bad nightmare OCI is I'll be able to give a full timescale. After looking at ompgsql, it looks like writing output modules is easy if you know what you're doing. ;) Then, I'll be able to provide support for this module (fixing bugs and so on) for a couple of years, so it won't be shoot and forget. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Tue Mar 3 15:26:26 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 15:26:26 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch><1236001365.28865.44.camel@rf10up.intern.adiscon.com> <200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> Just one quick note, more following: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Tuesday, March 03, 2009 3:29 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > Hi there. > > > > As I said, I need **excellent** performance. I definitely need > batch > > > operations, the ability to prepare the statements given as > arguments > > > on the configuration file, and not to commit entries one by one, > but > > > after a number of entries are ready or (better) after some not so > > > small time. According to the advise I got from experts around here, > > > I'll have to use Oracle Call Interface for this module, I don't > know > > > if there are any licensing issues. > > > > I can't comment on the licensing issue, I simply don't know what > > Oracle demands. > > I'm not sure how GPL-compatible it is to link to already existing > proprietary code. Anyways, first I code, then we test, then we (you, > actually) decide the legal aspects. Actually, not me ;) I leave this risk to the user. If someone pays the legal counselor, I'll add his POV to the project doc. Rainer From aoz.syn at gmail.com Tue Mar 3 16:15:10 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 3 Mar 2009 08:15:10 -0700 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to always use fqdn of sending devices? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> References: <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903030715x55403ed9k59253789295ffcba@mail.gmail.com> On Mon, Mar 2, 2009 at 00:06, Rainer Gerhards wrote: > I have a pragmatic suggestion: if you have package specific files, you > can send them to me. I will create a subdirectory for them. There will > be a README telling people that this stuff is (from my POV) > unmaintained, probably outdated and to be used with care. If a > maintainer (like Michael) later decides it was a bad idea to put the > files into the tarball, I'll also happily delete them. > > Does this sound like a workable compromise? It does, but I'm not sure how it will mesh with wanting to provide packages for other distros that aren't so responsive as Debian or up-to-date as Fedora. I'll be happy to provide an RPM specfile for -stable and -dev (since Fedora already does a -beta package) but that may not be sufficient for the general clicky-package group. From aoz.syn at gmail.com Tue Mar 3 16:18:12 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 3 Mar 2009 08:18:12 -0700 Subject: [rsyslog] Get rsyslog to always use fqdn of sending devices? In-Reply-To: <1236002254.28865.46.camel@rf10up.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44FBFE@grfint2.intern.adiscon.com> <49993125.2060603@ecker-software.de> <4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com> <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> <1236002254.28865.46.camel@rf10up.intern.adiscon.com> Message-ID: <4255c2570903030718t73f55871n26d83867c3a3e621@mail.gmail.com> On Mon, Mar 2, 2009 at 06:57, Rainer Gerhards wrote: > on twitter, I was pointed to rpmforge. Does this sound like something > that could be used? That definitely looks viable, I'll submit a request and see how it goes. From danson at rackspace.com Tue Mar 3 23:57:10 2009 From: danson at rackspace.com (Daniel Anson) Date: Tue, 3 Mar 2009 16:57:10 -0600 Subject: [rsyslog] Double quotes Problem Message-ID: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> Does anyone know of a quick and easy template to remove the double quote character from a %msg% before it is inserted into the database (MySQL in my case). I have a %msg% that looks like this: user pid=21214 uid=0 auid=4294967295 msg='PAM setcred: user="oracle" exe="/bin/su" (hostname=?, addr=?, terminal=? result=Success)' I am reading the %msg% from the MySQL database and returning it in JSON formatting. When it encounters a double-quote character, it causes issues. I can always fix the program that returns it in JSON, but I think rsyslog can pre-fix the %msg%. Daniel M. Anson Linux Systems Engineer Rackspace danson at rackspace.com Office: (210)312-5114 Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse at rackspace.com, and delete the original message. Your cooperation is appreciated. From david at lang.hm Wed Mar 4 00:54:14 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 15:54:14 -0800 (PST) Subject: [rsyslog] filtering by message size Message-ID: is it possible to filter by message size? I'm looking at a situation where I would like to send the message via UDP if it's below a given size and by TCP if it's larger. David Lang From david at lang.hm Wed Mar 4 01:42:20 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 16:42:20 -0800 (PST) Subject: [rsyslog] Double quotes Problem In-Reply-To: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> References: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> Message-ID: On Tue, 3 Mar 2009, Daniel Anson wrote: > Does anyone know of a quick and easy template to remove the double quote > character from a %msg% before it is inserted into the database (MySQL in > my case). I have a %msg% that looks like this: > > user pid=21214 uid=0 auid=4294967295 msg='PAM setcred: user="oracle" > exe="/bin/su" (hostname=?, addr=?, terminal=? result=Success)' > > I am reading the %msg% from the MySQL database and returning it in JSON > formatting. When it encounters a double-quote character, it causes > issues. I can always fix the program that returns it in JSON, but I > think rsyslog can pre-fix the %msg%. you will need to change the mySQL template in rsyslog I think you have two options. 1. you can put any valid SQL in the rsyslog config that does the insert, so write SQL that eliminates the quote 2. I think you can change the template to remove the quotes before sending it to MySQL (but this may end up removing quotes needed for MySQL to work) David Lang From rgerhards at hq.adiscon.com Wed Mar 4 07:13:27 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 07:13:27 +0100 Subject: [rsyslog] filtering by message size References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Oh, that's an interesting use case. It is not yet possible. I think we can implement (fairly simple) the size for a field (via the property replacer). However, that does not help you with the resulting size of a template string. I probably also need to check the supporting infrastructure for "greater than" comparisons... Would that help? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 12:54 AM > To: rsyslog-users > Subject: [rsyslog] filtering by message size > > is it possible to filter by message size? > > I'm looking at a situation where I would like to send the > message via UDP > if it's below a given size and by TCP if it's larger. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Wed Mar 4 08:06:03 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:06:03 -0800 (PST) Subject: [rsyslog] filtering by message size In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 4 Mar 2009, Rainer Gerhards wrote: > Oh, that's an interesting use case. It is not yet possible. I think we > can implement (fairly simple) the size for a field (via the property > replacer). However, that does not help you with the resulting size of a > template string. I probably also need to check the supporting > infrastructure for "greater than" comparisons... > > Would that help? yes, I can set the value to something conservative to account for the variable-length fields. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, March 04, 2009 12:54 AM >> To: rsyslog-users >> Subject: [rsyslog] filtering by message size >> >> is it possible to filter by message size? >> >> I'm looking at a situation where I would like to send the >> message via UDP >> if it's below a given size and by TCP if it's larger. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 4 08:10:56 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 08:10:56 +0100 Subject: [rsyslog] filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F30@GRFEXC.intern.adiscon.com> Let me see what I can do - it looks so trivial that I tend to think I have overlooked some subtlety ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:06 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > Oh, that's an interesting use case. It is not yet possible. > I think we > > can implement (fairly simple) the size for a field (via the property > > replacer). However, that does not help you with the > resulting size of a > > template string. I probably also need to check the supporting > > infrastructure for "greater than" comparisons... > > > > Would that help? > > yes, I can set the value to something conservative to account for the > variable-length fields. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >> Sent: Wednesday, March 04, 2009 12:54 AM > >> To: rsyslog-users > >> Subject: [rsyslog] filtering by message size > >> > >> is it possible to filter by message size? > >> > >> I'm looking at a situation where I would like to send the > >> message via UDP > >> if it's below a given size and by TCP if it's larger. > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Wed Mar 4 08:16:59 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:16:59 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: Ok, here is a diff that works. it cycles the source IP address from 32000-42000 (since we are just sending, and not creating a normal socket this should not matter) it needs LIBS = /usr/lib/libnet.a in the Makefile in tools to use it create a template that puts the hostname-ip ahead of what you want to send, similar to $template TraditionalFwdFormat,"%fromhost-ip% <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" *.* @10.0.0.100;TraditionalFwdFormat the one problem right now is that any logs sent from the local box will go out with a source IP of 127.0.0.1 I wasted a bit of time trying to setup filters to use a different template if $myhostname == $fromhost, but apparently the filtering doesn't allow comparing two properties, and then I realized that you have a very high-performance name cache now, so you could easily replace my trivial inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); line with a call to the name lookup and then the %fromhost-ip% could be replaced by %fromhost% in the template and everything would work sanely (assuming forward and reverse name resolution are sane ;-) I haven't tried to do IPv6 yet, I know that it requires more effort to set the IP layer options, but I don't know exactly what yet. I wanted to float this first to see what you think before spending much more time on it. David Lang From rgerhards at hq.adiscon.com Wed Mar 4 08:14:00 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 08:14:00 +0100 Subject: [rsyslog] UDP source forging. References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com><1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> David, Just a quick info: I'll initially create a separate branch for these changes, as I can not go through them in details right now. I'll keep that branch updated and the goal is to move it into the master branch as soon as possible. Thanks for all your hard work! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] UDP source forging. > > Ok, here is a diff that works. > > it cycles the source IP address from 32000-42000 (since we are just > sending, and not creating a normal socket this should not matter) > > it needs LIBS = /usr/lib/libnet.a in the Makefile in tools > > to use it create a template that puts the hostname-ip ahead > of what you > want to send, similar to > > $template TraditionalFwdFormat,"%fromhost-ip% > <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" > > *.* @10.0.0.100;TraditionalFwdFormat > > the one problem right now is that any logs sent from the > local box will go > out with a source IP of 127.0.0.1 > > I wasted a bit of time trying to setup filters to use a > different template > if $myhostname == $fromhost, but apparently the filtering > doesn't allow > comparing two properties, and then I realized that you have a very > high-performance name cache now, so you could easily replace > my trivial > inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); > line with a call to the name lookup and then the > %fromhost-ip% could be > replaced by %fromhost% in the template and everything would > work sanely > (assuming forward and reverse name resolution are sane ;-) > > I haven't tried to do IPv6 yet, I know that it requires more > effort to set > the IP layer options, but I don't know exactly what yet. > > I wanted to float this first to see what you think before > spending much > more time on it. > > David Lang > From david at lang.hm Wed Mar 4 08:32:16 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:32:16 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com><1235670387.28865.2.camel@rf10up.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 4 Mar 2009, Rainer Gerhards wrote: > David, > > Just a quick info: I'll initially create a separate branch for these > changes, as I can not go through them in details right now. I'll keep > that branch updated and the goal is to move it into the master branch as > soon as possible. Thanks for all your hard work! no problem, once you can comment on it I'll work on adding IPv6. one problem I will have at that point is that I don't have any systems that use it (and most of my systems don't even have it compiled into the kernel) one thing that would be very useful for people looking to create additional modules would be if there was a simple example module that did something, but didn't use all the callbacks and helper functions that you have created. trying to untangle those to figure out what's happening is pretty hard. the current imtemplate is close to what's needed, but it is just a little bit too trivial. it's not clear from that exactly where you would do things like opening sockets, initializing global variables, etc. I'm thinking that probably the most trivial example would be a stripped-down version of imudp and omfwd that just did the minimum needed to get the packets in and out. (possibly with one config option, just to show how it is done, but everything else hard-coded) Rainer doesn't need to be the person to do this, if there is someone else who understands the modules and has a little time it would sure help the rest of us. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, March 04, 2009 8:17 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] UDP source forging. >> >> Ok, here is a diff that works. >> >> it cycles the source IP address from 32000-42000 (since we are just >> sending, and not creating a normal socket this should not matter) >> >> it needs LIBS = /usr/lib/libnet.a in the Makefile in tools >> >> to use it create a template that puts the hostname-ip ahead >> of what you >> want to send, similar to >> >> $template TraditionalFwdFormat,"%fromhost-ip% >> <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" >> >> *.* @10.0.0.100;TraditionalFwdFormat >> >> the one problem right now is that any logs sent from the >> local box will go >> out with a source IP of 127.0.0.1 >> >> I wasted a bit of time trying to setup filters to use a >> different template >> if $myhostname == $fromhost, but apparently the filtering >> doesn't allow >> comparing two properties, and then I realized that you have a very >> high-performance name cache now, so you could easily replace >> my trivial >> inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); >> line with a call to the name lookup and then the >> %fromhost-ip% could be >> replaced by %fromhost% in the template and everything would >> work sanely >> (assuming forward and reverse name resolution are sane ;-) >> >> I haven't tried to do IPv6 yet, I know that it requires more >> effort to set >> the IP layer options, but I don't know exactly what yet. >> >> I wanted to float this first to see what you think before >> spending much >> more time on it. >> >> David Lang >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 4 09:49:48 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 09:49:48 +0100 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to alwaysuse fqdn of sending devices? References: <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com><4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> <4255c2570903030715x55403ed9k59253789295ffcba@mail.gmail.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F32@GRFEXC.intern.adiscon.com> RB, Not addressing all the meat of your message (I can't...), I'd like to spell out that if you have something that should go into the tarball, just mail me and I'll see it gets in. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: Tuesday, March 03, 2009 4:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] Packages in tarball - was: RE: Get rsyslog to > alwaysuse fqdn of sending devices? > > On Mon, Mar 2, 2009 at 00:06, Rainer Gerhards > wrote: > > I have a pragmatic suggestion: if you have package specific files, > you > > can send them to me. I will create a subdirectory for them. There > will > > be a README telling people that this stuff is (from my POV) > > unmaintained, probably outdated and to be used with care. If a > > maintainer (like Michael) later decides it was a bad idea to put the > > files into the tarball, I'll also happily delete them. > > > > Does this sound like a workable compromise? > > It does, but I'm not sure how it will mesh with wanting to provide > packages for other distros that aren't so responsive as Debian or > up-to-date as Fedora. I'll be happy to provide an RPM specfile for > -stable and -dev (since Fedora already does a -beta package) but that > may not be sufficient for the general clicky-package group. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From pieter.thysebaert at intec.ugent.be Wed Mar 4 10:06:48 2009 From: pieter.thysebaert at intec.ugent.be (pieter.thysebaert at intec.ugent.be) Date: Wed, 4 Mar 2009 10:06:48 +0100 (CET) Subject: [rsyslog] (no subject) Message-ID: Hello Rsyslog users, I have been trying to get Rsyslogd up and running on HP-UX 11.31 ia64. For what it's worth: my preliminary results can be found on http://wiki.rsyslog.com/index.php/HP-UX Best regards, Pieter From rgerhards at hq.adiscon.com Wed Mar 4 11:53:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 11:53:40 +0100 Subject: [rsyslog] rsyslog on HP-UX References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F33@GRFEXC.intern.adiscon.com> Hi Pieter, thanks for your effort. Some time ago, I did an initial port on HP-UX via their web offering. As far as I remember, it compiled well at that time. However, I do not know what has changed in the mean time and how it "feels" now on that platform. I'll see that I integrate your patch ASAP (but that may take a while). The important thing is that I cannot integrate it as-is but need to make sure it does not break the other platforms. I would appreciate if you could check out interim versions when I have them available. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of > pieter.thysebaert at intec.ugent.be > Sent: Wednesday, March 04, 2009 10:07 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] (no subject) > > Hello Rsyslog users, > > I have been trying to get Rsyslogd up and running on HP-UX 11.31 ia64. > > For what it's worth: my preliminary results can be found on > http://wiki.rsyslog.com/index.php/HP-UX > > Best regards, > Pieter > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From janisg at latnetdc.lv Wed Mar 4 16:20:21 2009 From: janisg at latnetdc.lv (Janis) Date: Wed, 04 Mar 2009 17:20:21 +0200 Subject: [rsyslog] Right regex format for property based filters Message-ID: <49AE9C35.4050605@latnetdc.lv> Hello list. I have a question regarding to rsyslog configuration. What is the correct syntax of property based filter with regex. I'm using this configuration right now, and would like to create date based logfiles for each host - hostA, hostB, hostC. But it doesn't work this way. $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile And when running rsyslog with -d, I got only false matches on this regex. I seems that it tries to match all the text inside quotes instead of regexp. As I have red in man page, and html docs, then regexp should be in POSIX RE format (tryed also everything enclosed in braces). For example, if I change regex like this: :HOSTNAME, regex, "host" -?TplFile Then it works and matches all the hosts (A,B,C), and creates the files for each (well it's the same as using contains). But that doesn't solve the problem, when there isn't equal start prefixes for all hosts. For example if I want to match hosts - dog,cat,cow. Best regards --janis From rgerhards at hq.adiscon.com Wed Mar 4 16:35:59 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 16:35:59 +0100 Subject: [rsyslog] Right regex format for property based filters References: <49AE9C35.4050605@latnetdc.lv> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F35@GRFEXC.intern.adiscon.com> Hi Janis, the regex is Posix BRE, nor ERE. I think the syntax you use is not supported in BRE (as a side-note, this reminds me that I wanted to check what it takes to upgrade them to use ERE, too). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Janis > Sent: Wednesday, March 04, 2009 4:20 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Right regex format for property based filters > > Hello list. > > I have a question regarding to rsyslog configuration. What is the > correct syntax of property based > filter with regex. > > I'm using this configuration right now, and would like to create date > based logfiles for each host - hostA, hostB, hostC. > But it doesn't work this way. > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%- > %$DAY%.log" > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > And when running rsyslog with -d, I got only false matches on this > regex. I seems that it tries to match all the text > inside quotes instead of regexp. As I have red in man page, and html > docs, then regexp should be in POSIX RE format > (tryed also everything enclosed in braces). For example, if I change > regex like this: > > :HOSTNAME, regex, "host" -?TplFile > > Then it works and matches all the hosts (A,B,C), and creates the files > for each (well it's the same as using contains). > But that doesn't solve the problem, when there isn't equal start > prefixes for all hosts. > For example if I want to match hosts - dog,cat,cow. > > Best regards > --janis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:03:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 18:03:17 +0100 Subject: [rsyslog] Right regex format for property based filters References: <49AE9C35.4050605@latnetdc.lv> <9B6E2A8877C38245BFB15CC491A11DA71F35@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F37@GRFEXC.intern.adiscon.com> ERE looks trivial - just seeing if I get it in... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, March 04, 2009 4:36 PM > To: janisg at latnetdc.lv; rsyslog-users > Subject: Re: [rsyslog] Right regex format for property based filters > > Hi Janis, > > the regex is Posix BRE, nor ERE. I think the syntax you use is not > supported in BRE (as a side-note, this reminds me that I wanted to > check > what it takes to upgrade them to use ERE, too). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Janis > > Sent: Wednesday, March 04, 2009 4:20 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] Right regex format for property based filters > > > > Hello list. > > > > I have a question regarding to rsyslog configuration. What is the > > correct syntax of property based > > filter with regex. > > > > I'm using this configuration right now, and would like to create date > > based logfiles for each host - hostA, hostB, hostC. > > But it doesn't work this way. > > > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%- > > %$DAY%.log" > > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > > > And when running rsyslog with -d, I got only false matches on this > > regex. I seems that it tries to match all the text > > inside quotes instead of regexp. As I have red in man page, and html > > docs, then regexp should be in POSIX RE format > > (tryed also everything enclosed in braces). For example, if I change > > regex like this: > > > > :HOSTNAME, regex, "host" -?TplFile > > > > Then it works and matches all the hosts (A,B,C), and creates the > files > > for each (well it's the same as using contains). > > But that doesn't solve the problem, when there isn't equal start > > prefixes for all hosts. > > For example if I want to match hosts - dog,cat,cow. > > > > Best regards > > --janis > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:38:44 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 04 Mar 2009 18:38:44 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <49AE9C35.4050605@latnetdc.lv> References: <49AE9C35.4050605@latnetdc.lv> Message-ID: <1236188324.27835.2.camel@rf10up.intern.adiscon.com> Janis, I have added ERE filter support to the devel branch and your use case described below now works - you just need to use "ereregexp" instead of "regexp". No release tarball yet, the patch is here: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952 I did some quick checks, but would appreciate if some others try it out. Rainer On Wed, 2009-03-04 at 17:20 +0200, Janis wrote: > Hello list. > > I have a question regarding to rsyslog configuration. What is the > correct syntax of property based > filter with regex. > > I'm using this configuration right now, and would like to create date > based logfiles for each host - hostA, hostB, hostC. > But it doesn't work this way. > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > And when running rsyslog with -d, I got only false matches on this > regex. I seems that it tries to match all the text > inside quotes instead of regexp. As I have red in man page, and html > docs, then regexp should be in POSIX RE format > (tryed also everything enclosed in braces). For example, if I change > regex like this: > > :HOSTNAME, regex, "host" -?TplFile > > Then it works and matches all the hosts (A,B,C), and creates the files > for each (well it's the same as using contains). > But that doesn't solve the problem, when there isn't equal start > prefixes for all hosts. > For example if I want to match hosts - dog,cat,cow. > > Best regards > --janis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:56:05 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 04 Mar 2009 18:56:05 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <1236188324.27835.2.camel@rf10up.intern.adiscon.com> References: <49AE9C35.4050605@latnetdc.lv> <1236188324.27835.2.camel@rf10up.intern.adiscon.com> Message-ID: <1236189365.27835.19.camel@rf10up.intern.adiscon.com> All, I introduced a memory leak with the ERE enhancement. It is fixed now. So be sure to apply all patches after the one I mentioned. For your convenience, I created a temporary tarball based on the fixed version. It is available at http://download.rsyslog.com/rsyslog/tmp.tar.gz The tarball claims to contain 4.1.4, but you should not count on that it is equal to the released version. I will *not* care any more about this tarball. But I think it is useful to have a version right at hand. Also, this doesn't require any autotools tricks ;) Rainer On Wed, 2009-03-04 at 18:38 +0100, Rainer Gerhards wrote: > Janis, > > I have added ERE filter support to the devel branch and your use case > described below now works - you just need to use "ereregexp" instead of > "regexp". No release tarball yet, the patch is here: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952 > > I did some quick checks, but would appreciate if some others try it out. > > Rainer > > On Wed, 2009-03-04 at 17:20 +0200, Janis wrote: > > Hello list. > > > > I have a question regarding to rsyslog configuration. What is the > > correct syntax of property based > > filter with regex. > > > > I'm using this configuration right now, and would like to create date > > based logfiles for each host - hostA, hostB, hostC. > > But it doesn't work this way. > > > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" > > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > > > And when running rsyslog with -d, I got only false matches on this > > regex. I seems that it tries to match all the text > > inside quotes instead of regexp. As I have red in man page, and html > > docs, then regexp should be in POSIX RE format > > (tryed also everything enclosed in braces). For example, if I change > > regex like this: > > > > :HOSTNAME, regex, "host" -?TplFile > > > > Then it works and matches all the hosts (A,B,C), and creates the files > > for each (well it's the same as using contains). > > But that doesn't solve the problem, when there isn't equal start > > prefixes for all hosts. > > For example if I want to match hosts - dog,cat,cow. > > > > Best regards > > --janis > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Wed Mar 4 20:39:56 2009 From: mbiebl at gmail.com (Michael Biebl) Date: Wed, 4 Mar 2009 20:39:56 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <1236189365.27835.19.camel@rf10up.intern.adiscon.com> References: <49AE9C35.4050605@latnetdc.lv> <1236188324.27835.2.camel@rf10up.intern.adiscon.com> <1236189365.27835.19.camel@rf10up.intern.adiscon.com> Message-ID: 2009/3/4 Rainer Gerhards : > All, > > I introduced a memory leak with the ERE enhancement. It is fixed now. So > be sure to apply all patches after the one I mentioned. > > For your convenience, I created a temporary tarball based on the fixed > version. It is available at > > http://download.rsyslog.com/rsyslog/tmp.tar.gz > > The tarball claims to contain 4.1.4, but you should not count on that it > is equal to the released version. I will *not* care any more about this > tarball. But I think it is useful to have a version right at hand. Also, > this doesn't require any autotools tricks ;) Rainer, gitweb has the nice snapshot feature, which allows to download a tarball for a given SHA1 [1] It doesn't contain the build system, so requires a "autoreconf -vfi" run, but otherwise it should work just fine. Cheers, Michael [1] http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=42db7de5968d2db0fa855a9f029f6bccc0a30650;sf=tgz -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Thu Mar 5 18:52:34 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 5 Mar 2009 18:52:34 +0100 Subject: [rsyslog] rsyslog on Solaris Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Hi all, I have spent some time integrating the Solaris patches the past days (actually, learning [installing] Solaris took the most time). Now I have an environment and the compile process works rather well. However, there seem to be some issue with building the archives. I have to admit I am a bit clueless. After my sig is a build log of the affected part. I would appreciate if someone could provide some hints. Thanks, Rainer Making all in runtime make[2]: Entering directory `/root/rsyslog/runtime' /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o librsyslog.la librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la-linkedlist.lo librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo librsyslog_la-datetime.lo librsyslog_la-srutils.lo librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo librsyslog_la-queue.lo librsyslog_la-cfsysline.lo librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la-parse.lo librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o .libs/librsyslog_la-template.o make[2]: *** [librsyslog.la] Error 1 make[2]: Leaving directory `/root/rsyslog/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog' make: *** [all] Error 2 From epiphani at gmail.com Thu Mar 5 19:01:14 2009 From: epiphani at gmail.com (Aaron Wiebe) Date: Thu, 5 Mar 2009 13:01:14 -0500 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Message-ID: Hey Rainer, On Thu, Mar 5, 2009 at 12:52 PM, Rainer Gerhards wrote: > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > make[2]: *** [librsyslog.la] Error 1 > make[2]: Leaving directory `/root/rsyslog/runtime' First guess, what is that 'false' doing there? That would make the command return nonzero to make, hence the error code. -Aaron From david at ecker-software.de Thu Mar 5 19:33:11 2009 From: david at ecker-software.de (David Ecker) Date: Thu, 05 Mar 2009 19:33:11 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Message-ID: <49B01AE7.8080406@ecker-software.de> Hi, found the following in a another forum: the problem is resolved. the $PATH didn't include the /usr/ccs/bin, so the configure script couldn't find the ar. just a guess, see http://www.fantasticunix.com/forum/general-solaris-discussion/212026-mono-solaris-8-a.html bye David Ecker Rainer Gerhards schrieb: > Hi all, > > I have spent some time integrating the Solaris patches the past days > (actually, learning [installing] Solaris took the most time). > > Now I have an environment and the compile process works rather well. > However, there seem to be some issue with building the archives. I have > to admit I am a bit clueless. After my sig is a build log of the > affected part. > > I would appreciate if someone could provide some hints. > > Thanks, > Rainer > > Making all in runtime > make[2]: Entering directory `/root/rsyslog/runtime' > /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > -Wmissing-format-attribute -g -o librsyslog.la > librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo > librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la-linkedlist.lo > librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo > librsyslog_la-datetime.lo librsyslog_la-srutils.lo > librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo > librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo > librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo > librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo > librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo > librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo > librsyslog_la-queue.lo librsyslog_la-cfsysline.lo > librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la-parse.lo > librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o > .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o > .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o > .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o > .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o > .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o > .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o > .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o > .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o > .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o > .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o > .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o > .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o > .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o > .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o > .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o > .libs/librsyslog_la-template.o > make[2]: *** [librsyslog.la] Error 1 > make[2]: Leaving directory `/root/rsyslog/runtime' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/rsyslog' > make: *** [all] Error 2 > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From thomas.mieslinger at 1und1.de Fri Mar 6 10:17:48 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 10:17:48 +0100 Subject: [rsyslog] wrong permissons on directories Message-ID: <49B0EA3C.1060104@1und1.de> Hi *, when creating directories through dynamic templates, the directory permissons are incomplete: rsyslog.conf: $template ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%-%$MONTH%-%$DAY%.log" resulting directories: ls -al /data/log drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ ls -al /data/log/zeusmw drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ # rsyslogd -version rsyslogd 3.21.3, compiled with: FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No Runtime Instrumentation (slow code): No (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) I'd be happy to know if thats a bug. Thanks Thomas From thomas.mieslinger at 1und1.de Fri Mar 6 10:22:54 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 10:22:54 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <49B01AE7.8080406@ecker-software.de> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> <49B01AE7.8080406@ecker-software.de> Message-ID: <49B0EB6E.1050209@1und1.de> Hi, is that code modified for Solaris already available in git? Could you please send me a pointer to a checkout location? Thanks Thomas From rgerhards at hq.adiscon.com Fri Mar 6 11:41:24 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 11:41:24 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> Yes, it is part of the regular git tree: http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris $ git clone git://git.adiscon.com/git/rsyslog.git then checkout the "solaris" branch: $ git checkout --track -b solaris origin/solaris Rainer PS: commands may be wrong ;) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 10:23 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Hi, > > is that code modified for Solaris already available in git? Could you > please send me a pointer to a checkout location? > > Thanks > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 12:17:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 12:17:40 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> Hi Thomas, can it be that your default umask gets into your way? In any case, you can set the permissions explicitely with $FileCreateMode $FileGroup $FileOwner And set the umask with $umask (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) Does this help? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 10:18 AM > To: rsyslog-users > Subject: [rsyslog] wrong permissons on directories > > Hi *, > > when creating directories through dynamic templates, the directory > permissons are incomplete: > > rsyslog.conf: > $template > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%- > %$MONTH%-%$DAY%.log" > > resulting directories: > ls -al /data/log > drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ > > ls -al /data/log/zeusmw > drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ > > # rsyslogd -version > rsyslogd 3.21.3, compiled with: > FEATURE_REGEXP: Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > GSSAPI Kerberos 5 support: Yes > FEATURE_DEBUG (debug build, slow code): No > Runtime Instrumentation (slow code): No > > (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > > I'd be happy to know if thats a bug. > > Thanks > Thomas > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 14:07:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 14:07:06 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> <49B01AE7.8080406@ecker-software.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F60@GRFEXC.intern.adiscon.com> Thanks to you and Aaron, It was a combination of ar not being present plus autoconfig then using false... So that was a purely environment-base thing. Now I am one step further and the next issue is a pthreads linker error message ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of David Ecker > Sent: Thursday, March 05, 2009 7:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Hi, > > found the following in a another forum: > > > the problem is resolved. the $PATH didn't include the /usr/ccs/bin, so > the configure script couldn't find the ar. > > > just a guess, see > http://www.fantasticunix.com/forum/general-solaris-discussion/212026- > mono-solaris-8-a.html > > bye > David Ecker > > Rainer Gerhards schrieb: > > Hi all, > > > > I have spent some time integrating the Solaris patches the past days > > (actually, learning [installing] Solaris took the most time). > > > > Now I have an environment and the compile process works rather well. > > However, there seem to be some issue with building the archives. I > have > > to admit I am a bit clueless. After my sig is a build log of the > > affected part. > > > > I would appreciate if someone could provide some hints. > > > > Thanks, > > Rainer > > > > Making all in runtime > > make[2]: Entering directory `/root/rsyslog/runtime' > > /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > > -Wmissing-format-attribute -g -o librsyslog.la > > librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo > > librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la- > linkedlist.lo > > librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo > > librsyslog_la-datetime.lo librsyslog_la-srutils.lo > > librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo > > librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo > > librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo > > librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo > > librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo > > librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo > > librsyslog_la-queue.lo librsyslog_la-cfsysline.lo > > librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la- > parse.lo > > librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt > > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > > .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o > > .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o > > .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o > > .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o > > .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o > > .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o > > .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o > > .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o > > .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o > > .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o > > .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o > > .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o > > .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o > > .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o > > .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o > > .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o > > .libs/librsyslog_la-template.o > > make[2]: *** [librsyslog.la] Error 1 > > make[2]: Leaving directory `/root/rsyslog/runtime' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/root/rsyslog' > > make: *** [all] Error 2 > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > From thomas.mieslinger at 1und1.de Fri Mar 6 14:37:49 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 14:37:49 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> Message-ID: <49B1272D.4010408@1und1.de> Rainer Gerhards wrote: > Yes, it is part of the regular git tree: > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > $ git clone git://git.adiscon.com/git/rsyslog.git > then checkout the "solaris" branch: > $ git checkout --track -b solaris origin/solaris That worked. Thanks. What is the minimal required autoconf/automake Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU automake) 1.10.1 which came which opensolaris. it complains about undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? Thanks Thomas From rgerhards at hq.adiscon.com Fri Mar 6 14:39:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 14:39:06 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> <49B1272D.4010408@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F62@GRFEXC.intern.adiscon.com> I am using 2.63 on Solaris 10 x64 and I just successfully compiled. I am about to write a few notes about the state of solaris development in a few moments. My twitter feed may also be useful for you: http://twitter.com/rgerhards My environment is described on http://wiki.rsyslog.com/index.php/Solaris Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 2:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Rainer Gerhards wrote: > > Yes, it is part of the regular git tree: > > > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > > $ git clone git://git.adiscon.com/git/rsyslog.git > > then checkout the "solaris" branch: > > $ git checkout --track -b solaris origin/solaris > > That worked. Thanks. What is the minimal required autoconf/automake > Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU > automake) 1.10.1 which came which opensolaris. it complains about > undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? > > Thanks Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Fri Mar 6 15:13:55 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 15:13:55 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> Message-ID: <49B12FA3.2030202@1und1.de> Thanks for the pointer to the documentation.. it is $DirCreateMode what I asked for... and now I ask for a change of the default documentation says: Default: 0644 Reality demands 0755. I changed it in my configuration. I'd be happy to see that changed in rsyslog. Thomas Rainer Gerhards wrote: > Hi Thomas, > > can it be that your default umask gets into your way? In any case, you > can set the permissions explicitely with > > $FileCreateMode > $FileGroup > $FileOwner > > And set the umask with > > $umask > > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > > Does this help? > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> Sent: Friday, March 06, 2009 10:18 AM >> To: rsyslog-users >> Subject: [rsyslog] wrong permissons on directories >> >> Hi *, >> >> when creating directories through dynamic templates, the directory >> permissons are incomplete: >> >> rsyslog.conf: >> $template >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%- >> %$MONTH%-%$DAY%.log" >> >> resulting directories: >> ls -al /data/log >> drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ >> >> ls -al /data/log/zeusmw >> drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ >> >> # rsyslogd -version >> rsyslogd 3.21.3, compiled with: >> FEATURE_REGEXP: Yes >> FEATURE_LARGEFILE: Yes >> FEATURE_NETZIP (message compression): Yes >> GSSAPI Kerberos 5 support: Yes >> FEATURE_DEBUG (debug build, slow code): No >> Runtime Instrumentation (slow code): No >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) >> >> I'd be happy to know if thats a bug. >> >> Thanks >> Thomas >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- Thomas Mieslinger IT Infrastructure Systems Telefon: +49-721-91374-4404 E-Mail: thomas.mieslinger at 1und1.de 1&1 Internet AG Brauerstra?e 48 76135 Karlsruhe Amtsgericht Montabaur HRB 6484 Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver Mauss, Jan Oetjen Aufsichtsratsvorsitzender: Michael Scheeren From rgerhards at hq.adiscon.com Fri Mar 6 15:19:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 15:19:06 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Thomas, do I correctly understand that you propose the default be changed? If so, I am hesitant to do that - wouldn't that potentially break existing deployments? On the other hand... how could that work... Umm... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 3:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > Thanks for the pointer to the documentation.. it is $DirCreateMode what > I asked for... > > and now I ask for a change of the default > documentation says: > Default: 0644 > > Reality demands 0755. I changed it in my configuration. I'd be happy to > see that changed in rsyslog. > > Thomas > > > > Rainer Gerhards wrote: > > Hi Thomas, > > > > can it be that your default umask gets into your way? In any case, > you > > can set the permissions explicitely with > > > > $FileCreateMode > > $FileGroup > > $FileOwner > > > > And set the umask with > > > > $umask > > > > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > > > > Does this help? > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> Sent: Friday, March 06, 2009 10:18 AM > >> To: rsyslog-users > >> Subject: [rsyslog] wrong permissons on directories > >> > >> Hi *, > >> > >> when creating directories through dynamic templates, the directory > >> permissons are incomplete: > >> > >> rsyslog.conf: > >> $template > >> > > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- > %$YEAR%- > >> %$MONTH%-%$DAY%.log" > >> > >> resulting directories: > >> ls -al /data/log > >> drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ > >> > >> ls -al /data/log/zeusmw > >> drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ > >> > >> # rsyslogd -version > >> rsyslogd 3.21.3, compiled with: > >> FEATURE_REGEXP: Yes > >> FEATURE_LARGEFILE: Yes > >> FEATURE_NETZIP (message compression): Yes > >> GSSAPI Kerberos 5 support: Yes > >> FEATURE_DEBUG (debug build, slow code): No > >> Runtime Instrumentation (slow code): No > >> > >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > >> > >> I'd be happy to know if thats a bug. > >> > >> Thanks > >> Thomas > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > -- > Thomas Mieslinger > IT Infrastructure Systems > Telefon: +49-721-91374-4404 > E-Mail: thomas.mieslinger at 1und1.de > > 1&1 Internet AG > Brauerstra?e 48 > 76135 Karlsruhe > > Amtsgericht Montabaur HRB 6484 > Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas > Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver > Mauss, Jan Oetjen > Aufsichtsratsvorsitzender: Michael Scheeren > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Fri Mar 6 15:54:24 2009 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 6 Mar 2009 15:54:24 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: FWIW, the Debian default rsyslog.conf ships with $DirCreateMode 0755 2009/3/6 Rainer Gerhards : > Thomas, > > do I correctly understand that you propose the default be changed? > > If so, I am hesitant to do that - wouldn't that potentially break existing deployments? On the other hand... how could that work... Umm... > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> Sent: Friday, March 06, 2009 3:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] wrong permissons on directories >> >> Thanks for the pointer to the documentation.. it is $DirCreateMode what >> I asked for... >> >> and now I ask for a change of the default >> documentation says: >> Default: 0644 >> >> Reality demands 0755. I changed it in my configuration. I'd be happy to >> see that changed in rsyslog. >> >> Thomas >> >> >> >> Rainer Gerhards wrote: >> > Hi Thomas, >> > >> > can it be that your default umask gets into your way? In any case, >> you >> > can set the permissions explicitely with >> > >> > $FileCreateMode >> > $FileGroup >> > $FileOwner >> > >> > And set the umask with >> > >> > $umask >> > >> > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) >> > >> > Does this help? >> > >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> >> Sent: Friday, March 06, 2009 10:18 AM >> >> To: rsyslog-users >> >> Subject: [rsyslog] wrong permissons on directories >> >> >> >> Hi *, >> >> >> >> when creating directories through dynamic templates, the directory >> >> permissons are incomplete: >> >> >> >> rsyslog.conf: >> >> $template >> >> >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- >> %$YEAR%- >> >> %$MONTH%-%$DAY%.log" >> >> >> >> resulting directories: >> >> ls -al /data/log >> >> drw-r--r-- 3 root root 4096 Mar ?5 15:53 zeusmw/ >> >> >> >> ls -al /data/log/zeusmw >> >> drw-r--r-- 2 root root 4096 Mar ?6 10:11 2009-03/ >> >> >> >> # rsyslogd -version >> >> rsyslogd 3.21.3, compiled with: >> >> ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >> ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?Yes >> >> ? ?FEATURE_NETZIP (message compression): ? Yes >> >> ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >> ? ?FEATURE_DEBUG (debug build, slow code): No >> >> ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) >> >> >> >> I'd be happy to know if thats a bug. >> >> >> >> Thanks >> >> Thomas >> >> >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> >> -- >> Thomas Mieslinger >> IT Infrastructure Systems >> Telefon: +49-721-91374-4404 >> E-Mail: thomas.mieslinger at 1und1.de >> >> 1&1 Internet AG >> Brauerstra?e 48 >> 76135 Karlsruhe >> >> Amtsgericht Montabaur HRB 6484 >> Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas >> Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver >> Mauss, Jan Oetjen >> Aufsichtsratsvorsitzender: Michael Scheeren >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From thomas.mieslinger at 1und1.de Fri Mar 6 16:17:30 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 16:17:30 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: <49B13E8A.2080308@1und1.de> I guess nobody did let rsyslog make directories. Rainer Gerhards wrote: > Thomas, > > do I correctly understand that you propose the default be changed? Yepp. > If so, I am hesitant to do that - wouldn't that potentially break existing deployments? hmm Maybe I haven't seen enough yet, but I can't imagine a deployment built on directory permissions 644.... > On the other hand... how could that work... Umm... They are all working as root out there :-) I think it would be good if you just double check it yourself that the directories get created with 644 and decicde on your findings. Thomas From rgerhards at hq.adiscon.com Fri Mar 6 16:40:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 16:40:12 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com><49B1272D.4010408@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F62@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F67@GRFEXC.intern.adiscon.com> I have just finished my "current state" writeup on rsyslog and solaris: http://blog.gerhards.net/2009/03/rsyslog-and-solaris.html I guess it contains some useful information ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, March 06, 2009 2:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > I am using 2.63 on Solaris 10 x64 and I just successfully compiled. I > am > about to write a few notes about the state of solaris development in a > few moments. My twitter feed may also be useful for you: > > http://twitter.com/rgerhards > > My environment is described on > > http://wiki.rsyslog.com/index.php/Solaris > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > > Sent: Friday, March 06, 2009 2:38 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog on Solaris > > > > Rainer Gerhards wrote: > > > Yes, it is part of the regular git tree: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > > > $ git clone git://git.adiscon.com/git/rsyslog.git > > > then checkout the "solaris" branch: > > > $ git checkout --track -b solaris origin/solaris > > > > That worked. Thanks. What is the minimal required autoconf/automake > > Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU > > automake) 1.10.1 which came which opensolaris. it complains about > > undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? > > > > Thanks Thomas > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 16:40:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 16:40:12 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com><49B12FA3.2030202@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> The more I think about it, the more it smells like a real bug. Has anyone objections changing the default? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Friday, March 06, 2009 3:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > FWIW, the Debian default rsyslog.conf ships with > > $DirCreateMode 0755 > > > 2009/3/6 Rainer Gerhards : > > Thomas, > > > > do I correctly understand that you propose the default be changed? > > > > If so, I am hesitant to do that - wouldn't that potentially break > existing deployments? On the other hand... how could that work... > Umm... > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> Sent: Friday, March 06, 2009 3:14 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] wrong permissons on directories > >> > >> Thanks for the pointer to the documentation.. it is $DirCreateMode > what > >> I asked for... > >> > >> and now I ask for a change of the default > >> documentation says: > >> Default: 0644 > >> > >> Reality demands 0755. I changed it in my configuration. I'd be happy > to > >> see that changed in rsyslog. > >> > >> Thomas > >> > >> > >> > >> Rainer Gerhards wrote: > >> > Hi Thomas, > >> > > >> > can it be that your default umask gets into your way? In any case, > >> you > >> > can set the permissions explicitely with > >> > > >> > $FileCreateMode > >> > $FileGroup > >> > $FileOwner > >> > > >> > And set the umask with > >> > > >> > $umask > >> > > >> > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > >> > > >> > Does this help? > >> > > >> > Rainer > >> > > >> >> -----Original Message----- > >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> >> Sent: Friday, March 06, 2009 10:18 AM > >> >> To: rsyslog-users > >> >> Subject: [rsyslog] wrong permissons on directories > >> >> > >> >> Hi *, > >> >> > >> >> when creating directories through dynamic templates, the > directory > >> >> permissons are incomplete: > >> >> > >> >> rsyslog.conf: > >> >> $template > >> >> > >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- > >> %$YEAR%- > >> >> %$MONTH%-%$DAY%.log" > >> >> > >> >> resulting directories: > >> >> ls -al /data/log > >> >> drw-r--r-- 3 root root 4096 Mar ?5 15:53 zeusmw/ > >> >> > >> >> ls -al /data/log/zeusmw > >> >> drw-r--r-- 2 root root 4096 Mar ?6 10:11 2009-03/ > >> >> > >> >> # rsyslogd -version > >> >> rsyslogd 3.21.3, compiled with: > >> >> ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >> >> ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?Yes > >> >> ? ?FEATURE_NETZIP (message compression): ? Yes > >> >> ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >> >> ? ?FEATURE_DEBUG (debug build, slow code): No > >> >> ? ?Runtime Instrumentation (slow code): ? ?No > >> >> > >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > >> >> > >> >> I'd be happy to know if thats a bug. > >> >> > >> >> Thanks > >> >> Thomas > >> >> > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > >> -- > >> Thomas Mieslinger > >> IT Infrastructure Systems > >> Telefon: +49-721-91374-4404 > >> E-Mail: thomas.mieslinger at 1und1.de > >> > >> 1&1 Internet AG > >> Brauerstra?e 48 > >> 76135 Karlsruhe > >> > >> Amtsgericht Montabaur HRB 6484 > >> Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas > >> Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver > >> Mauss, Jan Oetjen > >> Aufsichtsratsvorsitzender: Michael Scheeren > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 17:09:05 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 17:09:05 +0100 Subject: [rsyslog] Intro presentation Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F6C@GRFEXC.intern.adiscon.com> Hi all, I think about doing an online intro presentation to rsyslog that should be useful to new users, in addition to the doc. One may claim that updating the doc makes more sense, but this is a major effort, plus someone has volunteered to help with that (plus I'd like to experiment with online tutorials). So in short, I think I'd like to try this out. Question now: what do you think would be most useful? I think about 10 to 60 minutes of presentation, something that I should be able to create over some evenings than try to deliver. What would be the best candidates to go into such material? Feedback appreciated, Rainer From jules at visionintel.com Fri Mar 6 17:16:32 2009 From: jules at visionintel.com (jules at visionintel.com) Date: Fri, 06 Mar 2009 16:16:32 +0000 Subject: [rsyslog] Intro presentation Message-ID: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Remote loggin Sent from my Nokia phone -----Original Message----- From: Rainer Gerhards Sent: 06/03/2009 16:09:05 Subject: [rsyslog] Intro presentation Hi all, I think about doing an online intro presentation to rsyslog that should be useful to new users, in addition to the doc. One may claim that updating the doc makes more sense, but this is a major effort, plus someone has volunteered to help with that (plus I'd like to experiment with online tutorials). So in short, I think I'd like to try this out. Question now: what do you think would be most useful? I think about 10 to 60 minutes of presentation, something that I should be able to create over some evenings than try to deliver. What would be the best candidates to go into such material? Feedback appreciated, Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From aoz.syn at gmail.com Fri Mar 6 17:25:24 2009 From: aoz.syn at gmail.com (RB) Date: Fri, 6 Mar 2009 09:25:24 -0700 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903060825l37364ab2w738468329e628e82@mail.gmail.com> On Fri, Mar 6, 2009 at 08:40, Rainer Gerhards wrote: > The more I think about it, the more it smells like a real bug. Has anyone objections changing the default? None. It is unrealistic (and generally unusable) to have UNIX directory permissions without the execute bit (S_IX*). The only reason to do it would be to have an 'archive' directory of sorts, in which users may see names of children, but none of their permissions or contents. As has been noted, the only reason it's worked thus far is that most people either change the default or run the daemon as root, for whom those permissions aren't really a limiting factor. From u.a.martin at gmail.com Fri Mar 6 17:38:57 2009 From: u.a.martin at gmail.com (Ben Martin) Date: Fri, 6 Mar 2009 09:38:57 -0700 Subject: [rsyslog] Intro presentation In-Reply-To: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> References: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Message-ID: <661ae2b20903060838q1aa1f5d8g91c79cff9bc606ab@mail.gmail.com> Rainer I think a video tutorial is great idea. You might even start with a very brief discussion of the importance of centralized logging, from both the security and management perspective. Discussing the basic differences between v2 and v3 would also be helpful I think, as some distros (like CentOS) are still only packaging v2, while others (Debian) are installing v3 by default. - Ben On Fri, Mar 6, 2009 at 9:16 AM, wrote: > Remote loggin > > Sent from my Nokia phone > -----Original Message----- > From: Rainer Gerhards > Sent: ?06/03/2009 16:09:05 > Subject: ?[rsyslog] Intro presentation > > Hi all, > > I think about doing an online intro presentation to rsyslog that should > be useful to new users, in addition to the doc. One may claim that > updating the doc makes more sense, but this is a major effort, plus > someone has volunteered to help with that (plus I'd like to experiment > with online tutorials). So in short, I think I'd like to try this out. > > Question now: what do you think would be most useful? I think about 10 > to 60 minutes of presentation, something that I should be able to create > over some evenings than try to deliver. What would be the best > candidates to go into such material? > > Feedback appreciated, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Mar 6 18:21:14 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 18:21:14 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> <49B13E8A.2080308@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F70@GRFEXC.intern.adiscon.com> I guess the "root issue" is more a probably cause. I know that lot's of folks use rsyslog to create dirs. Will probably change the default, but in the beta first. Thanks for bringing this up. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 4:18 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > I guess nobody did let rsyslog make directories. > > Rainer Gerhards wrote: > > Thomas, > > > > do I correctly understand that you propose the default be changed? > > Yepp. > > > If so, I am hesitant to do that - wouldn't that potentially break > existing deployments? > > hmm Maybe I haven't seen enough yet, but I can't imagine a deployment > built on directory permissions 644.... > > > On the other hand... how could that work... Umm... > > They are all working as root out there :-) > > I think it would be good if you just double check it yourself that the > directories get created with 644 and decicde on your findings. > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Fri Mar 6 19:53:14 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 10:53:14 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly Message-ID: I'm running into problems trying to do filtering. it looks as if the log parsing is not properly filling in the properties. what I've run into so far when I use the property 'programname' the content that I see is what I would expect in 'hostname' when I use the property 'hostname' the content that I see is what I would expect in 'fromhost' I haven't checked all the other properties, but my guess is that somehow rsyslog is off-by-one in filling them in. David Lang From david at lang.hm Fri Mar 6 19:54:00 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 10:54:00 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: References: Message-ID: On Fri, 6 Mar 2009, david at lang.hm wrote: > I'm running into problems trying to do filtering. it looks as if the log > parsing is not properly filling in the properties. > > what I've run into so far > > when I use the property 'programname' the content that I see is what I would > expect in 'hostname' > > when I use the property 'hostname' the content that I see is what I would > expect in 'fromhost' > > I haven't checked all the other properties, but my guess is that somehow > rsyslog is off-by-one in filling them in. having said this, date, fromhost, and from-ip appear to be filled in correctly. David Lang From rgerhards at hq.adiscon.com Fri Mar 6 19:54:11 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 19:54:11 +0100 Subject: [rsyslog] properties not getting filled in correctly References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> That's why I am after the log samples :) I just termed a new acronym this afternoon: YAMSF - yet another malformed syslog format ;) http://blog.gerhards.net/2009/02/calling-for-log-samples.html I try hard to get the fields right, but often this is impossible, resulting in the issues you see. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, March 06, 2009 7:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Fri, 6 Mar 2009, david at lang.hm wrote: > > > I'm running into problems trying to do filtering. it looks as if the > log > > parsing is not properly filling in the properties. > > > > what I've run into so far > > > > when I use the property 'programname' the content that I see is what > I would > > expect in 'hostname' > > > > when I use the property 'hostname' the content that I see is what I > would > > expect in 'fromhost' > > > > I haven't checked all the other properties, but my guess is that > somehow > > rsyslog is off-by-one in filling them in. > > having said this, date, fromhost, and from-ip appear to be filled in > correctly. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Sat Mar 7 02:25:32 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 17:25:32 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 6 Mar 2009, Rainer Gerhards wrote: > That's why I am after the log samples :) I just termed a new acronym > this afternoon: > YAMSF - yet another malformed syslog format ;) > > http://blog.gerhards.net/2009/02/calling-for-log-samples.html > > I try hard to get the fields right, but often this is impossible, > resulting in the issues you see. these logs come from several different servers, including different OSs, but all are misparsed by rsyslog. I am not seeing anything obviously wrong with them <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Friday, March 06, 2009 7:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Fri, 6 Mar 2009, david at lang.hm wrote: >> >>> I'm running into problems trying to do filtering. it looks as if the >> log >>> parsing is not properly filling in the properties. >>> >>> what I've run into so far >>> >>> when I use the property 'programname' the content that I see is what >> I would >>> expect in 'hostname' >>> >>> when I use the property 'hostname' the content that I see is what I >> would >>> expect in 'fromhost' >>> >>> I haven't checked all the other properties, but my guess is that >> somehow >>> rsyslog is off-by-one in filling them in. >> >> having said this, date, fromhost, and from-ip appear to be filled in >> correctly. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Sat Mar 7 03:55:49 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 18:55:49 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 6 Mar 2009, david at lang.hm wrote: > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >> That's why I am after the log samples :) I just termed a new acronym >> this afternoon: >> YAMSF - yet another malformed syslog format ;) >> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >> >> I try hard to get the fields right, but often this is impossible, >> resulting in the issues you see. > > these logs come from several different servers, including different OSs, > but all are misparsed by rsyslog. > > I am not seeing anything obviously wrong with them > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw doing some more digging I see some very definant problems I created the following template $template DumpAll,"msg =%msg%\nrawmsg =%rawmsg%\nuxtradmsg =%uxtradmsg%\nhostname =%hostname%\nsource =%source%\nfromhost =%fromhost%\nfromhost-ip =%fromhost-ip%\nsyslogtag =%syslogtag%\nprogramname =%programname%\npri =%pri%\npri-text =%pri-text%\niut =%iut%\nsyslogfacility =%syslogfacility%\nsyslogfacility-text =%syslogfacility-text%\nsyslogseverity =%syslogseverity%\nsyslogseverity-text =%syslogseverity-text%\nsyslogpriority =%syslogpriority%\nsyslogpriority-text =%syslogpriority-text%\ntimegenerated =%timegenerated%\ntimereported =%timereported%\ntimestamp =%timestamp%\nprotocol-version =%protocol-version%\nstructured-data =%structured-data%\napp-name =%app-name%\nprocid =%procid%\nmsgid =%msgid%\ninputname =%inputname%\n\n" which creates a nice table for each log message showing what's in each property. things that I am seeing hostname and source are fromhost rather than the name/IP that's in the record. msg includes the programname programname and appname are what hostname should be David Lang msg = %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 rawmsg =<167>Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 uxtradmsg =Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 hostname =itascan1a-p source =itascan1a-p fromhost =itascan1a-p fromhost-ip =192.168.210.6 syslogtag =172.20.245.8 programname =172.20.245.8 pri =167 pri-text =local4.debug<167> iut =1 syslogfacility =20 syslogfacility-text =local4 syslogseverity =7 syslogseverity-text =debug syslogpriority =7 syslogpriority-text =debug timegenerated =Mar 7 02:33:47 timereported =Mar 6 18:33:47 timestamp =Mar 6 18:33:47 protocol-version =0 structured-data =- app-name =172.20.245.8 procid =- msgid =- inputname =imudp msg = plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 rawmsg =<29>Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 uxtradmsg =Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 hostname =itascan1a-p source =itascan1a-p fromhost =itascan1a-p fromhost-ip =192.168.210.6 syslogtag =methane1d-b programname =methane1d-b pri =29 pri-text =daemon.notice<29> iut =1 syslogfacility =3 syslogfacility-text =daemon syslogseverity =5 syslogseverity-text =notice syslogpriority =5 syslogpriority-text =notice timegenerated =Mar 7 02:33:47 timereported =Mar 6 18:33:47 timestamp =Mar 6 18:33:47 protocol-version =0 structured-data =- app-name =methane1d-b procid =- msgid =- inputname =imudp From rgerhards at hq.adiscon.com Sat Mar 7 10:47:54 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 7 Mar 2009 10:47:54 +0100 Subject: [rsyslog] properties not getting filled in correctly Message-ID: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> The messages indeed look ok. I'll feed them into my parser and will see what happens. rainer ----- Urspr?ngliche Nachricht ----- Von: "david at lang.hm" An: "rsyslog-users" Gesendet: 07.03.09 02:20 Betreff: Re: [rsyslog] properties not getting filled in correctly On Fri, 6 Mar 2009, Rainer Gerhards wrote: > That's why I am after the log samples :) I just termed a new acronym > this afternoon: > YAMSF - yet another malformed syslog format ;) > > http://blog.gerhards.net/2009/02/calling-for-log-samples.html > > I try hard to get the fields right, but often this is impossible, > resulting in the issues you see. these logs come from several different servers, including different OSs, but all are misparsed by rsyslog. I am not seeing anything obviously wrong with them <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Friday, March 06, 2009 7:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Fri, 6 Mar 2009, david at lang.hm wrote: >> >>> I'm running into problems trying to do filtering. it looks as if the >> log >>> parsing is not properly filling in the properties. >>> >>> what I've run into so far >>> >>> when I use the property 'programname' the content that I see is what >> I would >>> expect in 'hostname' >>> >>> when I use the property 'hostname' the content that I see is what I >> would >>> expect in 'fromhost' >>> >>> I haven't checked all the other properties, but my guess is that >> somehow >>> rsyslog is off-by-one in filling them in. >> >> having said this, date, fromhost, and from-ip appear to be filled in >> correctly. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 9 07:14:49 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 9 Mar 2009 07:14:49 +0100 Subject: [rsyslog] filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F7B@GRFEXC.intern.adiscon.com> Hi David, Sorry for the late reply. Of course, the change is not as trivial as I initially thought. It is very easy to add a length modifier to the property replacer, but you can not use the property replacer in property-based filters. Of course, I can modify those filters, but there no concept of a numerical value with these filters. The proper thing would be to do this in the script engine, where it was scheduled for, but the script engine does not yet support functions. Doh... I will look where I can best hack this into. My current thinking is that I will check what it takes to make the script engine support built-in (rather than loadable) functions, so that I could implement a set of core functions. I am not sure how much effort that is, but it doesn't look too scary (plus it would be really good to have this functionality, so it would be well-spent time). It that turns out not to be an option, I'll probably hack the script engine to support a unary operation "lengthof", that should be simple enough - but it is a dirty approach. I won't be able to do anything of this today, but I hope I can do either of the two within this week. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:06 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > Oh, that's an interesting use case. It is not yet possible. > I think we > > can implement (fairly simple) the size for a field (via the property > > replacer). However, that does not help you with the > resulting size of a > > template string. I probably also need to check the supporting > > infrastructure for "greater than" comparisons... > > > > Would that help? > > yes, I can set the value to something conservative to account for the > variable-length fields. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >> Sent: Wednesday, March 04, 2009 12:54 AM > >> To: rsyslog-users > >> Subject: [rsyslog] filtering by message size > >> > >> is it possible to filter by message size? > >> > >> I'm looking at a situation where I would like to send the > >> message via UDP > >> if it's below a given size and by TCP if it's larger. > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jackmarrow2 at gmail.com Tue Mar 10 11:15:09 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:15:09 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/3 Rainer Gerhards : > Well, you can see all change log entries by following the "change log" > menu item in the menu to the left ;) But it may even be more convenient > in that case that you get it directly from git as a single text file: > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=ba2a6c13e22b7 > f67401c7edb15ea17d31162bde4;hb=HEAD > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of jack marrow >> Sent: Tuesday, March 03, 2009 9:06 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] rsyslog changelog >> >> Hello, >> >> Is there a changelog for rsyslog, particularly showing the differences >> between the current version (3.x) and the 2.x version found in RHEL? >> >> Thanks, >> >> Jack >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > Thanks for this. One last question: on the receiving server side, can I see which logs came from which log file? From rgerhards at hq.adiscon.com Tue Mar 10 11:12:32 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 11:12:32 +0100 Subject: [rsyslog] rsyslog changelog References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> > One last question: on the receiving server side, can I see which logs > came from which log file? Usually, the log line should contain the host that sent the message. Does your's not? Rainer From jackmarrow2 at gmail.com Tue Mar 10 11:21:57 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:21:57 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/10 Rainer Gerhards : >> One last question: on the receiving server side, can I see which logs >> came from which log file? > > Usually, the log line should contain the host that sent the message. > Does your's not? > If a client sends /var/log/httpd/blah and /var/log/vsftpd/blah, does the receiving side simply receive the log contents or the filename as well? Is there a way to get both? From rgerhards at hq.adiscon.com Tue Mar 10 11:17:46 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 11:17:46 +0100 Subject: [rsyslog] rsyslog changelog References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Please post configs and elaborate a bit more about what you are trying to accomplish and what you have set up. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 10, 2009 11:22 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog changelog > > 2009/3/10 Rainer Gerhards : > >> One last question: on the receiving server side, can I see which > logs > >> came from which log file? > > > > Usually, the log line should contain the host that sent the message. > > Does your's not? > > > > If a client sends /var/log/httpd/blah and /var/log/vsftpd/blah, does > the receiving side simply receive the log contents or the filename as > well? Is there a way to get both? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 10 11:28:35 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:28:35 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/10 Rainer Gerhards : > Please post configs and elaborate a bit more about what you are trying > to accomplish and what you have set up. I am evaluating rsyslog at the moment. I would like to know if I can use it for log collection on the client for writing on the server. The server must know which log file is which. From david at lang.hm Tue Mar 10 16:21:45 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 10 Mar 2009 08:21:45 -0700 (PDT) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: On Sat, 7 Mar 2009, Rainer Gerhards wrote: > The messages indeed look ok. I'll feed them into my parser and will see what happens. any idea what's happening here yet? David Lang > rainer > > ----- Urspr?ngliche Nachricht ----- > Von: "david at lang.hm" > An: "rsyslog-users" > Gesendet: 07.03.09 02:20 > Betreff: Re: [rsyslog] properties not getting filled in correctly > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >> That's why I am after the log samples :) I just termed a new acronym >> this afternoon: >> YAMSF - yet another malformed syslog format ;) >> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >> >> I try hard to get the fields right, but often this is impossible, >> resulting in the issues you see. > > these logs come from several different servers, including different OSs, > but all are misparsed by rsyslog. > > I am not seeing anything obviously wrong with them > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > David Lang > >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Friday, March 06, 2009 7:54 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] properties not getting filled in correctly >>> >>> On Fri, 6 Mar 2009, david at lang.hm wrote: >>> >>>> I'm running into problems trying to do filtering. it looks as if the >>> log >>>> parsing is not properly filling in the properties. >>>> >>>> what I've run into so far >>>> >>>> when I use the property 'programname' the content that I see is what >>> I would >>>> expect in 'hostname' >>>> >>>> when I use the property 'hostname' the content that I see is what I >>> would >>>> expect in 'fromhost' >>>> >>>> I haven't checked all the other properties, but my guess is that >>> somehow >>>> rsyslog is off-by-one in filling them in. >>> >>> having said this, date, fromhost, and from-ip appear to be filled in >>> correctly. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Tue Mar 10 16:24:31 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 16:24:31 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FB1@GRFEXC.intern.adiscon.com> Not at the moment, I am currently looking into the scripting engine (for stringlength-based evaluations) I highly suggest http://twitter.com/rgerhards to keep track of what I am looking at. You do NOT need to be subscribed to twitter to use this service. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, March 10, 2009 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Sat, 7 Mar 2009, Rainer Gerhards wrote: > > > The messages indeed look ok. I'll feed them into my parser and will > see what happens. > > any idea what's happening here yet? > > David Lang > > > rainer > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "david at lang.hm" > > An: "rsyslog-users" > > Gesendet: 07.03.09 02:20 > > Betreff: Re: [rsyslog] properties not getting filled in correctly > > > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > > > >> That's why I am after the log samples :) I just termed a new acronym > >> this afternoon: > >> YAMSF - yet another malformed syslog format ;) > >> > >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >> > >> I try hard to get the fields right, but often this is impossible, > >> resulting in the issues you see. > > > > these logs come from several different servers, including different > OSs, > > but all are misparsed by rsyslog. > > > > I am not seeing anything obviously wrong with them > > > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > /192.168.243.37 destination=179.50.100.130/60029 > > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > duration=1 > > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > /192.168.22.8 destination=192.168.104.31/5667 > > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, > pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( > <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > delivery) > > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > > > David Lang > > > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Friday, March 06, 2009 7:54 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>> > >>>> I'm running into problems trying to do filtering. it looks as if > the > >>> log > >>>> parsing is not properly filling in the properties. > >>>> > >>>> what I've run into so far > >>>> > >>>> when I use the property 'programname' the content that I see is > what > >>> I would > >>>> expect in 'hostname' > >>>> > >>>> when I use the property 'hostname' the content that I see is what > I > >>> would > >>>> expect in 'fromhost' > >>>> > >>>> I haven't checked all the other properties, but my guess is that > >>> somehow > >>>> rsyslog is off-by-one in filling them in. > >>> > >>> having said this, date, fromhost, and from-ip appear to be filled > in > >>> correctly. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Tue Mar 10 18:14:20 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 10 Mar 2009 11:14:20 -0600 Subject: [rsyslog] rsyslog changelog In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903101014n37d41ea2s23db8accc90f96ba@mail.gmail.com> On Tue, Mar 10, 2009 at 04:28, jack marrow wrote: > 2009/3/10 Rainer Gerhards : >> Please post configs and elaborate a bit more about what you are trying >> to accomplish and what you have set up. > > I am evaluating rsyslog at the moment. > > I would like to know if I can use it for log collection on the client > for writing on the server. The server must know which log file is > which. This is more a "basic understanding of logging" question than one specific to rsyslog. Generally speaking, log daemons just log what client apps tell them to - httpd says, "I'm facility 6 and is my critical message". If the local log daemon is sending logs upstream, it will basically tell the upstream server "I'm myhostname and httpd (facility 6) just said with a critical priority". If all your daemons (httpd, vsftpd, etc.) log directly to the local syslog as opposed to a flat file, things should "just work". However, if you're configuring your "client" syslog instance to follow /var/log/httpd/access and retransmit that data to an upstream server, all that metadata (application name, facility, priority, etc) is lost. Hence, you must configure your client syslog to inject that data - with rsyslog, that would be done something like this: $ModLoad imfile.so $InputFileName /var/log/httpd/access $InputFileTag http_access $InputFilePollIntervalSeconds 5 $InputFileMonitor *.* @192.168.1.1 That sets up a monitor that polls /var/log/httpd/access every 5 seconds, prepends "http_access" to every line, and sends it via UDP to 192.168.1.1. From rgerhards at hq.adiscon.com Tue Mar 10 18:24:02 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 18:24:02 +0100 Subject: [rsyslog] RainerScript functions - was: RE: filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F7B@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FB3@GRFEXC.intern.adiscon.com> David, I have extended RainerScript with the ability to call functions. The current implementation is very much focused on the immediate needs and it has a VM instruction set design issue that prevents nested function calls from working. Also, it only supports build-in functions (not loadable modules), and the only build-in function so far is strlen() ;) - but it should do what you need. So far, it resides in its own git branch "rscript-func". I will continue to work on it (at least on the VM opcode issue), but would really appreciate some early feedback. With that version you can do things like if strlen($msg) > 80 then @@tcp-host if strlen($msg) <= 80 then @udp-host Note that the function argument can be any valid expression (but NOT another function call!), so the following is also valid (and maybe useful to get to a better guess): if strlen($msg & $syslogtag & $fromhost) > 80 then @@tcp-host Note that & is the string concatenation operator. Today's commit: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=e8499c6d33d09f6d8b42df72 da1661be0ef0f088 Feedback from you and all others is appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, March 09, 2009 7:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > Hi David, > > Sorry for the late reply. Of course, the change is not as trivial as I > initially thought. It is very easy to add a length modifier to the > property replacer, but you can not use the property replacer in > property-based filters. Of course, I can modify those filters, but > there > no concept of a numerical value with these filters. The proper thing > would be to do this in the script engine, where it was scheduled for, > but the script engine does not yet support functions. Doh... > > I will look where I can best hack this into. My current thinking is > that > I will check what it takes to make the script engine support built-in > (rather than loadable) functions, so that I could implement a set of > core functions. I am not sure how much effort that is, but it doesn't > look too scary (plus it would be really good to have this > functionality, > so it would be well-spent time). It that turns out not to be an option, > I'll probably hack the script engine to support a unary operation > "lengthof", that should be simple enough - but it is a dirty approach. > I > won't be able to do anything of this today, but I hope I can do either > of the two within this week. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > > Sent: Wednesday, March 04, 2009 8:06 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] filtering by message size > > > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > > > Oh, that's an interesting use case. It is not yet possible. > > I think we > > > can implement (fairly simple) the size for a field (via the > property > > > replacer). However, that does not help you with the > > resulting size of a > > > template string. I probably also need to check the supporting > > > infrastructure for "greater than" comparisons... > > > > > > Would that help? > > > > yes, I can set the value to something conservative to account for the > > variable-length fields. > > > > David Lang > > > > > Rainer > > > > > >> -----Original Message----- > > >> From: rsyslog-bounces at lists.adiscon.com > > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > > david at lang.hm > > >> Sent: Wednesday, March 04, 2009 12:54 AM > > >> To: rsyslog-users > > >> Subject: [rsyslog] filtering by message size > > >> > > >> is it possible to filter by message size? > > >> > > >> I'm looking at a situation where I would like to send the > > >> message via UDP > > >> if it's below a given size and by TCP if it's larger. > > >> > > >> David Lang > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 11 13:49:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 13:49:08 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> David, the issue is in v4 only (and so far UDP only, too). It was introduced by the optimizations, which pass some wrong parameters to the now-decoupled parser. Need to find root cause, though. Will keep you posted. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, March 10, 2009 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Sat, 7 Mar 2009, Rainer Gerhards wrote: > > > The messages indeed look ok. I'll feed them into my parser and will > see what happens. > > any idea what's happening here yet? > > David Lang > > > rainer > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "david at lang.hm" > > An: "rsyslog-users" > > Gesendet: 07.03.09 02:20 > > Betreff: Re: [rsyslog] properties not getting filled in correctly > > > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > > > >> That's why I am after the log samples :) I just termed a new acronym > >> this afternoon: > >> YAMSF - yet another malformed syslog format ;) > >> > >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >> > >> I try hard to get the fields right, but often this is impossible, > >> resulting in the issues you see. > > > > these logs come from several different servers, including different > OSs, > > but all are misparsed by rsyslog. > > > > I am not seeing anything obviously wrong with them > > > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > /192.168.243.37 destination=179.50.100.130/60029 > > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > duration=1 > > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > /192.168.22.8 destination=192.168.104.31/5667 > > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, > pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( > <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > delivery) > > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > > > David Lang > > > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Friday, March 06, 2009 7:54 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>> > >>>> I'm running into problems trying to do filtering. it looks as if > the > >>> log > >>>> parsing is not properly filling in the properties. > >>>> > >>>> what I've run into so far > >>>> > >>>> when I use the property 'programname' the content that I see is > what > >>> I would > >>>> expect in 'hostname' > >>>> > >>>> when I use the property 'hostname' the content that I see is what > I > >>> would > >>>> expect in 'fromhost' > >>>> > >>>> I haven't checked all the other properties, but my guess is that > >>> somehow > >>>> rsyslog is off-by-one in filling them in. > >>> > >>> having said this, date, fromhost, and from-ip appear to be filled > in > >>> correctly. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Wed Mar 11 13:51:18 2009 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Mar 2009 05:51:18 -0700 (PDT) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 11 Mar 2009, Rainer Gerhards wrote: > David, > > the issue is in v4 only (and so far UDP only, too). It was introduced by the > optimizations, which pass some wrong parameters to the now-decoupled parser. > Need to find root cause, though. > > Will keep you posted. thanks. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Tuesday, March 10, 2009 4:22 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Sat, 7 Mar 2009, Rainer Gerhards wrote: >> >>> The messages indeed look ok. I'll feed them into my parser and will >> see what happens. >> >> any idea what's happening here yet? >> >> David Lang >> >>> rainer >>> >>> ----- Urspr?ngliche Nachricht ----- >>> Von: "david at lang.hm" >>> An: "rsyslog-users" >>> Gesendet: 07.03.09 02:20 >>> Betreff: Re: [rsyslog] properties not getting filled in correctly >>> >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote: >>> >>>> That's why I am after the log samples :) I just termed a new acronym >>>> this afternoon: >>>> YAMSF - yet another malformed syslog format ;) >>>> >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >>>> >>>> I try hard to get the fields right, but often this is impossible, >>>> resulting in the issues you see. >>> >>> these logs come from several different servers, including different >> OSs, >>> but all are misparsed by rsyslog. >>> >>> I am not seeing anything obviously wrong with them >>> >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601 >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= >> /192.168.243.37 destination=179.50.100.130/60029 >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 >> duration=1 >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= >> /192.168.22.8 destination=192.168.104.31/5667 >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: >> to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( >> <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for >> delivery) >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw >>> >>> David Lang >>> >>>> Rainer >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>>>> Sent: Friday, March 06, 2009 7:54 PM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly >>>>> >>>>> On Fri, 6 Mar 2009, david at lang.hm wrote: >>>>> >>>>>> I'm running into problems trying to do filtering. it looks as if >> the >>>>> log >>>>>> parsing is not properly filling in the properties. >>>>>> >>>>>> what I've run into so far >>>>>> >>>>>> when I use the property 'programname' the content that I see is >> what >>>>> I would >>>>>> expect in 'hostname' >>>>>> >>>>>> when I use the property 'hostname' the content that I see is what >> I >>>>> would >>>>>> expect in 'fromhost' >>>>>> >>>>>> I haven't checked all the other properties, but my guess is that >>>>> somehow >>>>>> rsyslog is off-by-one in filling them in. >>>>> >>>>> having said this, date, fromhost, and from-ip appear to be filled >> in >>>>> correctly. >>>>> >>>>> David Lang >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 11 14:32:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 14:32:17 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FC3@GRFEXC.intern.adiscon.com> David, there is now a patch available: http://git.adiscon.com/?p=rsyslog.git;a=commit;h=59192611db992e7357337beb8e68 ec6cee5b3fec I will release a new devel today, and it will include the patch. I expect to release another one next week, which will then have the Solaris work plus the script engine with functions (feedback on that is still appreciated). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 11, 2009 1:51 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Wed, 11 Mar 2009, Rainer Gerhards wrote: > > > David, > > > > the issue is in v4 only (and so far UDP only, too). It was introduced > by the > > optimizations, which pass some wrong parameters to the now-decoupled > parser. > > Need to find root cause, though. > > > > Will keep you posted. > > thanks. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >> Sent: Tuesday, March 10, 2009 4:22 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] properties not getting filled in correctly > >> > >> On Sat, 7 Mar 2009, Rainer Gerhards wrote: > >> > >>> The messages indeed look ok. I'll feed them into my parser and will > >> see what happens. > >> > >> any idea what's happening here yet? > >> > >> David Lang > >> > >>> rainer > >>> > >>> ----- Urspr?ngliche Nachricht ----- > >>> Von: "david at lang.hm" > >>> An: "rsyslog-users" > >>> Gesendet: 07.03.09 02:20 > >>> Betreff: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >>> > >>>> That's why I am after the log samples :) I just termed a new > acronym > >>>> this afternoon: > >>>> YAMSF - yet another malformed syslog format ;) > >>>> > >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >>>> > >>>> I try hard to get the fields right, but often this is impossible, > >>>> resulting in the issues you see. > >>> > >>> these logs come from several different servers, including different > >> OSs, > >>> but all are misparsed by rsyslog. > >>> > >>> I am not seeing anything obviously wrong with them > >>> > >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > >> /192.168.243.37 destination=179.50.100.130/60029 > >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > >> duration=1 > >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > >> /192.168.22.8 destination=192.168.104.31/5667 > >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > >> to=, delay=00:00:01, xdelay=00:00:01, > mailer=esmtp, > >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, > stat=Sent ( > >> <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > >> delivery) > >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > >>> > >>> David Lang > >>> > >>>> Rainer > >>>> > >>>>> -----Original Message----- > >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>>>> Sent: Friday, March 06, 2009 7:54 PM > >>>>> To: rsyslog-users > >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>>>> > >>>>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>>>> > >>>>>> I'm running into problems trying to do filtering. it looks as if > >> the > >>>>> log > >>>>>> parsing is not properly filling in the properties. > >>>>>> > >>>>>> what I've run into so far > >>>>>> > >>>>>> when I use the property 'programname' the content that I see is > >> what > >>>>> I would > >>>>>> expect in 'hostname' > >>>>>> > >>>>>> when I use the property 'hostname' the content that I see is > what > >> I > >>>>> would > >>>>>> expect in 'fromhost' > >>>>>> > >>>>>> I haven't checked all the other properties, but my guess is that > >>>>> somehow > >>>>>> rsyslog is off-by-one in filling them in. > >>>>> > >>>>> having said this, date, fromhost, and from-ip appear to be filled > >> in > >>>>> correctly. > >>>>> > >>>>> David Lang > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 11 15:22:51 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 15:22:51 +0100 Subject: [rsyslog] rsyslog 4.1.5 (devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FC6@GRFEXC.intern.adiscon.com> Hi all, I have just released rsyslog 4.1.5, a member of the development branch. It offers ERE support in filter conditions as well as the ability to contain part of the repeated text in a "last message repeated n times" message. Also, it fixes a bug that caused invalid parsing when receiving messages via UDP. This is a recommended update for all development branch users. Change Log: http://www.rsyslog.com/Article349.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-150.phtml I hope this release is useful. As always, feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Thu Mar 12 10:53:30 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Mar 2009 10:53:30 +0100 Subject: [rsyslog] Intro presentation References: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FDC@GRFEXC.intern.adiscon.com> Hi all, I created a first video tutorial today, please see blog for questions: http://blog.gerhards.net/2009/03/rsyslog-video-tutorials.html For this test, I have used something that I had ready at hand, thus none of the suggested topics yet touched. Feedback to the questions raised in the blog post would be most welcome. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jules at visionintel.com > Sent: Friday, March 06, 2009 5:17 PM > To: rsyslog-users > Subject: Re: [rsyslog] Intro presentation > > Remote loggin > > Sent from my Nokia phone > -----Original Message----- > From: Rainer Gerhards > Sent: 06/03/2009 16:09:05 > Subject: [rsyslog] Intro presentation > > Hi all, > > I think about doing an online intro presentation to rsyslog that should > be useful to new users, in addition to the doc. One may claim that > updating the doc makes more sense, but this is a major effort, plus > someone has volunteered to help with that (plus I'd like to experiment > with online tutorials). So in short, I think I'd like to try this out. > > Question now: what do you think would be most useful? I think about 10 > to 60 minutes of presentation, something that I should be able to > create > over some evenings than try to deliver. What would be the best > candidates to go into such material? > > Feedback appreciated, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Mar 12 18:36:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Mar 2009 18:36:08 +0100 Subject: [rsyslog] rant on software (rsyslog) stability Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FE7@GRFEXC.intern.adiscon.com> Hi all, I was recently asked a couple of times if I could quickly create a "stable version" of this and that new feature. So I have finally taken out some time today (more than expected...) to pen down my position on it. The rant also tells a lot about rsyslog's branches, so I thought it is useful to circulate it on the mailing list: http://blog.gerhards.net/2009/03/how-software-gets-stable.html As always, feedback is appreciated. Rainer From mtant621 at charter.net Fri Mar 13 19:53:19 2009 From: mtant621 at charter.net (Michael Tant) Date: Fri, 13 Mar 2009 14:53:19 -0400 Subject: [rsyslog] Please Help! IPTables dumping to Console!!! Message-ID: I am running Fedora 10 linux with rsyslogd as my active logger. Recently I have had an issue with my iptables LOG target output going to the console and not going to the /var/log/messages file, even with the --log-level 6 argument. I have halfway resolved this issue by editing the /etc/rsyslog.conf file to include: kern.warning /var/log/iptables.log and appending --log-level 4 to my LOG target rules. This caused the output to go to the aforementioned file AND the console. I wish to still have the log data going to the iptables.log file, but wish to stop the dump to the console. I have reviewed the rsyslog.conf file, and the only statement which references /dev/console is kern.* but it is commented out with #. I am tempted to remove this statement to see if it helps, but I am unsure if this is safe, and furthermore convinced it will not change the outcome as this line is nothing more than a comment. Is there something somewhere I am perhaps missing? I don't fully understand the steps that move the log target output to the file, other than rsyslogd is in the middle somewhere with the kernel. Any suggestions would be greatly appreciated! Please send suggestion to mtant621 at chater.net I thank everyone for your help... Michael Tant From rvandolson at esri.com Sat Mar 14 00:18:14 2009 From: rvandolson at esri.com (Ray Van Dolson) Date: Fri, 13 Mar 2009 16:18:14 -0700 Subject: [rsyslog] Filtering on a group of IP's Message-ID: <20090313231814.GA7833@esri.com> I'm trying to shunt a bunch of logs from a group of IP's (about 10 IP's or so) to a fifo. Is the best way to do this with a property filter like the following? $template SplunkPipe,"|/logs/splunk/splunk.fifo" :fromhost-ip, isequal, "10.1.5.3" *.* -?SplunkPipe And how would I easily specify many 10 IP's? I'm thinking it would be slick to be able to find a "netgroup" that has the member IP's I want then just have my selector match against that netgroup. Is that sort of magic possible? Unfortunately I'm using rsyslog with RHEL5 which is only v2.0.6. Examples appreciated. :) Ray From david at lang.hm Sat Mar 14 00:48:22 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Mar 2009 16:48:22 -0700 (PDT) Subject: [rsyslog] Please Help! IPTables dumping to Console!!! In-Reply-To: References: Message-ID: On Fri, 13 Mar 2009, Michael Tant wrote: > Date: Fri, 13 Mar 2009 14:53:19 -0400 > From: Michael Tant > Reply-To: rsyslog-users > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Please Help! IPTables dumping to Console!!! > > I am running Fedora 10 linux with rsyslogd as my active logger. Recently I > have had an issue with my iptables LOG target output going to the console > and not going to the /var/log/messages file, even with the --log-level 6 > argument. I have halfway resolved this issue by editing the > /etc/rsyslog.conf file to include: kern.warning /var/log/iptables.log and > appending --log-level 4 to my LOG target rules. This caused the output to > go to the aforementioned file AND the console. > > I wish to still have the log data going to the iptables.log file, but wish > to stop the dump to the console. I have reviewed the rsyslog.conf file, and > the only statement which references /dev/console is kern.* but it is > commented out with #. I am tempted to remove this statement to see if it > helps, but I am unsure if this is safe, and furthermore convinced it will > not change the outcome as this line is nothing more than a comment. > > Is there something somewhere I am perhaps missing? I don't fully understand > the steps that move the log target output to the file, other than rsyslogd > is in the middle somewhere with the kernel. Any suggestions would be > greatly appreciated! Please send suggestion to mtant621 at chater.net there are a couple of possibilities here 1. you have something in /etc/rsyslog.conf that sends output to the console (or to root) the fix for this is to just remove/change the rsyslog.conf file 2. take a look in /etc/sysctl and see what you have log levels set to. some distros think that the iptables logs are important enough to spam everyone who's logged in, no matter what syslog is configured for. David Lang > I thank everyone for your help... > > Michael Tant > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From mtant621 at charter.net Sat Mar 14 16:19:13 2009 From: mtant621 at charter.net (Michael Tant) Date: Sat, 14 Mar 2009 11:19:13 -0400 Subject: [rsyslog] Still Dumping to Console Message-ID: I am still attempting to get the logging to stop dumping to console. IPtables is the only one doing this. I am currently logging to a different file by adding kern.warning /var/log/iptables.log to rsyslog.conf and --log-level 4 argument for the LOG targets. The data is making it to the file as specified, but is also being echoed to console if one of the tty's is displayed. It does echo to console in an X environment though, even a Konsole. I have check and found no logging references in the sysctl.conf file. I have completely removed the line: #kern.* /dev/console from the rsyslog.conf file, and have looked for auxilliary logging processes running and found none. I'm not skilled enough to fully understand the sysctl -a output so that could be the next possible culprit. If someone wants to take a look at that, rather than dumping it here and flooding you with huge email, you can find this at: http://fpaste.org/paste/6106 If there is something I'm overlooking or if there's some other way to fix this and force the correct behavior please let me know. As I don't quite have your skills with linux yet, please try to include as much information as you can, to assist with the fix. Again this is under Fedora 10. uname -a gives: Linux MTFedora 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb 23 13:21:22 EST 2009 i686 i686 i386 GNU/Linux if that gives any help. Thank You so much, Michael Tant From david at lang.hm Sat Mar 14 16:39:01 2009 From: david at lang.hm (david at lang.hm) Date: Sat, 14 Mar 2009 08:39:01 -0700 (PDT) Subject: [rsyslog] Still Dumping to Console In-Reply-To: References: Message-ID: On Sat, 14 Mar 2009, Michael Tant wrote: > I am still attempting to get the logging to stop dumping to console. > IPtables is the only one doing this. I am currently logging to a different > file by adding kern.warning /var/log/iptables.log to rsyslog.conf > and --log-level 4 argument for the LOG targets. The data is making it to > the file as specified, but is also being echoed to console if one of the > tty's is displayed. It does echo to console in an X environment though, > even a Konsole. I have check and found no logging references in the > sysctl.conf file. I have completely removed the line: #kern.* > /dev/console from the rsyslog.conf file, and have looked for auxilliary > logging processes running and found none. I'm not skilled enough to fully > understand the sysctl -a output so that could be the next possible culprit. > If someone wants to take a look at that, rather than dumping it here and > flooding you with huge email, you can find this at: > http://fpaste.org/paste/6106 > > If there is something I'm overlooking or if there's some other way to fix > this and force the correct behavior please let me know. As I don't quite > have your skills with linux yet, please try to include as much information > as you can, to assist with the fix. Again this is under Fedora 10. > uname -a gives: Linux MTFedora 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb > 23 13:21:22 EST 2009 i686 i686 i386 GNU/Linux if that gives any help. my ubuntu desktop has the following in /etc/sysctl.conf # the following stops low-level messages on console kernel.printk = 4 4 1 7 From rgerhards at hq.adiscon.com Sun Mar 15 11:20:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 15 Mar 2009 11:20:08 +0100 Subject: [rsyslog] webinar: "rsyslog templates" Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72000@GRFEXC.intern.adiscon.com> Hi all, in my effort to try video tutorials (or webinars as some pointed out ;)), I have now created a first live demo version, focused on templates. I hope it is useful: http://www.rsyslog.com/Article354.phtml Rainer From julianokyap at gmail.com Mon Mar 16 05:32:14 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 18:32:14 -1000 Subject: [rsyslog] Logging all messages from a remote server Message-ID: I'm having trouble logging ALL the syslog messages received from a server. I'm not sure if it's because it's from a non-standard piece of hardware (ie. not a Linux server). Logging to another server running syslogd works fine (but syslogd doesn't allow me to log messages from a remote server to a separate file and it's not my central syslogd server). I've tried several lines but none seem to work for me: if $fromhost == 'server' then /var/log/remote/server/all if $source == 'server' then /var/log/remote/server/all :FROMHOST, isequal, "server" /var/log/remote/server/all if $fromhost == 'server.domain.com' then /var/log/remote/server/all if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all .. Running Rsyslog 3.21.10. Thanks, Julian From david at lang.hm Mon Mar 16 06:16:04 2009 From: david at lang.hm (david at lang.hm) Date: Sun, 15 Mar 2009 22:16:04 -0700 (PDT) Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: References: Message-ID: On Sun, 15 Mar 2009, Julian Yap wrote: > I'm having trouble logging ALL the syslog messages received from a > server. I'm not sure if it's because it's from a non-standard piece > of hardware (ie. not a Linux server). Logging to another server > running syslogd works fine (but syslogd doesn't allow me to log > messages from a remote server to a separate file and it's not my > central syslogd server). > > I've tried several lines but none seem to work for me: > if $fromhost == 'server' then /var/log/remote/server/all > if $source == 'server' then /var/log/remote/server/all > :FROMHOST, isequal, "server" /var/log/remote/server/all > if $fromhost == 'server.domain.com' then /var/log/remote/server/all > if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all there are a few possible reasons that this could have problems is it that you have a high volume of logs and some just get dropped? if you just write everything to a file (*.* /var/log/test) does it have all the logs from this server? or is it missing some? do the logs from this server sometimes include the host and sometimes not? what is different between the logs that you match and the ones that you miss? David Lang From julianokyap at gmail.com Mon Mar 16 09:14:56 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 22:14:56 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: References: Message-ID: OK, I narrowed the issues down. Now I've faced strange issues like this before when using the $IncludeConfig directive. This is what I have just tested with in my /etc/rsyslog.conf file (and other lines) and it worked fine: ---- $IncludeConfig /etc/rsyslog.d/ :FROMHOST, isequal, "server" /var/log/remote/server/all ---- Now if I have a file /etc/rsyslog.d/testalert_for_another_server, things turn strange and only certain messages are logged from the first server.: ---- $ModLoad ommail $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $template DYNserver2, "/var/log/remote/server2.log" $template TraditionalFormatNoHostname,"%timegenerated% %syslogtag%%msg:::drop-last-lf%\n" if $hostname == 'server2.domain.com' then ?DYNserver2;TraditionalFormatNoHostname $ActionMailFrom rsyslog at domain.com $ActionMailTo server2_alert $template mailSubjectTestAlert,"INFO: Alert detected" $template mailBodyTestAlert,"Message is..." $ActionMailSubject mailSubjectTestAlert $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 if $hostname == 'server2.domain.com' and $msg contains 'Some message' then :ommail:;mailBodyTestAlert ---- Now if I add the contents of /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and remove file /etc/rsyslog.d/testalert_for_another_server) then things work fine... Now if I remove the previous changes to /etc/rsyslog.conf and modify /etc/rsyslog.d/testalert_for_another_server and remove the following lines then things work OK again: $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 - Julian On Sun, Mar 15, 2009 at 7:16 PM, wrote: > On Sun, 15 Mar 2009, Julian Yap wrote: > >> I'm having trouble logging ALL the syslog messages received from a >> server. ?I'm not sure if it's because it's from a non-standard piece >> of hardware (ie. not a Linux server). ?Logging to another server >> running syslogd works fine (but syslogd doesn't allow me to log >> messages from a remote server to a separate file and it's not my >> central syslogd server). >> >> I've tried several lines but none seem to work for me: >> if $fromhost == 'server' then /var/log/remote/server/all >> if $source == 'server' then /var/log/remote/server/all >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > > there are a few possible reasons that this could have problems > > is it that you have a high volume of logs and some just get dropped? > > if you just write everything to a file (*.* /var/log/test) does it have > all the logs from this server? or is it missing some? > > do the logs from this server sometimes include the host and sometimes not? > > what is different between the logs that you match and the ones that you > miss? > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 09:52:54 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 09:52:54 +0100 Subject: [rsyslog] Logging all messages from a remote server References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> The issue is that these statements $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 Modify the *next* action. So you need to specify them in front of the action. If you use the $includeConfig option, and have part of the action inside the include file and other parts (the statements) outside (or vice versa), you never know which action gets configured how. So place all of them together. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 9:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > OK, I narrowed the issues down. Now I've faced strange issues like > this before when using the $IncludeConfig directive. > > This is what I have just tested with in my /etc/rsyslog.conf file (and > other lines) and it worked fine: > ---- > $IncludeConfig /etc/rsyslog.d/ > :FROMHOST, isequal, "server" /var/log/remote/server/all > ---- > > Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > things turn strange and only certain messages are logged from the > first server.: > ---- > $ModLoad ommail > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > $template DYNserver2, "/var/log/remote/server2.log" > $template TraditionalFormatNoHostname,"%timegenerated% > %syslogtag%%msg:::drop-last-lf%\n" > > if $hostname == 'server2.domain.com' then > ?DYNserver2;TraditionalFormatNoHostname > > $ActionMailFrom rsyslog at domain.com > $ActionMailTo server2_alert > $template mailSubjectTestAlert,"INFO: Alert detected" > $template mailBodyTestAlert,"Message is..." > $ActionMailSubject mailSubjectTestAlert > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > if $hostname == 'server2.domain.com' and $msg contains 'Some message' > then :ommail:;mailBodyTestAlert > ---- > > Now if I add the contents of > /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and > remove file /etc/rsyslog.d/testalert_for_another_server) then things > work fine... > > Now if I remove the previous changes to /etc/rsyslog.conf and modify > /etc/rsyslog.d/testalert_for_another_server and remove the following > lines then things work OK again: > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > > - Julian > > > On Sun, Mar 15, 2009 at 7:16 PM, wrote: > > On Sun, 15 Mar 2009, Julian Yap wrote: > > > >> I'm having trouble logging ALL the syslog messages received from a > >> server. ?I'm not sure if it's because it's from a non-standard piece > >> of hardware (ie. not a Linux server). ?Logging to another server > >> running syslogd works fine (but syslogd doesn't allow me to log > >> messages from a remote server to a separate file and it's not my > >> central syslogd server). > >> > >> I've tried several lines but none seem to work for me: > >> if $fromhost == 'server' then /var/log/remote/server/all > >> if $source == 'server' then /var/log/remote/server/all > >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all > >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > > > > there are a few possible reasons that this could have problems > > > > is it that you have a high volume of logs and some just get dropped? > > > > if you just write everything to a file (*.* /var/log/test) does it > have > > all the logs from this server? or is it missing some? > > > > do the logs from this server sometimes include the host and sometimes > not? > > > > what is different between the logs that you match and the ones that > you > > miss? > > > > David Lang > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Mon Mar 16 10:04:37 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 23:04:37 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> Message-ID: Rainer, Would you recommend against using $includeConfig? In that case, it tends to lead to more unknown config issues. - Julian On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards wrote: > The issue is that these statements > > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > Modify the *next* action. So you need to specify them in front of the action. > If you use the $includeConfig option, and have part of the action inside the > include file and other parts (the statements) outside (or vice versa), you > never know which action gets configured how. So place all of them together. > > HTH > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> Sent: Monday, March 16, 2009 9:15 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> OK, I narrowed the issues down. ?Now I've faced strange issues like >> this before when using the $IncludeConfig directive. >> >> This is what I have just tested with in my /etc/rsyslog.conf file (and >> other lines) and it worked fine: >> ---- >> $IncludeConfig /etc/rsyslog.d/ >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all >> ---- >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, >> things turn strange and only certain messages are logged from the >> first server.: >> ---- >> $ModLoad ommail >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat >> >> $template DYNserver2, "/var/log/remote/server2.log" >> $template TraditionalFormatNoHostname,"%timegenerated% >> %syslogtag%%msg:::drop-last-lf%\n" >> >> if $hostname == 'server2.domain.com' then >> ?DYNserver2;TraditionalFormatNoHostname >> >> $ActionMailFrom rsyslog at domain.com >> $ActionMailTo server2_alert >> $template mailSubjectTestAlert,"INFO: Alert detected" >> $template mailBodyTestAlert,"Message is..." >> $ActionMailSubject mailSubjectTestAlert >> $ActionExecOnlyOnceEveryInterval 300 >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> $ActionExecOnlyEveryNthTime 3 >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some message' >> then :ommail:;mailBodyTestAlert >> ---- >> >> Now if I add the contents of >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and >> remove file /etc/rsyslog.d/testalert_for_another_server) then things >> work fine... >> >> Now if I remove the previous changes to /etc/rsyslog.conf and modify >> /etc/rsyslog.d/testalert_for_another_server and remove the following >> lines then things work OK again: >> $ActionExecOnlyOnceEveryInterval 300 >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> $ActionExecOnlyEveryNthTime 3 >> >> >> - Julian >> >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: >> > On Sun, 15 Mar 2009, Julian Yap wrote: >> > >> >> I'm having trouble logging ALL the syslog messages received from a >> >> server. ?I'm not sure if it's because it's from a non-standard piece >> >> of hardware (ie. not a Linux server). ?Logging to another server >> >> running syslogd works fine (but syslogd doesn't allow me to log >> >> messages from a remote server to a separate file and it's not my >> >> central syslogd server). >> >> >> >> I've tried several lines but none seem to work for me: >> >> if $fromhost == 'server' then /var/log/remote/server/all >> >> if $source == 'server' then /var/log/remote/server/all >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all >> > >> > there are a few possible reasons that this could have problems >> > >> > is it that you have a high volume of logs and some just get dropped? >> > >> > if you just write everything to a file (*.* /var/log/test) does it >> have >> > all the logs from this server? or is it missing some? >> > >> > do the logs from this server sometimes include the host and sometimes >> not? >> > >> > what is different between the logs that you match and the ones that >> you >> > miss? >> > >> > David Lang >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 10:08:36 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 10:08:36 +0100 Subject: [rsyslog] Logging all messages from a remote server References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 10:05 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > Rainer, > > Would you recommend against using $includeConfig? In that case, it > tends to lead to more unknown config issues. No, but do not split config directives that need to go together over several places. You need to put this together # this starts the definition of a single action $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 $... *.* action #this ends it So you need to put everything together. If you rip it apart, you will get undefined results. This is - to phrase it politely - not very well documented. You need to read the fine print, most of the $Action... params modify the *next* action - NOT *all* actions. So it is vitally important where they occur. Will try to make this clear as soon as I have a bit more time. Rainer > > - Julian > > On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards > wrote: > > The issue is that these statements > > > > $ActionExecOnlyOnceEveryInterval 300 > > $ActionExecOnlyEveryNthTimeTimeout 1200 > > $ActionExecOnlyEveryNthTime 3 > > > > Modify the *next* action. So you need to specify them in front of the > action. > > If you use the $includeConfig option, and have part of the action > inside the > > include file and other parts (the statements) outside (or vice > versa), you > > never know which action gets configured how. So place all of them > together. > > > > HTH > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> Sent: Monday, March 16, 2009 9:15 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> > >> OK, I narrowed the issues down. ?Now I've faced strange issues like > >> this before when using the $IncludeConfig directive. > >> > >> This is what I have just tested with in my /etc/rsyslog.conf file > (and > >> other lines) and it worked fine: > >> ---- > >> $IncludeConfig /etc/rsyslog.d/ > >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all > >> ---- > >> > >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > >> things turn strange and only certain messages are logged from the > >> first server.: > >> ---- > >> $ModLoad ommail > >> > >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat > >> > >> $template DYNserver2, "/var/log/remote/server2.log" > >> $template TraditionalFormatNoHostname,"%timegenerated% > >> %syslogtag%%msg:::drop-last-lf%\n" > >> > >> if $hostname == 'server2.domain.com' then > >> ?DYNserver2;TraditionalFormatNoHostname > >> > >> $ActionMailFrom rsyslog at domain.com > >> $ActionMailTo server2_alert > >> $template mailSubjectTestAlert,"INFO: Alert detected" > >> $template mailBodyTestAlert,"Message is..." > >> $ActionMailSubject mailSubjectTestAlert > >> $ActionExecOnlyOnceEveryInterval 300 > >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> $ActionExecOnlyEveryNthTime 3 > >> > >> if $hostname == 'server2.domain.com' and $msg contains 'Some > message' > >> then :ommail:;mailBodyTestAlert > >> ---- > >> > >> Now if I add the contents of > >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf > (and > >> remove file /etc/rsyslog.d/testalert_for_another_server) then things > >> work fine... > >> > >> Now if I remove the previous changes to /etc/rsyslog.conf and modify > >> /etc/rsyslog.d/testalert_for_another_server and remove the following > >> lines then things work OK again: > >> $ActionExecOnlyOnceEveryInterval 300 > >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> $ActionExecOnlyEveryNthTime 3 > >> > >> > >> - Julian > >> > >> > >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: > >> > On Sun, 15 Mar 2009, Julian Yap wrote: > >> > > >> >> I'm having trouble logging ALL the syslog messages received from > a > >> >> server. ?I'm not sure if it's because it's from a non-standard > piece > >> >> of hardware (ie. not a Linux server). ?Logging to another server > >> >> running syslogd works fine (but syslogd doesn't allow me to log > >> >> messages from a remote server to a separate file and it's not my > >> >> central syslogd server). > >> >> > >> >> I've tried several lines but none seem to work for me: > >> >> if $fromhost == 'server' then /var/log/remote/server/all > >> >> if $source == 'server' then /var/log/remote/server/all > >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> >> if $fromhost == 'server.domain.com' then > /var/log/remote/server/all > >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > >> > > >> > there are a few possible reasons that this could have problems > >> > > >> > is it that you have a high volume of logs and some just get > dropped? > >> > > >> > if you just write everything to a file (*.* /var/log/test) does it > >> have > >> > all the logs from this server? or is it missing some? > >> > > >> > do the logs from this server sometimes include the host and > sometimes > >> not? > >> > > >> > what is different between the logs that you match and the ones > that > >> you > >> > miss? > >> > > >> > David Lang > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Mon Mar 16 10:18:23 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 23:18:23 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> Message-ID: Thanks all. My config is working fine now. I can take some of the blame for requesting the $ActionExecOnlyEveryNthTime* params in the first place :P. Just to shed some light, my previous understanding (or what I initially gathered from the docs) was that the $Action params needed to just be in a block and the order of params didn't matter. So: #start Action $Action... $Action... $Action... #end Action So that was just what I gathered in my head. But it's all clear now. - Julian On Sun, Mar 15, 2009 at 11:08 PM, Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> Sent: Monday, March 16, 2009 10:05 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> Rainer, >> >> Would you recommend against using $includeConfig? ?In that case, it >> tends to lead to more unknown config issues. > > No, but do not split config directives that need to go together over several > places. You need to put this together > > # this starts the definition of a single action > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > $... > *.* ?action > #this ends it > > So you need to put everything together. If you rip it apart, you will get > undefined results. > > This is - to phrase it politely - not very well documented. You need to read > the fine print, most of the $Action... params modify the *next* action - NOT > *all* actions. So it is vitally important where they occur. > > Will try to make this clear as soon as I have a bit more time. > > > Rainer >> >> - Julian >> >> On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards >> wrote: >> > The issue is that these statements >> > >> > $ActionExecOnlyOnceEveryInterval 300 >> > $ActionExecOnlyEveryNthTimeTimeout 1200 >> > $ActionExecOnlyEveryNthTime 3 >> > >> > Modify the *next* action. So you need to specify them in front of the >> action. >> > If you use the $includeConfig option, and have part of the action >> inside the >> > include file and other parts (the statements) outside (or vice >> versa), you >> > never know which action gets configured how. So place all of them >> together. >> > >> > HTH >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> >> Sent: Monday, March 16, 2009 9:15 AM >> >> To: rsyslog-users >> >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> >> >> OK, I narrowed the issues down. ?Now I've faced strange issues like >> >> this before when using the $IncludeConfig directive. >> >> >> >> This is what I have just tested with in my /etc/rsyslog.conf file >> (and >> >> other lines) and it worked fine: >> >> ---- >> >> $IncludeConfig /etc/rsyslog.d/ >> >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all >> >> ---- >> >> >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, >> >> things turn strange and only certain messages are logged from the >> >> first server.: >> >> ---- >> >> $ModLoad ommail >> >> >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat >> >> >> >> $template DYNserver2, "/var/log/remote/server2.log" >> >> $template TraditionalFormatNoHostname,"%timegenerated% >> >> %syslogtag%%msg:::drop-last-lf%\n" >> >> >> >> if $hostname == 'server2.domain.com' then >> >> ?DYNserver2;TraditionalFormatNoHostname >> >> >> >> $ActionMailFrom rsyslog at domain.com >> >> $ActionMailTo server2_alert >> >> $template mailSubjectTestAlert,"INFO: Alert detected" >> >> $template mailBodyTestAlert,"Message is..." >> >> $ActionMailSubject mailSubjectTestAlert >> >> $ActionExecOnlyOnceEveryInterval 300 >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> >> $ActionExecOnlyEveryNthTime 3 >> >> >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some >> message' >> >> then :ommail:;mailBodyTestAlert >> >> ---- >> >> >> >> Now if I add the contents of >> >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf >> (and >> >> remove file /etc/rsyslog.d/testalert_for_another_server) then things >> >> work fine... >> >> >> >> Now if I remove the previous changes to /etc/rsyslog.conf and modify >> >> /etc/rsyslog.d/testalert_for_another_server and remove the following >> >> lines then things work OK again: >> >> $ActionExecOnlyOnceEveryInterval 300 >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> >> $ActionExecOnlyEveryNthTime 3 >> >> >> >> >> >> - Julian >> >> >> >> >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: >> >> > On Sun, 15 Mar 2009, Julian Yap wrote: >> >> > >> >> >> I'm having trouble logging ALL the syslog messages received from >> a >> >> >> server. ?I'm not sure if it's because it's from a non-standard >> piece >> >> >> of hardware (ie. not a Linux server). ?Logging to another server >> >> >> running syslogd works fine (but syslogd doesn't allow me to log >> >> >> messages from a remote server to a separate file and it's not my >> >> >> central syslogd server). >> >> >> >> >> >> I've tried several lines but none seem to work for me: >> >> >> if $fromhost == 'server' then /var/log/remote/server/all >> >> >> if $source == 'server' then /var/log/remote/server/all >> >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> >> >> if $fromhost == 'server.domain.com' then >> /var/log/remote/server/all >> >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all >> >> > >> >> > there are a few possible reasons that this could have problems >> >> > >> >> > is it that you have a high volume of logs and some just get >> dropped? >> >> > >> >> > if you just write everything to a file (*.* /var/log/test) does it >> >> have >> >> > all the logs from this server? or is it missing some? >> >> > >> >> > do the logs from this server sometimes include the host and >> sometimes >> >> not? >> >> > >> >> > what is different between the logs that you match and the ones >> that >> >> you >> >> > miss? >> >> > >> >> > David Lang >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com >> >> > >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 10:22:47 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 10:22:47 +0100 Subject: [rsyslog] Logging all messages from a remote server References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72007@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 10:18 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > Thanks all. My config is working fine now. > > I can take some of the blame for requesting the > $ActionExecOnlyEveryNthTime* params in the first place :P. > > Just to shed some light, my previous understanding (or what I > initially gathered from the docs) was that the $Action params needed > to just be in a block and the order of params didn't matter. > > So: > #start Action > $Action... > $Action... > $Action... > #end Action > > So that was just what I gathered in my head. But it's all clear now. Well, the order doesn't matter BUT (!) above you do NOT define an action - because the action itself is missing! So whatever action comes next, it will receive these parameters. Rainer > > - Julian > > On Sun, Mar 15, 2009 at 11:08 PM, Rainer Gerhards > wrote: > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> Sent: Monday, March 16, 2009 10:05 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> > >> Rainer, > >> > >> Would you recommend against using $includeConfig? ?In that case, it > >> tends to lead to more unknown config issues. > > > > No, but do not split config directives that need to go together over > several > > places. You need to put this together > > > > # this starts the definition of a single action > > $ActionExecOnlyOnceEveryInterval 300 > > $ActionExecOnlyEveryNthTimeTimeout 1200 > > $ActionExecOnlyEveryNthTime 3 > > $... > > *.* ?action > > #this ends it > > > > So you need to put everything together. If you rip it apart, you will > get > > undefined results. > > > > This is - to phrase it politely - not very well documented. You need > to read > > the fine print, most of the $Action... params modify the *next* > action - NOT > > *all* actions. So it is vitally important where they occur. > > > > Will try to make this clear as soon as I have a bit more time. > > > > > > Rainer > >> > >> - Julian > >> > >> On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards > >> wrote: > >> > The issue is that these statements > >> > > >> > $ActionExecOnlyOnceEveryInterval 300 > >> > $ActionExecOnlyEveryNthTimeTimeout 1200 > >> > $ActionExecOnlyEveryNthTime 3 > >> > > >> > Modify the *next* action. So you need to specify them in front of > the > >> action. > >> > If you use the $includeConfig option, and have part of the action > >> inside the > >> > include file and other parts (the statements) outside (or vice > >> versa), you > >> > never know which action gets configured how. So place all of them > >> together. > >> > > >> > HTH > >> > Rainer > >> > > >> >> -----Original Message----- > >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> >> Sent: Monday, March 16, 2009 9:15 AM > >> >> To: rsyslog-users > >> >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> >> > >> >> OK, I narrowed the issues down. ?Now I've faced strange issues > like > >> >> this before when using the $IncludeConfig directive. > >> >> > >> >> This is what I have just tested with in my /etc/rsyslog.conf file > >> (and > >> >> other lines) and it worked fine: > >> >> ---- > >> >> $IncludeConfig /etc/rsyslog.d/ > >> >> :FROMHOST, isequal, "server" > /var/log/remote/server/all > >> >> ---- > >> >> > >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > >> >> things turn strange and only certain messages are logged from the > >> >> first server.: > >> >> ---- > >> >> $ModLoad ommail > >> >> > >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat > >> >> > >> >> $template DYNserver2, "/var/log/remote/server2.log" > >> >> $template TraditionalFormatNoHostname,"%timegenerated% > >> >> %syslogtag%%msg:::drop-last-lf%\n" > >> >> > >> >> if $hostname == 'server2.domain.com' then > >> >> ?DYNserver2;TraditionalFormatNoHostname > >> >> > >> >> $ActionMailFrom rsyslog at domain.com > >> >> $ActionMailTo server2_alert > >> >> $template mailSubjectTestAlert,"INFO: Alert detected" > >> >> $template mailBodyTestAlert,"Message is..." > >> >> $ActionMailSubject mailSubjectTestAlert > >> >> $ActionExecOnlyOnceEveryInterval 300 > >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> >> $ActionExecOnlyEveryNthTime 3 > >> >> > >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some > >> message' > >> >> then :ommail:;mailBodyTestAlert > >> >> ---- > >> >> > >> >> Now if I add the contents of > >> >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf > >> (and > >> >> remove file /etc/rsyslog.d/testalert_for_another_server) then > things > >> >> work fine... > >> >> > >> >> Now if I remove the previous changes to /etc/rsyslog.conf and > modify > >> >> /etc/rsyslog.d/testalert_for_another_server and remove the > following > >> >> lines then things work OK again: > >> >> $ActionExecOnlyOnceEveryInterval 300 > >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> >> $ActionExecOnlyEveryNthTime 3 > >> >> > >> >> > >> >> - Julian > >> >> > >> >> > >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: > >> >> > On Sun, 15 Mar 2009, Julian Yap wrote: > >> >> > > >> >> >> I'm having trouble logging ALL the syslog messages received > from > >> a > >> >> >> server. ?I'm not sure if it's because it's from a non-standard > >> piece > >> >> >> of hardware (ie. not a Linux server). ?Logging to another > server > >> >> >> running syslogd works fine (but syslogd doesn't allow me to > log > >> >> >> messages from a remote server to a separate file and it's not > my > >> >> >> central syslogd server). > >> >> >> > >> >> >> I've tried several lines but none seem to work for me: > >> >> >> if $fromhost == 'server' then /var/log/remote/server/all > >> >> >> if $source == 'server' then /var/log/remote/server/all > >> >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> >> >> if $fromhost == 'server.domain.com' then > >> /var/log/remote/server/all > >> >> >> if $fromhost-ip == '192.168.0.60' then > /var/log/remote/server/all > >> >> > > >> >> > there are a few possible reasons that this could have problems > >> >> > > >> >> > is it that you have a high volume of logs and some just get > >> dropped? > >> >> > > >> >> > if you just write everything to a file (*.* /var/log/test) does > it > >> >> have > >> >> > all the logs from this server? or is it missing some? > >> >> > > >> >> > do the logs from this server sometimes include the host and > >> sometimes > >> >> not? > >> >> > > >> >> > what is different between the logs that you match and the ones > >> that > >> >> you > >> >> > miss? > >> >> > > >> >> > David Lang > >> >> > _______________________________________________ > >> >> > rsyslog mailing list > >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > http://www.rsyslog.com > >> >> > > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Mon Mar 16 16:34:09 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Mon, 16 Mar 2009 16:34:09 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean Message-ID: <49BE7171.3090601@1und1.de> Hi, I've configured rsyslog to use relp as transport protocol. sw version: rsyslog-relp-3.21.3-4 and rsyslog-3.21.3-4. in the log I see these messages: 2009-03-16T16:12:10.769408+01:00 zeus-log01-2 rsyslogd: [origin software="rsyslogd" swVersion="3.21.3" x-pid="3239" x-info="http://www.rsyslog.com"] restart 2009-03-16T16:12:10.769447+01:00 zeus-log01-2 rsyslogd: error -2077 trying to add listener 2009-03-16T16:12:10.769458+01:00 zeus-log01-2 rsyslogd: the last error occured in /data/etc/rsyslog/rsyslog.conf, line 6 2009-03-16T16:12:10.769470+01:00 zeus-log01-2 rsyslogd: the last error occured in /data/etc/rsyslog/rsyslog.conf, line 7 The config line in question read: ------snip # Global $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imrelp.so $UDPServerRun 514 $InputTCPServerRun 514 <-- line 6 $InputRELPServerRun 2514 <-- line 7 $DirCreateMode 0755 -------snap netstat -an| grep 514 shows all configured udp and tcp ports open. So where can I find a description of error -2077? Thanks in advance Thomas From rgerhards at hq.adiscon.com Mon Mar 16 16:39:21 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 16:39:21 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean References: <49BE7171.3090601@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> There should be informtion on that error on the web, but 2077 is "could not bind to port". A short reference can be found in git in file ./runtime/rsyslog.h Not sure where it orginates from in this case... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Monday, March 16, 2009 4:34 PM > To: rsyslog-users > Subject: [rsyslog] what does error -2077 trying to add listener mean > > Hi, > > I've configured rsyslog to use relp as transport protocol. > sw version: rsyslog-relp-3.21.3-4 and rsyslog-3.21.3-4. > > in the log I see these messages: > 2009-03-16T16:12:10.769408+01:00 zeus-log01-2 rsyslogd: [origin > software="rsyslogd" swVersion="3.21.3" x-pid="3239" > x-info="http://www.rsyslog.com"] restart > 2009-03-16T16:12:10.769447+01:00 zeus-log01-2 rsyslogd: error -2077 > trying to add listener > 2009-03-16T16:12:10.769458+01:00 zeus-log01-2 rsyslogd: the last error > occured in /data/etc/rsyslog/rsyslog.conf, line 6 > 2009-03-16T16:12:10.769470+01:00 zeus-log01-2 rsyslogd: the last error > occured in /data/etc/rsyslog/rsyslog.conf, line 7 > > The config line in question read: > > ------snip > # Global > $ModLoad imudp.so > $ModLoad imtcp.so > $ModLoad imrelp.so > $UDPServerRun 514 > $InputTCPServerRun 514 <-- line 6 > $InputRELPServerRun 2514 <-- line 7 > > $DirCreateMode 0755 > > -------snap > > netstat -an| grep 514 shows all configured udp and tcp ports open. > > So where can I find a description of error -2077? > > Thanks in advance > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Mon Mar 16 16:56:31 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Mon, 16 Mar 2009 16:56:31 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> References: <49BE7171.3090601@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> Message-ID: <49BE76AF.4030608@1und1.de> Hi Rainer, there is only one place where RS_RET_COULD_NOT_BIND is returned: runtime/nsd_ptcp.c numSocks = 0; /* num of sockets counter at start of array */ for(r = res; r != NULL ; r = r->ai_next) { sock = socket(r->ai_family, r->ai_socktype, r->ai_protocol); [ lots of magic ] } if(numSocks == 0) { dbgprintf("No TCP listen sockets could successfully be initialized"); ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); } I have no idea why the OS reports the Sockets open and messages get received, maybe there is a minor problem in the code, but somehow it works... Thomas From rgerhards at hq.adiscon.com Mon Mar 16 16:59:52 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 16:59:52 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean References: <49BE7171.3090601@1und1.de><9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> <49BE76AF.4030608@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7200F@GRFEXC.intern.adiscon.com> This sounds like some quirk with IPv6... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Monday, March 16, 2009 4:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] what does error -2077 trying to add listener > mean > > Hi Rainer, > > there is only one place where RS_RET_COULD_NOT_BIND is returned: > > runtime/nsd_ptcp.c > > numSocks = 0; /* num of sockets counter at start of array */ > for(r = res; r != NULL ; r = r->ai_next) { > sock = socket(r->ai_family, r->ai_socktype, r- > >ai_protocol); > > [ lots of magic ] > > } > > if(numSocks == 0) { > dbgprintf("No TCP listen sockets could successfully be > initialized"); > ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); > } > > I have no idea why the OS reports the Sockets open and messages get > received, maybe there is a minor problem in the code, but somehow it > works... > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 16 17:53:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 17:53:40 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch><1236001365.28865.44.camel@rf10up.intern.adiscon.com><200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> Sorry for the delay, it is currently quite busy here at my end :( I have now created a very rough skeleton template output module. You need to pull from git. It is contained in the master branch. So far, it does not perform useful work. I was a bit hesitant to add much more description, because I think this can either be brief and not matching what you need - or very elaborate (bbok-like), for what I currently do not have enough time. I suggest that you have a look at the template module, and then we simply try to get this going. It would be good if you could ask questions or tell me what needs to be placed inside the module. Or I can create yet another skeleton, based on ommysql, that has a bit more logic so that you can fill in the initial Oracle functionality. That will not offer superior performance, but I think it would be a good starting point to pursue the rest of this project. Please let me know what you think. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, March 03, 2009 3:26 PM > To: rsyslog-users > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > Just one quick note, more following: > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > > Sent: Tuesday, March 03, 2009 3:29 PM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > > > Hi there. > > > > > > As I said, I need **excellent** performance. I definitely need > > batch > > > > operations, the ability to prepare the statements given as > > arguments > > > > on the configuration file, and not to commit entries one by one, > > but > > > > after a number of entries are ready or (better) after some not so > > > > small time. According to the advise I got from experts around > here, > > > > I'll have to use Oracle Call Interface for this module, I don't > > know > > > > if there are any licensing issues. > > > > > > I can't comment on the licensing issue, I simply don't know what > > > Oracle demands. > > > > I'm not sure how GPL-compatible it is to link to already existing > > proprietary code. Anyways, first I code, then we test, then we (you, > > actually) decide the legal aspects. > > Actually, not me ;) I leave this risk to the user. If someone pays the > legal counselor, I'll add his POV to the project doc. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Tue Mar 17 20:44:57 2009 From: julianokyap at gmail.com (Julian Yap) Date: Tue, 17 Mar 2009 09:44:57 -1000 Subject: [rsyslog] Dynamic remote log files Message-ID: I have the following set up to generate Dynamic remote log files. $template DYNmessages, "/var/log/remote/%HOSTNAME%/messages" *.info,mail.none,authpriv.none,cron.none ?DYNmessages Unfortunately some devices log poorly without the hostname for some syslog messages. This means I'm ending up with lots of useless directories in /var/log/remote. If I log everything from a server to a file then it works fine: if $fromhost == 'server' then /var/log/remote/server/all As you can see the difference in file sizes as syslog messages are lost: # ls -l /var/log/remote/server/ total 1724 -rw------- 1 root root 980053 Mar 17 08:57 all -rw------- 1 root root 773533 Mar 17 08:57 messages I guess, I'm looking for config suggestions on setting up more robust dynamic logging for remote hosts. - Julian From aoz.syn at gmail.com Tue Mar 17 20:57:14 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 17 Mar 2009 13:57:14 -0600 Subject: [rsyslog] Dynamic remote log files In-Reply-To: References: Message-ID: <4255c2570903171257w4801cc3co8998ca883b5ae78@mail.gmail.com> On Tue, Mar 17, 2009 at 13:44, Julian Yap wrote: > I guess, I'm looking for config suggestions on setting up more robust > dynamic logging for remote hosts. The single most robust host-based structure I've found to use is 'fromhost-ip'. It's locally "generated" by the rsyslog daemon from the receiving socket and isn't affected by any of the message content. From Luis.Fernando.Munoz.Mejias at cern.ch Wed Mar 18 10:53:34 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Wed, 18 Mar 2009 10:53:34 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> Message-ID: <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> El Lunes, 16 de Marzo de 2009 17:53, Rainer Gerhards escribi?: > Please let me know what you think. I just came back from a week of holidays, I'm reviewing the skeleton, which looks pretty comprehensive. Thanks! -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Wed Mar 18 11:04:19 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 18 Mar 2009 11:04:19 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> When you are ready, I'd actually suggest that I create an "omoracle" git branch for you and do place a copy of ommysql into it. This, together with the comments from omtemplate, would probably one way to get a (non-optimal) quick start. I would suggest that we build a very basic oracle driver first and after we see it works well, then look into the performance optimization. Let me know what you think (and when you have time). I could actually create what I have proposed with little delay once you give a go. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Wednesday, March 18, 2009 10:54 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > El Lunes, 16 de Marzo de 2009 17:53, Rainer Gerhards escribi?: > > Please let me know what you think. > > I just came back from a week of holidays, I'm reviewing the skeleton, > which looks pretty comprehensive. Thanks! > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From kenneho.ndu at gmail.com Thu Mar 19 11:51:43 2009 From: kenneho.ndu at gmail.com (Kenneth Holter) Date: Thu, 19 Mar 2009 11:51:43 +0100 Subject: [rsyslog] rsyslog TCP session closing Message-ID: Hi. My rsyslog log host keep getting these messages in syslog: *rsyslogd:TCP session 66 will be closed, error ignored * The session numer (i.e. 66 in this case) varies. Are these messages of any importance? I'm guessing the sessions are closed due to being idle, and that the session will be re-established when the next syslog message are ready to be sent from the client. Regards, Kenneth From rgerhards at hq.adiscon.com Thu Mar 19 12:13:56 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 19 Mar 2009 12:13:56 +0100 Subject: [rsyslog] omfile reliability Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD37@GRFEXC.intern.adiscon.com> Hi all, as mentioned in some past posts, omfile did not really care if log data made it into the file system. The overall reaction was "don't care, discard message". I have now improved the situation very much (at least I hope so ;)). There is currently an experimental git branch omfile-errHandler which properly suspends the action if something goes wrong. The only thing it currently does not do is truncate partially written lines. I'll save this for some later release when I revamp to module as whole. I plan to merge this change into the main development branch soon and then do a new devel release. If you would like to play with the current version, I of course would appreciate that. If so, please let me know your results. Also, I found one strange thing while testing with the cifs (SMB) handler. It does not properly return a failure state, so I currently have no clue how to detect a failure condition in that case. Below, I post some excerpt from a forum thread related to the work [1]. If you happen to have any suggestions, please let me know. ===== good news and bad news: I have found a bug inside the code, and been able to fix that (not yet committed). However, I tried with the smb redirector (don't have nfs at hand) and it acks the writes, but does not ensure data is actually put onto the remote site. So there probably is no way to make sure we really have the data. Maybe the situation is better with NFS. below some excerpts from my twitter stream: # i have lots of garbage inside the log when I reconnect the network... looks like cifs driver can not really handle this situation1 minute ago from twhirl # it is interesting to see how the smb driver continues to accept data (at a very slow rate) while the network is off....9 minutes ago from twhirl # #rsyslog: issue is more complicated than I thought - probably a bug in dynafile creation processabout 1 hour ago from twhirl # ok, think I got a bug. FD is not set to indicated "closed" after actual close call - can lead to endless loopabout 2 hours ago from twhirl # as soon as I enter a new message, the missing content *is* writtenabout 2 hours ago from twhirl # after disconnect, nothing is written...about 2 hours ago from twhirl # very interesting... I get successful returns from write() to the network file - with plugged cable, lazy write, I guess...about 2 hours ago from twhirl # #rsyslog: OS buffering plays a big role in network-file retries - on the initial tries I do not see any error code at all! (w/o cable!!)about 2 hours ago from twhirl ==== Thanks, Rainer [1] http://kb.monitorware.com/log-to-nfs-and-buffer-if-unavailable-t8963-30.html# p15732 From Jefferson.Cowart at libraries.claremont.edu Thu Mar 19 23:18:00 2009 From: Jefferson.Cowart at libraries.claremont.edu (Jefferson Cowart) Date: Thu, 19 Mar 2009 15:18:00 -0700 Subject: [rsyslog] Separating Log files based on partial IP match Message-ID: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> I'm new to rsyslog, and I'm trying to set it up to centralize logging for a number of devices on my network. I'd like for it to log anything from my network switch to a single log file, my printers to another log file, etc. I'm able to separate the devices based on their IP address (e.g. my switches are in one IP subnet and my printers in another.) I see how to do per device logging on http://www.rsyslog.com/Article60.phtml, but I don't see a way to adjust that to do it based on IP subnet or anything like that. Unfortunately it looks like both FROMHOST and HOSTNAME are names not IPs, so it's not even clear if I could filter on that. Any help would be appreciated. Thanks. -- Thank You Jefferson Cowart Network and Systems Administrator Claremont University Consortium From david at lang.hm Fri Mar 20 00:44:36 2009 From: david at lang.hm (david at lang.hm) Date: Thu, 19 Mar 2009 16:44:36 -0700 (PDT) Subject: [rsyslog] Separating Log files based on partial IP match In-Reply-To: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> References: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> Message-ID: On Thu, 19 Mar 2009, Jefferson Cowart wrote: > I'm new to rsyslog, and I'm trying to set it up to centralize logging > for a number of devices on my network. I'd like for it to log anything > from my network switch to a single log file, my printers to another log > file, etc. I'm able to separate the devices based on their IP address > (e.g. my switches are in one IP subnet and my printers in another.) I > see how to do per device logging on > http://www.rsyslog.com/Article60.phtml, but I don't see a way to adjust > that to do it based on IP subnet or anything like that. Unfortunately it > looks like both FROMHOST and HOSTNAME are names not IPs, so it's not > even clear if I could filter on that. Any help would be appreciated. > Thanks. there is fromhost-ip that will give you the last-hop IP address I don't see an easy way to do it based on subnets, but take a look at the rscript stuff that just went into the development branch in the last week or so. that may give you the hooks needed to do the subnet calculation that will let you do what you want. David Lang From rgerhards at hq.adiscon.com Fri Mar 20 07:23:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 20 Mar 2009 07:23:12 +0100 Subject: [rsyslog] Separating Log files based on partial IP match References: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD3E@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, March 20, 2009 12:45 AM > To: rsyslog-users > Subject: Re: [rsyslog] Separating Log files based on partial IP match > > On Thu, 19 Mar 2009, Jefferson Cowart wrote: > > > I'm new to rsyslog, and I'm trying to set it up to centralize logging > > for a number of devices on my network. I'd like for it to log > anything > > from my network switch to a single log file, my printers to another > log > > file, etc. I'm able to separate the devices based on their IP address > > (e.g. my switches are in one IP subnet and my printers in another.) I > > see how to do per device logging on > > http://www.rsyslog.com/Article60.phtml, but I don't see a way to > adjust > > that to do it based on IP subnet or anything like that. Unfortunately > it > > looks like both FROMHOST and HOSTNAME are names not IPs, so it's not > > even clear if I could filter on that. Any help would be appreciated. > > Thanks. > > there is fromhost-ip that will give you the last-hop IP address > > I don't see an easy way to do it based on subnets, but take a look at > the > rscript stuff that just went into the development branch in the last > week > or so. that may give you the hooks needed to do the subnet calculation > that will let you do what you want. The only function currently supported is strlen(), but this is a very interesting use case to extend function support. I think I will add a couple of functions even without a full loadable interface, just to get some basic things done. If everything turns out to go smooth, I can hopefully do this next week. In the mean time, I would see if a property-based (regex) filter can do the job. For a classical class A,B,C net that should be easy to do. Rainer From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 20 18:01:31 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 20 Mar 2009 18:01:31 +0100 Subject: [rsyslog] Starting a native Oracle output module (was Re: Documentation on writing rsyslog modules?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> Message-ID: <200903201801.32116.Luis.Fernando.Munoz.Mejias@cern.ch> El Mi?rcoles, 18 de Marzo de 2009 11:04, Rainer Gerhards escribi?: > When you are ready, I'd actually suggest that I create an "omoracle" git > branch for you and do place a copy of ommysql into it. This, together with > the comments from omtemplate, would probably one way to get a (non-optimal) > quick start. > So, I'm starting it and I already have something that compiles. Next step is to have something I can test, then have something that makes something, then something that does the same but fast. > I would suggest that we build a very basic oracle driver first and after we > see it works well, then look into the performance optimization. > That's my idea, too. I want something that: 1) Connects to the DB at createInstance() time. 2) Runs the un-prepared statement passed as template on each syslog entry. 3) Disconnects only at freeInstance() time. Prepared statements and batch operations will be added later, indeed. But first, I'd like to know what ways I have to test my module, other than recompiling it, installing and restarting rsyslog for each change. > Let me know what you think (and when you have time). I'm already on it. I hope to deliver something for review next week. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 20 18:08:55 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 20 Mar 2009 18:08:55 +0100 Subject: [rsyslog] Starting a native Oracle output module (was Re:Documentation on writing rsyslog modules?) References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> <200903201801.32116.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD51@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Luis > Fernando Mu?oz Mej?as > Sent: Friday, March 20, 2009 6:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Starting a native Oracle output module > (was Re:Documentation on writing rsyslog modules?) > > El Mi?rcoles, 18 de Marzo de 2009 11:04, Rainer Gerhards escribi?: > > When you are ready, I'd actually suggest that I create an > "omoracle" git > > branch for you and do place a copy of ommysql into it. > This, together with > > the comments from omtemplate, would probably one way to get > a (non-optimal) > > quick start. > > > So, I'm starting it and I already have something that compiles. Next > step is to have something I can test, then have something that makes > something, then something that does the same but fast. > > > I would suggest that we build a very basic oracle driver > first and after we > > see it works well, then look into the performance optimization. > > > That's my idea, too. I want something that: > > 1) Connects to the DB at createInstance() time. > 2) Runs the un-prepared statement passed as template on each syslog > entry. > 3) Disconnects only at freeInstance() time. > > Prepared statements and batch operations will be added later, indeed. > > But first, I'd like to know what ways I have to test my module, other > than recompiling it, installing and restarting rsyslog for > each change. You can run rsyslog interactively, that's the key to a useful testing environment. In my development environment, I have a couple of conf files, and a shell script that starts rsyslogd in a variety of test settings (don't forget about running valgrind on it frequently, it safes you a lot of time ;)). I am not at my devel machine right now, but the core command looks something like cp "all plugins" runtime/.libs # or so ./tools/rsyslogd -dn -c 4 -f myconf.conf -M runtime/.libs ... Maybe some more... Then you run rsyslogd for your test, and press ctl-c when you are done. My cycle is Loop edit make run-script End Loop Does this help? Oh, and I have disabled the regular rsyslogd on that devel box. If you don't do, you probably need to add some extra quirks to it. I have also begun to work on some tcl-based tests yesterday, hope to have them in git mid next week. Rainer > > > Let me know what you think (and when you have time). > > I'm already on it. I hope to deliver something for review next week. :) > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 23 18:44:27 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 23 Mar 2009 18:44:27 +0100 Subject: [rsyslog] graph of rsyslog versions and branches Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD64@GRFEXC.intern.adiscon.com> Hi all, I created a condensed graph of rsyslog versions and branches today. I have done this as an example of how a software project evolves (what I'll write about soon), but I think it is also educating for folks interested in rsyslog. If you are interested, please find the entry point at my blog: http://blog.gerhards.net/2009/03/rsyslog-family-tree.html Rainer From pieter.thysebaert at intec.ugent.be Tue Mar 24 12:02:44 2009 From: pieter.thysebaert at intec.ugent.be (pieter.thysebaert at intec.ugent.be) Date: Tue, 24 Mar 2009 12:02:44 +0100 (CET) Subject: [rsyslog] imfile module - input line transformation Message-ID: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Hello rsyslog users, We are currently running a small rsyslog setup (i.e. TCP-based remote logging) in our test environment. This setup is also used to transfer Apache access logs, using the pipe operator in the Apache config and a Bash shell script which calls the "logger" tool to log a message to local rsyslog in a loop like # read first line #... while [ $result -eq 0 ]; do # log $line to $filename logger -p local0.info -t "APACHE" "$filename?$line" read line result=$? done The problem with this approach is twofold. First, we are experiencing performance issues under increased load (all Apache workers in status "L" on the Apache server status page when stress testing). Secondly, in order to resolve the first issue, we thought about moving to the file based input module which would make (we hope) Apache performance less depending on the logging infrastructure - as it would just log to the native filesystem as usual. However, as can be seen above, we're currently transforming the log messages to include the destination filename. On the remote rsyslog server (the receiving end), the messages are logged into a file whose name is dynamically derived from the first part of the log (the part before the first question mark). So my question is: can rsyslog be configured to 1. Read new lines from Apache access log as they become available 2. prepend an arbitrary string to the message (the destination filename) 3. log this transformed message instead of the original Or is there a more "best-practices" approach to do what I want (which is : filter messages on the remote end based on the tag and write them to a dynamically generated filename using regexps) Thanks, Pieter From Luis.Fernando.Munoz.Mejias at cern.ch Tue Mar 24 12:44:27 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?iso-8859-1?q?Mu=F1oz_Mej=EDas?=) Date: Tue, 24 Mar 2009 12:44:27 +0100 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <200903241244.27879.Luis.Fernando.Munoz.Mejias@cern.ch> Hi, > We are currently running a small rsyslog setup (i.e. TCP-based remote > logging) in our test environment. > > This setup is also used to transfer Apache access logs, using the pipe > operator in the Apache config and a Bash shell script which calls the > "logger" tool to log a message to local rsyslog in a loop like > > # read first line > #... > > while [ $result -eq 0 ]; do > # log $line to $filename > logger -p local0.info -t "APACHE" "$filename?$line" > read line > result=$? > done Why not use the CustomLog Apache directive to pipe directly the logger command: ... LogFormat "%b%l%a%h %b%l%a%h ..." logger_pipe CustomLog |/usr/bin/logger -p local0.info -t "apache" logger_pipe It should spawn only one logger process for the whole Apache host, and most likely reduce the load. > Secondly, in order to resolve the first issue, we thought about moving to > the file based input module which would make (we hope) Apache performance > less depending on the logging infrastructure - as it would just log to the > native filesystem as usual. However, as can be seen above, we're currently > transforming the log messages to include the destination filename. > On the remote rsyslog server (the receiving end), the messages are logged > into a file whose name is dynamically derived from the first part of the > log (the part before the first question mark). Again, you can use the LogFormat for that, and let Apache do the work without spawning processes over and over, which is most likely the slow part. > > So my question is: can rsyslog be configured to > 1. Read new lines from Apache access log as they become available > 2. prepend an arbitrary string to the message (the destination > filename) In principle, you should use a template for that (untested): $template TemplateName,"CONSTANT_ARBITRARY_STRING?%message%" if ($programname = "apache") then destination;TemplateName (Although I cannot assure how it behaves with TCP...) Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From THe_ZiPMaN+rsyslog at zipman.it Tue Mar 24 14:24:52 2009 From: THe_ZiPMaN+rsyslog at zipman.it (THe_ZiPMaN+rsyslog at zipman.it) Date: Tue, 24 Mar 2009 14:24:52 +0100 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <49C8DF24.8010507@zipman.it> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 pieter.thysebaert at intec.ugent.be wrote: | # read first line | #... | | while [ $result -eq 0 ]; do | # log $line to $filename | logger -p local0.info -t "APACHE" "$filename?$line" | read line | result=$? | done You are spawning a logger process for each log line... brrrr.... | Or is there a more "best-practices" approach to do what I want (which is : | filter messages on the remote end based on the tag and write them to a | dynamically generated filename using regexps) Personally I do this way: On the apache side for every VirtualHost: ErrorLog "|/usr/bin/logger -p local5.err -t http_example.com" CustomLog "|/usr/bin/logger -p local5.info -t http_example.com" combined On the rsyslog side: # Let the message "untouched" without adding any information for easy # parsing by webalizer & company $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" # Define an archiving policy that allows for simpler analisys and archiving # The number 58 should be tuned for your system. Obviously everything must # be on the same line. $template ArchiveApache,"/var/log/apache/%$YEAR%/%$MONTH%/%$DAY%/%syslogtag:F,58:1%_%syslogseverity-text%.log" # Define the destinations and prevent writing on other standard logs # Put this near the beginning of the conf file :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog :syslogtag,startswith,"http" ~ - -- Flavio Visentin GPG Key: http://www.zipman.it/gpgkey.asc There are only 10 types of people in this world: those who understand binary, and those who don't. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknI3yMACgkQusUmHkh1cnrISACfQNkWSda9yPICMM/ie78SGhLe FOMAniAk8S0coDfgCSNQp/IXGqCRfZd2 =IhIf -----END PGP SIGNATURE----- From aoz.syn at gmail.com Tue Mar 24 17:17:35 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 24 Mar 2009 10:17:35 -0600 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <4255c2570903240917l27354e08jc65a525b67e7c933@mail.gmail.com> > The problem with this approach is twofold. First, we are experiencing > performance issues under increased load (all Apache workers in status "L" > on the Apache server status page when stress testing). I am somewhat surprised neither of the responders did what seems obvious to me and bypass the pipe/execution altogether. Unless someone else here has had a problem doing so, there's no reason you couldn't just use a named pipe on both ends: [shell] mkfifo /var/run/htlog-1 [apache] CustomLog "/var/run/htlog-1" [rsyslog] $ModLoad imfile $InputFileName /var/run/htlog-1 $InputFileTag apache1 $InputFileRunMonitor That puts the logs in rsyslog with no extra executions or running processes; what you do after that for filtering is up to you. The nice thing about using a named pipe is that if the reading process dies, the buffer doesn't go away and you have less chance of losing messages. From erik at readmedia.com Tue Mar 24 20:32:41 2009 From: erik at readmedia.com (Erik Morton) Date: Tue, 24 Mar 2009 15:32:41 -0400 Subject: [rsyslog] Have I made rsyslog synchronous by mistake? Message-ID: <6829D0E0-079A-448C-8766-C190249425C1@readmedia.com> Hello there. I have rsyslog configured to forward logging messages from several application servers to a central log server. It's a Ruby on Rails app and I'm using the SyslogLogger gem to talk to rsyslog. From time to time under moderate volume my application, or more accurately one or more of my application containers, begins to freeze. I haven't been able to pin down the cause, but I did notice a couple of interesting things related to rsyslog. Very soon before the application begins to experience problems the central log file (to which all app servers forward) stops updating. This has happened every time the application has had problems. On a lark I decided to disable rsyslog and instead use the native rails logging framework. Each time this change has completely cleared up all the problems on the site. Obviously this is a grossly unscientific observation but I just can't ignore the coincidence. I'm thinking that I have borked the config of my installation to, somehow, cause this failure. Is it possible that I have configured rsyslog to somehow wait for a successful write to the log file instead of firing and forgetting? Am I required to create a local spool per http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html? Many thanks in advance. This is the configuration for the host: $ModLoad imtcp $InputTCPServerRun 200 *.info;mail.none;authpriv.none;cron.none;my_app.none / var/log/messages authpriv.* /var/log/secure mail.* -/var/log/ maillog cron.* /var/log/cron *.emerg * uucp,news.crit /var/log/spooler local7.* /var/log/ boot.log $outchannel my_app_rotate,/vol/logs/my_app.log,5242880,/usr/bin/ loganalysis /vol/logs/my_app.log !my_app *.* $my_app_rotate Each host then has this in rsyslog.conf !my_app *.* @@log_host:200 And I start rsyslogd on the central log host with SYSLOGD_OPTIONS="- t200 -m 0" From rgerhards at hq.adiscon.com Wed Mar 25 09:45:48 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 25 Mar 2009 09:45:48 +0100 Subject: [rsyslog] rsyslog branches (german content) Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD89@GRFEXC.intern.adiscon.com> Hi all, those of you who understand a bit of German may find this German blog post interesting: http://www.wissenslogs.de/wblogs/blog/mehr-als-bits-und-bytes/allgemein/2009- 03-24/software-evolution It talks about "software evolution" based on rsyslog's development process. While doing so, I think it captures also a lot of the spirit in which versions are created today for rsyslog. Sorry I have no English version currently... Rainer From Luis.Fernando.Munoz.Mejias at cern.ch Thu Mar 26 15:28:30 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Thu, 26 Mar 2009 15:28:30 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 Message-ID: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> Hi, I have a funny problem. Around here we have a number of nodes using old, syslogd, which report to their headnodes, which use rsyslog v3, wich keep relaying till I get a small copy on a test box. This test box uses, since yesterday, rsyslog v4. I noticed that for rsyslog v4, the last relay is considered to be the source host, the real source host is considered to be the syslogtag and everything else is inside the %msg% property. For the default template, I get messages like these: 2009-03-26T00:00:00+01:00 relayhost sourcehost1 cvs: GSSAPI userok: cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG GSS_C_CONF_FLAG 2009-03-26T00:00:00+01:00 relayhost sourcehost2 cvs: GSSAPI userok: cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG GSS_C_CONF_FLAG And, as I used to have a single file per host, I now have a single, huge "relayhost" file. Filters based on source or program name are broken, of course. What did I screw when upgrading? Thanks. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Thu Mar 26 15:30:45 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Mar 2009 15:30:45 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> You screw nothing - that's a bug in v4. You need to pull the latest devel from git ;) A new release is due soon. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Thursday, March 26, 2009 3:29 PM > To: rsyslog-users > Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > Hi, > > I have a funny problem. Around here we have a number of nodes using > old, syslogd, which report to their headnodes, which use rsyslog v3, > wich keep relaying till I get a small copy on a test box. This test box > uses, since yesterday, rsyslog v4. > > I noticed that for rsyslog v4, the last relay is considered to be the > source host, the real source host is considered to be the syslogtag and > everything else is inside the %msg% property. For the default template, > I get messages like these: > > 2009-03-26T00:00:00+01:00 relayhost sourcehost1 cvs: GSSAPI userok: > cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG > GSS_C_CONF_FLAG > 2009-03-26T00:00:00+01:00 relayhost sourcehost2 cvs: GSSAPI userok: > cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG > GSS_C_CONF_FLAG > > And, as I used to have a single file per host, I now have a single, > huge > "relayhost" file. Filters based on source or program name are broken, > of > course. > > What did I screw when upgrading? > > Thanks. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Luis.Fernando.Munoz.Mejias at cern.ch Thu Mar 26 16:24:24 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Thu, 26 Mar 2009 16:24:24 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> Message-ID: <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> El Jueves, 26 de Marzo de 2009 15:30, Rainer Gerhards escribi?: > You screw nothing - that's a bug in v4. You need to pull the latest devel > from git ;) I just tried (if it's "master" branch, I mean), with no success. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Thu Mar 26 17:04:39 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Mar 2009 17:04:39 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> It's the master branch and I am sure I fixed this... mhhh... Need to finally complete what I am working on right now, will look after that... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Thursday, March 26, 2009 4:24 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > El Jueves, 26 de Marzo de 2009 15:30, Rainer Gerhards escribi?: > > You screw nothing - that's a bug in v4. You need to pull the latest > devel > > from git ;) > > I just tried (if it's "master" branch, I mean), with no success. > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From kenneho.ndu at gmail.com Fri Mar 27 16:21:03 2009 From: kenneho.ndu at gmail.com (Kenneth Holter) Date: Fri, 27 Mar 2009 16:21:03 +0100 Subject: [rsyslog] Client syslog messages are logged twice Message-ID: Hi I'm running rsyslog v2.0.6, and have the following setup: rsyslog clients => rsyslog relay => rsyslog master <= rsyslog clients The /etc/rsyslog.conf file at the master has these lines in it: ** *$template DynaFile,"/var/log/syslog-clients/%HOSTNAME%/%$YEAR%/%$MONTH%/system-%HOSTNAME%-%$NOW%.log" *.* -?DynaFile * At my rsyslog master I see that many (or all) the client log messages are logged in two the different places, both under its hostname (i.e. %HOSTNAME% is replaced by the hostname) and under its IP-adresss (i.e. %HOSTNAME% is replaced by the IP-adress). So in effect, all the messages are logged in twice. I figured it might have something to do with reverse DNS, so I added the necessary entries the /etc/hosts-file, but with no success. Does anyone have a clue as to why this is happening? Regards, Kenneth From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 27 18:09:42 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 27 Mar 2009 18:09:42 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> Message-ID: <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> El Jueves, 26 de Marzo de 2009 17:04, Rainer Gerhards escribi?: > It's the master branch and I am sure I fixed this... I'm sorry to say it's not. I just pulled git master branch, rebuilt, reinstalled and no changes. 5 minutes ago I downgraded to v3.20, and my new log files appeared as I expected them to, and my filters work as expected. > mhhh... Need to finally complete what I am working on right now, will > look after that... Of course. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 27 18:21:59 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 27 Mar 2009 18:21:59 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch><200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> Can you send me an on-the-wire sample of those messages (I mean that are invalidly interpreted). I have now created the parser test suite and they would make a good addition, especially as I need to troubleshoot them ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Friday, March 27, 2009 6:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > El Jueves, 26 de Marzo de 2009 17:04, Rainer Gerhards escribi?: > > It's the master branch and I am sure I fixed this... > > I'm sorry to say it's not. I just pulled git master branch, rebuilt, > reinstalled and no changes. > > 5 minutes ago I downgraded to v3.20, and my new log files appeared as I > expected them to, and my filters work as expected. > > > mhhh... Need to finally complete what I am working on right now, will > > look after that... > > Of course. > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 27 19:23:15 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 27 Mar 2009 19:23:15 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> Message-ID: <200903271923.16168.Luis.Fernando.Munoz.Mejias@cern.ch> Rainer, > Can you send me an on-the-wire sample of those messages (I mean that are > invalidly interpreted). I have now created the parser test suite and they > would make a good addition, especially as I need to troubleshoot them ;) > > Rainer Before disclosing enough data I have to ask for permission. I can tell you that the last hop in this relay chain is using rsyslog v3, and that the format I got (tcpdump dixit) for these messages is always like this: <38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) And what gets actually logged for that is: 2009-03-27T19:06:53+01:00 last_hop_server source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) Then, last_hop_server becomes %hostname% and source_server becomes %syslogtag%. This last hop server is using rsyslog v3, so it seems to me I have to instruct v4 that the input is coming in a non-default format. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 27 22:38:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 27 Mar 2009 22:38:06 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 Message-ID: <000d01c9af24$6fb6e7ff$100013ac@intern.adiscon.com> These samples are enough, no need to disclose more. Single lines are sufficient, as long as they can repro the problem :) rainer ----- Urspr?ngliche Nachricht ----- Von: "Luis Fernando Mu?oz Mej?as" An: "rsyslog-users" Gesendet: 27.03.09 19:23 Betreff: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 Rainer, > Can you send me an on-the-wire sample of those messages (I mean that are > invalidly interpreted). I have now created the parser test suite and they > would make a good addition, especially as I need to troubleshoot them ;) > > Rainer Before disclosing enough data I have to ask for permission. I can tell you that the last hop in this relay chain is using rsyslog v3, and that the format I got (tcpdump dixit) for these messages is always like this: <38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) And what gets actually logged for that is: 2009-03-27T19:06:53+01:00 last_hop_server source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) Then, last_hop_server becomes %hostname% and source_server becomes %syslogtag%. This last hop server is using rsyslog v3, so it seems to me I have to instruct v4 that the input is coming in a non-default format. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 08:06:51 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 2 Mar 2009 08:06:51 +0100 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to always use fqdn of sending devices? References: <49993125.2060603@ecker-software.de><4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com><4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com><4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> Hi all, I have (obviously) no strong position in this. I do not object putting distro-specific files into a "contrib" directory and make them available with the tarball *as long as it is clear that I do not support them*. I concur to David that this may be useful and I also concur to Michael that it may cause some confusion. To me, the important point is that I can not support distro-specific things (at least outside of the core code) and that I will not want to create and release dependencies. So if we put some package files into the tarball, that means I will update them if I receivea patch or am asked to pull the, but I will neither verify them nor will I hold releases. So, in short, they will be unmaintained and often not matching the rest of the tarball. HOWEVER, I can see that there are cases where it would be useful to hae those files available. On the other hand, at least for Debian, I think it is possible to obtain the package files from Debian directly (but, granted, it may not have the newest ones, e.g. v4). I have a pragmatic suggestion: if you have package specific files, you can send them to me. I will create a subdirectory for them. There will be a README telling people that this stuff is (from my POV) unmaintained, probably outdated and to be used with care. If a maintainer (like Michael) later decides it was a bad idea to put the files into the tarball, I'll also happily delete them. Does this sound like a workable compromise? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Saturday, February 28, 2009 3:16 AM > To: rsyslog-users > Subject: Re: [rsyslog] Get rsyslog to always use fqdn of > sending devices? > > On Sat, 28 Feb 2009, Michael Biebl wrote: > > >>> > >>> If the fedora bits are kept in an entirely separate > upstream packaging > >>> branch, then I don't really care. > >>> But I wouldn't like to see them (or any debian related > files) shipped > >>> in a release tarball. > >> > >> so how am I (a debian user) supposed to create debian > compatible packages > >> for versions that you don't yet deal with? > >> > >> why couldn't you push the debian related files upstream > and maintain them > >> there? (submitting patches, or git pull requests for updates) > > > > Pretty simple: It's less work for me and Rainer and more flexible. > > Say I (for Debian) start adding the files upstream, so does > Fedora, BSD, etc... > > Now when Rainer wants to make a new release to not have any stale > > packaging files, he would have to ping all package > maintainer first to > > update the build files and push those changes. That simply doesn't > > scale. > > Packaging and upstream software releases should be decoupled. > > > > If you are really interested in the Debian Packaging, you > can grab the > > git repository from [1] and either work from there or at it as a > > "remote" to the rsyslog git repo and merge the debian specific bits. > > it's not that I'm interested in debian packaging, it's that I need to > install the stuff that you haven't decided to ship in debian > yet on my > debian system in such a way that I keep the package manager > happy (and > don't have it overwriting what I've compiled with an update > of an obsolete > version) > > it's not that the upstream version of the files need to be > perfect, but > they should be good enough to avoid the need for users to > have to fight > the packaging system and duplicate your efforts. > > I hate to have to pull in some stuff from your tree and > combine it with > stuff from the upstream tree because I don't know enough to > be sure that > I'm both pulling everything I need and not pulling something > that will > cause grief. > > you've made your decision, count this as one voice > disagreeing with that > decision. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Mon Mar 2 08:56:33 2009 From: david at lang.hm (david at lang.hm) Date: Sun, 1 Mar 2009 23:56:33 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 27 Feb 2009, david at lang.hm wrote: > On Thu, 26 Feb 2009, david at lang.hm wrote: > >> >> this works for reopening the socket each time, but if I uncomment the bind >> the sendto fails (error 22, invalid input) >> >> I haven't yet figured out what I'm missing on the bind that's causing this > > a little more testing and I find that the bind succeeds, but no traffic goes > out unless the source IP exists somewhere on the box (it can be bound to > lo:0, but it needs to exist) > > so the non-local-bind approach may not work :-( > > it's just hit midnight here, so I'm going to call it a night and try again > tomorrow. I abandoned this approach and spent the weekend learning how to do raw sockets. I found a library that makes it not that bad to do (at least for the IPv4 that I've done so far, IPv6 adds some wrinkles) the one thing thats not clear to me at this point is how to find the original source IP of the message. Is that available in a variable inside UDPSend, or is it something that I will have to get earlier in the process and then pass explicitly to UDPSend? David Lang From david at lang.hm Mon Mar 2 10:04:58 2009 From: david at lang.hm (david at lang.hm) Date: Mon, 2 Mar 2009 01:04:58 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: <1235670387.28865.2.camel@rf10up.intern.adiscon.com> References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: On Thu, 26 Feb 2009, Rainer Gerhards wrote: > On Sun, 2009-03-01 at 23:56 -0800, david at lang.hm wrote: >> On Fri, 27 Feb 2009, david at lang.hm wrote: >> >>> On Thu, 26 Feb 2009, david at lang.hm wrote: >>> >>>> >>>> this works for reopening the socket each time, but if I uncomment the bind >>>> the sendto fails (error 22, invalid input) >>>> >>>> I haven't yet figured out what I'm missing on the bind that's causing this >>> >>> a little more testing and I find that the bind succeeds, but no traffic goes >>> out unless the source IP exists somewhere on the box (it can be bound to >>> lo:0, but it needs to exist) >>> >>> so the non-local-bind approach may not work :-( >>> >>> it's just hit midnight here, so I'm going to call it a night and try again >>> tomorrow. >> >> I abandoned this approach and spent the weekend learning how to do raw >> sockets. I found a library that makes it not that bad to do (at least for >> the IPv4 that I've done so far, IPv6 adds some wrinkles) >> >> the one thing thats not clear to me at this point is how to find the >> original source IP of the message. Is that available in a variable inside >> UDPSend, or is it something that I will have to get earlier in the process >> and then pass explicitly to UDPSend? > > Actually, output modules do not receive access to the full message > object. This was originally done for security reasons (do not pass more > than needed). All they can receive is the strings that are passed to > them. So the module would need to be modified so that a second string > (like ommail) is passed and that string needs to be defined as the > to-be-spoofed IP (what also enables to rewrite the source IP). I will look into this. >> From all the discussion, it may make sense to start with a different > output plugin that may later be merged back into the original one... Ok, I won't try to have it do everything and just concentrate on doing the forging. forging on an all IPv4 network is very simple, on an all IPv6 network just a bit harder, it's not clear what to do for a mixed network (for a IPv6 destination and IPv4 source you can do a mapping, but what is the right thing to do for an IPv6 source with a IPv4 destination?) note that the other item (closing the output socket every X messages) should be pretty trivial to add into the existing module and is useful for both TCP and UDP. David Lang > Rainer >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 2 12:51:13 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 2 Mar 2009 12:51:13 +0100 Subject: [rsyslog] Weird fromhost property value References: <49A78F5C.3000400@net-m.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71EE7@GRFEXC.intern.adiscon.com> Can you retry with v4? That should be much cleaner now, maybe relp does not yet provide the resolved info (that is a protocol transport driver [lib] issue). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Patrick Shen > Sent: Friday, February 27, 2009 8:00 AM > To: rsyslog-users > Subject: [rsyslog] Weird fromhost property value > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi All, > > I've utilized rsyslog as my company's central logging server for half a > year. > > Today I encounterd a very weird issue about value of fromhost property. > We use dynamic templates to store logs from clients. > > The template is like below: > > $template > d_hosts,"/var/rsyslog/HOSTS/%fromhost%/%$year%/%$month%/%syslogfacility > -text%_%fromhost%_%$year%_%$month%_%$day > %.log" > > You can see we group logs by fromhost value. > > Today, I did 3 times test that a client named (sobek) sent logs to > central logging server by UDP, TCP and RELP. > > The FQDN of client node is "sobek.net-m.internal", short name is > "sobek", ip address is "172.21.101.13". > > After testing, I got when sending via UDP, the fromhost value is short > name. And via TCP, the value is FQDN. Via RELP, the value is IP > address. > > So I got a very weird directory organization at "/var/rsyslog/HOSTS". > > ####################################################################### > ### > drwxr-x--- 3 root syslog 80 Feb 27 07:24 172.21.101.13 <- RELP > drwxr-x--- 3 root syslog 80 Feb 27 05:58 sobek <- UDP > drwxr-x--- 3 root syslog 80 Feb 27 06:03 sobek.net-m.internal <- TCP > ####################################################################### > ### > > We are running rsyslog 3.20.0 both on client and server. So I wanna > know > if any other has encountered this before? > > Thanks, > Patrick > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFJp49ckHhYtFevC+MRApbbAJ9Dgxtw5mf+ax9D81OZPfh5E9aJPgCdEqF/ > FlkFDJpWr4k6pVV4AQiLhRw= > =cQzr > -----END PGP SIGNATURE----- > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 14:42:45 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:42:45 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200902161141.21380.Luis.Fernando.Munoz.Mejias@cern.ch> <577465F99B41C842AAFBE9ED71E70ABA44FC08@grfint2.intern.adiscon.com> <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <1236001365.28865.44.camel@rf10up.intern.adiscon.com> On Fri, 2009-02-27 at 18:48 +0100, Luis Fernando Mu?oz Mej?as wrote: > Rainer, > > Good and bad news... > > > > That sounds really great. Before you start coding or preparing > > > anything, let me check how well our DBs perform, because it's not > > > yet clear if they'll be able to cope with the high insertion rate we > > > expect. If we don't go for the Oracle database this work doesn't > > > make sense. I bet we'll want the Oracle, anyways. > > > > Sounds fair. > > Good news: I did my tests and, for many tasks I need to do, Oracle is > our way to go. So, I'm willing to write the module, with your > guidance/advise. > so far this sounds good ;) > As I said, I need **excellent** performance. I definitely need batch > operations, the ability to prepare the statements given as arguments on > the configuration file, and not to commit entries one by one, but after > a number of entries are ready or (better) after some not so small > time. According to the advise I got from experts around here, I'll have > to use Oracle Call Interface for this module, I don't know if there are > any licensing issues. I can't comment on the licensing issue, I simply don't know what Oracle demands. On thing to do it is let the output module handle the "combination work" together. The output module is called one per message, however, it does not mean the output must directly write them to the database. It may buffer them until the batch is large enough. But this currently needs to be implemented on the output module basis. Obviously, that will not make coding simpler. If we find a sponsor for the necessary non-trivial extension of the core engine, the output module's task may become much easier. If things go well, such a sponsor may show up... > > It seems I'll have to review how rsyslog's queing modules work... I would suggest not to move into them - but, of course, if you like to... Lol, this is the non-trival task I talked about, there are numerous subtleties and, of course, they are weakly documented (but the inline doc is quite good). > > > > For this evaluation, I already have a timestamp formatter that fits > > > into Oracle, something that can be used with the property replacer, > > > like %timereported:::date-oracle%. > > > The bad news is that this timestamp formatter works perfectly on > interactive sessions (sqlplus) but not on non-interactive ones, f.i, in > Python scripts. You need to call Oracle's to_timestamp(string, format), > and by bloating your code with this ugly function the rfc-3339 formatter > is good enough. So I won't submit this one. > Sounds fair ;) Do you have a time frame for your project? (and maybe a rough overview of the "big picture" - I am always soooo curios ;)) Rainer > Cheers. From rgerhards at hq.adiscon.com Mon Mar 2 14:57:34 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:57:34 +0100 Subject: [rsyslog] Get rsyslog to always use fqdn of sending devices? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44FB9E@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FBAF@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FBFE@grfint2.intern.adiscon.com> <49993125.2060603@ecker-software.de> <4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com> <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> Message-ID: <1236002254.28865.46.camel@rf10up.intern.adiscon.com> Hi RB, on twitter, I was pointed to rpmforge. Does this sound like something that could be used? Rainer On Thu, 2009-02-26 at 17:49 +0100, Rainer Gerhards wrote: > Hi RB, > > thanks for all your hard work. I am absolutely willing to help make > succeed in that. Just one question before we do down to details. Are > there any other options that we can pursue? I remember, quite some time > ago, that someone posted the idea that some well-known (non-RH, not > EPEL) repositories exist. Unfortunatley, I do no longer know which these > were. > > So the question is: are there any other such repositories where RHEL > users turn to and, if so, can we work with them to achieve our joint > goals? > > Sorry for some backtracking here... > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of RB > > Sent: Thursday, February 26, 2009 4:54 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Get rsyslog to always use fqdn of sending > > devices? > > > > On Tue, Feb 17, 2009 at 13:11, RB wrote: > > > Regardless, I'll take the flag and see what I can do to get a > > > readily-accessible reasonably current build available for CentOS-5. > > > > Good & bad news - the good news is the Fedora upstream is very > > responsive, the bad news is I got sidetracked after his response. > > > > I have been told that rsyslog cannot be put in EPEL since it is > > already packaged in RHEL, be that package good or bad. Tomas has > > offered to help with the SPEC should I have any problems, but it looks > > like we're on our own for the time being. > > > > RPM package distribution can be done to various depths. The simplest > > is to just provide both the SRPM and unsigned binary RPMs for a few > > chosen CPU architectures for each packaged release as an HTTP or FTP > > download. This would allow one-off installations (updates would be > > manual) and generally get the package 'out there' for use. Further > > steps would involve signing the binaries and possibly publishing a > > repo that users could subscribe to (using /etc/yum.* or equivalent) > > for automated updates. > > > > Distributing a binary package in whatever form is going to increase > > the load (however mildly) on the project - each release will involve > > compiling and distributing binaries and SRPMs, if not signing them as > > well. I can work with you [Rainer] to automate that process, but as a > > random user I should probably not be doing the compilation and signing > > myself. > > > > So, we have 4 basic questions: > > 1. What versions are desired? > > 2. Are there any rsyslog components or functionality not packaged in > > the Fedora distribution users here would like to see included? > > 3. Do we want to sign the packages? > > 4. Who will perform the compilation/signing? > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 2 14:54:00 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 02 Mar 2009 14:54:00 +0100 Subject: [rsyslog] Matching hostname and facility? In-Reply-To: <49A6D8CB.1010506@web-ster.com> References: <49A2E460.50604@web-ster.com> <49A5A521.8040107@web-ster.com> <49A6D8CB.1010506@web-ster.com> Message-ID: <1236002040.28865.45.camel@rf10up.intern.adiscon.com> On Thu, 2009-02-26 at 10:00 -0800, Scott Baker wrote: > On 02/25/2009 03:38 PM, (private) HKS wrote: > >> Does this syntax work on rsyslog 2.0.x, that's what my server has on it. > >> I've tried this syntax, but it's not logging. > >> > >> - Scott > > > > > > No, this will require 3+ - which you really should upgrade to anyway. > > That's what I figured... this is my CORE syslog server, so I'll need to > play a good upgrade proceedure. > > Is their documentation on configuration file changes going from 2.x to 3.x? There is a compatiblity guide, I guess this is what you are looking for: http://www.rsyslog.com/doc-v3compatibility.html Rainer > > - Scott > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 3 08:54:16 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 08:54:16 +0100 Subject: [rsyslog] Three questions! Message-ID: Hello! I have a few questions. 1. The man page on the website is really outdated. Is it possible for it to be updated automatically? 2. Is it possible for the man page for rsyslog.conf to be up there too? 3. Can rsyslog handle importing existing log files? e.g. sending the latest entries from /var/log/httpd/somename.acc across rsyslog to a logging server? Thanks! From jackmarrow2 at gmail.com Tue Mar 3 09:05:51 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 09:05:51 +0100 Subject: [rsyslog] rsyslog changelog Message-ID: Hello, Is there a changelog for rsyslog, particularly showing the differences between the current version (3.x) and the 2.x version found in RHEL? Thanks, Jack From rgerhards at hq.adiscon.com Tue Mar 3 09:05:47 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 09:05:47 +0100 Subject: [rsyslog] Three questions! References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F03@GRFEXC.intern.adiscon.com> Hi, you asked just in time. See note on doc here: http://blog.gerhards.net/2009/03/rsyslog-doc-state-of-art.html For the file import, you can do this with imfile: http://www.rsyslog.com/doc-imfile.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 03, 2009 8:54 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Three questions! > > Hello! > > I have a few questions. > > 1. The man page on the website is really outdated. Is it possible for > it to be updated automatically? > 2. Is it possible for the man page for rsyslog.conf to be up there too? > 3. Can rsyslog handle importing existing log files? e.g. sending the > latest entries from /var/log/httpd/somename.acc across rsyslog to a > logging server? > > Thanks! > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Tue Mar 3 09:08:23 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 00:08:23 -0800 (PST) Subject: [rsyslog] rsyslog changelog In-Reply-To: References: Message-ID: On Tue, 3 Mar 2009, jack marrow wrote: > Hello, > > Is there a changelog for rsyslog, particularly showing the differences > between the current version (3.x) and the 2.x version found in RHEL? the best way to see the differences would be through git, however the differences between 2.x and 3.x are going to be so massive that it's going to be hard to see anything useful. what are you looking for? David Lang From rgerhards at hq.adiscon.com Tue Mar 3 09:09:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 09:09:17 +0100 Subject: [rsyslog] rsyslog changelog References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Well, you can see all change log entries by following the "change log" menu item in the menu to the left ;) But it may even be more convenient in that case that you get it directly from git as a single text file: http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=ba2a6c13e22b7 f67401c7edb15ea17d31162bde4;hb=HEAD Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 03, 2009 9:06 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog changelog > > Hello, > > Is there a changelog for rsyslog, particularly showing the differences > between the current version (3.x) and the 2.x version found in RHEL? > > Thanks, > > Jack > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 3 09:16:08 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 3 Mar 2009 09:16:08 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: References: Message-ID: 2009/3/3 : > On Tue, 3 Mar 2009, jack marrow wrote: > >> Hello, >> >> Is there a changelog for rsyslog, particularly showing the differences >> between the current version (3.x) and the 2.x version found in RHEL? > > the best way to see the differences would be through git, however the > differences between 2.x and 3.x are going to be so massive that it's going > to be hard to see anything useful. > > what are you looking for? I need to know which features are in the RHEL 5 version (2.x) and which are in the upstream stable version (3.x). Is there a matrix somewhere? It would be good if there was. I am looking for imfile support, regular expressions (are these perl regular expressions or posix?). Plus the general major differences. Also are actions are supported? Thanks > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Tue Mar 3 13:01:29 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 04:01:29 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: On Mon, 2 Mar 2009, david at lang.hm wrote: > On Thu, 26 Feb 2009, Rainer Gerhards wrote: > >> Actually, output modules do not receive access to the full message >> object. This was originally done for security reasons (do not pass more >> than needed). All they can receive is the strings that are passed to >> them. So the module would need to be modified so that a second string >> (like ommail) is passed and that string needs to be defined as the >> to-be-spoofed IP (what also enables to rewrite the source IP). > > I will look into this. I haven't had time to figure this out yet. >>> From all the discussion, it may make sense to start with a different >> output plugin that may later be merged back into the original one... > > Ok, I won't try to have it do everything and just concentrate on doing the > forging. attached is a diff that turns the UDP forwarding into forging, currently with a fixed from address of 1.1.1.1 port 2 I also needed to modify the makefile to add LIBS = /usr/lib/libnet.a for it to compile in my research, I learned that syslog-ng uses this same library for their forging. so far I have avoided looking at the syslog-ng code (I wanted to understand what was happening on my own, and I also avoid any potential license issues until I can check on them) David Lang From Luis.Fernando.Munoz.Mejias at cern.ch Tue Mar 3 15:28:58 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Tue, 3 Mar 2009 15:28:58 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <1236001365.28865.44.camel@rf10up.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch> <1236001365.28865.44.camel@rf10up.intern.adiscon.com> Message-ID: <200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> Hi there. > > As I said, I need **excellent** performance. I definitely need batch > > operations, the ability to prepare the statements given as arguments > > on the configuration file, and not to commit entries one by one, but > > after a number of entries are ready or (better) after some not so > > small time. According to the advise I got from experts around here, > > I'll have to use Oracle Call Interface for this module, I don't know > > if there are any licensing issues. > > I can't comment on the licensing issue, I simply don't know what > Oracle demands. I'm not sure how GPL-compatible it is to link to already existing proprietary code. Anyways, first I code, then we test, then we (you, actually) decide the legal aspects. > On thing to do it is let the output module handle the "combination > work" together. The output module is called one per message, however, > it does not mean the output must directly write them to the > database. It may buffer them until the batch is large enough. But this > currently needs to be implemented on the output module basis. > Obviously, that will not make coding simpler. That's what I expected, indeed. > > It seems I'll have to review how rsyslog's queing modules work... > > I would suggest not to move into them - but, of course, if you like > to... Lol, this is the non-trival task I talked about, there are > numerous subtleties and, of course, they are weakly documented (but > the inline doc is quite good). OK. I'll just have a buffer of entries to be committed. > Do you have a time frame for your project? (and maybe a rough overview > of the "big picture" - I am always soooo curios ;)) Not a full timescale. Let's say that as soon as you can provide me with the documentation/skeleton module most (say 70%) of my work will be developing this output module. Then, when I understand what a bad nightmare OCI is I'll be able to give a full timescale. After looking at ompgsql, it looks like writing output modules is easy if you know what you're doing. ;) Then, I'll be able to provide support for this module (fixing bugs and so on) for a couple of years, so it won't be shoot and forget. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Tue Mar 3 15:26:26 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 3 Mar 2009 15:26:26 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch><1236001365.28865.44.camel@rf10up.intern.adiscon.com> <200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> Just one quick note, more following: > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Tuesday, March 03, 2009 3:29 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > Hi there. > > > > As I said, I need **excellent** performance. I definitely need > batch > > > operations, the ability to prepare the statements given as > arguments > > > on the configuration file, and not to commit entries one by one, > but > > > after a number of entries are ready or (better) after some not so > > > small time. According to the advise I got from experts around here, > > > I'll have to use Oracle Call Interface for this module, I don't > know > > > if there are any licensing issues. > > > > I can't comment on the licensing issue, I simply don't know what > > Oracle demands. > > I'm not sure how GPL-compatible it is to link to already existing > proprietary code. Anyways, first I code, then we test, then we (you, > actually) decide the legal aspects. Actually, not me ;) I leave this risk to the user. If someone pays the legal counselor, I'll add his POV to the project doc. Rainer From aoz.syn at gmail.com Tue Mar 3 16:15:10 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 3 Mar 2009 08:15:10 -0700 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to always use fqdn of sending devices? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> References: <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903030715x55403ed9k59253789295ffcba@mail.gmail.com> On Mon, Mar 2, 2009 at 00:06, Rainer Gerhards wrote: > I have a pragmatic suggestion: if you have package specific files, you > can send them to me. I will create a subdirectory for them. There will > be a README telling people that this stuff is (from my POV) > unmaintained, probably outdated and to be used with care. If a > maintainer (like Michael) later decides it was a bad idea to put the > files into the tarball, I'll also happily delete them. > > Does this sound like a workable compromise? It does, but I'm not sure how it will mesh with wanting to provide packages for other distros that aren't so responsive as Debian or up-to-date as Fedora. I'll be happy to provide an RPM specfile for -stable and -dev (since Fedora already does a -beta package) but that may not be sufficient for the general clicky-package group. From aoz.syn at gmail.com Tue Mar 3 16:18:12 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 3 Mar 2009 08:18:12 -0700 Subject: [rsyslog] Get rsyslog to always use fqdn of sending devices? In-Reply-To: <1236002254.28865.46.camel@rf10up.intern.adiscon.com> References: <577465F99B41C842AAFBE9ED71E70ABA44FBFE@grfint2.intern.adiscon.com> <49993125.2060603@ecker-software.de> <4255c2570902161448i731aa22as2b43e34feb049b55@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FC12@grfint2.intern.adiscon.com> <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com> <4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com> <9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com> <1236002254.28865.46.camel@rf10up.intern.adiscon.com> Message-ID: <4255c2570903030718t73f55871n26d83867c3a3e621@mail.gmail.com> On Mon, Mar 2, 2009 at 06:57, Rainer Gerhards wrote: > on twitter, I was pointed to rpmforge. Does this sound like something > that could be used? That definitely looks viable, I'll submit a request and see how it goes. From danson at rackspace.com Tue Mar 3 23:57:10 2009 From: danson at rackspace.com (Daniel Anson) Date: Tue, 3 Mar 2009 16:57:10 -0600 Subject: [rsyslog] Double quotes Problem Message-ID: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> Does anyone know of a quick and easy template to remove the double quote character from a %msg% before it is inserted into the database (MySQL in my case). I have a %msg% that looks like this: user pid=21214 uid=0 auid=4294967295 msg='PAM setcred: user="oracle" exe="/bin/su" (hostname=?, addr=?, terminal=? result=Success)' I am reading the %msg% from the MySQL database and returning it in JSON formatting. When it encounters a double-quote character, it causes issues. I can always fix the program that returns it in JSON, but I think rsyslog can pre-fix the %msg%. Daniel M. Anson Linux Systems Engineer Rackspace danson at rackspace.com Office: (210)312-5114 Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse at rackspace.com, and delete the original message. Your cooperation is appreciated. From david at lang.hm Wed Mar 4 00:54:14 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 15:54:14 -0800 (PST) Subject: [rsyslog] filtering by message size Message-ID: is it possible to filter by message size? I'm looking at a situation where I would like to send the message via UDP if it's below a given size and by TCP if it's larger. David Lang From david at lang.hm Wed Mar 4 01:42:20 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 16:42:20 -0800 (PST) Subject: [rsyslog] Double quotes Problem In-Reply-To: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> References: <11210_1236121117_n23MwVl0017808_96AF20FDF4301D419B33CCE8E3A0132B0B844DB5@SAT4MX07.RACKSPACE.CORP> Message-ID: On Tue, 3 Mar 2009, Daniel Anson wrote: > Does anyone know of a quick and easy template to remove the double quote > character from a %msg% before it is inserted into the database (MySQL in > my case). I have a %msg% that looks like this: > > user pid=21214 uid=0 auid=4294967295 msg='PAM setcred: user="oracle" > exe="/bin/su" (hostname=?, addr=?, terminal=? result=Success)' > > I am reading the %msg% from the MySQL database and returning it in JSON > formatting. When it encounters a double-quote character, it causes > issues. I can always fix the program that returns it in JSON, but I > think rsyslog can pre-fix the %msg%. you will need to change the mySQL template in rsyslog I think you have two options. 1. you can put any valid SQL in the rsyslog config that does the insert, so write SQL that eliminates the quote 2. I think you can change the template to remove the quotes before sending it to MySQL (but this may end up removing quotes needed for MySQL to work) David Lang From rgerhards at hq.adiscon.com Wed Mar 4 07:13:27 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 07:13:27 +0100 Subject: [rsyslog] filtering by message size References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Oh, that's an interesting use case. It is not yet possible. I think we can implement (fairly simple) the size for a field (via the property replacer). However, that does not help you with the resulting size of a template string. I probably also need to check the supporting infrastructure for "greater than" comparisons... Would that help? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 12:54 AM > To: rsyslog-users > Subject: [rsyslog] filtering by message size > > is it possible to filter by message size? > > I'm looking at a situation where I would like to send the > message via UDP > if it's below a given size and by TCP if it's larger. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Wed Mar 4 08:06:03 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:06:03 -0800 (PST) Subject: [rsyslog] filtering by message size In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 4 Mar 2009, Rainer Gerhards wrote: > Oh, that's an interesting use case. It is not yet possible. I think we > can implement (fairly simple) the size for a field (via the property > replacer). However, that does not help you with the resulting size of a > template string. I probably also need to check the supporting > infrastructure for "greater than" comparisons... > > Would that help? yes, I can set the value to something conservative to account for the variable-length fields. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, March 04, 2009 12:54 AM >> To: rsyslog-users >> Subject: [rsyslog] filtering by message size >> >> is it possible to filter by message size? >> >> I'm looking at a situation where I would like to send the >> message via UDP >> if it's below a given size and by TCP if it's larger. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 4 08:10:56 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 08:10:56 +0100 Subject: [rsyslog] filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F30@GRFEXC.intern.adiscon.com> Let me see what I can do - it looks so trivial that I tend to think I have overlooked some subtlety ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:06 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > Oh, that's an interesting use case. It is not yet possible. > I think we > > can implement (fairly simple) the size for a field (via the property > > replacer). However, that does not help you with the > resulting size of a > > template string. I probably also need to check the supporting > > infrastructure for "greater than" comparisons... > > > > Would that help? > > yes, I can set the value to something conservative to account for the > variable-length fields. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >> Sent: Wednesday, March 04, 2009 12:54 AM > >> To: rsyslog-users > >> Subject: [rsyslog] filtering by message size > >> > >> is it possible to filter by message size? > >> > >> I'm looking at a situation where I would like to send the > >> message via UDP > >> if it's below a given size and by TCP if it's larger. > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Wed Mar 4 08:16:59 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:16:59 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com> <577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com> <577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com> <1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: Ok, here is a diff that works. it cycles the source IP address from 32000-42000 (since we are just sending, and not creating a normal socket this should not matter) it needs LIBS = /usr/lib/libnet.a in the Makefile in tools to use it create a template that puts the hostname-ip ahead of what you want to send, similar to $template TraditionalFwdFormat,"%fromhost-ip% <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" *.* @10.0.0.100;TraditionalFwdFormat the one problem right now is that any logs sent from the local box will go out with a source IP of 127.0.0.1 I wasted a bit of time trying to setup filters to use a different template if $myhostname == $fromhost, but apparently the filtering doesn't allow comparing two properties, and then I realized that you have a very high-performance name cache now, so you could easily replace my trivial inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); line with a call to the name lookup and then the %fromhost-ip% could be replaced by %fromhost% in the template and everything would work sanely (assuming forward and reverse name resolution are sane ;-) I haven't tried to do IPv6 yet, I know that it requires more effort to set the IP layer options, but I don't know exactly what yet. I wanted to float this first to see what you think before spending much more time on it. David Lang From rgerhards at hq.adiscon.com Wed Mar 4 08:14:00 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 08:14:00 +0100 Subject: [rsyslog] UDP source forging. References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com><1235670387.28865.2.camel@rf10up.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> David, Just a quick info: I'll initially create a separate branch for these changes, as I can not go through them in details right now. I'll keep that branch updated and the goal is to move it into the master branch as soon as possible. Thanks for all your hard work! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] UDP source forging. > > Ok, here is a diff that works. > > it cycles the source IP address from 32000-42000 (since we are just > sending, and not creating a normal socket this should not matter) > > it needs LIBS = /usr/lib/libnet.a in the Makefile in tools > > to use it create a template that puts the hostname-ip ahead > of what you > want to send, similar to > > $template TraditionalFwdFormat,"%fromhost-ip% > <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" > > *.* @10.0.0.100;TraditionalFwdFormat > > the one problem right now is that any logs sent from the > local box will go > out with a source IP of 127.0.0.1 > > I wasted a bit of time trying to setup filters to use a > different template > if $myhostname == $fromhost, but apparently the filtering > doesn't allow > comparing two properties, and then I realized that you have a very > high-performance name cache now, so you could easily replace > my trivial > inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); > line with a call to the name lookup and then the > %fromhost-ip% could be > replaced by %fromhost% in the template and everything would > work sanely > (assuming forward and reverse name resolution are sane ;-) > > I haven't tried to do IPv6 yet, I know that it requires more > effort to set > the IP layer options, but I don't know exactly what yet. > > I wanted to float this first to see what you think before > spending much > more time on it. > > David Lang > From david at lang.hm Wed Mar 4 08:32:16 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 3 Mar 2009 23:32:16 -0800 (PST) Subject: [rsyslog] UDP source forging. In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> References: <4255c2570902231929sebf2f0cqf9f840389e6d6a0b@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44FCBE@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44FCC2@grfint2.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E8E@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71E9A@GRFEXC.intern.adiscon.com><1235670387.28865.2.camel@rf10up.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F31@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 4 Mar 2009, Rainer Gerhards wrote: > David, > > Just a quick info: I'll initially create a separate branch for these > changes, as I can not go through them in details right now. I'll keep > that branch updated and the goal is to move it into the master branch as > soon as possible. Thanks for all your hard work! no problem, once you can comment on it I'll work on adding IPv6. one problem I will have at that point is that I don't have any systems that use it (and most of my systems don't even have it compiled into the kernel) one thing that would be very useful for people looking to create additional modules would be if there was a simple example module that did something, but didn't use all the callbacks and helper functions that you have created. trying to untangle those to figure out what's happening is pretty hard. the current imtemplate is close to what's needed, but it is just a little bit too trivial. it's not clear from that exactly where you would do things like opening sockets, initializing global variables, etc. I'm thinking that probably the most trivial example would be a stripped-down version of imudp and omfwd that just did the minimum needed to get the packets in and out. (possibly with one config option, just to show how it is done, but everything else hard-coded) Rainer doesn't need to be the person to do this, if there is someone else who understands the modules and has a little time it would sure help the rest of us. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Wednesday, March 04, 2009 8:17 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] UDP source forging. >> >> Ok, here is a diff that works. >> >> it cycles the source IP address from 32000-42000 (since we are just >> sending, and not creating a normal socket this should not matter) >> >> it needs LIBS = /usr/lib/libnet.a in the Makefile in tools >> >> to use it create a template that puts the hostname-ip ahead >> of what you >> want to send, similar to >> >> $template TraditionalFwdFormat,"%fromhost-ip% >> <%pri%>%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" >> >> *.* @10.0.0.100;TraditionalFwdFormat >> >> the one problem right now is that any logs sent from the >> local box will go >> out with a source IP of 127.0.0.1 >> >> I wasted a bit of time trying to setup filters to use a >> different template >> if $myhostname == $fromhost, but apparently the filtering >> doesn't allow >> comparing two properties, and then I realized that you have a very >> high-performance name cache now, so you could easily replace >> my trivial >> inet_pton(AF_INET, source_text_ip, &(source_ip.sin_addr)); >> line with a call to the name lookup and then the >> %fromhost-ip% could be >> replaced by %fromhost% in the template and everything would >> work sanely >> (assuming forward and reverse name resolution are sane ;-) >> >> I haven't tried to do IPv6 yet, I know that it requires more >> effort to set >> the IP layer options, but I don't know exactly what yet. >> >> I wanted to float this first to see what you think before >> spending much >> more time on it. >> >> David Lang >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 4 09:49:48 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 09:49:48 +0100 Subject: [rsyslog] Packages in tarball - was: RE: Get rsyslog to alwaysuse fqdn of sending devices? References: <4255c2570902171211u26bc267brd13cdfb01728df70@mail.gmail.com><4255c2570902260753u53ab4c46le86afe27437d2ed9@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71E99@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71EDC@GRFEXC.intern.adiscon.com> <4255c2570903030715x55403ed9k59253789295ffcba@mail.gmail.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F32@GRFEXC.intern.adiscon.com> RB, Not addressing all the meat of your message (I can't...), I'd like to spell out that if you have something that should go into the tarball, just mail me and I'll see it gets in. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of RB > Sent: Tuesday, March 03, 2009 4:15 PM > To: rsyslog-users > Subject: Re: [rsyslog] Packages in tarball - was: RE: Get rsyslog to > alwaysuse fqdn of sending devices? > > On Mon, Mar 2, 2009 at 00:06, Rainer Gerhards > wrote: > > I have a pragmatic suggestion: if you have package specific files, > you > > can send them to me. I will create a subdirectory for them. There > will > > be a README telling people that this stuff is (from my POV) > > unmaintained, probably outdated and to be used with care. If a > > maintainer (like Michael) later decides it was a bad idea to put the > > files into the tarball, I'll also happily delete them. > > > > Does this sound like a workable compromise? > > It does, but I'm not sure how it will mesh with wanting to provide > packages for other distros that aren't so responsive as Debian or > up-to-date as Fedora. I'll be happy to provide an RPM specfile for > -stable and -dev (since Fedora already does a -beta package) but that > may not be sufficient for the general clicky-package group. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From pieter.thysebaert at intec.ugent.be Wed Mar 4 10:06:48 2009 From: pieter.thysebaert at intec.ugent.be (pieter.thysebaert at intec.ugent.be) Date: Wed, 4 Mar 2009 10:06:48 +0100 (CET) Subject: [rsyslog] (no subject) Message-ID: Hello Rsyslog users, I have been trying to get Rsyslogd up and running on HP-UX 11.31 ia64. For what it's worth: my preliminary results can be found on http://wiki.rsyslog.com/index.php/HP-UX Best regards, Pieter From rgerhards at hq.adiscon.com Wed Mar 4 11:53:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 11:53:40 +0100 Subject: [rsyslog] rsyslog on HP-UX References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F33@GRFEXC.intern.adiscon.com> Hi Pieter, thanks for your effort. Some time ago, I did an initial port on HP-UX via their web offering. As far as I remember, it compiled well at that time. However, I do not know what has changed in the mean time and how it "feels" now on that platform. I'll see that I integrate your patch ASAP (but that may take a while). The important thing is that I cannot integrate it as-is but need to make sure it does not break the other platforms. I would appreciate if you could check out interim versions when I have them available. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of > pieter.thysebaert at intec.ugent.be > Sent: Wednesday, March 04, 2009 10:07 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] (no subject) > > Hello Rsyslog users, > > I have been trying to get Rsyslogd up and running on HP-UX 11.31 ia64. > > For what it's worth: my preliminary results can be found on > http://wiki.rsyslog.com/index.php/HP-UX > > Best regards, > Pieter > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From janisg at latnetdc.lv Wed Mar 4 16:20:21 2009 From: janisg at latnetdc.lv (Janis) Date: Wed, 04 Mar 2009 17:20:21 +0200 Subject: [rsyslog] Right regex format for property based filters Message-ID: <49AE9C35.4050605@latnetdc.lv> Hello list. I have a question regarding to rsyslog configuration. What is the correct syntax of property based filter with regex. I'm using this configuration right now, and would like to create date based logfiles for each host - hostA, hostB, hostC. But it doesn't work this way. $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile And when running rsyslog with -d, I got only false matches on this regex. I seems that it tries to match all the text inside quotes instead of regexp. As I have red in man page, and html docs, then regexp should be in POSIX RE format (tryed also everything enclosed in braces). For example, if I change regex like this: :HOSTNAME, regex, "host" -?TplFile Then it works and matches all the hosts (A,B,C), and creates the files for each (well it's the same as using contains). But that doesn't solve the problem, when there isn't equal start prefixes for all hosts. For example if I want to match hosts - dog,cat,cow. Best regards --janis From rgerhards at hq.adiscon.com Wed Mar 4 16:35:59 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 16:35:59 +0100 Subject: [rsyslog] Right regex format for property based filters References: <49AE9C35.4050605@latnetdc.lv> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F35@GRFEXC.intern.adiscon.com> Hi Janis, the regex is Posix BRE, nor ERE. I think the syntax you use is not supported in BRE (as a side-note, this reminds me that I wanted to check what it takes to upgrade them to use ERE, too). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Janis > Sent: Wednesday, March 04, 2009 4:20 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Right regex format for property based filters > > Hello list. > > I have a question regarding to rsyslog configuration. What is the > correct syntax of property based > filter with regex. > > I'm using this configuration right now, and would like to create date > based logfiles for each host - hostA, hostB, hostC. > But it doesn't work this way. > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%- > %$DAY%.log" > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > And when running rsyslog with -d, I got only false matches on this > regex. I seems that it tries to match all the text > inside quotes instead of regexp. As I have red in man page, and html > docs, then regexp should be in POSIX RE format > (tryed also everything enclosed in braces). For example, if I change > regex like this: > > :HOSTNAME, regex, "host" -?TplFile > > Then it works and matches all the hosts (A,B,C), and creates the files > for each (well it's the same as using contains). > But that doesn't solve the problem, when there isn't equal start > prefixes for all hosts. > For example if I want to match hosts - dog,cat,cow. > > Best regards > --janis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:03:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 4 Mar 2009 18:03:17 +0100 Subject: [rsyslog] Right regex format for property based filters References: <49AE9C35.4050605@latnetdc.lv> <9B6E2A8877C38245BFB15CC491A11DA71F35@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F37@GRFEXC.intern.adiscon.com> ERE looks trivial - just seeing if I get it in... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Wednesday, March 04, 2009 4:36 PM > To: janisg at latnetdc.lv; rsyslog-users > Subject: Re: [rsyslog] Right regex format for property based filters > > Hi Janis, > > the regex is Posix BRE, nor ERE. I think the syntax you use is not > supported in BRE (as a side-note, this reminds me that I wanted to > check > what it takes to upgrade them to use ERE, too). > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Janis > > Sent: Wednesday, March 04, 2009 4:20 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] Right regex format for property based filters > > > > Hello list. > > > > I have a question regarding to rsyslog configuration. What is the > > correct syntax of property based > > filter with regex. > > > > I'm using this configuration right now, and would like to create date > > based logfiles for each host - hostA, hostB, hostC. > > But it doesn't work this way. > > > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%- > > %$DAY%.log" > > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > > > And when running rsyslog with -d, I got only false matches on this > > regex. I seems that it tries to match all the text > > inside quotes instead of regexp. As I have red in man page, and html > > docs, then regexp should be in POSIX RE format > > (tryed also everything enclosed in braces). For example, if I change > > regex like this: > > > > :HOSTNAME, regex, "host" -?TplFile > > > > Then it works and matches all the hosts (A,B,C), and creates the > files > > for each (well it's the same as using contains). > > But that doesn't solve the problem, when there isn't equal start > > prefixes for all hosts. > > For example if I want to match hosts - dog,cat,cow. > > > > Best regards > > --janis > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:38:44 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 04 Mar 2009 18:38:44 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <49AE9C35.4050605@latnetdc.lv> References: <49AE9C35.4050605@latnetdc.lv> Message-ID: <1236188324.27835.2.camel@rf10up.intern.adiscon.com> Janis, I have added ERE filter support to the devel branch and your use case described below now works - you just need to use "ereregexp" instead of "regexp". No release tarball yet, the patch is here: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952 I did some quick checks, but would appreciate if some others try it out. Rainer On Wed, 2009-03-04 at 17:20 +0200, Janis wrote: > Hello list. > > I have a question regarding to rsyslog configuration. What is the > correct syntax of property based > filter with regex. > > I'm using this configuration right now, and would like to create date > based logfiles for each host - hostA, hostB, hostC. > But it doesn't work this way. > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > And when running rsyslog with -d, I got only false matches on this > regex. I seems that it tries to match all the text > inside quotes instead of regexp. As I have red in man page, and html > docs, then regexp should be in POSIX RE format > (tryed also everything enclosed in braces). For example, if I change > regex like this: > > :HOSTNAME, regex, "host" -?TplFile > > Then it works and matches all the hosts (A,B,C), and creates the files > for each (well it's the same as using contains). > But that doesn't solve the problem, when there isn't equal start > prefixes for all hosts. > For example if I want to match hosts - dog,cat,cow. > > Best regards > --janis > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 4 18:56:05 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 04 Mar 2009 18:56:05 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <1236188324.27835.2.camel@rf10up.intern.adiscon.com> References: <49AE9C35.4050605@latnetdc.lv> <1236188324.27835.2.camel@rf10up.intern.adiscon.com> Message-ID: <1236189365.27835.19.camel@rf10up.intern.adiscon.com> All, I introduced a memory leak with the ERE enhancement. It is fixed now. So be sure to apply all patches after the one I mentioned. For your convenience, I created a temporary tarball based on the fixed version. It is available at http://download.rsyslog.com/rsyslog/tmp.tar.gz The tarball claims to contain 4.1.4, but you should not count on that it is equal to the released version. I will *not* care any more about this tarball. But I think it is useful to have a version right at hand. Also, this doesn't require any autotools tricks ;) Rainer On Wed, 2009-03-04 at 18:38 +0100, Rainer Gerhards wrote: > Janis, > > I have added ERE filter support to the devel branch and your use case > described below now works - you just need to use "ereregexp" instead of > "regexp". No release tarball yet, the patch is here: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=5005bce38763051b5b12e48ac60c3ff17097a952 > > I did some quick checks, but would appreciate if some others try it out. > > Rainer > > On Wed, 2009-03-04 at 17:20 +0200, Janis wrote: > > Hello list. > > > > I have a question regarding to rsyslog configuration. What is the > > correct syntax of property based > > filter with regex. > > > > I'm using this configuration right now, and would like to create date > > based logfiles for each host - hostA, hostB, hostC. > > But it doesn't work this way. > > > > $template TplFile,"/var/log/hosts/%HOSTNAME%-%$YEAR%-%$MONTH%-%$DAY%.log" > > :HOSTNAME, regex, "hostA|hostB|hostC" -?TplFile > > > > And when running rsyslog with -d, I got only false matches on this > > regex. I seems that it tries to match all the text > > inside quotes instead of regexp. As I have red in man page, and html > > docs, then regexp should be in POSIX RE format > > (tryed also everything enclosed in braces). For example, if I change > > regex like this: > > > > :HOSTNAME, regex, "host" -?TplFile > > > > Then it works and matches all the hosts (A,B,C), and creates the files > > for each (well it's the same as using contains). > > But that doesn't solve the problem, when there isn't equal start > > prefixes for all hosts. > > For example if I want to match hosts - dog,cat,cow. > > > > Best regards > > --janis > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Wed Mar 4 20:39:56 2009 From: mbiebl at gmail.com (Michael Biebl) Date: Wed, 4 Mar 2009 20:39:56 +0100 Subject: [rsyslog] Right regex format for property based filters In-Reply-To: <1236189365.27835.19.camel@rf10up.intern.adiscon.com> References: <49AE9C35.4050605@latnetdc.lv> <1236188324.27835.2.camel@rf10up.intern.adiscon.com> <1236189365.27835.19.camel@rf10up.intern.adiscon.com> Message-ID: 2009/3/4 Rainer Gerhards : > All, > > I introduced a memory leak with the ERE enhancement. It is fixed now. So > be sure to apply all patches after the one I mentioned. > > For your convenience, I created a temporary tarball based on the fixed > version. It is available at > > http://download.rsyslog.com/rsyslog/tmp.tar.gz > > The tarball claims to contain 4.1.4, but you should not count on that it > is equal to the released version. I will *not* care any more about this > tarball. But I think it is useful to have a version right at hand. Also, > this doesn't require any autotools tricks ;) Rainer, gitweb has the nice snapshot feature, which allows to download a tarball for a given SHA1 [1] It doesn't contain the build system, so requires a "autoreconf -vfi" run, but otherwise it should work just fine. Cheers, Michael [1] http://git.adiscon.com/?p=rsyslog.git;a=snapshot;h=42db7de5968d2db0fa855a9f029f6bccc0a30650;sf=tgz -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rgerhards at hq.adiscon.com Thu Mar 5 18:52:34 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 5 Mar 2009 18:52:34 +0100 Subject: [rsyslog] rsyslog on Solaris Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Hi all, I have spent some time integrating the Solaris patches the past days (actually, learning [installing] Solaris took the most time). Now I have an environment and the compile process works rather well. However, there seem to be some issue with building the archives. I have to admit I am a bit clueless. After my sig is a build log of the affected part. I would appreciate if someone could provide some hints. Thanks, Rainer Making all in runtime make[2]: Entering directory `/root/rsyslog/runtime' /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g -o librsyslog.la librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la-linkedlist.lo librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo librsyslog_la-datetime.lo librsyslog_la-srutils.lo librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo librsyslog_la-queue.lo librsyslog_la-cfsysline.lo librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la-parse.lo librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o .libs/librsyslog_la-template.o make[2]: *** [librsyslog.la] Error 1 make[2]: Leaving directory `/root/rsyslog/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog' make: *** [all] Error 2 From epiphani at gmail.com Thu Mar 5 19:01:14 2009 From: epiphani at gmail.com (Aaron Wiebe) Date: Thu, 5 Mar 2009 13:01:14 -0500 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Message-ID: Hey Rainer, On Thu, Mar 5, 2009 at 12:52 PM, Rainer Gerhards wrote: > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > make[2]: *** [librsyslog.la] Error 1 > make[2]: Leaving directory `/root/rsyslog/runtime' First guess, what is that 'false' doing there? That would make the command return nonzero to make, hence the error code. -Aaron From david at ecker-software.de Thu Mar 5 19:33:11 2009 From: david at ecker-software.de (David Ecker) Date: Thu, 05 Mar 2009 19:33:11 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> Message-ID: <49B01AE7.8080406@ecker-software.de> Hi, found the following in a another forum: the problem is resolved. the $PATH didn't include the /usr/ccs/bin, so the configure script couldn't find the ar. just a guess, see http://www.fantasticunix.com/forum/general-solaris-discussion/212026-mono-solaris-8-a.html bye David Ecker Rainer Gerhards schrieb: > Hi all, > > I have spent some time integrating the Solaris patches the past days > (actually, learning [installing] Solaris took the most time). > > Now I have an environment and the compile process works rather well. > However, there seem to be some issue with building the archives. I have > to admit I am a bit clueless. After my sig is a build log of the > affected part. > > I would appreciate if someone could provide some hints. > > Thanks, > Rainer > > Making all in runtime > make[2]: Entering directory `/root/rsyslog/runtime' > /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > -Wmissing-format-attribute -g -o librsyslog.la > librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo > librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la-linkedlist.lo > librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo > librsyslog_la-datetime.lo librsyslog_la-srutils.lo > librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo > librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo > librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo > librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo > librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo > librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo > librsyslog_la-queue.lo librsyslog_la-cfsysline.lo > librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la-parse.lo > librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o > .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o > .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o > .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o > .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o > .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o > .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o > .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o > .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o > .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o > .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o > .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o > .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o > .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o > .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o > .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o > .libs/librsyslog_la-template.o > make[2]: *** [librsyslog.la] Error 1 > make[2]: Leaving directory `/root/rsyslog/runtime' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/rsyslog' > make: *** [all] Error 2 > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From thomas.mieslinger at 1und1.de Fri Mar 6 10:17:48 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 10:17:48 +0100 Subject: [rsyslog] wrong permissons on directories Message-ID: <49B0EA3C.1060104@1und1.de> Hi *, when creating directories through dynamic templates, the directory permissons are incomplete: rsyslog.conf: $template ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%-%$MONTH%-%$DAY%.log" resulting directories: ls -al /data/log drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ ls -al /data/log/zeusmw drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ # rsyslogd -version rsyslogd 3.21.3, compiled with: FEATURE_REGEXP: Yes FEATURE_LARGEFILE: Yes FEATURE_NETZIP (message compression): Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No Runtime Instrumentation (slow code): No (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) I'd be happy to know if thats a bug. Thanks Thomas From thomas.mieslinger at 1und1.de Fri Mar 6 10:22:54 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 10:22:54 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <49B01AE7.8080406@ecker-software.de> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> <49B01AE7.8080406@ecker-software.de> Message-ID: <49B0EB6E.1050209@1und1.de> Hi, is that code modified for Solaris already available in git? Could you please send me a pointer to a checkout location? Thanks Thomas From rgerhards at hq.adiscon.com Fri Mar 6 11:41:24 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 11:41:24 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> Yes, it is part of the regular git tree: http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris $ git clone git://git.adiscon.com/git/rsyslog.git then checkout the "solaris" branch: $ git checkout --track -b solaris origin/solaris Rainer PS: commands may be wrong ;) > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 10:23 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Hi, > > is that code modified for Solaris already available in git? Could you > please send me a pointer to a checkout location? > > Thanks > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 12:17:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 12:17:40 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> Hi Thomas, can it be that your default umask gets into your way? In any case, you can set the permissions explicitely with $FileCreateMode $FileGroup $FileOwner And set the umask with $umask (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) Does this help? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 10:18 AM > To: rsyslog-users > Subject: [rsyslog] wrong permissons on directories > > Hi *, > > when creating directories through dynamic templates, the directory > permissons are incomplete: > > rsyslog.conf: > $template > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%- > %$MONTH%-%$DAY%.log" > > resulting directories: > ls -al /data/log > drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ > > ls -al /data/log/zeusmw > drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ > > # rsyslogd -version > rsyslogd 3.21.3, compiled with: > FEATURE_REGEXP: Yes > FEATURE_LARGEFILE: Yes > FEATURE_NETZIP (message compression): Yes > GSSAPI Kerberos 5 support: Yes > FEATURE_DEBUG (debug build, slow code): No > Runtime Instrumentation (slow code): No > > (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > > I'd be happy to know if thats a bug. > > Thanks > Thomas > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 14:07:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 14:07:06 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com> <49B01AE7.8080406@ecker-software.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F60@GRFEXC.intern.adiscon.com> Thanks to you and Aaron, It was a combination of ar not being present plus autoconfig then using false... So that was a purely environment-base thing. Now I am one step further and the next issue is a pthreads linker error message ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of David Ecker > Sent: Thursday, March 05, 2009 7:33 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Hi, > > found the following in a another forum: > > > the problem is resolved. the $PATH didn't include the /usr/ccs/bin, so > the configure script couldn't find the ar. > > > just a guess, see > http://www.fantasticunix.com/forum/general-solaris-discussion/212026- > mono-solaris-8-a.html > > bye > David Ecker > > Rainer Gerhards schrieb: > > Hi all, > > > > I have spent some time integrating the Solaris patches the past days > > (actually, learning [installing] Solaris took the most time). > > > > Now I have an environment and the compile process works rather well. > > However, there seem to be some issue with building the archives. I > have > > to admit I am a bit clueless. After my sig is a build log of the > > affected part. > > > > I would appreciate if someone could provide some hints. > > > > Thanks, > > Rainer > > > > Making all in runtime > > make[2]: Entering directory `/root/rsyslog/runtime' > > /bin/bash ../libtool --tag=CC --mode=link gcc -g -O2 -W -Wall > > -Wformat-security -Wshadow -Wcast-align -Wpointer-arith > > -Wmissing-format-attribute -g -o librsyslog.la > > librsyslog_la-rsyslog.lo librsyslog_la-glbl.lo librsyslog_la-conf.lo > > librsyslog_la-parser.lo librsyslog_la-msg.lo librsyslog_la- > linkedlist.lo > > librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo > > librsyslog_la-datetime.lo librsyslog_la-srutils.lo > > librsyslog_la-errmsg.lo librsyslog_la-debug.lo librsyslog_la-obj.lo > > librsyslog_la-modules.lo librsyslog_la-sync.lo librsyslog_la-expr.lo > > librsyslog_la-ctok.lo librsyslog_la-ctok_token.lo > > librsyslog_la-stream.lo librsyslog_la-var.lo librsyslog_la-wtp.lo > > librsyslog_la-wti.lo librsyslog_la-sysvar.lo librsyslog_la-vm.lo > > librsyslog_la-vmstk.lo librsyslog_la-vmprg.lo librsyslog_la-vmop.lo > > librsyslog_la-queue.lo librsyslog_la-cfsysline.lo > > librsyslog_la-action.lo librsyslog_la-threads.lo librsyslog_la- > parse.lo > > librsyslog_la-outchannel.lo librsyslog_la-template.lo -lrt > > false cru .libs/librsyslog.a .libs/librsyslog_la-rsyslog.o > > .libs/librsyslog_la-glbl.o .libs/librsyslog_la-conf.o > > .libs/librsyslog_la-parser.o .libs/librsyslog_la-msg.o > > .libs/librsyslog_la-linkedlist.o .libs/librsyslog_la-objomsr.o > > .libs/librsyslog_la-stringbuf.o .libs/librsyslog_la-datetime.o > > .libs/librsyslog_la-srutils.o .libs/librsyslog_la-errmsg.o > > .libs/librsyslog_la-debug.o .libs/librsyslog_la-obj.o > > .libs/librsyslog_la-modules.o .libs/librsyslog_la-sync.o > > .libs/librsyslog_la-expr.o .libs/librsyslog_la-ctok.o > > .libs/librsyslog_la-ctok_token.o .libs/librsyslog_la-stream.o > > .libs/librsyslog_la-var.o .libs/librsyslog_la-wtp.o > > .libs/librsyslog_la-wti.o .libs/librsyslog_la-sysvar.o > > .libs/librsyslog_la-vm.o .libs/librsyslog_la-vmstk.o > > .libs/librsyslog_la-vmprg.o .libs/librsyslog_la-vmop.o > > .libs/librsyslog_la-queue.o .libs/librsyslog_la-cfsysline.o > > .libs/librsyslog_la-action.o .libs/librsyslog_la-threads.o > > .libs/librsyslog_la-parse.o .libs/librsyslog_la-outchannel.o > > .libs/librsyslog_la-template.o > > make[2]: *** [librsyslog.la] Error 1 > > make[2]: Leaving directory `/root/rsyslog/runtime' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/root/rsyslog' > > make: *** [all] Error 2 > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > From thomas.mieslinger at 1und1.de Fri Mar 6 14:37:49 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 14:37:49 +0100 Subject: [rsyslog] rsyslog on Solaris In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> Message-ID: <49B1272D.4010408@1und1.de> Rainer Gerhards wrote: > Yes, it is part of the regular git tree: > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > $ git clone git://git.adiscon.com/git/rsyslog.git > then checkout the "solaris" branch: > $ git checkout --track -b solaris origin/solaris That worked. Thanks. What is the minimal required autoconf/automake Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU automake) 1.10.1 which came which opensolaris. it complains about undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? Thanks Thomas From rgerhards at hq.adiscon.com Fri Mar 6 14:39:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 14:39:06 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com> <49B1272D.4010408@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F62@GRFEXC.intern.adiscon.com> I am using 2.63 on Solaris 10 x64 and I just successfully compiled. I am about to write a few notes about the state of solaris development in a few moments. My twitter feed may also be useful for you: http://twitter.com/rgerhards My environment is described on http://wiki.rsyslog.com/index.php/Solaris Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 2:38 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > Rainer Gerhards wrote: > > Yes, it is part of the regular git tree: > > > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > > $ git clone git://git.adiscon.com/git/rsyslog.git > > then checkout the "solaris" branch: > > $ git checkout --track -b solaris origin/solaris > > That worked. Thanks. What is the minimal required autoconf/automake > Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU > automake) 1.10.1 which came which opensolaris. it complains about > undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? > > Thanks Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Fri Mar 6 15:13:55 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 15:13:55 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> Message-ID: <49B12FA3.2030202@1und1.de> Thanks for the pointer to the documentation.. it is $DirCreateMode what I asked for... and now I ask for a change of the default documentation says: Default: 0644 Reality demands 0755. I changed it in my configuration. I'd be happy to see that changed in rsyslog. Thomas Rainer Gerhards wrote: > Hi Thomas, > > can it be that your default umask gets into your way? In any case, you > can set the permissions explicitely with > > $FileCreateMode > $FileGroup > $FileOwner > > And set the umask with > > $umask > > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > > Does this help? > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> Sent: Friday, March 06, 2009 10:18 AM >> To: rsyslog-users >> Subject: [rsyslog] wrong permissons on directories >> >> Hi *, >> >> when creating directories through dynamic templates, the directory >> permissons are incomplete: >> >> rsyslog.conf: >> $template >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all-%$YEAR%- >> %$MONTH%-%$DAY%.log" >> >> resulting directories: >> ls -al /data/log >> drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ >> >> ls -al /data/log/zeusmw >> drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ >> >> # rsyslogd -version >> rsyslogd 3.21.3, compiled with: >> FEATURE_REGEXP: Yes >> FEATURE_LARGEFILE: Yes >> FEATURE_NETZIP (message compression): Yes >> GSSAPI Kerberos 5 support: Yes >> FEATURE_DEBUG (debug build, slow code): No >> Runtime Instrumentation (slow code): No >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) >> >> I'd be happy to know if thats a bug. >> >> Thanks >> Thomas >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- Thomas Mieslinger IT Infrastructure Systems Telefon: +49-721-91374-4404 E-Mail: thomas.mieslinger at 1und1.de 1&1 Internet AG Brauerstra?e 48 76135 Karlsruhe Amtsgericht Montabaur HRB 6484 Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver Mauss, Jan Oetjen Aufsichtsratsvorsitzender: Michael Scheeren From rgerhards at hq.adiscon.com Fri Mar 6 15:19:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 15:19:06 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Thomas, do I correctly understand that you propose the default be changed? If so, I am hesitant to do that - wouldn't that potentially break existing deployments? On the other hand... how could that work... Umm... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 3:14 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > Thanks for the pointer to the documentation.. it is $DirCreateMode what > I asked for... > > and now I ask for a change of the default > documentation says: > Default: 0644 > > Reality demands 0755. I changed it in my configuration. I'd be happy to > see that changed in rsyslog. > > Thomas > > > > Rainer Gerhards wrote: > > Hi Thomas, > > > > can it be that your default umask gets into your way? In any case, > you > > can set the permissions explicitely with > > > > $FileCreateMode > > $FileGroup > > $FileOwner > > > > And set the umask with > > > > $umask > > > > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > > > > Does this help? > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> Sent: Friday, March 06, 2009 10:18 AM > >> To: rsyslog-users > >> Subject: [rsyslog] wrong permissons on directories > >> > >> Hi *, > >> > >> when creating directories through dynamic templates, the directory > >> permissons are incomplete: > >> > >> rsyslog.conf: > >> $template > >> > > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- > %$YEAR%- > >> %$MONTH%-%$DAY%.log" > >> > >> resulting directories: > >> ls -al /data/log > >> drw-r--r-- 3 root root 4096 Mar 5 15:53 zeusmw/ > >> > >> ls -al /data/log/zeusmw > >> drw-r--r-- 2 root root 4096 Mar 6 10:11 2009-03/ > >> > >> # rsyslogd -version > >> rsyslogd 3.21.3, compiled with: > >> FEATURE_REGEXP: Yes > >> FEATURE_LARGEFILE: Yes > >> FEATURE_NETZIP (message compression): Yes > >> GSSAPI Kerberos 5 support: Yes > >> FEATURE_DEBUG (debug build, slow code): No > >> Runtime Instrumentation (slow code): No > >> > >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > >> > >> I'd be happy to know if thats a bug. > >> > >> Thanks > >> Thomas > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > -- > Thomas Mieslinger > IT Infrastructure Systems > Telefon: +49-721-91374-4404 > E-Mail: thomas.mieslinger at 1und1.de > > 1&1 Internet AG > Brauerstra?e 48 > 76135 Karlsruhe > > Amtsgericht Montabaur HRB 6484 > Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas > Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver > Mauss, Jan Oetjen > Aufsichtsratsvorsitzender: Michael Scheeren > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From mbiebl at gmail.com Fri Mar 6 15:54:24 2009 From: mbiebl at gmail.com (Michael Biebl) Date: Fri, 6 Mar 2009 15:54:24 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: FWIW, the Debian default rsyslog.conf ships with $DirCreateMode 0755 2009/3/6 Rainer Gerhards : > Thomas, > > do I correctly understand that you propose the default be changed? > > If so, I am hesitant to do that - wouldn't that potentially break existing deployments? On the other hand... how could that work... Umm... > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> Sent: Friday, March 06, 2009 3:14 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] wrong permissons on directories >> >> Thanks for the pointer to the documentation.. it is $DirCreateMode what >> I asked for... >> >> and now I ask for a change of the default >> documentation says: >> Default: 0644 >> >> Reality demands 0755. I changed it in my configuration. I'd be happy to >> see that changed in rsyslog. >> >> Thomas >> >> >> >> Rainer Gerhards wrote: >> > Hi Thomas, >> > >> > can it be that your default umask gets into your way? In any case, >> you >> > can set the permissions explicitely with >> > >> > $FileCreateMode >> > $FileGroup >> > $FileOwner >> > >> > And set the umask with >> > >> > $umask >> > >> > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) >> > >> > Does this help? >> > >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger >> >> Sent: Friday, March 06, 2009 10:18 AM >> >> To: rsyslog-users >> >> Subject: [rsyslog] wrong permissons on directories >> >> >> >> Hi *, >> >> >> >> when creating directories through dynamic templates, the directory >> >> permissons are incomplete: >> >> >> >> rsyslog.conf: >> >> $template >> >> >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- >> %$YEAR%- >> >> %$MONTH%-%$DAY%.log" >> >> >> >> resulting directories: >> >> ls -al /data/log >> >> drw-r--r-- 3 root root 4096 Mar ?5 15:53 zeusmw/ >> >> >> >> ls -al /data/log/zeusmw >> >> drw-r--r-- 2 root root 4096 Mar ?6 10:11 2009-03/ >> >> >> >> # rsyslogd -version >> >> rsyslogd 3.21.3, compiled with: >> >> ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >> ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?Yes >> >> ? ?FEATURE_NETZIP (message compression): ? Yes >> >> ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >> ? ?FEATURE_DEBUG (debug build, slow code): No >> >> ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) >> >> >> >> I'd be happy to know if thats a bug. >> >> >> >> Thanks >> >> Thomas >> >> >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> >> -- >> Thomas Mieslinger >> IT Infrastructure Systems >> Telefon: +49-721-91374-4404 >> E-Mail: thomas.mieslinger at 1und1.de >> >> 1&1 Internet AG >> Brauerstra?e 48 >> 76135 Karlsruhe >> >> Amtsgericht Montabaur HRB 6484 >> Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas >> Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver >> Mauss, Jan Oetjen >> Aufsichtsratsvorsitzender: Michael Scheeren >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From thomas.mieslinger at 1und1.de Fri Mar 6 16:17:30 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Fri, 06 Mar 2009 16:17:30 +0100 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: <49B13E8A.2080308@1und1.de> I guess nobody did let rsyslog make directories. Rainer Gerhards wrote: > Thomas, > > do I correctly understand that you propose the default be changed? Yepp. > If so, I am hesitant to do that - wouldn't that potentially break existing deployments? hmm Maybe I haven't seen enough yet, but I can't imagine a deployment built on directory permissions 644.... > On the other hand... how could that work... Umm... They are all working as root out there :-) I think it would be good if you just double check it yourself that the directories get created with 644 and decicde on your findings. Thomas From rgerhards at hq.adiscon.com Fri Mar 6 16:40:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 16:40:12 +0100 Subject: [rsyslog] rsyslog on Solaris References: <9B6E2A8877C38245BFB15CC491A11DA71F51@GRFEXC.intern.adiscon.com><49B01AE7.8080406@ecker-software.de> <49B0EB6E.1050209@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5C@GRFEXC.intern.adiscon.com><49B1272D.4010408@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F62@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F67@GRFEXC.intern.adiscon.com> I have just finished my "current state" writeup on rsyslog and solaris: http://blog.gerhards.net/2009/03/rsyslog-and-solaris.html I guess it contains some useful information ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Friday, March 06, 2009 2:39 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog on Solaris > > I am using 2.63 on Solaris 10 x64 and I just successfully compiled. I > am > about to write a few notes about the state of solaris development in a > few moments. My twitter feed may also be useful for you: > > http://twitter.com/rgerhards > > My environment is described on > > http://wiki.rsyslog.com/index.php/Solaris > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > > Sent: Friday, March 06, 2009 2:38 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] rsyslog on Solaris > > > > Rainer Gerhards wrote: > > > Yes, it is part of the regular git tree: > > > > > http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/solaris > > > $ git clone git://git.adiscon.com/git/rsyslog.git > > > then checkout the "solaris" branch: > > > $ git checkout --track -b solaris origin/solaris > > > > That worked. Thanks. What is the minimal required autoconf/automake > > Verison? I'm using autoconf (GNU Autoconf) 2.63 and automake (GNU > > automake) 1.10.1 which came which opensolaris. it complains about > > undefined macros like AM_INIT_AUTOMAKE. Which Versions are you using? > > > > Thanks Thomas > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 16:40:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 16:40:12 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com><49B12FA3.2030202@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> The more I think about it, the more it smells like a real bug. Has anyone objections changing the default? Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Biebl > Sent: Friday, March 06, 2009 3:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > FWIW, the Debian default rsyslog.conf ships with > > $DirCreateMode 0755 > > > 2009/3/6 Rainer Gerhards : > > Thomas, > > > > do I correctly understand that you propose the default be changed? > > > > If so, I am hesitant to do that - wouldn't that potentially break > existing deployments? On the other hand... how could that work... > Umm... > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> Sent: Friday, March 06, 2009 3:14 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] wrong permissons on directories > >> > >> Thanks for the pointer to the documentation.. it is $DirCreateMode > what > >> I asked for... > >> > >> and now I ask for a change of the default > >> documentation says: > >> Default: 0644 > >> > >> Reality demands 0755. I changed it in my configuration. I'd be happy > to > >> see that changed in rsyslog. > >> > >> Thomas > >> > >> > >> > >> Rainer Gerhards wrote: > >> > Hi Thomas, > >> > > >> > can it be that your default umask gets into your way? In any case, > >> you > >> > can set the permissions explicitely with > >> > > >> > $FileCreateMode > >> > $FileGroup > >> > $FileOwner > >> > > >> > And set the umask with > >> > > >> > $umask > >> > > >> > (see http://www.rsyslog.com/doc-rsyslog_conf_global.html) > >> > > >> > Does this help? > >> > > >> > Rainer > >> > > >> >> -----Original Message----- > >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> >> bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > >> >> Sent: Friday, March 06, 2009 10:18 AM > >> >> To: rsyslog-users > >> >> Subject: [rsyslog] wrong permissons on directories > >> >> > >> >> Hi *, > >> >> > >> >> when creating directories through dynamic templates, the > directory > >> >> permissons are incomplete: > >> >> > >> >> rsyslog.conf: > >> >> $template > >> >> > >> > ZeusMwAllLogFileService,"/data/log/zeusmw/%$YEAR%-%$MONTH%/all- > >> %$YEAR%- > >> >> %$MONTH%-%$DAY%.log" > >> >> > >> >> resulting directories: > >> >> ls -al /data/log > >> >> drw-r--r-- 3 root root 4096 Mar ?5 15:53 zeusmw/ > >> >> > >> >> ls -al /data/log/zeusmw > >> >> drw-r--r-- 2 root root 4096 Mar ?6 10:11 2009-03/ > >> >> > >> >> # rsyslogd -version > >> >> rsyslogd 3.21.3, compiled with: > >> >> ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >> >> ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?Yes > >> >> ? ?FEATURE_NETZIP (message compression): ? Yes > >> >> ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >> >> ? ?FEATURE_DEBUG (debug build, slow code): No > >> >> ? ?Runtime Instrumentation (slow code): ? ?No > >> >> > >> >> (its the rsyslog-3.21.3-4 fedora 10 package compiled on rhel5) > >> >> > >> >> I'd be happy to know if thats a bug. > >> >> > >> >> Thanks > >> >> Thomas > >> >> > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > >> -- > >> Thomas Mieslinger > >> IT Infrastructure Systems > >> Telefon: +49-721-91374-4404 > >> E-Mail: thomas.mieslinger at 1und1.de > >> > >> 1&1 Internet AG > >> Brauerstra?e 48 > >> 76135 Karlsruhe > >> > >> Amtsgericht Montabaur HRB 6484 > >> Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas > >> Gottschlich, Robert Hoffmann, Markus Huhn, Henning Kettler, Oliver > >> Mauss, Jan Oetjen > >> Aufsichtsratsvorsitzender: Michael Scheeren > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Fri Mar 6 17:09:05 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 17:09:05 +0100 Subject: [rsyslog] Intro presentation Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F6C@GRFEXC.intern.adiscon.com> Hi all, I think about doing an online intro presentation to rsyslog that should be useful to new users, in addition to the doc. One may claim that updating the doc makes more sense, but this is a major effort, plus someone has volunteered to help with that (plus I'd like to experiment with online tutorials). So in short, I think I'd like to try this out. Question now: what do you think would be most useful? I think about 10 to 60 minutes of presentation, something that I should be able to create over some evenings than try to deliver. What would be the best candidates to go into such material? Feedback appreciated, Rainer From jules at visionintel.com Fri Mar 6 17:16:32 2009 From: jules at visionintel.com (jules at visionintel.com) Date: Fri, 06 Mar 2009 16:16:32 +0000 Subject: [rsyslog] Intro presentation Message-ID: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Remote loggin Sent from my Nokia phone -----Original Message----- From: Rainer Gerhards Sent: 06/03/2009 16:09:05 Subject: [rsyslog] Intro presentation Hi all, I think about doing an online intro presentation to rsyslog that should be useful to new users, in addition to the doc. One may claim that updating the doc makes more sense, but this is a major effort, plus someone has volunteered to help with that (plus I'd like to experiment with online tutorials). So in short, I think I'd like to try this out. Question now: what do you think would be most useful? I think about 10 to 60 minutes of presentation, something that I should be able to create over some evenings than try to deliver. What would be the best candidates to go into such material? Feedback appreciated, Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From aoz.syn at gmail.com Fri Mar 6 17:25:24 2009 From: aoz.syn at gmail.com (RB) Date: Fri, 6 Mar 2009 09:25:24 -0700 Subject: [rsyslog] wrong permissons on directories In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> References: <49B0EA3C.1060104@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F68@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903060825l37364ab2w738468329e628e82@mail.gmail.com> On Fri, Mar 6, 2009 at 08:40, Rainer Gerhards wrote: > The more I think about it, the more it smells like a real bug. Has anyone objections changing the default? None. It is unrealistic (and generally unusable) to have UNIX directory permissions without the execute bit (S_IX*). The only reason to do it would be to have an 'archive' directory of sorts, in which users may see names of children, but none of their permissions or contents. As has been noted, the only reason it's worked thus far is that most people either change the default or run the daemon as root, for whom those permissions aren't really a limiting factor. From u.a.martin at gmail.com Fri Mar 6 17:38:57 2009 From: u.a.martin at gmail.com (Ben Martin) Date: Fri, 6 Mar 2009 09:38:57 -0700 Subject: [rsyslog] Intro presentation In-Reply-To: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> References: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Message-ID: <661ae2b20903060838q1aa1f5d8g91c79cff9bc606ab@mail.gmail.com> Rainer I think a video tutorial is great idea. You might even start with a very brief discussion of the importance of centralized logging, from both the security and management perspective. Discussing the basic differences between v2 and v3 would also be helpful I think, as some distros (like CentOS) are still only packaging v2, while others (Debian) are installing v3 by default. - Ben On Fri, Mar 6, 2009 at 9:16 AM, wrote: > Remote loggin > > Sent from my Nokia phone > -----Original Message----- > From: Rainer Gerhards > Sent: ?06/03/2009 16:09:05 > Subject: ?[rsyslog] Intro presentation > > Hi all, > > I think about doing an online intro presentation to rsyslog that should > be useful to new users, in addition to the doc. One may claim that > updating the doc makes more sense, but this is a major effort, plus > someone has volunteered to help with that (plus I'd like to experiment > with online tutorials). So in short, I think I'd like to try this out. > > Question now: what do you think would be most useful? I think about 10 > to 60 minutes of presentation, something that I should be able to create > over some evenings than try to deliver. What would be the best > candidates to go into such material? > > Feedback appreciated, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Fri Mar 6 18:21:14 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 18:21:14 +0100 Subject: [rsyslog] wrong permissons on directories References: <49B0EA3C.1060104@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F5F@GRFEXC.intern.adiscon.com> <49B12FA3.2030202@1und1.de><9B6E2A8877C38245BFB15CC491A11DA71F63@GRFEXC.intern.adiscon.com> <49B13E8A.2080308@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F70@GRFEXC.intern.adiscon.com> I guess the "root issue" is more a probably cause. I know that lot's of folks use rsyslog to create dirs. Will probably change the default, but in the beta first. Thanks for bringing this up. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Friday, March 06, 2009 4:18 PM > To: rsyslog-users > Subject: Re: [rsyslog] wrong permissons on directories > > I guess nobody did let rsyslog make directories. > > Rainer Gerhards wrote: > > Thomas, > > > > do I correctly understand that you propose the default be changed? > > Yepp. > > > If so, I am hesitant to do that - wouldn't that potentially break > existing deployments? > > hmm Maybe I haven't seen enough yet, but I can't imagine a deployment > built on directory permissions 644.... > > > On the other hand... how could that work... Umm... > > They are all working as root out there :-) > > I think it would be good if you just double check it yourself that the > directories get created with 644 and decicde on your findings. > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Fri Mar 6 19:53:14 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 10:53:14 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly Message-ID: I'm running into problems trying to do filtering. it looks as if the log parsing is not properly filling in the properties. what I've run into so far when I use the property 'programname' the content that I see is what I would expect in 'hostname' when I use the property 'hostname' the content that I see is what I would expect in 'fromhost' I haven't checked all the other properties, but my guess is that somehow rsyslog is off-by-one in filling them in. David Lang From david at lang.hm Fri Mar 6 19:54:00 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 10:54:00 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: References: Message-ID: On Fri, 6 Mar 2009, david at lang.hm wrote: > I'm running into problems trying to do filtering. it looks as if the log > parsing is not properly filling in the properties. > > what I've run into so far > > when I use the property 'programname' the content that I see is what I would > expect in 'hostname' > > when I use the property 'hostname' the content that I see is what I would > expect in 'fromhost' > > I haven't checked all the other properties, but my guess is that somehow > rsyslog is off-by-one in filling them in. having said this, date, fromhost, and from-ip appear to be filled in correctly. David Lang From rgerhards at hq.adiscon.com Fri Mar 6 19:54:11 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 6 Mar 2009 19:54:11 +0100 Subject: [rsyslog] properties not getting filled in correctly References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> That's why I am after the log samples :) I just termed a new acronym this afternoon: YAMSF - yet another malformed syslog format ;) http://blog.gerhards.net/2009/02/calling-for-log-samples.html I try hard to get the fields right, but often this is impossible, resulting in the issues you see. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, March 06, 2009 7:54 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Fri, 6 Mar 2009, david at lang.hm wrote: > > > I'm running into problems trying to do filtering. it looks as if the > log > > parsing is not properly filling in the properties. > > > > what I've run into so far > > > > when I use the property 'programname' the content that I see is what > I would > > expect in 'hostname' > > > > when I use the property 'hostname' the content that I see is what I > would > > expect in 'fromhost' > > > > I haven't checked all the other properties, but my guess is that > somehow > > rsyslog is off-by-one in filling them in. > > having said this, date, fromhost, and from-ip appear to be filled in > correctly. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Sat Mar 7 02:25:32 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 17:25:32 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 6 Mar 2009, Rainer Gerhards wrote: > That's why I am after the log samples :) I just termed a new acronym > this afternoon: > YAMSF - yet another malformed syslog format ;) > > http://blog.gerhards.net/2009/02/calling-for-log-samples.html > > I try hard to get the fields right, but often this is impossible, > resulting in the issues you see. these logs come from several different servers, including different OSs, but all are misparsed by rsyslog. I am not seeing anything obviously wrong with them <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Friday, March 06, 2009 7:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Fri, 6 Mar 2009, david at lang.hm wrote: >> >>> I'm running into problems trying to do filtering. it looks as if the >> log >>> parsing is not properly filling in the properties. >>> >>> what I've run into so far >>> >>> when I use the property 'programname' the content that I see is what >> I would >>> expect in 'hostname' >>> >>> when I use the property 'hostname' the content that I see is what I >> would >>> expect in 'fromhost' >>> >>> I haven't checked all the other properties, but my guess is that >> somehow >>> rsyslog is off-by-one in filling them in. >> >> having said this, date, fromhost, and from-ip appear to be filled in >> correctly. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From david at lang.hm Sat Mar 7 03:55:49 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 6 Mar 2009 18:55:49 -0800 (PST) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71F72@GRFEXC.intern.adiscon.com> Message-ID: On Fri, 6 Mar 2009, david at lang.hm wrote: > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >> That's why I am after the log samples :) I just termed a new acronym >> this afternoon: >> YAMSF - yet another malformed syslog format ;) >> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >> >> I try hard to get the fields right, but often this is impossible, >> resulting in the issues you see. > > these logs come from several different servers, including different OSs, > but all are misparsed by rsyslog. > > I am not seeing anything obviously wrong with them > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw doing some more digging I see some very definant problems I created the following template $template DumpAll,"msg =%msg%\nrawmsg =%rawmsg%\nuxtradmsg =%uxtradmsg%\nhostname =%hostname%\nsource =%source%\nfromhost =%fromhost%\nfromhost-ip =%fromhost-ip%\nsyslogtag =%syslogtag%\nprogramname =%programname%\npri =%pri%\npri-text =%pri-text%\niut =%iut%\nsyslogfacility =%syslogfacility%\nsyslogfacility-text =%syslogfacility-text%\nsyslogseverity =%syslogseverity%\nsyslogseverity-text =%syslogseverity-text%\nsyslogpriority =%syslogpriority%\nsyslogpriority-text =%syslogpriority-text%\ntimegenerated =%timegenerated%\ntimereported =%timereported%\ntimestamp =%timestamp%\nprotocol-version =%protocol-version%\nstructured-data =%structured-data%\napp-name =%app-name%\nprocid =%procid%\nmsgid =%msgid%\ninputname =%inputname%\n\n" which creates a nice table for each log message showing what's in each property. things that I am seeing hostname and source are fromhost rather than the name/IP that's in the record. msg includes the programname programname and appname are what hostname should be David Lang msg = %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 rawmsg =<167>Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 uxtradmsg =Mar 6 18:33:47 172.20.245.8 %PIX-7-710005: UDP request discarded from BOK37UAT/3683 to test_app:255.255.255.255/61601 hostname =itascan1a-p source =itascan1a-p fromhost =itascan1a-p fromhost-ip =192.168.210.6 syslogtag =172.20.245.8 programname =172.20.245.8 pri =167 pri-text =local4.debug<167> iut =1 syslogfacility =20 syslogfacility-text =local4 syslogseverity =7 syslogseverity-text =debug syslogpriority =7 syslogpriority-text =debug timegenerated =Mar 7 02:33:47 timereported =Mar 6 18:33:47 timestamp =Mar 6 18:33:47 protocol-version =0 structured-data =- app-name =172.20.245.8 procid =- msgid =- inputname =imudp msg = plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 rawmsg =<29>Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 uxtradmsg =Mar 6 18:33:47 methane1d-b plug-gw[28055]: disconnect host= /192.168.242.212 destination=179.50.100.130/12773 in=0 out=0 duration=0 hostname =itascan1a-p source =itascan1a-p fromhost =itascan1a-p fromhost-ip =192.168.210.6 syslogtag =methane1d-b programname =methane1d-b pri =29 pri-text =daemon.notice<29> iut =1 syslogfacility =3 syslogfacility-text =daemon syslogseverity =5 syslogseverity-text =notice syslogpriority =5 syslogpriority-text =notice timegenerated =Mar 7 02:33:47 timereported =Mar 6 18:33:47 timestamp =Mar 6 18:33:47 protocol-version =0 structured-data =- app-name =methane1d-b procid =- msgid =- inputname =imudp From rgerhards at hq.adiscon.com Sat Mar 7 10:47:54 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 7 Mar 2009 10:47:54 +0100 Subject: [rsyslog] properties not getting filled in correctly Message-ID: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> The messages indeed look ok. I'll feed them into my parser and will see what happens. rainer ----- Urspr?ngliche Nachricht ----- Von: "david at lang.hm" An: "rsyslog-users" Gesendet: 07.03.09 02:20 Betreff: Re: [rsyslog] properties not getting filled in correctly On Fri, 6 Mar 2009, Rainer Gerhards wrote: > That's why I am after the log samples :) I just termed a new acronym > this afternoon: > YAMSF - yet another malformed syslog format ;) > > http://blog.gerhards.net/2009/02/calling-for-log-samples.html > > I try hard to get the fields right, but often this is impossible, > resulting in the issues you see. these logs come from several different servers, including different OSs, but all are misparsed by rsyslog. I am not seeing anything obviously wrong with them <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Friday, March 06, 2009 7:54 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Fri, 6 Mar 2009, david at lang.hm wrote: >> >>> I'm running into problems trying to do filtering. it looks as if the >> log >>> parsing is not properly filling in the properties. >>> >>> what I've run into so far >>> >>> when I use the property 'programname' the content that I see is what >> I would >>> expect in 'hostname' >>> >>> when I use the property 'hostname' the content that I see is what I >> would >>> expect in 'fromhost' >>> >>> I haven't checked all the other properties, but my guess is that >> somehow >>> rsyslog is off-by-one in filling them in. >> >> having said this, date, fromhost, and from-ip appear to be filled in >> correctly. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 9 07:14:49 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 9 Mar 2009 07:14:49 +0100 Subject: [rsyslog] filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71F7B@GRFEXC.intern.adiscon.com> Hi David, Sorry for the late reply. Of course, the change is not as trivial as I initially thought. It is very easy to add a length modifier to the property replacer, but you can not use the property replacer in property-based filters. Of course, I can modify those filters, but there no concept of a numerical value with these filters. The proper thing would be to do this in the script engine, where it was scheduled for, but the script engine does not yet support functions. Doh... I will look where I can best hack this into. My current thinking is that I will check what it takes to make the script engine support built-in (rather than loadable) functions, so that I could implement a set of core functions. I am not sure how much effort that is, but it doesn't look too scary (plus it would be really good to have this functionality, so it would be well-spent time). It that turns out not to be an option, I'll probably hack the script engine to support a unary operation "lengthof", that should be simple enough - but it is a dirty approach. I won't be able to do anything of this today, but I hope I can do either of the two within this week. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 04, 2009 8:06 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > Oh, that's an interesting use case. It is not yet possible. > I think we > > can implement (fairly simple) the size for a field (via the property > > replacer). However, that does not help you with the > resulting size of a > > template string. I probably also need to check the supporting > > infrastructure for "greater than" comparisons... > > > > Would that help? > > yes, I can set the value to something conservative to account for the > variable-length fields. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > david at lang.hm > >> Sent: Wednesday, March 04, 2009 12:54 AM > >> To: rsyslog-users > >> Subject: [rsyslog] filtering by message size > >> > >> is it possible to filter by message size? > >> > >> I'm looking at a situation where I would like to send the > >> message via UDP > >> if it's below a given size and by TCP if it's larger. > >> > >> David Lang > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From jackmarrow2 at gmail.com Tue Mar 10 11:15:09 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:15:09 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/3 Rainer Gerhards : > Well, you can see all change log entries by following the "change log" > menu item in the menu to the left ;) But it may even be more convenient > in that case that you get it directly from git as a single text file: > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=ba2a6c13e22b7 > f67401c7edb15ea17d31162bde4;hb=HEAD > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of jack marrow >> Sent: Tuesday, March 03, 2009 9:06 AM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] rsyslog changelog >> >> Hello, >> >> Is there a changelog for rsyslog, particularly showing the differences >> between the current version (3.x) and the 2.x version found in RHEL? >> >> Thanks, >> >> Jack >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > Thanks for this. One last question: on the receiving server side, can I see which logs came from which log file? From rgerhards at hq.adiscon.com Tue Mar 10 11:12:32 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 11:12:32 +0100 Subject: [rsyslog] rsyslog changelog References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> > One last question: on the receiving server side, can I see which logs > came from which log file? Usually, the log line should contain the host that sent the message. Does your's not? Rainer From jackmarrow2 at gmail.com Tue Mar 10 11:21:57 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:21:57 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/10 Rainer Gerhards : >> One last question: on the receiving server side, can I see which logs >> came from which log file? > > Usually, the log line should contain the host that sent the message. > Does your's not? > If a client sends /var/log/httpd/blah and /var/log/vsftpd/blah, does the receiving side simply receive the log contents or the filename as well? Is there a way to get both? From rgerhards at hq.adiscon.com Tue Mar 10 11:17:46 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 11:17:46 +0100 Subject: [rsyslog] rsyslog changelog References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Please post configs and elaborate a bit more about what you are trying to accomplish and what you have set up. > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jack marrow > Sent: Tuesday, March 10, 2009 11:22 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog changelog > > 2009/3/10 Rainer Gerhards : > >> One last question: on the receiving server side, can I see which > logs > >> came from which log file? > > > > Usually, the log line should contain the host that sent the message. > > Does your's not? > > > > If a client sends /var/log/httpd/blah and /var/log/vsftpd/blah, does > the receiving side simply receive the log contents or the filename as > well? Is there a way to get both? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From jackmarrow2 at gmail.com Tue Mar 10 11:28:35 2009 From: jackmarrow2 at gmail.com (jack marrow) Date: Tue, 10 Mar 2009 11:28:35 +0100 Subject: [rsyslog] rsyslog changelog In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Message-ID: 2009/3/10 Rainer Gerhards : > Please post configs and elaborate a bit more about what you are trying > to accomplish and what you have set up. I am evaluating rsyslog at the moment. I would like to know if I can use it for log collection on the client for writing on the server. The server must know which log file is which. From david at lang.hm Tue Mar 10 16:21:45 2009 From: david at lang.hm (david at lang.hm) Date: Tue, 10 Mar 2009 08:21:45 -0700 (PDT) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: On Sat, 7 Mar 2009, Rainer Gerhards wrote: > The messages indeed look ok. I'll feed them into my parser and will see what happens. any idea what's happening here yet? David Lang > rainer > > ----- Urspr?ngliche Nachricht ----- > Von: "david at lang.hm" > An: "rsyslog-users" > Gesendet: 07.03.09 02:20 > Betreff: Re: [rsyslog] properties not getting filled in correctly > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >> That's why I am after the log samples :) I just termed a new acronym >> this afternoon: >> YAMSF - yet another malformed syslog format ;) >> >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >> >> I try hard to get the fields right, but often this is impossible, >> resulting in the issues you see. > > these logs come from several different servers, including different OSs, > but all are misparsed by rsyslog. > > I am not seeing anything obviously wrong with them > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= /192.168.243.37 destination=179.50.100.130/60029 > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 duration=1 > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= /192.168.22.8 destination=192.168.104.31/5667 > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for delivery) > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > David Lang > >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>> Sent: Friday, March 06, 2009 7:54 PM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] properties not getting filled in correctly >>> >>> On Fri, 6 Mar 2009, david at lang.hm wrote: >>> >>>> I'm running into problems trying to do filtering. it looks as if the >>> log >>>> parsing is not properly filling in the properties. >>>> >>>> what I've run into so far >>>> >>>> when I use the property 'programname' the content that I see is what >>> I would >>>> expect in 'hostname' >>>> >>>> when I use the property 'hostname' the content that I see is what I >>> would >>>> expect in 'fromhost' >>>> >>>> I haven't checked all the other properties, but my guess is that >>> somehow >>>> rsyslog is off-by-one in filling them in. >>> >>> having said this, date, fromhost, and from-ip appear to be filled in >>> correctly. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Tue Mar 10 16:24:31 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 16:24:31 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FB1@GRFEXC.intern.adiscon.com> Not at the moment, I am currently looking into the scripting engine (for stringlength-based evaluations) I highly suggest http://twitter.com/rgerhards to keep track of what I am looking at. You do NOT need to be subscribed to twitter to use this service. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, March 10, 2009 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Sat, 7 Mar 2009, Rainer Gerhards wrote: > > > The messages indeed look ok. I'll feed them into my parser and will > see what happens. > > any idea what's happening here yet? > > David Lang > > > rainer > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "david at lang.hm" > > An: "rsyslog-users" > > Gesendet: 07.03.09 02:20 > > Betreff: Re: [rsyslog] properties not getting filled in correctly > > > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > > > >> That's why I am after the log samples :) I just termed a new acronym > >> this afternoon: > >> YAMSF - yet another malformed syslog format ;) > >> > >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >> > >> I try hard to get the fields right, but often this is impossible, > >> resulting in the issues you see. > > > > these logs come from several different servers, including different > OSs, > > but all are misparsed by rsyslog. > > > > I am not seeing anything obviously wrong with them > > > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > /192.168.243.37 destination=179.50.100.130/60029 > > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > duration=1 > > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > /192.168.22.8 destination=192.168.104.31/5667 > > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, > pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( > <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > delivery) > > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > > > David Lang > > > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Friday, March 06, 2009 7:54 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>> > >>>> I'm running into problems trying to do filtering. it looks as if > the > >>> log > >>>> parsing is not properly filling in the properties. > >>>> > >>>> what I've run into so far > >>>> > >>>> when I use the property 'programname' the content that I see is > what > >>> I would > >>>> expect in 'hostname' > >>>> > >>>> when I use the property 'hostname' the content that I see is what > I > >>> would > >>>> expect in 'fromhost' > >>>> > >>>> I haven't checked all the other properties, but my guess is that > >>> somehow > >>>> rsyslog is off-by-one in filling them in. > >>> > >>> having said this, date, fromhost, and from-ip appear to be filled > in > >>> correctly. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From aoz.syn at gmail.com Tue Mar 10 18:14:20 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 10 Mar 2009 11:14:20 -0600 Subject: [rsyslog] rsyslog changelog In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA71F04@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA0@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FA2@GRFEXC.intern.adiscon.com> Message-ID: <4255c2570903101014n37d41ea2s23db8accc90f96ba@mail.gmail.com> On Tue, Mar 10, 2009 at 04:28, jack marrow wrote: > 2009/3/10 Rainer Gerhards : >> Please post configs and elaborate a bit more about what you are trying >> to accomplish and what you have set up. > > I am evaluating rsyslog at the moment. > > I would like to know if I can use it for log collection on the client > for writing on the server. The server must know which log file is > which. This is more a "basic understanding of logging" question than one specific to rsyslog. Generally speaking, log daemons just log what client apps tell them to - httpd says, "I'm facility 6 and is my critical message". If the local log daemon is sending logs upstream, it will basically tell the upstream server "I'm myhostname and httpd (facility 6) just said with a critical priority". If all your daemons (httpd, vsftpd, etc.) log directly to the local syslog as opposed to a flat file, things should "just work". However, if you're configuring your "client" syslog instance to follow /var/log/httpd/access and retransmit that data to an upstream server, all that metadata (application name, facility, priority, etc) is lost. Hence, you must configure your client syslog to inject that data - with rsyslog, that would be done something like this: $ModLoad imfile.so $InputFileName /var/log/httpd/access $InputFileTag http_access $InputFilePollIntervalSeconds 5 $InputFileMonitor *.* @192.168.1.1 That sets up a monitor that polls /var/log/httpd/access every 5 seconds, prepends "http_access" to every line, and sends it via UDP to 192.168.1.1. From rgerhards at hq.adiscon.com Tue Mar 10 18:24:02 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 10 Mar 2009 18:24:02 +0100 Subject: [rsyslog] RainerScript functions - was: RE: filtering by message size References: <9B6E2A8877C38245BFB15CC491A11DA71F2F@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71F7B@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FB3@GRFEXC.intern.adiscon.com> David, I have extended RainerScript with the ability to call functions. The current implementation is very much focused on the immediate needs and it has a VM instruction set design issue that prevents nested function calls from working. Also, it only supports build-in functions (not loadable modules), and the only build-in function so far is strlen() ;) - but it should do what you need. So far, it resides in its own git branch "rscript-func". I will continue to work on it (at least on the VM opcode issue), but would really appreciate some early feedback. With that version you can do things like if strlen($msg) > 80 then @@tcp-host if strlen($msg) <= 80 then @udp-host Note that the function argument can be any valid expression (but NOT another function call!), so the following is also valid (and maybe useful to get to a better guess): if strlen($msg & $syslogtag & $fromhost) > 80 then @@tcp-host Note that & is the string concatenation operator. Today's commit: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=e8499c6d33d09f6d8b42df72 da1661be0ef0f088 Feedback from you and all others is appreciated. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, March 09, 2009 7:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] filtering by message size > > Hi David, > > Sorry for the late reply. Of course, the change is not as trivial as I > initially thought. It is very easy to add a length modifier to the > property replacer, but you can not use the property replacer in > property-based filters. Of course, I can modify those filters, but > there > no concept of a numerical value with these filters. The proper thing > would be to do this in the script engine, where it was scheduled for, > but the script engine does not yet support functions. Doh... > > I will look where I can best hack this into. My current thinking is > that > I will check what it takes to make the script engine support built-in > (rather than loadable) functions, so that I could implement a set of > core functions. I am not sure how much effort that is, but it doesn't > look too scary (plus it would be really good to have this > functionality, > so it would be well-spent time). It that turns out not to be an option, > I'll probably hack the script engine to support a unary operation > "lengthof", that should be simple enough - but it is a dirty approach. > I > won't be able to do anything of this today, but I hope I can do either > of the two within this week. > > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com > > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm > > Sent: Wednesday, March 04, 2009 8:06 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] filtering by message size > > > > On Wed, 4 Mar 2009, Rainer Gerhards wrote: > > > > > Oh, that's an interesting use case. It is not yet possible. > > I think we > > > can implement (fairly simple) the size for a field (via the > property > > > replacer). However, that does not help you with the > > resulting size of a > > > template string. I probably also need to check the supporting > > > infrastructure for "greater than" comparisons... > > > > > > Would that help? > > > > yes, I can set the value to something conservative to account for the > > variable-length fields. > > > > David Lang > > > > > Rainer > > > > > >> -----Original Message----- > > >> From: rsyslog-bounces at lists.adiscon.com > > >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of > > david at lang.hm > > >> Sent: Wednesday, March 04, 2009 12:54 AM > > >> To: rsyslog-users > > >> Subject: [rsyslog] filtering by message size > > >> > > >> is it possible to filter by message size? > > >> > > >> I'm looking at a situation where I would like to send the > > >> message via UDP > > >> if it's below a given size and by TCP if it's larger. > > >> > > >> David Lang > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 11 13:49:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 13:49:08 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> David, the issue is in v4 only (and so far UDP only, too). It was introduced by the optimizations, which pass some wrong parameters to the now-decoupled parser. Need to find root cause, though. Will keep you posted. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, March 10, 2009 4:22 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Sat, 7 Mar 2009, Rainer Gerhards wrote: > > > The messages indeed look ok. I'll feed them into my parser and will > see what happens. > > any idea what's happening here yet? > > David Lang > > > rainer > > > > ----- Urspr?ngliche Nachricht ----- > > Von: "david at lang.hm" > > An: "rsyslog-users" > > Gesendet: 07.03.09 02:20 > > Betreff: Re: [rsyslog] properties not getting filled in correctly > > > > On Fri, 6 Mar 2009, Rainer Gerhards wrote: > > > >> That's why I am after the log samples :) I just termed a new acronym > >> this afternoon: > >> YAMSF - yet another malformed syslog format ;) > >> > >> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >> > >> I try hard to get the fields right, but often this is impossible, > >> resulting in the issues you see. > > > > these logs come from several different servers, including different > OSs, > > but all are misparsed by rsyslog. > > > > I am not seeing anything obviously wrong with them > > > > <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > > <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > /192.168.243.37 destination=179.50.100.130/60029 > > <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > duration=1 > > <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > /192.168.22.8 destination=192.168.104.31/5667 > > <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, > pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( > <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > delivery) > > <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > > > > David Lang > > > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>> Sent: Friday, March 06, 2009 7:54 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>> > >>>> I'm running into problems trying to do filtering. it looks as if > the > >>> log > >>>> parsing is not properly filling in the properties. > >>>> > >>>> what I've run into so far > >>>> > >>>> when I use the property 'programname' the content that I see is > what > >>> I would > >>>> expect in 'hostname' > >>>> > >>>> when I use the property 'hostname' the content that I see is what > I > >>> would > >>>> expect in 'fromhost' > >>>> > >>>> I haven't checked all the other properties, but my guess is that > >>> somehow > >>>> rsyslog is off-by-one in filling them in. > >>> > >>> having said this, date, fromhost, and from-ip appear to be filled > in > >>> correctly. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From david at lang.hm Wed Mar 11 13:51:18 2009 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Mar 2009 05:51:18 -0700 (PDT) Subject: [rsyslog] properties not getting filled in correctly In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> Message-ID: On Wed, 11 Mar 2009, Rainer Gerhards wrote: > David, > > the issue is in v4 only (and so far UDP only, too). It was introduced by the > optimizations, which pass some wrong parameters to the now-decoupled parser. > Need to find root cause, though. > > Will keep you posted. thanks. David Lang > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >> Sent: Tuesday, March 10, 2009 4:22 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] properties not getting filled in correctly >> >> On Sat, 7 Mar 2009, Rainer Gerhards wrote: >> >>> The messages indeed look ok. I'll feed them into my parser and will >> see what happens. >> >> any idea what's happening here yet? >> >> David Lang >> >>> rainer >>> >>> ----- Urspr?ngliche Nachricht ----- >>> Von: "david at lang.hm" >>> An: "rsyslog-users" >>> Gesendet: 07.03.09 02:20 >>> Betreff: Re: [rsyslog] properties not getting filled in correctly >>> >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote: >>> >>>> That's why I am after the log samples :) I just termed a new acronym >>>> this afternoon: >>>> YAMSF - yet another malformed syslog format ;) >>>> >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html >>>> >>>> I try hard to get the fields right, but often this is impossible, >>>> resulting in the issues you see. >>> >>> these logs come from several different servers, including different >> OSs, >>> but all are misparsed by rsyslog. >>> >>> I am not seeing anything obviously wrong with them >>> >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601 >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= >> /192.168.243.37 destination=179.50.100.130/60029 >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 >> duration=1 >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= >> /192.168.22.8 destination=192.168.104.31/5667 >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: >> to=, delay=00:00:01, xdelay=00:00:01, mailer=esmtp, >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, stat=Sent ( >> <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for >> delivery) >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw >>> >>> David Lang >>> >>>> Rainer >>>> >>>>> -----Original Message----- >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm >>>>> Sent: Friday, March 06, 2009 7:54 PM >>>>> To: rsyslog-users >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly >>>>> >>>>> On Fri, 6 Mar 2009, david at lang.hm wrote: >>>>> >>>>>> I'm running into problems trying to do filtering. it looks as if >> the >>>>> log >>>>>> parsing is not properly filling in the properties. >>>>>> >>>>>> what I've run into so far >>>>>> >>>>>> when I use the property 'programname' the content that I see is >> what >>>>> I would >>>>>> expect in 'hostname' >>>>>> >>>>>> when I use the property 'hostname' the content that I see is what >> I >>>>> would >>>>>> expect in 'fromhost' >>>>>> >>>>>> I haven't checked all the other properties, but my guess is that >>>>> somehow >>>>>> rsyslog is off-by-one in filling them in. >>>>> >>>>> having said this, date, fromhost, and from-ip appear to be filled >> in >>>>> correctly. >>>>> >>>>> David Lang >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Wed Mar 11 14:32:17 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 14:32:17 +0100 Subject: [rsyslog] properties not getting filled in correctly References: <000101c99f09$3bc4177d$100013ac@intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71FBD@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FC3@GRFEXC.intern.adiscon.com> David, there is now a patch available: http://git.adiscon.com/?p=rsyslog.git;a=commit;h=59192611db992e7357337beb8e68 ec6cee5b3fec I will release a new devel today, and it will include the patch. I expect to release another one next week, which will then have the Solaris work plus the script engine with functions (feedback on that is still appreciated). Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Wednesday, March 11, 2009 1:51 PM > To: rsyslog-users > Subject: Re: [rsyslog] properties not getting filled in correctly > > On Wed, 11 Mar 2009, Rainer Gerhards wrote: > > > David, > > > > the issue is in v4 only (and so far UDP only, too). It was introduced > by the > > optimizations, which pass some wrong parameters to the now-decoupled > parser. > > Need to find root cause, though. > > > > Will keep you posted. > > thanks. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >> Sent: Tuesday, March 10, 2009 4:22 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] properties not getting filled in correctly > >> > >> On Sat, 7 Mar 2009, Rainer Gerhards wrote: > >> > >>> The messages indeed look ok. I'll feed them into my parser and will > >> see what happens. > >> > >> any idea what's happening here yet? > >> > >> David Lang > >> > >>> rainer > >>> > >>> ----- Urspr?ngliche Nachricht ----- > >>> Von: "david at lang.hm" > >>> An: "rsyslog-users" > >>> Gesendet: 07.03.09 02:20 > >>> Betreff: Re: [rsyslog] properties not getting filled in correctly > >>> > >>> On Fri, 6 Mar 2009, Rainer Gerhards wrote: > >>> > >>>> That's why I am after the log samples :) I just termed a new > acronym > >>>> this afternoon: > >>>> YAMSF - yet another malformed syslog format ;) > >>>> > >>>> http://blog.gerhards.net/2009/02/calling-for-log-samples.html > >>>> > >>>> I try hard to get the fields right, but often this is impossible, > >>>> resulting in the issues you see. > >>> > >>> these logs come from several different servers, including different > >> OSs, > >>> but all are misparsed by rsyslog. > >>> > >>> I am not seeing anything obviously wrong with them > >>> > >>> <167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: UDP request > >> discarded from SERVER1/2741 to test_app:255.255.255.255/61601 > >>> <29>Mar 6 16:57:54 methane1d-b plug-gw[25213]: connect host= > >> /192.168.243.37 destination=179.50.100.130/60029 > >>> <29>Mar 6 16:57:54 methane1a-b plug-gw[29368]: disconnect host= > >> /192.168.242.119 destination=179.50.100.52/14733 in=357 out=71 > >> duration=1 > >>> <29>Mar 6 16:57:54 happy1-b plug-gw[30259]: connect host= > >> /192.168.22.8 destination=192.168.104.31/5667 > >>> <22>Mar 6 16:57:54 192.168.242.66 sendmail[13328]: n270vrSH013326: > >> to=, delay=00:00:01, xdelay=00:00:01, > mailer=esmtp, > >> pri=37052, relay=mx1.hotmail.com. [65.54.244.8], dsn=2.0.0, > stat=Sent ( > >> <200903070057.n270vrL174106 at w31.diginsite.com> Queued mail for > >> delivery) > >>> <29>Mar 6 16:57:54 corpmail1-p netacl[3839]: permit host= > >> /10.201.7.120 service=telnetd execute=/usr/local/etc/tn-gw > >>> > >>> David Lang > >>> > >>>> Rainer > >>>> > >>>>> -----Original Message----- > >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm > >>>>> Sent: Friday, March 06, 2009 7:54 PM > >>>>> To: rsyslog-users > >>>>> Subject: Re: [rsyslog] properties not getting filled in correctly > >>>>> > >>>>> On Fri, 6 Mar 2009, david at lang.hm wrote: > >>>>> > >>>>>> I'm running into problems trying to do filtering. it looks as if > >> the > >>>>> log > >>>>>> parsing is not properly filling in the properties. > >>>>>> > >>>>>> what I've run into so far > >>>>>> > >>>>>> when I use the property 'programname' the content that I see is > >> what > >>>>> I would > >>>>>> expect in 'hostname' > >>>>>> > >>>>>> when I use the property 'hostname' the content that I see is > what > >> I > >>>>> would > >>>>>> expect in 'fromhost' > >>>>>> > >>>>>> I haven't checked all the other properties, but my guess is that > >>>>> somehow > >>>>>> rsyslog is off-by-one in filling them in. > >>>>> > >>>>> having said this, date, fromhost, and from-ip appear to be filled > >> in > >>>>> correctly. > >>>>> > >>>>> David Lang > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Wed Mar 11 15:22:51 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 11 Mar 2009 15:22:51 +0100 Subject: [rsyslog] rsyslog 4.1.5 (devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FC6@GRFEXC.intern.adiscon.com> Hi all, I have just released rsyslog 4.1.5, a member of the development branch. It offers ERE support in filter conditions as well as the ability to contain part of the repeated text in a "last message repeated n times" message. Also, it fixes a bug that caused invalid parsing when receiving messages via UDP. This is a recommended update for all development branch users. Change Log: http://www.rsyslog.com/Article349.phtml Download: http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-150.phtml I hope this release is useful. As always, feedback is appreciated. Rainer From rgerhards at hq.adiscon.com Thu Mar 12 10:53:30 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Mar 2009 10:53:30 +0100 Subject: [rsyslog] Intro presentation References: <49b14c65.1c185e0a.1a42.ffff8549@mx.google.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FDC@GRFEXC.intern.adiscon.com> Hi all, I created a first video tutorial today, please see blog for questions: http://blog.gerhards.net/2009/03/rsyslog-video-tutorials.html For this test, I have used something that I had ready at hand, thus none of the suggested topics yet touched. Feedback to the questions raised in the blog post would be most welcome. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of jules at visionintel.com > Sent: Friday, March 06, 2009 5:17 PM > To: rsyslog-users > Subject: Re: [rsyslog] Intro presentation > > Remote loggin > > Sent from my Nokia phone > -----Original Message----- > From: Rainer Gerhards > Sent: 06/03/2009 16:09:05 > Subject: [rsyslog] Intro presentation > > Hi all, > > I think about doing an online intro presentation to rsyslog that should > be useful to new users, in addition to the doc. One may claim that > updating the doc makes more sense, but this is a major effort, plus > someone has volunteered to help with that (plus I'd like to experiment > with online tutorials). So in short, I think I'd like to try this out. > > Question now: what do you think would be most useful? I think about 10 > to 60 minutes of presentation, something that I should be able to > create > over some evenings than try to deliver. What would be the best > candidates to go into such material? > > Feedback appreciated, > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Thu Mar 12 18:36:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Mar 2009 18:36:08 +0100 Subject: [rsyslog] rant on software (rsyslog) stability Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71FE7@GRFEXC.intern.adiscon.com> Hi all, I was recently asked a couple of times if I could quickly create a "stable version" of this and that new feature. So I have finally taken out some time today (more than expected...) to pen down my position on it. The rant also tells a lot about rsyslog's branches, so I thought it is useful to circulate it on the mailing list: http://blog.gerhards.net/2009/03/how-software-gets-stable.html As always, feedback is appreciated. Rainer From mtant621 at charter.net Fri Mar 13 19:53:19 2009 From: mtant621 at charter.net (Michael Tant) Date: Fri, 13 Mar 2009 14:53:19 -0400 Subject: [rsyslog] Please Help! IPTables dumping to Console!!! Message-ID: I am running Fedora 10 linux with rsyslogd as my active logger. Recently I have had an issue with my iptables LOG target output going to the console and not going to the /var/log/messages file, even with the --log-level 6 argument. I have halfway resolved this issue by editing the /etc/rsyslog.conf file to include: kern.warning /var/log/iptables.log and appending --log-level 4 to my LOG target rules. This caused the output to go to the aforementioned file AND the console. I wish to still have the log data going to the iptables.log file, but wish to stop the dump to the console. I have reviewed the rsyslog.conf file, and the only statement which references /dev/console is kern.* but it is commented out with #. I am tempted to remove this statement to see if it helps, but I am unsure if this is safe, and furthermore convinced it will not change the outcome as this line is nothing more than a comment. Is there something somewhere I am perhaps missing? I don't fully understand the steps that move the log target output to the file, other than rsyslogd is in the middle somewhere with the kernel. Any suggestions would be greatly appreciated! Please send suggestion to mtant621 at chater.net I thank everyone for your help... Michael Tant From rvandolson at esri.com Sat Mar 14 00:18:14 2009 From: rvandolson at esri.com (Ray Van Dolson) Date: Fri, 13 Mar 2009 16:18:14 -0700 Subject: [rsyslog] Filtering on a group of IP's Message-ID: <20090313231814.GA7833@esri.com> I'm trying to shunt a bunch of logs from a group of IP's (about 10 IP's or so) to a fifo. Is the best way to do this with a property filter like the following? $template SplunkPipe,"|/logs/splunk/splunk.fifo" :fromhost-ip, isequal, "10.1.5.3" *.* -?SplunkPipe And how would I easily specify many 10 IP's? I'm thinking it would be slick to be able to find a "netgroup" that has the member IP's I want then just have my selector match against that netgroup. Is that sort of magic possible? Unfortunately I'm using rsyslog with RHEL5 which is only v2.0.6. Examples appreciated. :) Ray From david at lang.hm Sat Mar 14 00:48:22 2009 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Mar 2009 16:48:22 -0700 (PDT) Subject: [rsyslog] Please Help! IPTables dumping to Console!!! In-Reply-To: References: Message-ID: On Fri, 13 Mar 2009, Michael Tant wrote: > Date: Fri, 13 Mar 2009 14:53:19 -0400 > From: Michael Tant > Reply-To: rsyslog-users > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Please Help! IPTables dumping to Console!!! > > I am running Fedora 10 linux with rsyslogd as my active logger. Recently I > have had an issue with my iptables LOG target output going to the console > and not going to the /var/log/messages file, even with the --log-level 6 > argument. I have halfway resolved this issue by editing the > /etc/rsyslog.conf file to include: kern.warning /var/log/iptables.log and > appending --log-level 4 to my LOG target rules. This caused the output to > go to the aforementioned file AND the console. > > I wish to still have the log data going to the iptables.log file, but wish > to stop the dump to the console. I have reviewed the rsyslog.conf file, and > the only statement which references /dev/console is kern.* but it is > commented out with #. I am tempted to remove this statement to see if it > helps, but I am unsure if this is safe, and furthermore convinced it will > not change the outcome as this line is nothing more than a comment. > > Is there something somewhere I am perhaps missing? I don't fully understand > the steps that move the log target output to the file, other than rsyslogd > is in the middle somewhere with the kernel. Any suggestions would be > greatly appreciated! Please send suggestion to mtant621 at chater.net there are a couple of possibilities here 1. you have something in /etc/rsyslog.conf that sends output to the console (or to root) the fix for this is to just remove/change the rsyslog.conf file 2. take a look in /etc/sysctl and see what you have log levels set to. some distros think that the iptables logs are important enough to spam everyone who's logged in, no matter what syslog is configured for. David Lang > I thank everyone for your help... > > Michael Tant > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From mtant621 at charter.net Sat Mar 14 16:19:13 2009 From: mtant621 at charter.net (Michael Tant) Date: Sat, 14 Mar 2009 11:19:13 -0400 Subject: [rsyslog] Still Dumping to Console Message-ID: I am still attempting to get the logging to stop dumping to console. IPtables is the only one doing this. I am currently logging to a different file by adding kern.warning /var/log/iptables.log to rsyslog.conf and --log-level 4 argument for the LOG targets. The data is making it to the file as specified, but is also being echoed to console if one of the tty's is displayed. It does echo to console in an X environment though, even a Konsole. I have check and found no logging references in the sysctl.conf file. I have completely removed the line: #kern.* /dev/console from the rsyslog.conf file, and have looked for auxilliary logging processes running and found none. I'm not skilled enough to fully understand the sysctl -a output so that could be the next possible culprit. If someone wants to take a look at that, rather than dumping it here and flooding you with huge email, you can find this at: http://fpaste.org/paste/6106 If there is something I'm overlooking or if there's some other way to fix this and force the correct behavior please let me know. As I don't quite have your skills with linux yet, please try to include as much information as you can, to assist with the fix. Again this is under Fedora 10. uname -a gives: Linux MTFedora 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb 23 13:21:22 EST 2009 i686 i686 i386 GNU/Linux if that gives any help. Thank You so much, Michael Tant From david at lang.hm Sat Mar 14 16:39:01 2009 From: david at lang.hm (david at lang.hm) Date: Sat, 14 Mar 2009 08:39:01 -0700 (PDT) Subject: [rsyslog] Still Dumping to Console In-Reply-To: References: Message-ID: On Sat, 14 Mar 2009, Michael Tant wrote: > I am still attempting to get the logging to stop dumping to console. > IPtables is the only one doing this. I am currently logging to a different > file by adding kern.warning /var/log/iptables.log to rsyslog.conf > and --log-level 4 argument for the LOG targets. The data is making it to > the file as specified, but is also being echoed to console if one of the > tty's is displayed. It does echo to console in an X environment though, > even a Konsole. I have check and found no logging references in the > sysctl.conf file. I have completely removed the line: #kern.* > /dev/console from the rsyslog.conf file, and have looked for auxilliary > logging processes running and found none. I'm not skilled enough to fully > understand the sysctl -a output so that could be the next possible culprit. > If someone wants to take a look at that, rather than dumping it here and > flooding you with huge email, you can find this at: > http://fpaste.org/paste/6106 > > If there is something I'm overlooking or if there's some other way to fix > this and force the correct behavior please let me know. As I don't quite > have your skills with linux yet, please try to include as much information > as you can, to assist with the fix. Again this is under Fedora 10. > uname -a gives: Linux MTFedora 2.6.27.19-170.2.35.fc10.i686 #1 SMP Mon Feb > 23 13:21:22 EST 2009 i686 i686 i386 GNU/Linux if that gives any help. my ubuntu desktop has the following in /etc/sysctl.conf # the following stops low-level messages on console kernel.printk = 4 4 1 7 From rgerhards at hq.adiscon.com Sun Mar 15 11:20:08 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sun, 15 Mar 2009 11:20:08 +0100 Subject: [rsyslog] webinar: "rsyslog templates" Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72000@GRFEXC.intern.adiscon.com> Hi all, in my effort to try video tutorials (or webinars as some pointed out ;)), I have now created a first live demo version, focused on templates. I hope it is useful: http://www.rsyslog.com/Article354.phtml Rainer From julianokyap at gmail.com Mon Mar 16 05:32:14 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 18:32:14 -1000 Subject: [rsyslog] Logging all messages from a remote server Message-ID: I'm having trouble logging ALL the syslog messages received from a server. I'm not sure if it's because it's from a non-standard piece of hardware (ie. not a Linux server). Logging to another server running syslogd works fine (but syslogd doesn't allow me to log messages from a remote server to a separate file and it's not my central syslogd server). I've tried several lines but none seem to work for me: if $fromhost == 'server' then /var/log/remote/server/all if $source == 'server' then /var/log/remote/server/all :FROMHOST, isequal, "server" /var/log/remote/server/all if $fromhost == 'server.domain.com' then /var/log/remote/server/all if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all .. Running Rsyslog 3.21.10. Thanks, Julian From david at lang.hm Mon Mar 16 06:16:04 2009 From: david at lang.hm (david at lang.hm) Date: Sun, 15 Mar 2009 22:16:04 -0700 (PDT) Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: References: Message-ID: On Sun, 15 Mar 2009, Julian Yap wrote: > I'm having trouble logging ALL the syslog messages received from a > server. I'm not sure if it's because it's from a non-standard piece > of hardware (ie. not a Linux server). Logging to another server > running syslogd works fine (but syslogd doesn't allow me to log > messages from a remote server to a separate file and it's not my > central syslogd server). > > I've tried several lines but none seem to work for me: > if $fromhost == 'server' then /var/log/remote/server/all > if $source == 'server' then /var/log/remote/server/all > :FROMHOST, isequal, "server" /var/log/remote/server/all > if $fromhost == 'server.domain.com' then /var/log/remote/server/all > if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all there are a few possible reasons that this could have problems is it that you have a high volume of logs and some just get dropped? if you just write everything to a file (*.* /var/log/test) does it have all the logs from this server? or is it missing some? do the logs from this server sometimes include the host and sometimes not? what is different between the logs that you match and the ones that you miss? David Lang From julianokyap at gmail.com Mon Mar 16 09:14:56 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 22:14:56 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: References: Message-ID: OK, I narrowed the issues down. Now I've faced strange issues like this before when using the $IncludeConfig directive. This is what I have just tested with in my /etc/rsyslog.conf file (and other lines) and it worked fine: ---- $IncludeConfig /etc/rsyslog.d/ :FROMHOST, isequal, "server" /var/log/remote/server/all ---- Now if I have a file /etc/rsyslog.d/testalert_for_another_server, things turn strange and only certain messages are logged from the first server.: ---- $ModLoad ommail $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $template DYNserver2, "/var/log/remote/server2.log" $template TraditionalFormatNoHostname,"%timegenerated% %syslogtag%%msg:::drop-last-lf%\n" if $hostname == 'server2.domain.com' then ?DYNserver2;TraditionalFormatNoHostname $ActionMailFrom rsyslog at domain.com $ActionMailTo server2_alert $template mailSubjectTestAlert,"INFO: Alert detected" $template mailBodyTestAlert,"Message is..." $ActionMailSubject mailSubjectTestAlert $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 if $hostname == 'server2.domain.com' and $msg contains 'Some message' then :ommail:;mailBodyTestAlert ---- Now if I add the contents of /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and remove file /etc/rsyslog.d/testalert_for_another_server) then things work fine... Now if I remove the previous changes to /etc/rsyslog.conf and modify /etc/rsyslog.d/testalert_for_another_server and remove the following lines then things work OK again: $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 - Julian On Sun, Mar 15, 2009 at 7:16 PM, wrote: > On Sun, 15 Mar 2009, Julian Yap wrote: > >> I'm having trouble logging ALL the syslog messages received from a >> server. ?I'm not sure if it's because it's from a non-standard piece >> of hardware (ie. not a Linux server). ?Logging to another server >> running syslogd works fine (but syslogd doesn't allow me to log >> messages from a remote server to a separate file and it's not my >> central syslogd server). >> >> I've tried several lines but none seem to work for me: >> if $fromhost == 'server' then /var/log/remote/server/all >> if $source == 'server' then /var/log/remote/server/all >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > > there are a few possible reasons that this could have problems > > is it that you have a high volume of logs and some just get dropped? > > if you just write everything to a file (*.* /var/log/test) does it have > all the logs from this server? or is it missing some? > > do the logs from this server sometimes include the host and sometimes not? > > what is different between the logs that you match and the ones that you > miss? > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 09:52:54 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 09:52:54 +0100 Subject: [rsyslog] Logging all messages from a remote server References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> The issue is that these statements $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 Modify the *next* action. So you need to specify them in front of the action. If you use the $includeConfig option, and have part of the action inside the include file and other parts (the statements) outside (or vice versa), you never know which action gets configured how. So place all of them together. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 9:15 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > OK, I narrowed the issues down. Now I've faced strange issues like > this before when using the $IncludeConfig directive. > > This is what I have just tested with in my /etc/rsyslog.conf file (and > other lines) and it worked fine: > ---- > $IncludeConfig /etc/rsyslog.d/ > :FROMHOST, isequal, "server" /var/log/remote/server/all > ---- > > Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > things turn strange and only certain messages are logged from the > first server.: > ---- > $ModLoad ommail > > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > $template DYNserver2, "/var/log/remote/server2.log" > $template TraditionalFormatNoHostname,"%timegenerated% > %syslogtag%%msg:::drop-last-lf%\n" > > if $hostname == 'server2.domain.com' then > ?DYNserver2;TraditionalFormatNoHostname > > $ActionMailFrom rsyslog at domain.com > $ActionMailTo server2_alert > $template mailSubjectTestAlert,"INFO: Alert detected" > $template mailBodyTestAlert,"Message is..." > $ActionMailSubject mailSubjectTestAlert > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > if $hostname == 'server2.domain.com' and $msg contains 'Some message' > then :ommail:;mailBodyTestAlert > ---- > > Now if I add the contents of > /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and > remove file /etc/rsyslog.d/testalert_for_another_server) then things > work fine... > > Now if I remove the previous changes to /etc/rsyslog.conf and modify > /etc/rsyslog.d/testalert_for_another_server and remove the following > lines then things work OK again: > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > > - Julian > > > On Sun, Mar 15, 2009 at 7:16 PM, wrote: > > On Sun, 15 Mar 2009, Julian Yap wrote: > > > >> I'm having trouble logging ALL the syslog messages received from a > >> server. ?I'm not sure if it's because it's from a non-standard piece > >> of hardware (ie. not a Linux server). ?Logging to another server > >> running syslogd works fine (but syslogd doesn't allow me to log > >> messages from a remote server to a separate file and it's not my > >> central syslogd server). > >> > >> I've tried several lines but none seem to work for me: > >> if $fromhost == 'server' then /var/log/remote/server/all > >> if $source == 'server' then /var/log/remote/server/all > >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all > >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > > > > there are a few possible reasons that this could have problems > > > > is it that you have a high volume of logs and some just get dropped? > > > > if you just write everything to a file (*.* /var/log/test) does it > have > > all the logs from this server? or is it missing some? > > > > do the logs from this server sometimes include the host and sometimes > not? > > > > what is different between the logs that you match and the ones that > you > > miss? > > > > David Lang > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Mon Mar 16 10:04:37 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 23:04:37 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> Message-ID: Rainer, Would you recommend against using $includeConfig? In that case, it tends to lead to more unknown config issues. - Julian On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards wrote: > The issue is that these statements > > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > > Modify the *next* action. So you need to specify them in front of the action. > If you use the $includeConfig option, and have part of the action inside the > include file and other parts (the statements) outside (or vice versa), you > never know which action gets configured how. So place all of them together. > > HTH > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> Sent: Monday, March 16, 2009 9:15 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> OK, I narrowed the issues down. ?Now I've faced strange issues like >> this before when using the $IncludeConfig directive. >> >> This is what I have just tested with in my /etc/rsyslog.conf file (and >> other lines) and it worked fine: >> ---- >> $IncludeConfig /etc/rsyslog.d/ >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all >> ---- >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, >> things turn strange and only certain messages are logged from the >> first server.: >> ---- >> $ModLoad ommail >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat >> >> $template DYNserver2, "/var/log/remote/server2.log" >> $template TraditionalFormatNoHostname,"%timegenerated% >> %syslogtag%%msg:::drop-last-lf%\n" >> >> if $hostname == 'server2.domain.com' then >> ?DYNserver2;TraditionalFormatNoHostname >> >> $ActionMailFrom rsyslog at domain.com >> $ActionMailTo server2_alert >> $template mailSubjectTestAlert,"INFO: Alert detected" >> $template mailBodyTestAlert,"Message is..." >> $ActionMailSubject mailSubjectTestAlert >> $ActionExecOnlyOnceEveryInterval 300 >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> $ActionExecOnlyEveryNthTime 3 >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some message' >> then :ommail:;mailBodyTestAlert >> ---- >> >> Now if I add the contents of >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf (and >> remove file /etc/rsyslog.d/testalert_for_another_server) then things >> work fine... >> >> Now if I remove the previous changes to /etc/rsyslog.conf and modify >> /etc/rsyslog.d/testalert_for_another_server and remove the following >> lines then things work OK again: >> $ActionExecOnlyOnceEveryInterval 300 >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> $ActionExecOnlyEveryNthTime 3 >> >> >> - Julian >> >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: >> > On Sun, 15 Mar 2009, Julian Yap wrote: >> > >> >> I'm having trouble logging ALL the syslog messages received from a >> >> server. ?I'm not sure if it's because it's from a non-standard piece >> >> of hardware (ie. not a Linux server). ?Logging to another server >> >> running syslogd works fine (but syslogd doesn't allow me to log >> >> messages from a remote server to a separate file and it's not my >> >> central syslogd server). >> >> >> >> I've tried several lines but none seem to work for me: >> >> if $fromhost == 'server' then /var/log/remote/server/all >> >> if $source == 'server' then /var/log/remote/server/all >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> >> if $fromhost == 'server.domain.com' then /var/log/remote/server/all >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all >> > >> > there are a few possible reasons that this could have problems >> > >> > is it that you have a high volume of logs and some just get dropped? >> > >> > if you just write everything to a file (*.* /var/log/test) does it >> have >> > all the logs from this server? or is it missing some? >> > >> > do the logs from this server sometimes include the host and sometimes >> not? >> > >> > what is different between the logs that you match and the ones that >> you >> > miss? >> > >> > David Lang >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 10:08:36 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 10:08:36 +0100 Subject: [rsyslog] Logging all messages from a remote server References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 10:05 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > Rainer, > > Would you recommend against using $includeConfig? In that case, it > tends to lead to more unknown config issues. No, but do not split config directives that need to go together over several places. You need to put this together # this starts the definition of a single action $ActionExecOnlyOnceEveryInterval 300 $ActionExecOnlyEveryNthTimeTimeout 1200 $ActionExecOnlyEveryNthTime 3 $... *.* action #this ends it So you need to put everything together. If you rip it apart, you will get undefined results. This is - to phrase it politely - not very well documented. You need to read the fine print, most of the $Action... params modify the *next* action - NOT *all* actions. So it is vitally important where they occur. Will try to make this clear as soon as I have a bit more time. Rainer > > - Julian > > On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards > wrote: > > The issue is that these statements > > > > $ActionExecOnlyOnceEveryInterval 300 > > $ActionExecOnlyEveryNthTimeTimeout 1200 > > $ActionExecOnlyEveryNthTime 3 > > > > Modify the *next* action. So you need to specify them in front of the > action. > > If you use the $includeConfig option, and have part of the action > inside the > > include file and other parts (the statements) outside (or vice > versa), you > > never know which action gets configured how. So place all of them > together. > > > > HTH > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> Sent: Monday, March 16, 2009 9:15 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> > >> OK, I narrowed the issues down. ?Now I've faced strange issues like > >> this before when using the $IncludeConfig directive. > >> > >> This is what I have just tested with in my /etc/rsyslog.conf file > (and > >> other lines) and it worked fine: > >> ---- > >> $IncludeConfig /etc/rsyslog.d/ > >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all > >> ---- > >> > >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > >> things turn strange and only certain messages are logged from the > >> first server.: > >> ---- > >> $ModLoad ommail > >> > >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat > >> > >> $template DYNserver2, "/var/log/remote/server2.log" > >> $template TraditionalFormatNoHostname,"%timegenerated% > >> %syslogtag%%msg:::drop-last-lf%\n" > >> > >> if $hostname == 'server2.domain.com' then > >> ?DYNserver2;TraditionalFormatNoHostname > >> > >> $ActionMailFrom rsyslog at domain.com > >> $ActionMailTo server2_alert > >> $template mailSubjectTestAlert,"INFO: Alert detected" > >> $template mailBodyTestAlert,"Message is..." > >> $ActionMailSubject mailSubjectTestAlert > >> $ActionExecOnlyOnceEveryInterval 300 > >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> $ActionExecOnlyEveryNthTime 3 > >> > >> if $hostname == 'server2.domain.com' and $msg contains 'Some > message' > >> then :ommail:;mailBodyTestAlert > >> ---- > >> > >> Now if I add the contents of > >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf > (and > >> remove file /etc/rsyslog.d/testalert_for_another_server) then things > >> work fine... > >> > >> Now if I remove the previous changes to /etc/rsyslog.conf and modify > >> /etc/rsyslog.d/testalert_for_another_server and remove the following > >> lines then things work OK again: > >> $ActionExecOnlyOnceEveryInterval 300 > >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> $ActionExecOnlyEveryNthTime 3 > >> > >> > >> - Julian > >> > >> > >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: > >> > On Sun, 15 Mar 2009, Julian Yap wrote: > >> > > >> >> I'm having trouble logging ALL the syslog messages received from > a > >> >> server. ?I'm not sure if it's because it's from a non-standard > piece > >> >> of hardware (ie. not a Linux server). ?Logging to another server > >> >> running syslogd works fine (but syslogd doesn't allow me to log > >> >> messages from a remote server to a separate file and it's not my > >> >> central syslogd server). > >> >> > >> >> I've tried several lines but none seem to work for me: > >> >> if $fromhost == 'server' then /var/log/remote/server/all > >> >> if $source == 'server' then /var/log/remote/server/all > >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> >> if $fromhost == 'server.domain.com' then > /var/log/remote/server/all > >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all > >> > > >> > there are a few possible reasons that this could have problems > >> > > >> > is it that you have a high volume of logs and some just get > dropped? > >> > > >> > if you just write everything to a file (*.* /var/log/test) does it > >> have > >> > all the logs from this server? or is it missing some? > >> > > >> > do the logs from this server sometimes include the host and > sometimes > >> not? > >> > > >> > what is different between the logs that you match and the ones > that > >> you > >> > miss? > >> > > >> > David Lang > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Mon Mar 16 10:18:23 2009 From: julianokyap at gmail.com (Julian Yap) Date: Sun, 15 Mar 2009 23:18:23 -1000 Subject: [rsyslog] Logging all messages from a remote server In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> Message-ID: Thanks all. My config is working fine now. I can take some of the blame for requesting the $ActionExecOnlyEveryNthTime* params in the first place :P. Just to shed some light, my previous understanding (or what I initially gathered from the docs) was that the $Action params needed to just be in a block and the order of params didn't matter. So: #start Action $Action... $Action... $Action... #end Action So that was just what I gathered in my head. But it's all clear now. - Julian On Sun, Mar 15, 2009 at 11:08 PM, Rainer Gerhards wrote: >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> Sent: Monday, March 16, 2009 10:05 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> Rainer, >> >> Would you recommend against using $includeConfig? ?In that case, it >> tends to lead to more unknown config issues. > > No, but do not split config directives that need to go together over several > places. You need to put this together > > # this starts the definition of a single action > $ActionExecOnlyOnceEveryInterval 300 > $ActionExecOnlyEveryNthTimeTimeout 1200 > $ActionExecOnlyEveryNthTime 3 > $... > *.* ?action > #this ends it > > So you need to put everything together. If you rip it apart, you will get > undefined results. > > This is - to phrase it politely - not very well documented. You need to read > the fine print, most of the $Action... params modify the *next* action - NOT > *all* actions. So it is vitally important where they occur. > > Will try to make this clear as soon as I have a bit more time. > > > Rainer >> >> - Julian >> >> On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards >> wrote: >> > The issue is that these statements >> > >> > $ActionExecOnlyOnceEveryInterval 300 >> > $ActionExecOnlyEveryNthTimeTimeout 1200 >> > $ActionExecOnlyEveryNthTime 3 >> > >> > Modify the *next* action. So you need to specify them in front of the >> action. >> > If you use the $includeConfig option, and have part of the action >> inside the >> > include file and other parts (the statements) outside (or vice >> versa), you >> > never know which action gets configured how. So place all of them >> together. >> > >> > HTH >> > Rainer >> > >> >> -----Original Message----- >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> >> bounces at lists.adiscon.com] On Behalf Of Julian Yap >> >> Sent: Monday, March 16, 2009 9:15 AM >> >> To: rsyslog-users >> >> Subject: Re: [rsyslog] Logging all messages from a remote server >> >> >> >> OK, I narrowed the issues down. ?Now I've faced strange issues like >> >> this before when using the $IncludeConfig directive. >> >> >> >> This is what I have just tested with in my /etc/rsyslog.conf file >> (and >> >> other lines) and it worked fine: >> >> ---- >> >> $IncludeConfig /etc/rsyslog.d/ >> >> :FROMHOST, isequal, "server" ? ? ? ? ? ? /var/log/remote/server/all >> >> ---- >> >> >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, >> >> things turn strange and only certain messages are logged from the >> >> first server.: >> >> ---- >> >> $ModLoad ommail >> >> >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat >> >> >> >> $template DYNserver2, "/var/log/remote/server2.log" >> >> $template TraditionalFormatNoHostname,"%timegenerated% >> >> %syslogtag%%msg:::drop-last-lf%\n" >> >> >> >> if $hostname == 'server2.domain.com' then >> >> ?DYNserver2;TraditionalFormatNoHostname >> >> >> >> $ActionMailFrom rsyslog at domain.com >> >> $ActionMailTo server2_alert >> >> $template mailSubjectTestAlert,"INFO: Alert detected" >> >> $template mailBodyTestAlert,"Message is..." >> >> $ActionMailSubject mailSubjectTestAlert >> >> $ActionExecOnlyOnceEveryInterval 300 >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> >> $ActionExecOnlyEveryNthTime 3 >> >> >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some >> message' >> >> then :ommail:;mailBodyTestAlert >> >> ---- >> >> >> >> Now if I add the contents of >> >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf >> (and >> >> remove file /etc/rsyslog.d/testalert_for_another_server) then things >> >> work fine... >> >> >> >> Now if I remove the previous changes to /etc/rsyslog.conf and modify >> >> /etc/rsyslog.d/testalert_for_another_server and remove the following >> >> lines then things work OK again: >> >> $ActionExecOnlyOnceEveryInterval 300 >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 >> >> $ActionExecOnlyEveryNthTime 3 >> >> >> >> >> >> - Julian >> >> >> >> >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: >> >> > On Sun, 15 Mar 2009, Julian Yap wrote: >> >> > >> >> >> I'm having trouble logging ALL the syslog messages received from >> a >> >> >> server. ?I'm not sure if it's because it's from a non-standard >> piece >> >> >> of hardware (ie. not a Linux server). ?Logging to another server >> >> >> running syslogd works fine (but syslogd doesn't allow me to log >> >> >> messages from a remote server to a separate file and it's not my >> >> >> central syslogd server). >> >> >> >> >> >> I've tried several lines but none seem to work for me: >> >> >> if $fromhost == 'server' then /var/log/remote/server/all >> >> >> if $source == 'server' then /var/log/remote/server/all >> >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all >> >> >> if $fromhost == 'server.domain.com' then >> /var/log/remote/server/all >> >> >> if $fromhost-ip == '192.168.0.60' then /var/log/remote/server/all >> >> > >> >> > there are a few possible reasons that this could have problems >> >> > >> >> > is it that you have a high volume of logs and some just get >> dropped? >> >> > >> >> > if you just write everything to a file (*.* /var/log/test) does it >> >> have >> >> > all the logs from this server? or is it missing some? >> >> > >> >> > do the logs from this server sometimes include the host and >> sometimes >> >> not? >> >> > >> >> > what is different between the logs that you match and the ones >> that >> >> you >> >> > miss? >> >> > >> >> > David Lang >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com >> >> > >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 16 10:22:47 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 10:22:47 +0100 Subject: [rsyslog] Logging all messages from a remote server References: <9B6E2A8877C38245BFB15CC491A11DA72005@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA72006@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72007@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Julian Yap > Sent: Monday, March 16, 2009 10:18 AM > To: rsyslog-users > Subject: Re: [rsyslog] Logging all messages from a remote server > > Thanks all. My config is working fine now. > > I can take some of the blame for requesting the > $ActionExecOnlyEveryNthTime* params in the first place :P. > > Just to shed some light, my previous understanding (or what I > initially gathered from the docs) was that the $Action params needed > to just be in a block and the order of params didn't matter. > > So: > #start Action > $Action... > $Action... > $Action... > #end Action > > So that was just what I gathered in my head. But it's all clear now. Well, the order doesn't matter BUT (!) above you do NOT define an action - because the action itself is missing! So whatever action comes next, it will receive these parameters. Rainer > > - Julian > > On Sun, Mar 15, 2009 at 11:08 PM, Rainer Gerhards > wrote: > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> Sent: Monday, March 16, 2009 10:05 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> > >> Rainer, > >> > >> Would you recommend against using $includeConfig? ?In that case, it > >> tends to lead to more unknown config issues. > > > > No, but do not split config directives that need to go together over > several > > places. You need to put this together > > > > # this starts the definition of a single action > > $ActionExecOnlyOnceEveryInterval 300 > > $ActionExecOnlyEveryNthTimeTimeout 1200 > > $ActionExecOnlyEveryNthTime 3 > > $... > > *.* ?action > > #this ends it > > > > So you need to put everything together. If you rip it apart, you will > get > > undefined results. > > > > This is - to phrase it politely - not very well documented. You need > to read > > the fine print, most of the $Action... params modify the *next* > action - NOT > > *all* actions. So it is vitally important where they occur. > > > > Will try to make this clear as soon as I have a bit more time. > > > > > > Rainer > >> > >> - Julian > >> > >> On Sun, Mar 15, 2009 at 10:52 PM, Rainer Gerhards > >> wrote: > >> > The issue is that these statements > >> > > >> > $ActionExecOnlyOnceEveryInterval 300 > >> > $ActionExecOnlyEveryNthTimeTimeout 1200 > >> > $ActionExecOnlyEveryNthTime 3 > >> > > >> > Modify the *next* action. So you need to specify them in front of > the > >> action. > >> > If you use the $includeConfig option, and have part of the action > >> inside the > >> > include file and other parts (the statements) outside (or vice > >> versa), you > >> > never know which action gets configured how. So place all of them > >> together. > >> > > >> > HTH > >> > Rainer > >> > > >> >> -----Original Message----- > >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> >> bounces at lists.adiscon.com] On Behalf Of Julian Yap > >> >> Sent: Monday, March 16, 2009 9:15 AM > >> >> To: rsyslog-users > >> >> Subject: Re: [rsyslog] Logging all messages from a remote server > >> >> > >> >> OK, I narrowed the issues down. ?Now I've faced strange issues > like > >> >> this before when using the $IncludeConfig directive. > >> >> > >> >> This is what I have just tested with in my /etc/rsyslog.conf file > >> (and > >> >> other lines) and it worked fine: > >> >> ---- > >> >> $IncludeConfig /etc/rsyslog.d/ > >> >> :FROMHOST, isequal, "server" > /var/log/remote/server/all > >> >> ---- > >> >> > >> >> Now if I have a file /etc/rsyslog.d/testalert_for_another_server, > >> >> things turn strange and only certain messages are logged from the > >> >> first server.: > >> >> ---- > >> >> $ModLoad ommail > >> >> > >> >> $ActionFileDefaultTemplate ? ? ?RSYSLOG_TraditionalFileFormat > >> >> > >> >> $template DYNserver2, "/var/log/remote/server2.log" > >> >> $template TraditionalFormatNoHostname,"%timegenerated% > >> >> %syslogtag%%msg:::drop-last-lf%\n" > >> >> > >> >> if $hostname == 'server2.domain.com' then > >> >> ?DYNserver2;TraditionalFormatNoHostname > >> >> > >> >> $ActionMailFrom rsyslog at domain.com > >> >> $ActionMailTo server2_alert > >> >> $template mailSubjectTestAlert,"INFO: Alert detected" > >> >> $template mailBodyTestAlert,"Message is..." > >> >> $ActionMailSubject mailSubjectTestAlert > >> >> $ActionExecOnlyOnceEveryInterval 300 > >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> >> $ActionExecOnlyEveryNthTime 3 > >> >> > >> >> if $hostname == 'server2.domain.com' and $msg contains 'Some > >> message' > >> >> then :ommail:;mailBodyTestAlert > >> >> ---- > >> >> > >> >> Now if I add the contents of > >> >> /etc/rsyslog.d/testalert_for_another_server to /etc/rsyslog.conf > >> (and > >> >> remove file /etc/rsyslog.d/testalert_for_another_server) then > things > >> >> work fine... > >> >> > >> >> Now if I remove the previous changes to /etc/rsyslog.conf and > modify > >> >> /etc/rsyslog.d/testalert_for_another_server and remove the > following > >> >> lines then things work OK again: > >> >> $ActionExecOnlyOnceEveryInterval 300 > >> >> $ActionExecOnlyEveryNthTimeTimeout 1200 > >> >> $ActionExecOnlyEveryNthTime 3 > >> >> > >> >> > >> >> - Julian > >> >> > >> >> > >> >> On Sun, Mar 15, 2009 at 7:16 PM, ? wrote: > >> >> > On Sun, 15 Mar 2009, Julian Yap wrote: > >> >> > > >> >> >> I'm having trouble logging ALL the syslog messages received > from > >> a > >> >> >> server. ?I'm not sure if it's because it's from a non-standard > >> piece > >> >> >> of hardware (ie. not a Linux server). ?Logging to another > server > >> >> >> running syslogd works fine (but syslogd doesn't allow me to > log > >> >> >> messages from a remote server to a separate file and it's not > my > >> >> >> central syslogd server). > >> >> >> > >> >> >> I've tried several lines but none seem to work for me: > >> >> >> if $fromhost == 'server' then /var/log/remote/server/all > >> >> >> if $source == 'server' then /var/log/remote/server/all > >> >> >> :FROMHOST, isequal, "server" /var/log/remote/server/all > >> >> >> if $fromhost == 'server.domain.com' then > >> /var/log/remote/server/all > >> >> >> if $fromhost-ip == '192.168.0.60' then > /var/log/remote/server/all > >> >> > > >> >> > there are a few possible reasons that this could have problems > >> >> > > >> >> > is it that you have a high volume of logs and some just get > >> dropped? > >> >> > > >> >> > if you just write everything to a file (*.* /var/log/test) does > it > >> >> have > >> >> > all the logs from this server? or is it missing some? > >> >> > > >> >> > do the logs from this server sometimes include the host and > >> sometimes > >> >> not? > >> >> > > >> >> > what is different between the logs that you match and the ones > >> that > >> >> you > >> >> > miss? > >> >> > > >> >> > David Lang > >> >> > _______________________________________________ > >> >> > rsyslog mailing list > >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> > http://www.rsyslog.com > >> >> > > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Mon Mar 16 16:34:09 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Mon, 16 Mar 2009 16:34:09 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean Message-ID: <49BE7171.3090601@1und1.de> Hi, I've configured rsyslog to use relp as transport protocol. sw version: rsyslog-relp-3.21.3-4 and rsyslog-3.21.3-4. in the log I see these messages: 2009-03-16T16:12:10.769408+01:00 zeus-log01-2 rsyslogd: [origin software="rsyslogd" swVersion="3.21.3" x-pid="3239" x-info="http://www.rsyslog.com"] restart 2009-03-16T16:12:10.769447+01:00 zeus-log01-2 rsyslogd: error -2077 trying to add listener 2009-03-16T16:12:10.769458+01:00 zeus-log01-2 rsyslogd: the last error occured in /data/etc/rsyslog/rsyslog.conf, line 6 2009-03-16T16:12:10.769470+01:00 zeus-log01-2 rsyslogd: the last error occured in /data/etc/rsyslog/rsyslog.conf, line 7 The config line in question read: ------snip # Global $ModLoad imudp.so $ModLoad imtcp.so $ModLoad imrelp.so $UDPServerRun 514 $InputTCPServerRun 514 <-- line 6 $InputRELPServerRun 2514 <-- line 7 $DirCreateMode 0755 -------snap netstat -an| grep 514 shows all configured udp and tcp ports open. So where can I find a description of error -2077? Thanks in advance Thomas From rgerhards at hq.adiscon.com Mon Mar 16 16:39:21 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 16:39:21 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean References: <49BE7171.3090601@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> There should be informtion on that error on the web, but 2077 is "could not bind to port". A short reference can be found in git in file ./runtime/rsyslog.h Not sure where it orginates from in this case... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Monday, March 16, 2009 4:34 PM > To: rsyslog-users > Subject: [rsyslog] what does error -2077 trying to add listener mean > > Hi, > > I've configured rsyslog to use relp as transport protocol. > sw version: rsyslog-relp-3.21.3-4 and rsyslog-3.21.3-4. > > in the log I see these messages: > 2009-03-16T16:12:10.769408+01:00 zeus-log01-2 rsyslogd: [origin > software="rsyslogd" swVersion="3.21.3" x-pid="3239" > x-info="http://www.rsyslog.com"] restart > 2009-03-16T16:12:10.769447+01:00 zeus-log01-2 rsyslogd: error -2077 > trying to add listener > 2009-03-16T16:12:10.769458+01:00 zeus-log01-2 rsyslogd: the last error > occured in /data/etc/rsyslog/rsyslog.conf, line 6 > 2009-03-16T16:12:10.769470+01:00 zeus-log01-2 rsyslogd: the last error > occured in /data/etc/rsyslog/rsyslog.conf, line 7 > > The config line in question read: > > ------snip > # Global > $ModLoad imudp.so > $ModLoad imtcp.so > $ModLoad imrelp.so > $UDPServerRun 514 > $InputTCPServerRun 514 <-- line 6 > $InputRELPServerRun 2514 <-- line 7 > > $DirCreateMode 0755 > > -------snap > > netstat -an| grep 514 shows all configured udp and tcp ports open. > > So where can I find a description of error -2077? > > Thanks in advance > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From thomas.mieslinger at 1und1.de Mon Mar 16 16:56:31 2009 From: thomas.mieslinger at 1und1.de (Thomas Mieslinger) Date: Mon, 16 Mar 2009 16:56:31 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> References: <49BE7171.3090601@1und1.de> <9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> Message-ID: <49BE76AF.4030608@1und1.de> Hi Rainer, there is only one place where RS_RET_COULD_NOT_BIND is returned: runtime/nsd_ptcp.c numSocks = 0; /* num of sockets counter at start of array */ for(r = res; r != NULL ; r = r->ai_next) { sock = socket(r->ai_family, r->ai_socktype, r->ai_protocol); [ lots of magic ] } if(numSocks == 0) { dbgprintf("No TCP listen sockets could successfully be initialized"); ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); } I have no idea why the OS reports the Sockets open and messages get received, maybe there is a minor problem in the code, but somehow it works... Thomas From rgerhards at hq.adiscon.com Mon Mar 16 16:59:52 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 16:59:52 +0100 Subject: [rsyslog] what does error -2077 trying to add listener mean References: <49BE7171.3090601@1und1.de><9B6E2A8877C38245BFB15CC491A11DA7200E@GRFEXC.intern.adiscon.com> <49BE76AF.4030608@1und1.de> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7200F@GRFEXC.intern.adiscon.com> This sounds like some quirk with IPv6... > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Thomas Mieslinger > Sent: Monday, March 16, 2009 4:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] what does error -2077 trying to add listener > mean > > Hi Rainer, > > there is only one place where RS_RET_COULD_NOT_BIND is returned: > > runtime/nsd_ptcp.c > > numSocks = 0; /* num of sockets counter at start of array */ > for(r = res; r != NULL ; r = r->ai_next) { > sock = socket(r->ai_family, r->ai_socktype, r- > >ai_protocol); > > [ lots of magic ] > > } > > if(numSocks == 0) { > dbgprintf("No TCP listen sockets could successfully be > initialized"); > ABORT_FINALIZE(RS_RET_COULD_NOT_BIND); > } > > I have no idea why the OS reports the Sockets open and messages get > received, maybe there is a minor problem in the code, but somehow it > works... > > Thomas > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From rgerhards at hq.adiscon.com Mon Mar 16 17:53:40 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Mar 2009 17:53:40 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200902271848.56481.Luis.Fernando.Munoz.Mejias@cern.ch><1236001365.28865.44.camel@rf10up.intern.adiscon.com><200903031528.59095.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> Sorry for the delay, it is currently quite busy here at my end :( I have now created a very rough skeleton template output module. You need to pull from git. It is contained in the master branch. So far, it does not perform useful work. I was a bit hesitant to add much more description, because I think this can either be brief and not matching what you need - or very elaborate (bbok-like), for what I currently do not have enough time. I suggest that you have a look at the template module, and then we simply try to get this going. It would be good if you could ask questions or tell me what needs to be placed inside the module. Or I can create yet another skeleton, based on ommysql, that has a bit more logic so that you can fill in the initial Oracle functionality. That will not offer superior performance, but I think it would be a good starting point to pursue the rest of this project. Please let me know what you think. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Tuesday, March 03, 2009 3:26 PM > To: rsyslog-users > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > Just one quick note, more following: > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > > Sent: Tuesday, March 03, 2009 3:29 PM > > To: rsyslog at lists.adiscon.com > > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > > > Hi there. > > > > > > As I said, I need **excellent** performance. I definitely need > > batch > > > > operations, the ability to prepare the statements given as > > arguments > > > > on the configuration file, and not to commit entries one by one, > > but > > > > after a number of entries are ready or (better) after some not so > > > > small time. According to the advise I got from experts around > here, > > > > I'll have to use Oracle Call Interface for this module, I don't > > know > > > > if there are any licensing issues. > > > > > > I can't comment on the licensing issue, I simply don't know what > > > Oracle demands. > > > > I'm not sure how GPL-compatible it is to link to already existing > > proprietary code. Anyways, first I code, then we test, then we (you, > > actually) decide the legal aspects. > > Actually, not me ;) I leave this risk to the user. If someone pays the > legal counselor, I'll add his POV to the project doc. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From julianokyap at gmail.com Tue Mar 17 20:44:57 2009 From: julianokyap at gmail.com (Julian Yap) Date: Tue, 17 Mar 2009 09:44:57 -1000 Subject: [rsyslog] Dynamic remote log files Message-ID: I have the following set up to generate Dynamic remote log files. $template DYNmessages, "/var/log/remote/%HOSTNAME%/messages" *.info,mail.none,authpriv.none,cron.none ?DYNmessages Unfortunately some devices log poorly without the hostname for some syslog messages. This means I'm ending up with lots of useless directories in /var/log/remote. If I log everything from a server to a file then it works fine: if $fromhost == 'server' then /var/log/remote/server/all As you can see the difference in file sizes as syslog messages are lost: # ls -l /var/log/remote/server/ total 1724 -rw------- 1 root root 980053 Mar 17 08:57 all -rw------- 1 root root 773533 Mar 17 08:57 messages I guess, I'm looking for config suggestions on setting up more robust dynamic logging for remote hosts. - Julian From aoz.syn at gmail.com Tue Mar 17 20:57:14 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 17 Mar 2009 13:57:14 -0600 Subject: [rsyslog] Dynamic remote log files In-Reply-To: References: Message-ID: <4255c2570903171257w4801cc3co8998ca883b5ae78@mail.gmail.com> On Tue, Mar 17, 2009 at 13:44, Julian Yap wrote: > I guess, I'm looking for config suggestions on setting up more robust > dynamic logging for remote hosts. The single most robust host-based structure I've found to use is 'fromhost-ip'. It's locally "generated" by the rsyslog daemon from the receiving socket and isn't affected by any of the message content. From Luis.Fernando.Munoz.Mejias at cern.ch Wed Mar 18 10:53:34 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Wed, 18 Mar 2009 10:53:34 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> Message-ID: <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> El Lunes, 16 de Marzo de 2009 17:53, Rainer Gerhards escribi?: > Please let me know what you think. I just came back from a week of holidays, I'm reviewing the skeleton, which looks pretty comprehensive. Thanks! -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Wed Mar 18 11:04:19 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 18 Mar 2009 11:04:19 +0100 Subject: [rsyslog] Documentation on writing rsyslog modules? References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA71F25@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA72010@GRFEXC.intern.adiscon.com> <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> When you are ready, I'd actually suggest that I create an "omoracle" git branch for you and do place a copy of ommysql into it. This, together with the comments from omtemplate, would probably one way to get a (non-optimal) quick start. I would suggest that we build a very basic oracle driver first and after we see it works well, then look into the performance optimization. Let me know what you think (and when you have time). I could actually create what I have proposed with little delay once you give a go. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Wednesday, March 18, 2009 10:54 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Documentation on writing rsyslog modules? > > El Lunes, 16 de Marzo de 2009 17:53, Rainer Gerhards escribi?: > > Please let me know what you think. > > I just came back from a week of holidays, I'm reviewing the skeleton, > which looks pretty comprehensive. Thanks! > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From kenneho.ndu at gmail.com Thu Mar 19 11:51:43 2009 From: kenneho.ndu at gmail.com (Kenneth Holter) Date: Thu, 19 Mar 2009 11:51:43 +0100 Subject: [rsyslog] rsyslog TCP session closing Message-ID: Hi. My rsyslog log host keep getting these messages in syslog: *rsyslogd:TCP session 66 will be closed, error ignored * The session numer (i.e. 66 in this case) varies. Are these messages of any importance? I'm guessing the sessions are closed due to being idle, and that the session will be re-established when the next syslog message are ready to be sent from the client. Regards, Kenneth From rgerhards at hq.adiscon.com Thu Mar 19 12:13:56 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 19 Mar 2009 12:13:56 +0100 Subject: [rsyslog] omfile reliability Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD37@GRFEXC.intern.adiscon.com> Hi all, as mentioned in some past posts, omfile did not really care if log data made it into the file system. The overall reaction was "don't care, discard message". I have now improved the situation very much (at least I hope so ;)). There is currently an experimental git branch omfile-errHandler which properly suspends the action if something goes wrong. The only thing it currently does not do is truncate partially written lines. I'll save this for some later release when I revamp to module as whole. I plan to merge this change into the main development branch soon and then do a new devel release. If you would like to play with the current version, I of course would appreciate that. If so, please let me know your results. Also, I found one strange thing while testing with the cifs (SMB) handler. It does not properly return a failure state, so I currently have no clue how to detect a failure condition in that case. Below, I post some excerpt from a forum thread related to the work [1]. If you happen to have any suggestions, please let me know. ===== good news and bad news: I have found a bug inside the code, and been able to fix that (not yet committed). However, I tried with the smb redirector (don't have nfs at hand) and it acks the writes, but does not ensure data is actually put onto the remote site. So there probably is no way to make sure we really have the data. Maybe the situation is better with NFS. below some excerpts from my twitter stream: # i have lots of garbage inside the log when I reconnect the network... looks like cifs driver can not really handle this situation1 minute ago from twhirl # it is interesting to see how the smb driver continues to accept data (at a very slow rate) while the network is off....9 minutes ago from twhirl # #rsyslog: issue is more complicated than I thought - probably a bug in dynafile creation processabout 1 hour ago from twhirl # ok, think I got a bug. FD is not set to indicated "closed" after actual close call - can lead to endless loopabout 2 hours ago from twhirl # as soon as I enter a new message, the missing content *is* writtenabout 2 hours ago from twhirl # after disconnect, nothing is written...about 2 hours ago from twhirl # very interesting... I get successful returns from write() to the network file - with plugged cable, lazy write, I guess...about 2 hours ago from twhirl # #rsyslog: OS buffering plays a big role in network-file retries - on the initial tries I do not see any error code at all! (w/o cable!!)about 2 hours ago from twhirl ==== Thanks, Rainer [1] http://kb.monitorware.com/log-to-nfs-and-buffer-if-unavailable-t8963-30.html# p15732 From Jefferson.Cowart at libraries.claremont.edu Thu Mar 19 23:18:00 2009 From: Jefferson.Cowart at libraries.claremont.edu (Jefferson Cowart) Date: Thu, 19 Mar 2009 15:18:00 -0700 Subject: [rsyslog] Separating Log files based on partial IP match Message-ID: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> I'm new to rsyslog, and I'm trying to set it up to centralize logging for a number of devices on my network. I'd like for it to log anything from my network switch to a single log file, my printers to another log file, etc. I'm able to separate the devices based on their IP address (e.g. my switches are in one IP subnet and my printers in another.) I see how to do per device logging on http://www.rsyslog.com/Article60.phtml, but I don't see a way to adjust that to do it based on IP subnet or anything like that. Unfortunately it looks like both FROMHOST and HOSTNAME are names not IPs, so it's not even clear if I could filter on that. Any help would be appreciated. Thanks. -- Thank You Jefferson Cowart Network and Systems Administrator Claremont University Consortium From david at lang.hm Fri Mar 20 00:44:36 2009 From: david at lang.hm (david at lang.hm) Date: Thu, 19 Mar 2009 16:44:36 -0700 (PDT) Subject: [rsyslog] Separating Log files based on partial IP match In-Reply-To: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> References: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> Message-ID: On Thu, 19 Mar 2009, Jefferson Cowart wrote: > I'm new to rsyslog, and I'm trying to set it up to centralize logging > for a number of devices on my network. I'd like for it to log anything > from my network switch to a single log file, my printers to another log > file, etc. I'm able to separate the devices based on their IP address > (e.g. my switches are in one IP subnet and my printers in another.) I > see how to do per device logging on > http://www.rsyslog.com/Article60.phtml, but I don't see a way to adjust > that to do it based on IP subnet or anything like that. Unfortunately it > looks like both FROMHOST and HOSTNAME are names not IPs, so it's not > even clear if I could filter on that. Any help would be appreciated. > Thanks. there is fromhost-ip that will give you the last-hop IP address I don't see an easy way to do it based on subnets, but take a look at the rscript stuff that just went into the development branch in the last week or so. that may give you the hooks needed to do the subnet calculation that will let you do what you want. David Lang From rgerhards at hq.adiscon.com Fri Mar 20 07:23:12 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 20 Mar 2009 07:23:12 +0100 Subject: [rsyslog] Separating Log files based on partial IP match References: <8DA250A32C13714E8B102E9AAF260B4B08A74D3D@aquila.libs.claremont.edu> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD3E@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, March 20, 2009 12:45 AM > To: rsyslog-users > Subject: Re: [rsyslog] Separating Log files based on partial IP match > > On Thu, 19 Mar 2009, Jefferson Cowart wrote: > > > I'm new to rsyslog, and I'm trying to set it up to centralize logging > > for a number of devices on my network. I'd like for it to log > anything > > from my network switch to a single log file, my printers to another > log > > file, etc. I'm able to separate the devices based on their IP address > > (e.g. my switches are in one IP subnet and my printers in another.) I > > see how to do per device logging on > > http://www.rsyslog.com/Article60.phtml, but I don't see a way to > adjust > > that to do it based on IP subnet or anything like that. Unfortunately > it > > looks like both FROMHOST and HOSTNAME are names not IPs, so it's not > > even clear if I could filter on that. Any help would be appreciated. > > Thanks. > > there is fromhost-ip that will give you the last-hop IP address > > I don't see an easy way to do it based on subnets, but take a look at > the > rscript stuff that just went into the development branch in the last > week > or so. that may give you the hooks needed to do the subnet calculation > that will let you do what you want. The only function currently supported is strlen(), but this is a very interesting use case to extend function support. I think I will add a couple of functions even without a full loadable interface, just to get some basic things done. If everything turns out to go smooth, I can hopefully do this next week. In the mean time, I would see if a property-based (regex) filter can do the job. For a classical class A,B,C net that should be easy to do. Rainer From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 20 18:01:31 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 20 Mar 2009 18:01:31 +0100 Subject: [rsyslog] Starting a native Oracle output module (was Re: Documentation on writing rsyslog modules?) In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch> <200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> Message-ID: <200903201801.32116.Luis.Fernando.Munoz.Mejias@cern.ch> El Mi?rcoles, 18 de Marzo de 2009 11:04, Rainer Gerhards escribi?: > When you are ready, I'd actually suggest that I create an "omoracle" git > branch for you and do place a copy of ommysql into it. This, together with > the comments from omtemplate, would probably one way to get a (non-optimal) > quick start. > So, I'm starting it and I already have something that compiles. Next step is to have something I can test, then have something that makes something, then something that does the same but fast. > I would suggest that we build a very basic oracle driver first and after we > see it works well, then look into the performance optimization. > That's my idea, too. I want something that: 1) Connects to the DB at createInstance() time. 2) Runs the un-prepared statement passed as template on each syslog entry. 3) Disconnects only at freeInstance() time. Prepared statements and batch operations will be added later, indeed. But first, I'd like to know what ways I have to test my module, other than recompiling it, installing and restarting rsyslog for each change. > Let me know what you think (and when you have time). I'm already on it. I hope to deliver something for review next week. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 20 18:08:55 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 20 Mar 2009 18:08:55 +0100 Subject: [rsyslog] Starting a native Oracle output module (was Re:Documentation on writing rsyslog modules?) References: <200902111459.20042.Luis.Fernando.Munoz.Mejias@cern.ch><200903181053.35152.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702AD1A@GRFEXC.intern.adiscon.com> <200903201801.32116.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD51@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Luis > Fernando Mu?oz Mej?as > Sent: Friday, March 20, 2009 6:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Starting a native Oracle output module > (was Re:Documentation on writing rsyslog modules?) > > El Mi?rcoles, 18 de Marzo de 2009 11:04, Rainer Gerhards escribi?: > > When you are ready, I'd actually suggest that I create an > "omoracle" git > > branch for you and do place a copy of ommysql into it. > This, together with > > the comments from omtemplate, would probably one way to get > a (non-optimal) > > quick start. > > > So, I'm starting it and I already have something that compiles. Next > step is to have something I can test, then have something that makes > something, then something that does the same but fast. > > > I would suggest that we build a very basic oracle driver > first and after we > > see it works well, then look into the performance optimization. > > > That's my idea, too. I want something that: > > 1) Connects to the DB at createInstance() time. > 2) Runs the un-prepared statement passed as template on each syslog > entry. > 3) Disconnects only at freeInstance() time. > > Prepared statements and batch operations will be added later, indeed. > > But first, I'd like to know what ways I have to test my module, other > than recompiling it, installing and restarting rsyslog for > each change. You can run rsyslog interactively, that's the key to a useful testing environment. In my development environment, I have a couple of conf files, and a shell script that starts rsyslogd in a variety of test settings (don't forget about running valgrind on it frequently, it safes you a lot of time ;)). I am not at my devel machine right now, but the core command looks something like cp "all plugins" runtime/.libs # or so ./tools/rsyslogd -dn -c 4 -f myconf.conf -M runtime/.libs ... Maybe some more... Then you run rsyslogd for your test, and press ctl-c when you are done. My cycle is Loop edit make run-script End Loop Does this help? Oh, and I have disabled the regular rsyslogd on that devel box. If you don't do, you probably need to add some extra quirks to it. I have also begun to work on some tcl-based tests yesterday, hope to have them in git mid next week. Rainer > > > Let me know what you think (and when you have time). > > I'm already on it. I hope to deliver something for review next week. :) > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > From rgerhards at hq.adiscon.com Mon Mar 23 18:44:27 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 23 Mar 2009 18:44:27 +0100 Subject: [rsyslog] graph of rsyslog versions and branches Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD64@GRFEXC.intern.adiscon.com> Hi all, I created a condensed graph of rsyslog versions and branches today. I have done this as an example of how a software project evolves (what I'll write about soon), but I think it is also educating for folks interested in rsyslog. If you are interested, please find the entry point at my blog: http://blog.gerhards.net/2009/03/rsyslog-family-tree.html Rainer From pieter.thysebaert at intec.ugent.be Tue Mar 24 12:02:44 2009 From: pieter.thysebaert at intec.ugent.be (pieter.thysebaert at intec.ugent.be) Date: Tue, 24 Mar 2009 12:02:44 +0100 (CET) Subject: [rsyslog] imfile module - input line transformation Message-ID: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Hello rsyslog users, We are currently running a small rsyslog setup (i.e. TCP-based remote logging) in our test environment. This setup is also used to transfer Apache access logs, using the pipe operator in the Apache config and a Bash shell script which calls the "logger" tool to log a message to local rsyslog in a loop like # read first line #... while [ $result -eq 0 ]; do # log $line to $filename logger -p local0.info -t "APACHE" "$filename?$line" read line result=$? done The problem with this approach is twofold. First, we are experiencing performance issues under increased load (all Apache workers in status "L" on the Apache server status page when stress testing). Secondly, in order to resolve the first issue, we thought about moving to the file based input module which would make (we hope) Apache performance less depending on the logging infrastructure - as it would just log to the native filesystem as usual. However, as can be seen above, we're currently transforming the log messages to include the destination filename. On the remote rsyslog server (the receiving end), the messages are logged into a file whose name is dynamically derived from the first part of the log (the part before the first question mark). So my question is: can rsyslog be configured to 1. Read new lines from Apache access log as they become available 2. prepend an arbitrary string to the message (the destination filename) 3. log this transformed message instead of the original Or is there a more "best-practices" approach to do what I want (which is : filter messages on the remote end based on the tag and write them to a dynamically generated filename using regexps) Thanks, Pieter From Luis.Fernando.Munoz.Mejias at cern.ch Tue Mar 24 12:44:27 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?iso-8859-1?q?Mu=F1oz_Mej=EDas?=) Date: Tue, 24 Mar 2009 12:44:27 +0100 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <200903241244.27879.Luis.Fernando.Munoz.Mejias@cern.ch> Hi, > We are currently running a small rsyslog setup (i.e. TCP-based remote > logging) in our test environment. > > This setup is also used to transfer Apache access logs, using the pipe > operator in the Apache config and a Bash shell script which calls the > "logger" tool to log a message to local rsyslog in a loop like > > # read first line > #... > > while [ $result -eq 0 ]; do > # log $line to $filename > logger -p local0.info -t "APACHE" "$filename?$line" > read line > result=$? > done Why not use the CustomLog Apache directive to pipe directly the logger command: ... LogFormat "%b%l%a%h %b%l%a%h ..." logger_pipe CustomLog |/usr/bin/logger -p local0.info -t "apache" logger_pipe It should spawn only one logger process for the whole Apache host, and most likely reduce the load. > Secondly, in order to resolve the first issue, we thought about moving to > the file based input module which would make (we hope) Apache performance > less depending on the logging infrastructure - as it would just log to the > native filesystem as usual. However, as can be seen above, we're currently > transforming the log messages to include the destination filename. > On the remote rsyslog server (the receiving end), the messages are logged > into a file whose name is dynamically derived from the first part of the > log (the part before the first question mark). Again, you can use the LogFormat for that, and let Apache do the work without spawning processes over and over, which is most likely the slow part. > > So my question is: can rsyslog be configured to > 1. Read new lines from Apache access log as they become available > 2. prepend an arbitrary string to the message (the destination > filename) In principle, you should use a template for that (untested): $template TemplateName,"CONSTANT_ARBITRARY_STRING?%message%" if ($programname = "apache") then destination;TemplateName (Although I cannot assure how it behaves with TCP...) Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From THe_ZiPMaN+rsyslog at zipman.it Tue Mar 24 14:24:52 2009 From: THe_ZiPMaN+rsyslog at zipman.it (THe_ZiPMaN+rsyslog at zipman.it) Date: Tue, 24 Mar 2009 14:24:52 +0100 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <49C8DF24.8010507@zipman.it> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 pieter.thysebaert at intec.ugent.be wrote: | # read first line | #... | | while [ $result -eq 0 ]; do | # log $line to $filename | logger -p local0.info -t "APACHE" "$filename?$line" | read line | result=$? | done You are spawning a logger process for each log line... brrrr.... | Or is there a more "best-practices" approach to do what I want (which is : | filter messages on the remote end based on the tag and write them to a | dynamically generated filename using regexps) Personally I do this way: On the apache side for every VirtualHost: ErrorLog "|/usr/bin/logger -p local5.err -t http_example.com" CustomLog "|/usr/bin/logger -p local5.info -t http_example.com" combined On the rsyslog side: # Let the message "untouched" without adding any information for easy # parsing by webalizer & company $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" # Define an archiving policy that allows for simpler analisys and archiving # The number 58 should be tuned for your system. Obviously everything must # be on the same line. $template ArchiveApache,"/var/log/apache/%$YEAR%/%$MONTH%/%$DAY%/%syslogtag:F,58:1%_%syslogseverity-text%.log" # Define the destinations and prevent writing on other standard logs # Put this near the beginning of the conf file :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog :syslogtag,startswith,"http" ~ - -- Flavio Visentin GPG Key: http://www.zipman.it/gpgkey.asc There are only 10 types of people in this world: those who understand binary, and those who don't. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknI3yMACgkQusUmHkh1cnrISACfQNkWSda9yPICMM/ie78SGhLe FOMAniAk8S0coDfgCSNQp/IXGqCRfZd2 =IhIf -----END PGP SIGNATURE----- From aoz.syn at gmail.com Tue Mar 24 17:17:35 2009 From: aoz.syn at gmail.com (RB) Date: Tue, 24 Mar 2009 10:17:35 -0600 Subject: [rsyslog] imfile module - input line transformation In-Reply-To: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> References: <114f002337ab7226a7b4545f4d353483.squirrel@webserver6.intec.ugent.be> Message-ID: <4255c2570903240917l27354e08jc65a525b67e7c933@mail.gmail.com> > The problem with this approach is twofold. First, we are experiencing > performance issues under increased load (all Apache workers in status "L" > on the Apache server status page when stress testing). I am somewhat surprised neither of the responders did what seems obvious to me and bypass the pipe/execution altogether. Unless someone else here has had a problem doing so, there's no reason you couldn't just use a named pipe on both ends: [shell] mkfifo /var/run/htlog-1 [apache] CustomLog "/var/run/htlog-1" [rsyslog] $ModLoad imfile $InputFileName /var/run/htlog-1 $InputFileTag apache1 $InputFileRunMonitor That puts the logs in rsyslog with no extra executions or running processes; what you do after that for filtering is up to you. The nice thing about using a named pipe is that if the reading process dies, the buffer doesn't go away and you have less chance of losing messages. From erik at readmedia.com Tue Mar 24 20:32:41 2009 From: erik at readmedia.com (Erik Morton) Date: Tue, 24 Mar 2009 15:32:41 -0400 Subject: [rsyslog] Have I made rsyslog synchronous by mistake? Message-ID: <6829D0E0-079A-448C-8766-C190249425C1@readmedia.com> Hello there. I have rsyslog configured to forward logging messages from several application servers to a central log server. It's a Ruby on Rails app and I'm using the SyslogLogger gem to talk to rsyslog. From time to time under moderate volume my application, or more accurately one or more of my application containers, begins to freeze. I haven't been able to pin down the cause, but I did notice a couple of interesting things related to rsyslog. Very soon before the application begins to experience problems the central log file (to which all app servers forward) stops updating. This has happened every time the application has had problems. On a lark I decided to disable rsyslog and instead use the native rails logging framework. Each time this change has completely cleared up all the problems on the site. Obviously this is a grossly unscientific observation but I just can't ignore the coincidence. I'm thinking that I have borked the config of my installation to, somehow, cause this failure. Is it possible that I have configured rsyslog to somehow wait for a successful write to the log file instead of firing and forgetting? Am I required to create a local spool per http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html? Many thanks in advance. This is the configuration for the host: $ModLoad imtcp $InputTCPServerRun 200 *.info;mail.none;authpriv.none;cron.none;my_app.none / var/log/messages authpriv.* /var/log/secure mail.* -/var/log/ maillog cron.* /var/log/cron *.emerg * uucp,news.crit /var/log/spooler local7.* /var/log/ boot.log $outchannel my_app_rotate,/vol/logs/my_app.log,5242880,/usr/bin/ loganalysis /vol/logs/my_app.log !my_app *.* $my_app_rotate Each host then has this in rsyslog.conf !my_app *.* @@log_host:200 And I start rsyslogd on the central log host with SYSLOGD_OPTIONS="- t200 -m 0" From rgerhards at hq.adiscon.com Wed Mar 25 09:45:48 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 25 Mar 2009 09:45:48 +0100 Subject: [rsyslog] rsyslog branches (german content) Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702AD89@GRFEXC.intern.adiscon.com> Hi all, those of you who understand a bit of German may find this German blog post interesting: http://www.wissenslogs.de/wblogs/blog/mehr-als-bits-und-bytes/allgemein/2009- 03-24/software-evolution It talks about "software evolution" based on rsyslog's development process. While doing so, I think it captures also a lot of the spirit in which versions are created today for rsyslog. Sorry I have no English version currently... Rainer From Luis.Fernando.Munoz.Mejias at cern.ch Thu Mar 26 15:28:30 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Thu, 26 Mar 2009 15:28:30 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 Message-ID: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> Hi, I have a funny problem. Around here we have a number of nodes using old, syslogd, which report to their headnodes, which use rsyslog v3, wich keep relaying till I get a small copy on a test box. This test box uses, since yesterday, rsyslog v4. I noticed that for rsyslog v4, the last relay is considered to be the source host, the real source host is considered to be the syslogtag and everything else is inside the %msg% property. For the default template, I get messages like these: 2009-03-26T00:00:00+01:00 relayhost sourcehost1 cvs: GSSAPI userok: cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG GSS_C_CONF_FLAG 2009-03-26T00:00:00+01:00 relayhost sourcehost2 cvs: GSSAPI userok: cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG GSS_C_CONF_FLAG And, as I used to have a single file per host, I now have a single, huge "relayhost" file. Filters based on source or program name are broken, of course. What did I screw when upgrading? Thanks. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Thu Mar 26 15:30:45 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Mar 2009 15:30:45 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> You screw nothing - that's a bug in v4. You need to pull the latest devel from git ;) A new release is due soon. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Thursday, March 26, 2009 3:29 PM > To: rsyslog-users > Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > Hi, > > I have a funny problem. Around here we have a number of nodes using > old, syslogd, which report to their headnodes, which use rsyslog v3, > wich keep relaying till I get a small copy on a test box. This test box > uses, since yesterday, rsyslog v4. > > I noticed that for rsyslog v4, the last relay is considered to be the > source host, the real source host is considered to be the syslogtag and > everything else is inside the %msg% property. For the default template, > I get messages like these: > > 2009-03-26T00:00:00+01:00 relayhost sourcehost1 cvs: GSSAPI userok: > cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG > GSS_C_CONF_FLAG > 2009-03-26T00:00:00+01:00 relayhost sourcehost2 cvs: GSSAPI userok: > cvsadmin GSS_C_MUTUAL_FLAG GSS_C_REPLAY_FLAG GSS_C_INTEG_FLAG > GSS_C_CONF_FLAG > > And, as I used to have a single file per host, I now have a single, > huge > "relayhost" file. Filters based on source or program name are broken, > of > course. > > What did I screw when upgrading? > > Thanks. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Luis.Fernando.Munoz.Mejias at cern.ch Thu Mar 26 16:24:24 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Thu, 26 Mar 2009 16:24:24 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> Message-ID: <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> El Jueves, 26 de Marzo de 2009 15:30, Rainer Gerhards escribi?: > You screw nothing - that's a bug in v4. You need to pull the latest devel > from git ;) I just tried (if it's "master" branch, I mean), with no success. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Thu Mar 26 17:04:39 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Mar 2009 17:04:39 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702ADA7@GRFEXC.intern.adiscon.com> <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> It's the master branch and I am sure I fixed this... mhhh... Need to finally complete what I am working on right now, will look after that... Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Thursday, March 26, 2009 4:24 PM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > El Jueves, 26 de Marzo de 2009 15:30, Rainer Gerhards escribi?: > > You screw nothing - that's a bug in v4. You need to pull the latest > devel > > from git ;) > > I just tried (if it's "master" branch, I mean), with no success. > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From kenneho.ndu at gmail.com Fri Mar 27 16:21:03 2009 From: kenneho.ndu at gmail.com (Kenneth Holter) Date: Fri, 27 Mar 2009 16:21:03 +0100 Subject: [rsyslog] Client syslog messages are logged twice Message-ID: Hi I'm running rsyslog v2.0.6, and have the following setup: rsyslog clients => rsyslog relay => rsyslog master <= rsyslog clients The /etc/rsyslog.conf file at the master has these lines in it: ** *$template DynaFile,"/var/log/syslog-clients/%HOSTNAME%/%$YEAR%/%$MONTH%/system-%HOSTNAME%-%$NOW%.log" *.* -?DynaFile * At my rsyslog master I see that many (or all) the client log messages are logged in two the different places, both under its hostname (i.e. %HOSTNAME% is replaced by the hostname) and under its IP-adresss (i.e. %HOSTNAME% is replaced by the IP-adress). So in effect, all the messages are logged in twice. I figured it might have something to do with reverse DNS, so I added the necessary entries the /etc/hosts-file, but with no success. Does anyone have a clue as to why this is happening? Regards, Kenneth From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 27 18:09:42 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 27 Mar 2009 18:09:42 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> Message-ID: <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> El Jueves, 26 de Marzo de 2009 17:04, Rainer Gerhards escribi?: > It's the master branch and I am sure I fixed this... I'm sorry to say it's not. I just pulled git master branch, rebuilt, reinstalled and no changes. 5 minutes ago I downgraded to v3.20, and my new log files appeared as I expected them to, and my filters work as expected. > mhhh... Need to finally complete what I am working on right now, will > look after that... Of course. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 27 18:21:59 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 27 Mar 2009 18:21:59 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch><200903261624.24335.Luis.Fernando.Munoz.Mejias@cern.ch><9B6E2A8877C38245BFB15CC491A11DA702ADA9@GRFEXC.intern.adiscon.com> <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> Can you send me an on-the-wire sample of those messages (I mean that are invalidly interpreted). I have now created the parser test suite and they would make a good addition, especially as I need to troubleshoot them ;) Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as > Sent: Friday, March 27, 2009 6:10 PM > To: rsyslog-users > Subject: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 > > El Jueves, 26 de Marzo de 2009 17:04, Rainer Gerhards escribi?: > > It's the master branch and I am sure I fixed this... > > I'm sorry to say it's not. I just pulled git master branch, rebuilt, > reinstalled and no changes. > > 5 minutes ago I downgraded to v3.20, and my new log files appeared as I > expected them to, and my filters work as expected. > > > mhhh... Need to finally complete what I am working on right now, will > > look after that... > > Of course. > > Cheers. > -- > Luis Fernando Mu?oz Mej?as > Luis.Fernando.Munoz.Mejias at cern.ch > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com From Luis.Fernando.Munoz.Mejias at cern.ch Fri Mar 27 19:23:15 2009 From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=) Date: Fri, 27 Mar 2009 19:23:15 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> References: <200903261528.30218.Luis.Fernando.Munoz.Mejias@cern.ch> <200903271809.43048.Luis.Fernando.Munoz.Mejias@cern.ch> <9B6E2A8877C38245BFB15CC491A11DA702ADBA@GRFEXC.intern.adiscon.com> Message-ID: <200903271923.16168.Luis.Fernando.Munoz.Mejias@cern.ch> Rainer, > Can you send me an on-the-wire sample of those messages (I mean that are > invalidly interpreted). I have now created the parser test suite and they > would make a good addition, especially as I need to troubleshoot them ;) > > Rainer Before disclosing enough data I have to ask for permission. I can tell you that the last hop in this relay chain is using rsyslog v3, and that the format I got (tcpdump dixit) for these messages is always like this: <38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) And what gets actually logged for that is: 2009-03-27T19:06:53+01:00 last_hop_server source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) Then, last_hop_server becomes %hostname% and source_server becomes %syslogtag%. This last hop server is using rsyslog v3, so it seems to me I have to instruct v4 that the input is coming in a non-default format. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch From rgerhards at hq.adiscon.com Fri Mar 27 22:38:06 2009 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 27 Mar 2009 22:38:06 +0100 Subject: [rsyslog] Weird problems when combining rsyslog 3 and 4 Message-ID: <000d01c9af24$6fb6e7ff$100013ac@intern.adiscon.com> These samples are enough, no need to disclose more. Single lines are sufficient, as long as they can repro the problem :) rainer ----- Urspr?ngliche Nachricht ----- Von: "Luis Fernando Mu?oz Mej?as" An: "rsyslog-users" Gesendet: 27.03.09 19:23 Betreff: Re: [rsyslog] Weird problems when combining rsyslog 3 and 4 Rainer, > Can you send me an on-the-wire sample of those messages (I mean that are > invalidly interpreted). I have now created the parser test suite and they > would make a good addition, especially as I need to troubleshoot them ;) > > Rainer Before disclosing enough data I have to ask for permission. I can tell you that the last hop in this relay chain is using rsyslog v3, and that the format I got (tcpdump dixit) for these messages is always like this: <38>Mar 27 19:06:53 source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) And what gets actually logged for that is: 2009-03-27T19:06:53+01:00 last_hop_server source_server sshd(pam_unix)[12750]: session opened for user foo by (uid=0) Then, last_hop_server becomes %hostname% and source_server becomes %syslogtag%. This last hop server is using rsyslog v3, so it seems to me I have to instruct v4 that the input is coming in a non-default format. Cheers. -- Luis Fernando Mu?oz Mej?as Luis.Fernando.Munoz.Mejias at cern.ch _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com