From mlemoine at huewebstudio.com  Mon Jan  3 22:43:16 2011
From: mlemoine at huewebstudio.com (Mathieu Lemoine)
Date: Mon, 3 Jan 2011 16:43:16 -0500
Subject: [rsyslog] reliability/security (RELP/TLS) trade off
Message-ID: <AANLkTikHbUfvizJrovzSp19ojow+LCkwn2=S4nZDQYMM@mail.gmail.com>

Hello,

I have looked for an answer in both Rainer's Blog and the rsyslog doc
(on rsyslog.com).

I am using rsyslog in a cloud environment (namely Amazon EC2).

Since I have several web servers and need centralized aggregation for
both easier management and reliable gross stats, I forward the syslog
messages toward a central instance.
Since the cloud environment should be considered as hostile (potential
MITM attacks and so on) and the system log are critical data, SSL/TLS
appears to be the best solution.
However It appears that TLS in rsyslog is available only for PTCP
transport and not RELP, at lest if I believe what's written on
http://www.rsyslog.com/doc/rsyslog_tls.html .
Moreover, although PTCP has increased reliability ("decreased
unreliability"?), as mentioned in various Rainer's blogposts, it is
still far to be reliable.
Although I don't need to be audit-grade reliable, I would still prefer
to have reasonable reliability and as far as I understand PTCP is not
even close to it.

My question is therefore the following :
Is there a solution appart from stunnel to have RELP over TLS ? and
btw, what about TLS authentication ?
It seems that currently we are faced with a trade-off between
reliability and security.
Since cloud-type architecture will be (in my opinion) more and more
used, I guess that could be something very relevant to have in
rsyslog.

Btw, referring to
http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html
Rainer speaks of double-acking.
Although the overhead and delay involved with double acking are very
low, I though re-acking with silent drop would be more efficient :
The receiver sends only one ack, if it is lost and the message is
repeated by the client (detected by the mean of, e.g., a sequence
number) the receiver drop the message (this avoid duplication) and
re-send the ACK.
This could be improved by sending NACK in case of "hole" in the
sequence numbers, triggering instant resending of previously dropped
messages.
Sequential number collisions could then be addressed with sliding windows.
If the message was dropped, it is simply resent by the client (after
an ACK timeout or a NACK reception) and if the ACK is lost, the
duplicated message does not appear but the ACK is sent again.
Imho, the sequence numbers and sliding windows do not yield neither
development nor complexity overhead because they still need to be
implemented with double-acking, in case of a losed reACK.

Well that was just my two cents on this idea.

It would be great if someone could keep me posted with respect to the
RELP/TLS issue.

Regards,
Mathieu Lemoine

From david at lang.hm  Mon Jan  3 22:48:01 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 3 Jan 2011 13:48:01 -0800 (PST)
Subject: [rsyslog] reliability/security (RELP/TLS) trade off
In-Reply-To: <AANLkTikHbUfvizJrovzSp19ojow+LCkwn2=S4nZDQYMM@mail.gmail.com>
References: <AANLkTikHbUfvizJrovzSp19ojow+LCkwn2=S4nZDQYMM@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1101031347120.14678@asgard.lang.hm>

as I understand it, the answer right now is to use stunnel or something 
similar (you could use ipsec or any other VPN between machines)

David Lang

On Mon, 3 Jan 2011, Mathieu Lemoine wrote:

> Date: Mon, 3 Jan 2011 16:43:16 -0500
> From: Mathieu Lemoine <mlemoine at huewebstudio.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] reliability/security (RELP/TLS) trade off
> 
> Hello,
>
> I have looked for an answer in both Rainer's Blog and the rsyslog doc
> (on rsyslog.com).
>
> I am using rsyslog in a cloud environment (namely Amazon EC2).
>
> Since I have several web servers and need centralized aggregation for
> both easier management and reliable gross stats, I forward the syslog
> messages toward a central instance.
> Since the cloud environment should be considered as hostile (potential
> MITM attacks and so on) and the system log are critical data, SSL/TLS
> appears to be the best solution.
> However It appears that TLS in rsyslog is available only for PTCP
> transport and not RELP, at lest if I believe what's written on
> http://www.rsyslog.com/doc/rsyslog_tls.html .
> Moreover, although PTCP has increased reliability ("decreased
> unreliability"?), as mentioned in various Rainer's blogposts, it is
> still far to be reliable.
> Although I don't need to be audit-grade reliable, I would still prefer
> to have reasonable reliability and as far as I understand PTCP is not
> even close to it.
>
> My question is therefore the following :
> Is there a solution appart from stunnel to have RELP over TLS ? and
> btw, what about TLS authentication ?
> It seems that currently we are faced with a trade-off between
> reliability and security.
> Since cloud-type architecture will be (in my opinion) more and more
> used, I guess that could be something very relevant to have in
> rsyslog.
>
> Btw, referring to
> http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html
> Rainer speaks of double-acking.
> Although the overhead and delay involved with double acking are very
> low, I though re-acking with silent drop would be more efficient :
> The receiver sends only one ack, if it is lost and the message is
> repeated by the client (detected by the mean of, e.g., a sequence
> number) the receiver drop the message (this avoid duplication) and
> re-send the ACK.
> This could be improved by sending NACK in case of "hole" in the
> sequence numbers, triggering instant resending of previously dropped
> messages.
> Sequential number collisions could then be addressed with sliding windows.
> If the message was dropped, it is simply resent by the client (after
> an ACK timeout or a NACK reception) and if the ACK is lost, the
> duplicated message does not appear but the ACK is sent again.
> Imho, the sequence numbers and sliding windows do not yield neither
> development nor complexity overhead because they still need to be
> implemented with double-acking, in case of a losed reACK.
>
> Well that was just my two cents on this idea.
>
> It would be great if someone could keep me posted with respect to the
> RELP/TLS issue.
>
> Regards,
> Mathieu Lemoine
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From victor.pereira at bigrails.com  Wed Jan  5 17:23:50 2011
From: victor.pereira at bigrails.com (Victor Pereira)
Date: Wed, 5 Jan 2011 17:23:50 +0100
Subject: [rsyslog] mongodb plugin to rsyslogd
Message-ID: <AANLkTim2VYTvEwgMR1aVak_XVM8z2pFW+QmthHjY0R10@mail.gmail.com>

hi, i did develop a plugin to output syslog messages to mongodb

i just sent a message to the forum but i don't know how active the forum
is..

anyway, the link for my post is
http://kb.monitorware.com/mongodb-backend-rsyslogd-t10655.html

or the repository is there https://github.com/vpereira/rsyslogd-mongo

regards,

VP

From champ at softwink.com  Wed Jan  5 18:01:32 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Wed, 5 Jan 2011 12:01:32 -0500
Subject: [rsyslog] Berlin & liblognorm ....
Message-ID: <20110105170132.GA31783@bundy.vistech.net>


	I've made it back from Berlin,  Germany.   27C3 and Berlin
Sides was a blast.  I gave a talk on Sagan @ Berlin Sides on the 
12/30/2010.   Unfortunately,  the talks where not recorded (video).  
Anyways,  part of my talk was to "pimp out" Liblognorm.   That went 
well,   and hopefully it'll bring a little bit of attention to the 
project.  

	This morning I commited Sagan with Liblognorm support into SVN. 
It still needs a bit of work and a new rule set update,  but it looks 
very promising.  No major issues (so far!) to report.  

	I'll post my slides ASAP....   

-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110105/db0ccd5b/attachment.pgp>

From jason at jasonantman.com  Fri Jan  7 15:13:49 2011
From: jason at jasonantman.com (Jason Antman)
Date: Fri, 07 Jan 2011 09:13:49 -0500
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
 problems logging to MySQL
In-Reply-To: <4D126285.50002@jasonantman.com>
References: <4D126285.50002@jasonantman.com>
Message-ID: <4D271F9D.7060302@jasonantman.com>

Since I haven't gotten any response to this... can anyone at least give
me a yes or no answer:

Has anyone tried calling omruleset from within a ruleset bound to an
input module? Is this supported?

I'm getting all sorts of segfaults, malloc errors, etc. on Centos 5.5
x86_64, and before I try and debug this, I'd at least like to know
whether or not I'm trying something that either nobody else has ever
done, or isn't supposed to work.

Thanks,
Jason Antman
Rutgers University


Jason Antman wrote:
> Greetings,
>
> I'm trying to implement multiple rulesets with per-ruleset queues in
> order to handle messages going to MySQL. Rsyslog is doing template/regex
> based parsing for the MySQL inserts - it's not raw log messages, or the
> Adiscon format. I seem to be running into some errors.
>
> I tried creating a disk-assisted queue, stopping mysql for a minute or
> so, grabbing some relatively unique strings from the text files
> containing the raw log data, starting MySQL back up, and then searching
> for the strings - which I presumed would eventually be flushed from the
> queue to the database.
>
> Not only did I never see any log data collected when MySQL is down end
> up in the database, but after about 3 minutes with MySQL down, not only
> did I not get any disk queue files, but I saw slightly garbled versions
> of the log messages ending up in /var/log/messages - which isn't even
> mentioned anywhere in the rulesets!
>
> I'm binding imtcp and imudp to a ruleset ("remote") that first logs
> everything to dynfiles based on hostname, and then (for a few specific
> source IPs) passes messages on to omruleset for further processing.
>
> Aside from some repetitive rules for string matching, my ruleset looks like:
> $RuleSet DHCP-parsing
>
> # create a Disk-Assisted LinkedList main queue specific to this ruleset
> $WorkDirectory /var/rsyslog/work
> $RulesetCreateMainQueue on
> $MainMsgQueueFilename DHCPqueue
> $MainMsgQueueType LinkedList
> $MainMsgQueueMaxDiskSpace 500M
> $MainMsgQueueSaveOnShutdown on # persist queue contents to disk on shutdown
>
> # TESTING ONLY
> *.* /var/log/dhcp-parsing
> # END TESTING ONLY
>
> $ActionResumeRetryCount -1 # retry infinitely
>
> :msg, startswith, " DHCPREQUEST for"
> :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
> & :ommysql:localhost,test,syslog,foo;DHCPREQUESTIP
> & ~ ### DISCARD
>
> :msg, startswith, " DHCPACK on"
> :ommysql:localhost,test,syslogger,syslogger;DHCPACKonIP
> & :ommysql:localhost,test,syslog,foo;DHCPACKonMAC
> & ~ ### DISCARD
>
> :msg, startswith, " DHCPINFORM from"
> :ommysql:localhost,test,syslog,foo;DHCPINFORM
> & ~ ### DISCARD
>
> if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no client
> hardware address' ) ) \
> then :ommysql:localhost,test,syslog,foo;DHCPACKtoMAC
> & :ommysql:localhost,test,syslog,foo;DHCPACKtoIP
> & ~ ### DISCARD
>
>
> The garbage that I was seeing in /var/log/messages looked like (as you
> can see, right after the timestamp, it's obviously not properly
> formatted, though prior to killing mysql, it looked fine in the correct
> log files):
> Dec 22 15:20:41 : dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> (Courtney-PC) via eth0
> Dec 22 15:20:41   dhcpd: DHCPREQUEST for 123.456.78.123 from
> 01:23:45:67:89:ab (Courtney-PC) via 123.456.78.123
> Dec 22 15:20:41  dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> (Courtney-PC) via 123.456.78.123
> Dec 22 15:20:41   dhcpd: DHCPOFFER on 123.456.78.123 to
> 01:23:45:67:89:ab (LeVoyageur) via 123.456.78.123
> Dec 22 15:20:41   dhcpd: DHCPREQUEST for 123.456.78.123 (123.456.78.123)
> from 01:23:45:67:89:ab (LeVoyageur) via 123.456.78.123
> Dec 22 15:20:41   dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> (LeVoyageur) via 123.456.78.123
> Dec 22 15:20:41   dhcpd: DHCPDISCOVER from 01:23:45:67:89:ab via
> 123.456.78.123
> Dec 22 15:20:41   dhcpd: DHCPREQUEST for 123.456.78.123 from
> 01:23:45:67:89:ab (F52F2867C1364CC) via eth0
> Dec 22 15:20:41   dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> (F52F2867C1364CC) via eth0
> Dec 22 15:20:42   dhcpd: DHCPDISCOVER from 01:23:45:67:89:ab via
> 123.456.78.123
> Dec 22 15:20:42 l dhcpd: DHCPOFFER on 123.456.78.123 to
> 01:23:45:67:89:ab via 123.456.78.123
> Dec 22 15:20:42   dhcpd: DHCPREQUEST for 123.456.78.123 from
> 01:23:45:67:89:ab via eth0
> Dec 22 15:20:42 l dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> via eth0
> Dec 22 15:20:42  dhcpd: DHCPREQUEST for 123.456.78.123 from
> 01:23:45:67:89:ab via 123.456.78.123
> Dec 22 15:20:42  ast dhcpd: DHCPACK on 123.456.78.123 to
> 01:23:45:67:89:ab via 172.31.16
> Dec 22 15:20:47  dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> via 123.456.78.123
> Dec 22 15:20:47 l dhcpd: DHCPREQUEST for 123.456.78.123 from
> 01:23:45:67:89:ab (Blue-PC) via eth0
> Dec 22 15:20:47 eER from 01:23:45:67:89:ab via 172.dhcpd: dhcpd: DHCPACK
> on 123.456.78.123 to 01:23:45:67:89:ab (Blue-PC) via eth0
> Dec 22 15:20:47 l.25.114 dhcpd: DHCPREQUEST for 123.456.78.123 from
> 01:23:45:67:89:ab (Blue-PC) via 123.456.78.123
> Dec 22 15:20:47  dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
> (Blue-PC) via 123.456.78.123
>
>
> Any ideas? I chose rsyslog in the hopes that I'd be able to replace our
> current cron-triggered log parsing scripts with rsyslog builtin
> templates and regex functions, but it seems like I'm running into more
> and more problems trying to develop complex rules that can also handle
> decent message volume and things like restarts of mysql.
>
> Thanks for any advice,
> Jason
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
>   


From champ at softwink.com  Fri Jan  7 15:33:13 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Fri, 7 Jan 2011 09:33:13 -0500
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
 problems logging to MySQL
In-Reply-To: <4D271F9D.7060302@jasonantman.com>
References: <4D126285.50002@jasonantman.com> <4D271F9D.7060302@jasonantman.com>
Message-ID: <20110107143313.GC15088@bundy.vistech.net>

On Fri, Jan 07, 2011 at 09:13:49AM -0500, Jason Antman wrote:
> Since I haven't gotten any response to this... can anyone at least give
> me a yes or no answer:

	I think Rainer might still be on vacation.   It might be a bit
before he can look at it.  Hopefully someone else might have a answer
for you.


-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110107/3a5de2ff/attachment.pgp>

From david at lang.hm  Fri Jan  7 19:38:23 2011
From: david at lang.hm (david at lang.hm)
Date: Fri, 7 Jan 2011 10:38:23 -0800 (PST)
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
 problems logging to MySQL
In-Reply-To: <4D271F9D.7060302@jasonantman.com>
References: <4D126285.50002@jasonantman.com> <4D271F9D.7060302@jasonantman.com>
Message-ID: <alpine.DEB.2.00.1101071037100.14678@asgard.lang.hm>

On Fri, 7 Jan 2011, Jason Antman wrote:

> Since I haven't gotten any response to this... can anyone at least give
> me a yes or no answer:
>
> Has anyone tried calling omruleset from within a ruleset bound to an
> input module? Is this supported?

I have not done anything with rulesets yet.

> I'm getting all sorts of segfaults, malloc errors, etc. on Centos 5.5
> x86_64, and before I try and debug this, I'd at least like to know
> whether or not I'm trying something that either nobody else has ever
> done, or isn't supposed to work.

I know it sounds bad, but have you tried downloading the latest .git 
version and see if the problems have been fixed since the version you are 
using?

David Lang

> Thanks,
> Jason Antman
> Rutgers University
>
>
> Jason Antman wrote:
>> Greetings,
>>
>> I'm trying to implement multiple rulesets with per-ruleset queues in
>> order to handle messages going to MySQL. Rsyslog is doing template/regex
>> based parsing for the MySQL inserts - it's not raw log messages, or the
>> Adiscon format. I seem to be running into some errors.
>>
>> I tried creating a disk-assisted queue, stopping mysql for a minute or
>> so, grabbing some relatively unique strings from the text files
>> containing the raw log data, starting MySQL back up, and then searching
>> for the strings - which I presumed would eventually be flushed from the
>> queue to the database.
>>
>> Not only did I never see any log data collected when MySQL is down end
>> up in the database, but after about 3 minutes with MySQL down, not only
>> did I not get any disk queue files, but I saw slightly garbled versions
>> of the log messages ending up in /var/log/messages - which isn't even
>> mentioned anywhere in the rulesets!
>>
>> I'm binding imtcp and imudp to a ruleset ("remote") that first logs
>> everything to dynfiles based on hostname, and then (for a few specific
>> source IPs) passes messages on to omruleset for further processing.
>>
>> Aside from some repetitive rules for string matching, my ruleset looks like:
>> $RuleSet DHCP-parsing
>>
>> # create a Disk-Assisted LinkedList main queue specific to this ruleset
>> $WorkDirectory /var/rsyslog/work
>> $RulesetCreateMainQueue on
>> $MainMsgQueueFilename DHCPqueue
>> $MainMsgQueueType LinkedList
>> $MainMsgQueueMaxDiskSpace 500M
>> $MainMsgQueueSaveOnShutdown on # persist queue contents to disk on shutdown
>>
>> # TESTING ONLY
>> *.* /var/log/dhcp-parsing
>> # END TESTING ONLY
>>
>> $ActionResumeRetryCount -1 # retry infinitely
>>
>> :msg, startswith, " DHCPREQUEST for"
>> :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
>> & :ommysql:localhost,test,syslog,foo;DHCPREQUESTIP
>> & ~ ### DISCARD
>>
>> :msg, startswith, " DHCPACK on"
>> :ommysql:localhost,test,syslogger,syslogger;DHCPACKonIP
>> & :ommysql:localhost,test,syslog,foo;DHCPACKonMAC
>> & ~ ### DISCARD
>>
>> :msg, startswith, " DHCPINFORM from"
>> :ommysql:localhost,test,syslog,foo;DHCPINFORM
>> & ~ ### DISCARD
>>
>> if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no client
>> hardware address' ) ) \
>> then :ommysql:localhost,test,syslog,foo;DHCPACKtoMAC
>> & :ommysql:localhost,test,syslog,foo;DHCPACKtoIP
>> & ~ ### DISCARD
>>
>>
>> The garbage that I was seeing in /var/log/messages looked like (as you
>> can see, right after the timestamp, it's obviously not properly
>> formatted, though prior to killing mysql, it looked fine in the correct
>> log files):
>> Dec 22 15:20:41 : dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> (Courtney-PC) via eth0
>> Dec 22 15:20:41   dhcpd: DHCPREQUEST for 123.456.78.123 from
>> 01:23:45:67:89:ab (Courtney-PC) via 123.456.78.123
>> Dec 22 15:20:41  dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> (Courtney-PC) via 123.456.78.123
>> Dec 22 15:20:41   dhcpd: DHCPOFFER on 123.456.78.123 to
>> 01:23:45:67:89:ab (LeVoyageur) via 123.456.78.123
>> Dec 22 15:20:41   dhcpd: DHCPREQUEST for 123.456.78.123 (123.456.78.123)
>> from 01:23:45:67:89:ab (LeVoyageur) via 123.456.78.123
>> Dec 22 15:20:41   dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> (LeVoyageur) via 123.456.78.123
>> Dec 22 15:20:41   dhcpd: DHCPDISCOVER from 01:23:45:67:89:ab via
>> 123.456.78.123
>> Dec 22 15:20:41   dhcpd: DHCPREQUEST for 123.456.78.123 from
>> 01:23:45:67:89:ab (F52F2867C1364CC) via eth0
>> Dec 22 15:20:41   dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> (F52F2867C1364CC) via eth0
>> Dec 22 15:20:42   dhcpd: DHCPDISCOVER from 01:23:45:67:89:ab via
>> 123.456.78.123
>> Dec 22 15:20:42 l dhcpd: DHCPOFFER on 123.456.78.123 to
>> 01:23:45:67:89:ab via 123.456.78.123
>> Dec 22 15:20:42   dhcpd: DHCPREQUEST for 123.456.78.123 from
>> 01:23:45:67:89:ab via eth0
>> Dec 22 15:20:42 l dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> via eth0
>> Dec 22 15:20:42  dhcpd: DHCPREQUEST for 123.456.78.123 from
>> 01:23:45:67:89:ab via 123.456.78.123
>> Dec 22 15:20:42  ast dhcpd: DHCPACK on 123.456.78.123 to
>> 01:23:45:67:89:ab via 172.31.16
>> Dec 22 15:20:47  dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> via 123.456.78.123
>> Dec 22 15:20:47 l dhcpd: DHCPREQUEST for 123.456.78.123 from
>> 01:23:45:67:89:ab (Blue-PC) via eth0
>> Dec 22 15:20:47 eER from 01:23:45:67:89:ab via 172.dhcpd: dhcpd: DHCPACK
>> on 123.456.78.123 to 01:23:45:67:89:ab (Blue-PC) via eth0
>> Dec 22 15:20:47 l.25.114 dhcpd: DHCPREQUEST for 123.456.78.123 from
>> 01:23:45:67:89:ab (Blue-PC) via 123.456.78.123
>> Dec 22 15:20:47  dhcpd: DHCPACK on 123.456.78.123 to 01:23:45:67:89:ab
>> (Blue-PC) via 123.456.78.123
>>
>>
>> Any ideas? I chose rsyslog in the hopes that I'd be able to replace our
>> current cron-triggered log parsing scripts with rsyslog builtin
>> templates and regex functions, but it seems like I'm running into more
>> and more problems trying to develop complex rules that can also handle
>> decent message volume and things like restarts of mysql.
>>
>> Thanks for any advice,
>> Jason
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From jason at jasonantman.com  Fri Jan  7 20:08:28 2011
From: jason at jasonantman.com (Jason Antman)
Date: Fri, 07 Jan 2011 14:08:28 -0500
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
 problems logging to MySQL
In-Reply-To: <20110107143313.GC15088@bundy.vistech.net>
References: <4D126285.50002@jasonantman.com> <4D271F9D.7060302@jasonantman.com>
	<20110107143313.GC15088@bundy.vistech.net>
Message-ID: <4D2764AC.9030605@jasonantman.com>

Ok.

Just as a quick overview (I haven't analyzed enough of the debugging
information that I collected to submit a bug report), rsyslog becomes
unstable when omruleset is called from within a ruleset. Crashes were a
mix of segfaults and malloc/realloc errors. With my original config
(complex mix of multiple ommysql calls per if statement, etc.) triggered
a crash within the first few seconds of running, every time. I created a
much smaller sample config (one ruleset bound to imudp/imtcp, two
rulesets called from there each with two if statement rules) and it runs
for about 30 seconds before dieing.

Perhaps there's some interaction somewhere between omruleset and other
output modules??

If I remove the omruleset calls and put everything from them in the main
ruleset (bound to imudp and imtcp), it runs without any problems.

I'm running 5.6.2 on CentOS 5.5 x86_64.

Thanks,
Jason

Sample config that segfaults is below:
====== BEGIN CODE====

#### GLOBAL DIRECTIVES ####

$FileOwner root
$FileGroup root
$FileCreateMode 0640
$DirOwner root
$DirGroup root
$DirCreateMode 0750

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

$WorkDirectory /var/rsyslog/work

# Provides logging to MySQL - define before any rules that use it
$ModLoad ommysql
$ModLoad omruleset

# templates - include first
$IncludeConfig /etc/rsyslog.d/templates.conf
$IncludeConfig /etc/rsyslog.d/dhcp-templates.conf

#### Imports - ORDER MATTERS HERE ####

$RuleSet BSC-ruleset

*.* /var/log/TESTING/rules-BSC-ruleset

if $msg contains 'user_login_suc' then
/var/log/TESTING/rules-BSC-ruleset-logout_suc
& :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login
& :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login-web
& ~

$RuleSet DHCP-parsing

*.* /var/log/TESTING/rules-DHCP-parsing

:msg, startswith, " DHCPREQUEST for"
/var/log/TESTING/rules-DHCP-parsing-requestfor
& :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
& :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTIP
& ~ ### DISCARD

if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no client
hardware address' ) ) then /var/log/TESTING/rules-DHCP-parsing-ackto \
& :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoMAC
& :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoIP
& ~ ### DISCARD

$RuleSet remote

*.* /var/log/TESTING/rules-remote

$ActionOmrulesetRulesetName BSC-ruleset
if $fromhost-ip == '128.6.30.195' or $fromhost-ip == '128.6.30.196' \
then /var/log/TESTING/rules-remote-BSC
& :omruleset:
& ~

$ActionOmrulesetRulesetName DHCP-parsing
if ( $fromhost-ip == '172.16.25.114' ) or ( $fromhost-ip ==
'172.16.25.116' ) or ( $fromhost-ip == '128.6.17.217' ) then
/var/log/TESTING/rules-remote-dhcp
& :omruleset:

$RuleSet local
*.*                /var/log/TESTING/local

##### END IMPORTS

#### Default Ruleset ####
# since we bind TCP and UDP to remote, this should only handle local
$DefaultRuleset local

#### MODULES ####

$ModLoad imuxsock.so    # provides support for local system logging
(e.g. via logger command)
$ModLoad imklog.so      # provides kernel logging support (previously
done by rklogd)
#$ModLoad immark.so     # provides --MARK-- message capability

#### BIND INPUTS ####

# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerAddress *
$InputUDPServerBindRuleset remote # bind UDP to the remote ruleset
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerBindRuleset remote # bind tcp to the remote ruleset
$InputTCPServerRun 514


====== END CODE ====

Champ Clark III [Softwink] wrote:
> On Fri, Jan 07, 2011 at 09:13:49AM -0500, Jason Antman wrote:
>   
>> Since I haven't gotten any response to this... can anyone at least give
>> me a yes or no answer:
>>     
>
> 	I think Rainer might still be on vacation.   It might be a bit
> before he can look at it.  Hopefully someone else might have a answer
> for you.
>
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


From rgerhards at hq.adiscon.com  Fri Jan  7 23:19:54 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 7 Jan 2011 23:19:54 +0100
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
	problems logging to MySQL
References: <4D126285.50002@jasonantman.com>
	<4D271F9D.7060302@jasonantman.com><20110107143313.GC15088@bundy.vistech.net>
	<4D2764AC.9030605@jasonantman.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD99C@GRFEXC.intern.adiscon.com>

I am preparing for my flight back home tomorrow. If you can, let valgrind run
on the instance, this will probably give a lot of insight.

To the best of my knowledge, omruleset is not in widespread use, at least I
don't know any user with a complex config. As such, I've not yet looked much
at it.

Will check early next week, valgrind would probably be helpful.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jason Antman
> Sent: Friday, January 07, 2011 8:08 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Multiple rulesets and queues - strange behavior,
> problems logging to MySQL
> 
> Ok.
> 
> Just as a quick overview (I haven't analyzed enough of the debugging
> information that I collected to submit a bug report), rsyslog becomes
> unstable when omruleset is called from within a ruleset. Crashes were a
> mix of segfaults and malloc/realloc errors. With my original config
> (complex mix of multiple ommysql calls per if statement, etc.)
> triggered
> a crash within the first few seconds of running, every time. I created
> a
> much smaller sample config (one ruleset bound to imudp/imtcp, two
> rulesets called from there each with two if statement rules) and it
> runs
> for about 30 seconds before dieing.
> 
> Perhaps there's some interaction somewhere between omruleset and other
> output modules??
> 
> If I remove the omruleset calls and put everything from them in the
> main
> ruleset (bound to imudp and imtcp), it runs without any problems.
> 
> I'm running 5.6.2 on CentOS 5.5 x86_64.
> 
> Thanks,
> Jason
> 
> Sample config that segfaults is below:
> ====== BEGIN CODE====
> 
> #### GLOBAL DIRECTIVES ####
> 
> $FileOwner root
> $FileGroup root
> $FileCreateMode 0640
> $DirOwner root
> $DirGroup root
> $DirCreateMode 0750
> 
> # Use default timestamp format
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
> 
> $WorkDirectory /var/rsyslog/work
> 
> # Provides logging to MySQL - define before any rules that use it
> $ModLoad ommysql
> $ModLoad omruleset
> 
> # templates - include first
> $IncludeConfig /etc/rsyslog.d/templates.conf
> $IncludeConfig /etc/rsyslog.d/dhcp-templates.conf
> 
> #### Imports - ORDER MATTERS HERE ####
> 
> $RuleSet BSC-ruleset
> 
> *.* /var/log/TESTING/rules-BSC-ruleset
> 
> if $msg contains 'user_login_suc' then
> /var/log/TESTING/rules-BSC-ruleset-logout_suc
> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login
> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login-web
> & ~
> 
> $RuleSet DHCP-parsing
> 
> *.* /var/log/TESTING/rules-DHCP-parsing
> 
> :msg, startswith, " DHCPREQUEST for"
> /var/log/TESTING/rules-DHCP-parsing-requestfor
> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTIP
> & ~ ### DISCARD
> 
> if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no client
> hardware address' ) ) then /var/log/TESTING/rules-DHCP-parsing-ackto \
> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoMAC
> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoIP
> & ~ ### DISCARD
> 
> $RuleSet remote
> 
> *.* /var/log/TESTING/rules-remote
> 
> $ActionOmrulesetRulesetName BSC-ruleset
> if $fromhost-ip == '128.6.30.195' or $fromhost-ip == '128.6.30.196' \
> then /var/log/TESTING/rules-remote-BSC
> & :omruleset:
> & ~
> 
> $ActionOmrulesetRulesetName DHCP-parsing
> if ( $fromhost-ip == '172.16.25.114' ) or ( $fromhost-ip ==
> '172.16.25.116' ) or ( $fromhost-ip == '128.6.17.217' ) then
> /var/log/TESTING/rules-remote-dhcp
> & :omruleset:
> 
> $RuleSet local
> *.*                /var/log/TESTING/local
> 
> ##### END IMPORTS
> 
> #### Default Ruleset ####
> # since we bind TCP and UDP to remote, this should only handle local
> $DefaultRuleset local
> 
> #### MODULES ####
> 
> $ModLoad imuxsock.so    # provides support for local system logging
> (e.g. via logger command)
> $ModLoad imklog.so      # provides kernel logging support (previously
> done by rklogd)
> #$ModLoad immark.so     # provides --MARK-- message capability
> 
> #### BIND INPUTS ####
> 
> # Provides UDP syslog reception
> $ModLoad imudp.so
> $UDPServerAddress *
> $InputUDPServerBindRuleset remote # bind UDP to the remote ruleset
> $UDPServerRun 514
> 
> # Provides TCP syslog reception
> $ModLoad imtcp.so
> $InputTCPServerBindRuleset remote # bind tcp to the remote ruleset
> $InputTCPServerRun 514
> 
> 
> ====== END CODE ====
> 
> Champ Clark III [Softwink] wrote:
> > On Fri, Jan 07, 2011 at 09:13:49AM -0500, Jason Antman wrote:
> >
> >> Since I haven't gotten any response to this... can anyone at least
> give
> >> me a yes or no answer:
> >>
> >
> > 	I think Rainer might still be on vacation.   It might be a bit
> > before he can look at it.  Hopefully someone else might have a answer
> > for you.
> >
> >
> >
> > ---------------------------------------------------------------------
> ---
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Sun Jan  9 18:56:38 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 9 Jan 2011 18:56:38 +0100
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
	problems logging to MySQL
References: <4D126285.50002@jasonantman.com>
	<4D271F9D.7060302@jasonantman.com><20110107143313.GC15088@bundy.vistech.net>
	<4D2764AC.9030605@jasonantman.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD99E@GRFEXC.intern.adiscon.com>

Jason,

any chance you can reproduce this with a single input and a single additional
ruleset? Did you try this?

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jason Antman
> Sent: Friday, January 07, 2011 8:08 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Multiple rulesets and queues - strange behavior,
> problems logging to MySQL
> 
> Ok.
> 
> Just as a quick overview (I haven't analyzed enough of the debugging
> information that I collected to submit a bug report), rsyslog becomes
> unstable when omruleset is called from within a ruleset. Crashes were a
> mix of segfaults and malloc/realloc errors. With my original config
> (complex mix of multiple ommysql calls per if statement, etc.)
> triggered
> a crash within the first few seconds of running, every time. I created
> a
> much smaller sample config (one ruleset bound to imudp/imtcp, two
> rulesets called from there each with two if statement rules) and it
> runs
> for about 30 seconds before dieing.
> 
> Perhaps there's some interaction somewhere between omruleset and other
> output modules??
> 
> If I remove the omruleset calls and put everything from them in the
> main
> ruleset (bound to imudp and imtcp), it runs without any problems.
> 
> I'm running 5.6.2 on CentOS 5.5 x86_64.
> 
> Thanks,
> Jason
> 
> Sample config that segfaults is below:
> ====== BEGIN CODE====
> 
> #### GLOBAL DIRECTIVES ####
> 
> $FileOwner root
> $FileGroup root
> $FileCreateMode 0640
> $DirOwner root
> $DirGroup root
> $DirCreateMode 0750
> 
> # Use default timestamp format
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
> 
> $WorkDirectory /var/rsyslog/work
> 
> # Provides logging to MySQL - define before any rules that use it
> $ModLoad ommysql
> $ModLoad omruleset
> 
> # templates - include first
> $IncludeConfig /etc/rsyslog.d/templates.conf
> $IncludeConfig /etc/rsyslog.d/dhcp-templates.conf
> 
> #### Imports - ORDER MATTERS HERE ####
> 
> $RuleSet BSC-ruleset
> 
> *.* /var/log/TESTING/rules-BSC-ruleset
> 
> if $msg contains 'user_login_suc' then
> /var/log/TESTING/rules-BSC-ruleset-logout_suc
> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login
> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login-web
> & ~
> 
> $RuleSet DHCP-parsing
> 
> *.* /var/log/TESTING/rules-DHCP-parsing
> 
> :msg, startswith, " DHCPREQUEST for"
> /var/log/TESTING/rules-DHCP-parsing-requestfor
> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTIP
> & ~ ### DISCARD
> 
> if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no client
> hardware address' ) ) then /var/log/TESTING/rules-DHCP-parsing-ackto \
> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoMAC
> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoIP
> & ~ ### DISCARD
> 
> $RuleSet remote
> 
> *.* /var/log/TESTING/rules-remote
> 
> $ActionOmrulesetRulesetName BSC-ruleset
> if $fromhost-ip == '128.6.30.195' or $fromhost-ip == '128.6.30.196' \
> then /var/log/TESTING/rules-remote-BSC
> & :omruleset:
> & ~
> 
> $ActionOmrulesetRulesetName DHCP-parsing
> if ( $fromhost-ip == '172.16.25.114' ) or ( $fromhost-ip ==
> '172.16.25.116' ) or ( $fromhost-ip == '128.6.17.217' ) then
> /var/log/TESTING/rules-remote-dhcp
> & :omruleset:
> 
> $RuleSet local
> *.*                /var/log/TESTING/local
> 
> ##### END IMPORTS
> 
> #### Default Ruleset ####
> # since we bind TCP and UDP to remote, this should only handle local
> $DefaultRuleset local
> 
> #### MODULES ####
> 
> $ModLoad imuxsock.so    # provides support for local system logging
> (e.g. via logger command)
> $ModLoad imklog.so      # provides kernel logging support (previously
> done by rklogd)
> #$ModLoad immark.so     # provides --MARK-- message capability
> 
> #### BIND INPUTS ####
> 
> # Provides UDP syslog reception
> $ModLoad imudp.so
> $UDPServerAddress *
> $InputUDPServerBindRuleset remote # bind UDP to the remote ruleset
> $UDPServerRun 514
> 
> # Provides TCP syslog reception
> $ModLoad imtcp.so
> $InputTCPServerBindRuleset remote # bind tcp to the remote ruleset
> $InputTCPServerRun 514
> 
> 
> ====== END CODE ====
> 
> Champ Clark III [Softwink] wrote:
> > On Fri, Jan 07, 2011 at 09:13:49AM -0500, Jason Antman wrote:
> >
> >> Since I haven't gotten any response to this... can anyone at least
> give
> >> me a yes or no answer:
> >>
> >
> > 	I think Rainer might still be on vacation.   It might be a bit
> > before he can look at it.  Hopefully someone else might have a answer
> > for you.
> >
> >
> >
> > ---------------------------------------------------------------------
> ---
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From champ at softwink.com  Mon Jan 10 02:37:22 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Sun, 9 Jan 2011 20:37:22 -0500
Subject: [rsyslog] Strange liblognorm issue(s)...
In-Reply-To: <20101020175951.GA13070@bundy.vistech.net>
References: <20101020175951.GA13070@bundy.vistech.net>
Message-ID: <20110110013722.GA24485@bundy.vistech.net>


I was working on some liblognorm rules,  and ran into something rather 
strange.  First,  here's a example rule (that works)..  This is my 
input being examined:

sshd[24833]: Invalid user champtest from 192.168.1.1

--<snip>----
prefix=sshd[%pid:number%]:
rule=: Invalid user champtest from %src-ip:ipv4%
--<snip>----

	This works fine... 

Normlize: [cee at 115 src-ip="98.224.46.168" pid="24500"]

So I add to it to grab the username,  like thus:

--<snip>----
prefix=sshd[%pid:number%]:
rule=: Invalid user %user:word% from %src-ip:ipv4%
--<snip>----

This produces a segfault.  However,  this works:

--<snip>----
prefix=sshd[%pid:number%]:
rule=: %invalid:word% user %user:word% %from:word% %src-ip:ipv4%
--<snip>----

Normlize: [cee at 115 src-ip="98.224.46.168" from="from" user="champtest" invalid="Invalid" pid="24766"]

I changed the rule a little bit (dropped the prefix).  This works:

--<snip>----
prefix=
rule=:sshd[%pid:number%]: %invalid:word% user %user:word% %from:word% %src-ip:ipv4%
--<snip>----

	This works fine:

Normlize: [cee at 115 src-ip="98.224.46.168" from="from" user="champtest" invalid="Invalid" pid="24797"]

	This causes a seg fault: 

--<snip>----
prefix=
rule=:sshd[%pid:number%]: Invalid user %user:word% %from:word% %src-ip:ipv4%
--<snip>----

	Can someone attempt to reproduce?  It's really strange!  I can
provide straces/gdb dumps if need be. 

-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110109/2da5ac9c/attachment.pgp>

From rgerhards at hq.adiscon.com  Mon Jan 10 08:17:58 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 10 Jan 2011 08:17:58 +0100
Subject: [rsyslog] Strange liblognorm issue(s)...
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9A1@GRFEXC.intern.adiscon.com>

being back to the office, working through my mail backlog (mostly in sequence
;)). It would be a good idea to create a bug tracker. I guess it is
definitely related to how the parse tree is built.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Monday, January 10, 2011 2:37 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Strange liblognorm issue(s)...
> 
> 
> I was working on some liblognorm rules,  and ran into something rather
> strange.  First,  here's a example rule (that works)..  This is my
> input being examined:
> 
> sshd[24833]: Invalid user champtest from 192.168.1.1
> 
> --<snip>----
> prefix=sshd[%pid:number%]:
> rule=: Invalid user champtest from %src-ip:ipv4%
> --<snip>----
> 
> 	This works fine...
> 
> Normlize: [cee at 115 src-ip="98.224.46.168" pid="24500"]
> 
> So I add to it to grab the username,  like thus:
> 
> --<snip>----
> prefix=sshd[%pid:number%]:
> rule=: Invalid user %user:word% from %src-ip:ipv4%
> --<snip>----
> 
> This produces a segfault.  However,  this works:
> 
> --<snip>----
> prefix=sshd[%pid:number%]:
> rule=: %invalid:word% user %user:word% %from:word% %src-ip:ipv4%
> --<snip>----
> 
> Normlize: [cee at 115 src-ip="98.224.46.168" from="from" user="champtest"
> invalid="Invalid" pid="24766"]
> 
> I changed the rule a little bit (dropped the prefix).  This works:
> 
> --<snip>----
> prefix=
> rule=:sshd[%pid:number%]: %invalid:word% user %user:word% %from:word%
> %src-ip:ipv4%
> --<snip>----
> 
> 	This works fine:
> 
> Normlize: [cee at 115 src-ip="98.224.46.168" from="from" user="champtest"
> invalid="Invalid" pid="24797"]
> 
> 	This causes a seg fault:
> 
> --<snip>----
> prefix=
> rule=:sshd[%pid:number%]: Invalid user %user:word% %from:word% %src-
> ip:ipv4%
> --<snip>----
> 
> 	Can someone attempt to reproduce?  It's really strange!  I can
> provide straces/gdb dumps if need be.
> 
> --
>         Champ Clark III | Softwink, Inc | 800-538-9357 x 101
>                      http://www.softwink.com
> 
> GPG Key ID: 58A2A58F
> Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
> If it wasn't for C, we'd be using BASI, PASAL and OBOL.

From rgerhards at hq.adiscon.com  Mon Jan 10 13:42:08 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 10 Jan 2011 13:42:08 +0100
Subject: [rsyslog] Losing UDP packages
In-Reply-To: <20101229170805.GB14863@fly.srk.fer.hr>
References: <20101228145725.GA6780@fly.srk.fer.hr>	<9B6E2A8877C38245BFB15CC491A11DA71DD979@GRFEXC.intern.adiscon.com>
	<20101229170805.GB14863@fly.srk.fer.hr>
Message-ID: <4D2AFEA0.3060603@hq.adiscon.com>

Hi,

thanks for all your hard work! I have replaced my initial implementation 
with your patch. It is available starting with 5.7.3 (as it changes 
quite a bit, it can not go immediately into the stable version). I have 
removed support from v4, because the changes required are too large to 
justify supporting it in v4.

So far, I have just done code review and very rough testing. I will let 
the new code run in my lab within the next couple of days, but I thought 
it is such a good addition that I merged it ASAP.

Thanks again for your help!

Rainer


On 12/29/2010 06:08 PM, Dra?en Ka?ar wrote:
> Rainer Gerhards wrote:
>> I am on vacation right now. But I think what happens is that the worker
>> threads inherit the priority setting from the UDP listener thread. You
>> probably need to change thread creation in ./runtime/wtp.c.
>
> I hoped there would be a better method. Anyway, I've added thread
> attributes in every pthread_create call, since changing just the one in
> wtp.c wasn't enough.
>
> There is one pthread_create() in plugins/imsolaris/sun_cddl.c which I
> didn't touch because it seems buggy. It's using create_door_thr as
> pthread_attr_t, but create_door_thr is never initialized, as far as I can
> see.
>
> The updated patch against rsyslog 5.6.2 is attached. I have only UDP
> thread in real-time mode now.
>
> I'm not sure if the code which gets the default thread properties should
> go in rsyslog.c or somewhere else. It can be safely moved anywhere in the
> initialization sequence, before the first pthread_create is called.
>
> About configure check: the proper way to check for the functionality would
> be to check for _XOPEN_REALTIME_THREADS preprocessor macro. That's what's
> supposed to be defined if real-time thread functionality is available.
> However, there is no that symbol anywhere in /usr/include on Solaris 10
> (update 6 is what I checked).
>
> The equivalent run-time check sysconf(_SC_XOPEN_REALTIME_THREADS) is
> returning 1, though, so the lack of _XOPEN_REALTIME_THREADS macro is a
> Solaris bug, as far as I can tell.
>
> Therefore I'm checking for the availability of pthread_setschedparam() and
> then have all real-time thread code in #ifdef HAVE_PTHREAD_SETSCHEDPARAM
> blocks. It's not ideal, but I hope it works.
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From champ at softwink.com  Mon Jan 10 18:45:40 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Mon, 10 Jan 2011 12:45:40 -0500
Subject: [rsyslog] Strange liblognorm issue(s)...
In-Reply-To: <20110110013722.GA24485@bundy.vistech.net>
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
Message-ID: <20110110174540.GA11336@bundy.vistech.net>


	I can reproduce with normalizer.c .... Here's the information:
This is the syslog "input" I'm using: 

sshd[1234]: Invalid user champ from 192.168.0.1

	Here's the rule that causes the segfault:

prefix=
rule=:sshd[%pid:number%]: Invalid user %user:word% from %src-ip:ipv4%

	When normalizer is run..  here's the output:

backup src # cat trigger | ./normalizer -r testrule.txt 
To normalize: 'sshd[1234]: Invalid user champ from 192.168.0.1'
Segmentation fault

	If I change the rule to this: 

prefix=
rule=:sshd[%pid:number%]: %invalid:word% user %user:word% from %src-ip:ipv4%

	It works fine: 

backup src # cat trigger | ./normalizer -r testrule.txt 
To normalize: 'sshd[1234]: Invalid user champ from 192.168.0.1'
normalized: '[cee at 115 src-ip="192.168.0.1" user="champ" invalid="Invalid" pid="1234"]'

	Doing a little debugging,  it appears the segfault happens here
(in the normalizer.c code).... 

	ln_normalize(ctx, str, &event);

	This is using the stock normalizer.c that shipps with
liblognorm-0.1.0.  Let me know if there's any other testing you want 
me to do,  but it appears to be easily reproduced. 

-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110110/d5a6c8f5/attachment.pgp>

From rgerhards at hq.adiscon.com  Mon Jan 10 18:47:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 10 Jan 2011 18:47:37 +0100
Subject: [rsyslog] Strange liblognorm issue(s)...
References: <20101020175951.GA13070@bundy.vistech.net><20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9B3@GRFEXC.intern.adiscon.com>

Thanks, this is very useful. I made good progress today and will probably be
able to have at least a quick lock tomorrow and a more in-depth look (if at
all required) on Wednesday.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Monday, January 10, 2011 6:46 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Strange liblognorm issue(s)...
> 
> 
> 	I can reproduce with normalizer.c .... Here's the information:
> This is the syslog "input" I'm using:
> 
> sshd[1234]: Invalid user champ from 192.168.0.1
> 
> 	Here's the rule that causes the segfault:
> 
> prefix=
> rule=:sshd[%pid:number%]: Invalid user %user:word% from %src-ip:ipv4%
> 
> 	When normalizer is run..  here's the output:
> 
> backup src # cat trigger | ./normalizer -r testrule.txt
> To normalize: 'sshd[1234]: Invalid user champ from 192.168.0.1'
> Segmentation fault
> 
> 	If I change the rule to this:
> 
> prefix=
> rule=:sshd[%pid:number%]: %invalid:word% user %user:word% from %src-
> ip:ipv4%
> 
> 	It works fine:
> 
> backup src # cat trigger | ./normalizer -r testrule.txt
> To normalize: 'sshd[1234]: Invalid user champ from 192.168.0.1'
> normalized: '[cee at 115 src-ip="192.168.0.1" user="champ"
> invalid="Invalid" pid="1234"]'
> 
> 	Doing a little debugging,  it appears the segfault happens here
> (in the normalizer.c code)....
> 
> 	ln_normalize(ctx, str, &event);
> 
> 	This is using the stock normalizer.c that shipps with
> liblognorm-0.1.0.  Let me know if there's any other testing you want
> me to do,  but it appears to be easily reproduced.
> 
> --
>         Champ Clark III | Softwink, Inc | 800-538-9357 x 101
>                      http://www.softwink.com
> 
> GPG Key ID: 58A2A58F
> Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
> If it wasn't for C, we'd be using BASI, PASAL and OBOL.

From champ at softwink.com  Mon Jan 10 18:52:39 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Mon, 10 Jan 2011 12:52:39 -0500
Subject: [rsyslog] Strange liblognorm issue(s)...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9B3@GRFEXC.intern.adiscon.com>
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9B3@GRFEXC.intern.adiscon.com>
Message-ID: <20110110175239.GA11909@bundy.vistech.net>

On Mon, Jan 10, 2011 at 06:47:37PM +0100, Rainer Gerhards wrote:
> Thanks, this is very useful. I made good progress today and will probably be
> able to have at least a quick lock tomorrow and a more in-depth look (if at
> all required) on Wednesday.

	Thanks Rainer,  I just verified via gdb with Sagan as well.... 

--<snip>-------
#40 0x0000000000000001 in ?? ()
#41 0x0000000000b08260 in ?? ()
#42 0x00000d352e373d5f in main (argc=<value optimized out>, 
    argv=<value optimized out>) at sagan.c:965
--<snip>----------

	Line 965: 

ln_normalize(ctx, str, &lnevent);

	Let me know if I can help in any way,  and thanks. 


-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110110/8806c6e4/attachment.pgp>

From rgerhards at hq.adiscon.com  Mon Jan 10 18:57:00 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 10 Jan 2011 18:57:00 +0100
Subject: [rsyslog] Strange liblognorm issue(s)...
References: <20101020175951.GA13070@bundy.vistech.net><20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>

I couldn't stand it, copied & pasted the info. Same segfault here :) valgrind
tells me it is ptree.c/627 (628 if you remove the comment). This is inside
the parse tree as expected. Let me see if I find out tomorrow what goes on.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Monday, January 10, 2011 6:46 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Strange liblognorm issue(s)...
> 
> 
> 	I can reproduce with normalizer.c .... Here's the information:
> This is the syslog "input" I'm using:
> 
> sshd[1234]: Invalid user champ from 192.168.0.1
> 
> 	Here's the rule that causes the segfault:
> 
> prefix=
> rule=:sshd[%pid:number%]: Invalid user %user:word% from %src-ip:ipv4%
> 
> 	When normalizer is run..  here's the output:
> 
> backup src # cat trigger | ./normalizer -r testrule.txt
> To normalize: 'sshd[1234]: Invalid user champ from 192.168.0.1'
> Segmentation fault
> 
> 	If I change the rule to this:
> 
> prefix=
> rule=:sshd[%pid:number%]: %invalid:word% user %user:word% from %src-
> ip:ipv4%
> 
> 	It works fine:
> 
> backup src # cat trigger | ./normalizer -r testrule.txt
> To normalize: 'sshd[1234]: Invalid user champ from 192.168.0.1'
> normalized: '[cee at 115 src-ip="192.168.0.1" user="champ"
> invalid="Invalid" pid="1234"]'
> 
> 	Doing a little debugging,  it appears the segfault happens here
> (in the normalizer.c code)....
> 
> 	ln_normalize(ctx, str, &event);
> 
> 	This is using the stock normalizer.c that shipps with
> liblognorm-0.1.0.  Let me know if there's any other testing you want
> me to do,  but it appears to be easily reproduced.
> 
> --
>         Champ Clark III | Softwink, Inc | 800-538-9357 x 101
>                      http://www.softwink.com
> 
> GPG Key ID: 58A2A58F
> Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
> If it wasn't for C, we'd be using BASI, PASAL and OBOL.

From champ at softwink.com  Mon Jan 10 19:09:14 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Mon, 10 Jan 2011 13:09:14 -0500
Subject: [rsyslog] Strange liblognorm issue(s)...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
Message-ID: <20110110180914.GB11909@bundy.vistech.net>

On Mon, Jan 10, 2011 at 06:57:00PM +0100, Rainer Gerhards wrote:
> I couldn't stand it, copied & pasted the info. Same segfault here :) valgrind
> tells me it is ptree.c/627 (628 if you remove the comment). This is inside
> the parse tree as expected. Let me see if I find out tomorrow what goes on.

	No rush.  Just wanting to point it out as I ran into it
yesterday.  It was driving me nuts :)

-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110110/4af7ca4e/attachment.pgp>

From jason at jasonantman.com  Mon Jan 10 19:12:40 2011
From: jason at jasonantman.com (Jason Antman)
Date: Mon, 10 Jan 2011 13:12:40 -0500
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
 problems logging to MySQL
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD99E@GRFEXC.intern.adiscon.com>
References: <4D126285.50002@jasonantman.com>	<4D271F9D.7060302@jasonantman.com><20110107143313.GC15088@bundy.vistech.net>	<4D2764AC.9030605@jasonantman.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD99E@GRFEXC.intern.adiscon.com>
Message-ID: <4D2B4C18.4000704@jasonantman.com>

Haven't given that a try yet. I just finished working up a stable 
(albeit slow and very sub-optimal) configuration with everything in one 
ruleset, and ran that on my development server for the past 3 days. Once 
I have that migrated to production I'll give one input/one additional 
ruleset a try, probably sometime early tomorrow (by east coast US time).

Thanks,
Jason

Rainer Gerhards wrote:
> Jason,
>
> any chance you can reproduce this with a single input and a single additional
> ruleset? Did you try this?
>
> Rainer
>
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Jason Antman
>> Sent: Friday, January 07, 2011 8:08 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Multiple rulesets and queues - strange behavior,
>> problems logging to MySQL
>>
>> Ok.
>>
>> Just as a quick overview (I haven't analyzed enough of the debugging
>> information that I collected to submit a bug report), rsyslog becomes
>> unstable when omruleset is called from within a ruleset. Crashes were a
>> mix of segfaults and malloc/realloc errors. With my original config
>> (complex mix of multiple ommysql calls per if statement, etc.)
>> triggered
>> a crash within the first few seconds of running, every time. I created
>> a
>> much smaller sample config (one ruleset bound to imudp/imtcp, two
>> rulesets called from there each with two if statement rules) and it
>> runs
>> for about 30 seconds before dieing.
>>
>> Perhaps there's some interaction somewhere between omruleset and other
>> output modules??
>>
>> If I remove the omruleset calls and put everything from them in the
>> main
>> ruleset (bound to imudp and imtcp), it runs without any problems.
>>
>> I'm running 5.6.2 on CentOS 5.5 x86_64.
>>
>> Thanks,
>> Jason
>>
>> Sample config that segfaults is below:
>> ====== BEGIN CODE====
>>
>> #### GLOBAL DIRECTIVES ####
>>
>> $FileOwner root
>> $FileGroup root
>> $FileCreateMode 0640
>> $DirOwner root
>> $DirGroup root
>> $DirCreateMode 0750
>>
>> # Use default timestamp format
>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>>
>> $WorkDirectory /var/rsyslog/work
>>
>> # Provides logging to MySQL - define before any rules that use it
>> $ModLoad ommysql
>> $ModLoad omruleset
>>
>> # templates - include first
>> $IncludeConfig /etc/rsyslog.d/templates.conf
>> $IncludeConfig /etc/rsyslog.d/dhcp-templates.conf
>>
>> #### Imports - ORDER MATTERS HERE ####
>>
>> $RuleSet BSC-ruleset
>>
>> *.* /var/log/TESTING/rules-BSC-ruleset
>>
>> if $msg contains 'user_login_suc' then
>> /var/log/TESTING/rules-BSC-ruleset-logout_suc
>> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login
>> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login-web
>> & ~
>>
>> $RuleSet DHCP-parsing
>>
>> *.* /var/log/TESTING/rules-DHCP-parsing
>>
>> :msg, startswith, " DHCPREQUEST for"
>> /var/log/TESTING/rules-DHCP-parsing-requestfor
>> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
>> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTIP
>> & ~ ### DISCARD
>>
>> if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no client
>> hardware address' ) ) then /var/log/TESTING/rules-DHCP-parsing-ackto \
>> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoMAC
>> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoIP
>> & ~ ### DISCARD
>>
>> $RuleSet remote
>>
>> *.* /var/log/TESTING/rules-remote
>>
>> $ActionOmrulesetRulesetName BSC-ruleset
>> if $fromhost-ip == '128.6.30.195' or $fromhost-ip == '128.6.30.196' \
>> then /var/log/TESTING/rules-remote-BSC
>> & :omruleset:
>> & ~
>>
>> $ActionOmrulesetRulesetName DHCP-parsing
>> if ( $fromhost-ip == '172.16.25.114' ) or ( $fromhost-ip ==
>> '172.16.25.116' ) or ( $fromhost-ip == '128.6.17.217' ) then
>> /var/log/TESTING/rules-remote-dhcp
>> & :omruleset:
>>
>> $RuleSet local
>> *.*                /var/log/TESTING/local
>>
>> ##### END IMPORTS
>>
>> #### Default Ruleset ####
>> # since we bind TCP and UDP to remote, this should only handle local
>> $DefaultRuleset local
>>
>> #### MODULES ####
>>
>> $ModLoad imuxsock.so    # provides support for local system logging
>> (e.g. via logger command)
>> $ModLoad imklog.so      # provides kernel logging support (previously
>> done by rklogd)
>> #$ModLoad immark.so     # provides --MARK-- message capability
>>
>> #### BIND INPUTS ####
>>
>> # Provides UDP syslog reception
>> $ModLoad imudp.so
>> $UDPServerAddress *
>> $InputUDPServerBindRuleset remote # bind UDP to the remote ruleset
>> $UDPServerRun 514
>>
>> # Provides TCP syslog reception
>> $ModLoad imtcp.so
>> $InputTCPServerBindRuleset remote # bind tcp to the remote ruleset
>> $InputTCPServerRun 514
>>
>>
>> ====== END CODE ====
>>
>> Champ Clark III [Softwink] wrote:
>>     
>>> On Fri, Jan 07, 2011 at 09:13:49AM -0500, Jason Antman wrote:
>>>
>>>       
>>>> Since I haven't gotten any response to this... can anyone at least
>>>>         
>> give
>>     
>>>> me a yes or no answer:
>>>>
>>>>         
>>> 	I think Rainer might still be on vacation.   It might be a bit
>>> before he can look at it.  Hopefully someone else might have a answer
>>> for you.
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>>       
>> ---
>>     
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
>   


From rgerhards at hq.adiscon.com  Mon Jan 10 21:08:06 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 10 Jan 2011 21:08:06 +0100
Subject: [rsyslog] Multiple rulesets and queues - strange behavior,
	problems logging to MySQL
References: <4D126285.50002@jasonantman.com>	<4D271F9D.7060302@jasonantman.com><20110107143313.GC15088@bundy.vistech.net>	<4D2764AC.9030605@jasonantman.com><9B6E2A8877C38245BFB15CC491A11DA71DD99E@GRFEXC.intern.adiscon.com>
	<4D2B4C18.4000704@jasonantman.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9B5@GRFEXC.intern.adiscon.com>

FYI: I have briefly tried to reproduce the issue with a very simple config
today, but that didn't work out. It would be great if you could reduce the
config somewhat further. A debug log would probably also be useful.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jason Antman
> Sent: Monday, January 10, 2011 7:13 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Multiple rulesets and queues - strange behavior,
> problems logging to MySQL
> 
> Haven't given that a try yet. I just finished working up a stable
> (albeit slow and very sub-optimal) configuration with everything in one
> ruleset, and ran that on my development server for the past 3 days.
> Once
> I have that migrated to production I'll give one input/one additional
> ruleset a try, probably sometime early tomorrow (by east coast US
> time).
> 
> Thanks,
> Jason
> 
> Rainer Gerhards wrote:
> > Jason,
> >
> > any chance you can reproduce this with a single input and a single
> additional
> > ruleset? Did you try this?
> >
> > Rainer
> >
> >
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Jason Antman
> >> Sent: Friday, January 07, 2011 8:08 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] Multiple rulesets and queues - strange
> behavior,
> >> problems logging to MySQL
> >>
> >> Ok.
> >>
> >> Just as a quick overview (I haven't analyzed enough of the debugging
> >> information that I collected to submit a bug report), rsyslog
> becomes
> >> unstable when omruleset is called from within a ruleset. Crashes
> were a
> >> mix of segfaults and malloc/realloc errors. With my original config
> >> (complex mix of multiple ommysql calls per if statement, etc.)
> >> triggered
> >> a crash within the first few seconds of running, every time. I
> created
> >> a
> >> much smaller sample config (one ruleset bound to imudp/imtcp, two
> >> rulesets called from there each with two if statement rules) and it
> >> runs
> >> for about 30 seconds before dieing.
> >>
> >> Perhaps there's some interaction somewhere between omruleset and
> other
> >> output modules??
> >>
> >> If I remove the omruleset calls and put everything from them in the
> >> main
> >> ruleset (bound to imudp and imtcp), it runs without any problems.
> >>
> >> I'm running 5.6.2 on CentOS 5.5 x86_64.
> >>
> >> Thanks,
> >> Jason
> >>
> >> Sample config that segfaults is below:
> >> ====== BEGIN CODE====
> >>
> >> #### GLOBAL DIRECTIVES ####
> >>
> >> $FileOwner root
> >> $FileGroup root
> >> $FileCreateMode 0640
> >> $DirOwner root
> >> $DirGroup root
> >> $DirCreateMode 0750
> >>
> >> # Use default timestamp format
> >> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
> >>
> >> $WorkDirectory /var/rsyslog/work
> >>
> >> # Provides logging to MySQL - define before any rules that use it
> >> $ModLoad ommysql
> >> $ModLoad omruleset
> >>
> >> # templates - include first
> >> $IncludeConfig /etc/rsyslog.d/templates.conf
> >> $IncludeConfig /etc/rsyslog.d/dhcp-templates.conf
> >>
> >> #### Imports - ORDER MATTERS HERE ####
> >>
> >> $RuleSet BSC-ruleset
> >>
> >> *.* /var/log/TESTING/rules-BSC-ruleset
> >>
> >> if $msg contains 'user_login_suc' then
> >> /var/log/TESTING/rules-BSC-ruleset-logout_suc
> >> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login
> >> & :ommysql:localhost,wireless_logs,syslogger,syslogger;BSC-login-web
> >> & ~
> >>
> >> $RuleSet DHCP-parsing
> >>
> >> *.* /var/log/TESTING/rules-DHCP-parsing
> >>
> >> :msg, startswith, " DHCPREQUEST for"
> >> /var/log/TESTING/rules-DHCP-parsing-requestfor
> >> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTMAC
> >> & :ommysql:localhost,test,syslogger,syslogger;DHCPREQUESTIP
> >> & ~ ### DISCARD
> >>
> >> if $msg startswith ' DHCPACK to' and ( not ( $msg contains 'no
> client
> >> hardware address' ) ) then /var/log/TESTING/rules-DHCP-parsing-ackto
> \
> >> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoMAC
> >> & :ommysql:localhost,test,syslogger,syslogger;DHCPACKtoIP
> >> & ~ ### DISCARD
> >>
> >> $RuleSet remote
> >>
> >> *.* /var/log/TESTING/rules-remote
> >>
> >> $ActionOmrulesetRulesetName BSC-ruleset
> >> if $fromhost-ip == '128.6.30.195' or $fromhost-ip == '128.6.30.196'
> \
> >> then /var/log/TESTING/rules-remote-BSC
> >> & :omruleset:
> >> & ~
> >>
> >> $ActionOmrulesetRulesetName DHCP-parsing
> >> if ( $fromhost-ip == '172.16.25.114' ) or ( $fromhost-ip ==
> >> '172.16.25.116' ) or ( $fromhost-ip == '128.6.17.217' ) then
> >> /var/log/TESTING/rules-remote-dhcp
> >> & :omruleset:
> >>
> >> $RuleSet local
> >> *.*                /var/log/TESTING/local
> >>
> >> ##### END IMPORTS
> >>
> >> #### Default Ruleset ####
> >> # since we bind TCP and UDP to remote, this should only handle local
> >> $DefaultRuleset local
> >>
> >> #### MODULES ####
> >>
> >> $ModLoad imuxsock.so    # provides support for local system logging
> >> (e.g. via logger command)
> >> $ModLoad imklog.so      # provides kernel logging support
> (previously
> >> done by rklogd)
> >> #$ModLoad immark.so     # provides --MARK-- message capability
> >>
> >> #### BIND INPUTS ####
> >>
> >> # Provides UDP syslog reception
> >> $ModLoad imudp.so
> >> $UDPServerAddress *
> >> $InputUDPServerBindRuleset remote # bind UDP to the remote ruleset
> >> $UDPServerRun 514
> >>
> >> # Provides TCP syslog reception
> >> $ModLoad imtcp.so
> >> $InputTCPServerBindRuleset remote # bind tcp to the remote ruleset
> >> $InputTCPServerRun 514
> >>
> >>
> >> ====== END CODE ====
> >>
> >> Champ Clark III [Softwink] wrote:
> >>
> >>> On Fri, Jan 07, 2011 at 09:13:49AM -0500, Jason Antman wrote:
> >>>
> >>>
> >>>> Since I haven't gotten any response to this... can anyone at least
> >>>>
> >> give
> >>
> >>>> me a yes or no answer:
> >>>>
> >>>>
> >>> 	I think Rainer might still be on vacation.   It might be a bit
> >>> before he can look at it.  Hopefully someone else might have a
> answer
> >>> for you.
> >>>
> >>>
> >>>
> >>> -------------------------------------------------------------------
> --
> >>>
> >> ---
> >>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>>
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Tue Jan 11 15:46:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Jan 2011 15:46:24 +0100
Subject: [rsyslog] Strange liblognorm issue(s)...
References: <20101020175951.GA13070@bundy.vistech.net><20110110013722.GA24485@bundy.vistech.net><20110110174540.GA11336@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
	<20110110180914.GB11909@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>

I found it, a dumb mistake probably rooted in year-long training with
zero-terminated strings ;)

fix:
http://git.adiscon.com/?p=liblognorm.git;a=commit;h=c36358fb071f269b54e3b8891
f300ee15a231eb3

It crashed iff a log prefix for a node was generated with *exactly* 16 bytes
of size.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Monday, January 10, 2011 7:09 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Strange liblognorm issue(s)...
> 
> On Mon, Jan 10, 2011 at 06:57:00PM +0100, Rainer Gerhards wrote:
> > I couldn't stand it, copied & pasted the info. Same segfault here :)
> valgrind
> > tells me it is ptree.c/627 (628 if you remove the comment). This is
> inside
> > the parse tree as expected. Let me see if I find out tomorrow what
> goes on.
> 
> 	No rush.  Just wanting to point it out as I ran into it
> yesterday.  It was driving me nuts :)
> 
> --
>         Champ Clark III | Softwink, Inc | 800-538-9357 x 101
>                      http://www.softwink.com
> 
> GPG Key ID: 58A2A58F
> Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
> If it wasn't for C, we'd be using BASI, PASAL and OBOL.

From champ at softwink.com  Tue Jan 11 16:01:15 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Tue, 11 Jan 2011 10:01:15 -0500
Subject: [rsyslog] Strange liblognorm issue(s)...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
	<20110110180914.GB11909@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>
Message-ID: <20110111150115.GA5622@bundy.vistech.net>

On Tue, Jan 11, 2011 at 03:46:24PM +0100, Rainer Gerhards wrote:
> I found it, a dumb mistake probably rooted in year-long training with
> zero-terminated strings ;)
> 
> fix:
> http://git.adiscon.com/?p=liblognorm.git;a=commit;h=c36358fb071f269b54e3b8891
> f300ee15a231eb3
> 
> It crashed iff a log prefix for a node was generated with *exactly* 16 bytes
> of size.

	Ah!  Oh the fun of developing.....  :) Thanks for the fix.  I'll
be doing some intense "testing" of liblognorm the next few days.  Will 
let you know if anything else comes up! 


-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110111/64fde6fb/attachment.pgp>

From rgerhards at hq.adiscon.com  Tue Jan 11 16:19:53 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Jan 2011 16:19:53 +0100
Subject: [rsyslog] Strange liblognorm issue(s)...
References: <20101020175951.GA13070@bundy.vistech.net><20110110013722.GA24485@bundy.vistech.net><20110110174540.GA11336@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com><20110110180914.GB11909@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>
	<20110111150115.GA5622@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Tuesday, January 11, 2011 4:01 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Strange liblognorm issue(s)...
> 
> On Tue, Jan 11, 2011 at 03:46:24PM +0100, Rainer Gerhards wrote:
> > I found it, a dumb mistake probably rooted in year-long training with
> > zero-terminated strings ;)
> >
> > fix:
> >
> http://git.adiscon.com/?p=liblognorm.git;a=commit;h=c36358fb071f269b54e
> 3b8891
> > f300ee15a231eb3
> >
> > It crashed iff a log prefix for a node was generated with *exactly*
> 16 bytes
> > of size.
> 
> 	Ah!  Oh the fun of developing.....  :) Thanks for the fix.  I'll
> be doing some intense "testing" of liblognorm the next few days.  Will
> let you know if anything else comes up!

Yes, that would be excellent! While I still have a backlog, I hope to be able
to focus on liblognorm, libee, libestr and its integration into rsyslog the
next few weeks.

BTW: if you have suggestions for parsers (like ipv4), please let me know. In
order to prevent false positives, I definitely need to write a couple of more
specific parsers (probably one of the next things to do).

Rainer

From champ at softwink.com  Tue Jan 11 16:38:20 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Tue, 11 Jan 2011 10:38:20 -0500
Subject: [rsyslog] Strange liblognorm issue(s)...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com>
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
	<20110110180914.GB11909@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>
	<20110111150115.GA5622@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com>
Message-ID: <20110111153820.GA8396@bundy.vistech.net>

> Yes, that would be excellent! While I still have a backlog, I hope to be able
> to focus on liblognorm, libee, libestr and its integration into rsyslog the
> next few weeks.
> 
> BTW: if you have suggestions for parsers (like ipv4), please let me know. In
> order to prevent false positives, I definitely need to write a couple of more
> specific parsers (probably one of the next things to do).

	Pulled down the latest git,  and tested.  Worked great :)  I'm 
not sure if it's in there yet,  but a ipv6 will probably be requested at
some point.   I'll let you know if I run into anything. 

-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110111/2f9c11bc/attachment.pgp>

From rgerhards at hq.adiscon.com  Tue Jan 11 16:41:12 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Jan 2011 16:41:12 +0100
Subject: [rsyslog] custom parsers
References: <20101020175951.GA13070@bundy.vistech.net><20110110013722.GA24485@bundy.vistech.net><20110110174540.GA11336@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com><20110110180914.GB11909@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com><20110111150115.GA5622@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com>
	<20110111153820.GA8396@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9C3@GRFEXC.intern.adiscon.com>

All: I'll probably create a dedicated liblogging list soon -- sorry for the
noise

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Tuesday, January 11, 2011 4:38 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Strange liblognorm issue(s)...
> 
> > Yes, that would be excellent! While I still have a backlog, I hope to
> be able
> > to focus on liblognorm, libee, libestr and its integration into
> rsyslog the
> > next few weeks.
> >
> > BTW: if you have suggestions for parsers (like ipv4), please let me
> know. In
> > order to prevent false positives, I definitely need to write a couple
> of more
> > specific parsers (probably one of the next things to do).
> 
> 	Pulled down the latest git,  and tested.  Worked great :)  I'm
> not sure if it's in there yet,  but a ipv6 will probably be requested
> at
> some point.   I'll let you know if I run into anything.

IPv6 is definitely something needed. I think also a couple of date formats.
The quality and speed of the normalization process greatly depends on the
parsers. I hopefully will find time to write a paper on the method used, then
it will become clear. But it is very important to have good parsers *and*
priorites in which they are used (priorities are on the todo list as well).

Rainer
> 
> --
>         Champ Clark III | Softwink, Inc | 800-538-9357 x 101
>                      http://www.softwink.com
> 
> GPG Key ID: 58A2A58F
> Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
> If it wasn't for C, we'd be using BASI, PASAL and OBOL.

From champ at softwink.com  Tue Jan 11 16:45:42 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Tue, 11 Jan 2011 10:45:42 -0500
Subject: [rsyslog] custom parsers
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9C3@GRFEXC.intern.adiscon.com>
References: <20101020175951.GA13070@bundy.vistech.net>
	<20110110013722.GA24485@bundy.vistech.net>
	<20110110174540.GA11336@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
	<20110110180914.GB11909@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>
	<20110111150115.GA5622@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com>
	<20110111153820.GA8396@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9C3@GRFEXC.intern.adiscon.com>
Message-ID: <20110111154542.GA8942@bundy.vistech.net>

On Tue, Jan 11, 2011 at 04:41:12PM +0100, Rainer Gerhards wrote:
> All: I'll probably create a dedicated liblogging list soon -- sorry for the
> noise

	Just make sure you annouce it.....


-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110111/bbb0b8a1/attachment.pgp>

From rgerhards at hq.adiscon.com  Tue Jan 11 17:54:53 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Jan 2011 17:54:53 +0100
Subject: [rsyslog] custom parsers
References: <20101020175951.GA13070@bundy.vistech.net><20110110013722.GA24485@bundy.vistech.net><20110110174540.GA11336@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com><20110110180914.GB11909@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com><20110111150115.GA5622@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com><20110111153820.GA8396@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9C3@GRFEXC.intern.adiscon.com>
	<20110111154542.GA8942@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9C6@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Champ Clark III [Softwink]
> Sent: Tuesday, January 11, 2011 4:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] custom parsers
> 
> On Tue, Jan 11, 2011 at 04:41:12PM +0100, Rainer Gerhards wrote:
> > All: I'll probably create a dedicated liblogging list soon -- sorry
> for the
> > noise
> 
> 	Just make sure you annouce it.....

will do -- and I guess it was unwise to quote as I did. Probably this was
lost:

> 	Pulled down the latest git,  and tested.  Worked great :)  I'm
> not sure if it's in there yet,  but a ipv6 will probably be requested
> at
> some point.   I'll let you know if I run into anything.

IPv6 is definitely something needed. I think also a couple of date formats.
The quality and speed of the normalization process greatly depends on the
parsers. I hopefully will find time to write a paper on the method used, then
it will become clear. But it is very important to have good parsers *and*
priorites in which they are used (priorities are on the todo list as well).

Rainer

From champ at softwink.com  Wed Jan 12 22:52:57 2011
From: champ at softwink.com (Champ Clark III [Softwink])
Date: Wed, 12 Jan 2011 16:52:57 -0500
Subject: [rsyslog] custom parsers
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9C6@GRFEXC.intern.adiscon.com>
References: <20110110174540.GA11336@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com>
	<20110110180914.GB11909@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com>
	<20110111150115.GA5622@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com>
	<20110111153820.GA8396@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9C3@GRFEXC.intern.adiscon.com>
	<20110111154542.GA8942@bundy.vistech.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9C6@GRFEXC.intern.adiscon.com>
Message-ID: <20110112215257.GA30213@bundy.vistech.net>

> > 	Just make sure you annouce it.....
> 
> will do -- and I guess it was unwise to quote as I did. Probably this was
> lost:
> 
> > 	Pulled down the latest git,  and tested.  Worked great :)  I'm
> > not sure if it's in there yet,  but a ipv6 will probably be requested
> > at
> > some point.   I'll let you know if I run into anything.
> 
> IPv6 is definitely something needed. I think also a couple of date formats.
> The quality and speed of the normalization process greatly depends on the
> parsers. I hopefully will find time to write a paper on the method used, then
> it will become clear. But it is very important to have good parsers *and*
> priorites in which they are used (priorities are on the todo list as well).

	I got into writing some parser rules for a Sonicwall (IPS
device) today.  I've run into a couple of small,  but not impacting
functionality,  issues.  Rather than clutter this list with them,  I'll
wait till you get another mailman/mailing list setup for it.  
	
	Basically,  just ideas for a couple more parser types and some
rule writting things.... 

-- 
        Champ Clark III | Softwink, Inc | 800-538-9357 x 101
                     http://www.softwink.com

GPG Key ID: 58A2A58F
Key fingerprint = 7734 2A1C 007D 581E BDF7  6AD5 0F1F 655F 58A2 A58F
If it wasn't for C, we'd be using BASI, PASAL and OBOL.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110112/d581c717/attachment.pgp>

From rgerhards at hq.adiscon.com  Thu Jan 13 09:27:35 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Jan 2011 09:27:35 +0100
Subject: [rsyslog] question about the parser interface.
In-Reply-To: <alpine.DEB.2.00.1012132043470.19517@asgard.lang.hm>
References: <alpine.DEB.2.00.1012132006340.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012132043470.19517@asgard.lang.hm>
Message-ID: <4D2EB777.60200@hq.adiscon.com>

David,

sorry for the delay (I have to admit I did not notice in December that 
you actually needed my help in integrating into the build system -- 
looks like I really needed my vacation ;)).

I have now merged the patch into a new branch v5-devel-david and 
integrated the new module into the build system (this is done via a 
separate commit, so it should be fairly obvious what you need to do if 
there is more need :)). I have also tried to compile and that works.

However, I have *not at all* looked at the code or tested anything. I 
guess it's best if you can continue working on it. Once you tell me it 
is ready, I can merge it into the main v5-devel branch. It would be 
great if you could provide at least some minimal documentation before we 
do that (have a look at the other modules' docs at ./doc/<modname>.html).

Thanks,
Rainer

On 12/14/2010 05:46 AM, david at lang.hm wrote:
> If the message can be modified by the parser, I think this is
> appriximatly what is needed for the cisco problem.
>
> I'm not sure exactly what to do to tie it in to the build system, so
> this one isn't even compile tested.
>
> David Lang
>
> On Mon, 13 Dec 2010, david at lang.hm wrote:
>
>> Date: Mon, 13 Dec 2010 20:26:35 -0800 (PST)
>> From: david at lang.hm
>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>> Subject: [rsyslog] question about the parser interface.
>>
>> Is it possible for a parser to just modify the input string and then
>> let it fall through for another parser to handle the modified string?
>>
>> I have two rather simple parsers I want to write that fall in this
>> category
>>
>> 1. Cisco with name resolution
>>
>> A cisco without name resolution turned on logs
>> <pri> timestamp IPaddr %tag msg
>>
>> a cisco with name resolution turned on logs
>> <pri> timestamp name : %tag msg
>>
>> I want to detect the bare : in the syslog field followed by the % at
>> the start of the next tag, and if I find them, just memmove everything
>> up (so that the % ends up where the : was, shortening the string by
>> two characters), then let if fall through for normal processing.
>>
>> 2. AIX forwarding messages
>>
>> AIX defaults to messages in the format
>>
>> <pri> timestamp Message Forwarded From hostname syslogtag msg
>>
>> I want to look for 'Message Forwarded From' starting in the hostname
>> field, and if I find them, memmove everything up so that the hostname
>> is in the right place, and again let everything fall through to the
>> normal parser for handling.
>>
>> I really don't want to have to duplicate the normal parser in each of
>> these parsers as they are just (almost) trivial cleanups of the log
>> message before it's handled normally.
>>
>> David Lang


From david at lang.hm  Thu Jan 13 09:33:42 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 13 Jan 2011 00:33:42 -0800 (PST)
Subject: [rsyslog] question about the parser interface.
In-Reply-To: <4D2EB777.60200@hq.adiscon.com>
References: <alpine.DEB.2.00.1012132006340.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012132043470.19517@asgard.lang.hm>
	<4D2EB777.60200@hq.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101130031590.21529@asgard.lang.hm>

am I correct in thinking that I can have the parser modify it's input 
string and then fall through and have following parsers process the 
modified string?

If this is possible, then this sort of thing is very simple, if not I end 
up duplicating a lot of the default parser with just a small modification 
at the beginning.

David Lang

On Thu, 13 Jan 2011, Rainer Gerhards wrote:

> Date: Thu, 13 Jan 2011 09:27:35 +0100
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: [rsyslog] question about the parser interface.
> 
> David,
>
> sorry for the delay (I have to admit I did not notice in December that you 
> actually needed my help in integrating into the build system -- looks like I 
> really needed my vacation ;)).
>
> I have now merged the patch into a new branch v5-devel-david and integrated 
> the new module into the build system (this is done via a separate commit, so 
> it should be fairly obvious what you need to do if there is more need :)). I 
> have also tried to compile and that works.
>
> However, I have *not at all* looked at the code or tested anything. I guess 
> it's best if you can continue working on it. Once you tell me it is ready, I 
> can merge it into the main v5-devel branch. It would be great if you could 
> provide at least some minimal documentation before we do that (have a look at 
> the other modules' docs at ./doc/<modname>.html).
>
> Thanks,
> Rainer
>
> On 12/14/2010 05:46 AM, david at lang.hm wrote:
>> If the message can be modified by the parser, I think this is
>> appriximatly what is needed for the cisco problem.
>> 
>> I'm not sure exactly what to do to tie it in to the build system, so
>> this one isn't even compile tested.
>> 
>> David Lang
>> 
>> On Mon, 13 Dec 2010, david at lang.hm wrote:
>> 
>>> Date: Mon, 13 Dec 2010 20:26:35 -0800 (PST)
>>> From: david at lang.hm
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: [rsyslog] question about the parser interface.
>>> 
>>> Is it possible for a parser to just modify the input string and then
>>> let it fall through for another parser to handle the modified string?
>>> 
>>> I have two rather simple parsers I want to write that fall in this
>>> category
>>> 
>>> 1. Cisco with name resolution
>>> 
>>> A cisco without name resolution turned on logs
>>> <pri> timestamp IPaddr %tag msg
>>> 
>>> a cisco with name resolution turned on logs
>>> <pri> timestamp name : %tag msg
>>> 
>>> I want to detect the bare : in the syslog field followed by the % at
>>> the start of the next tag, and if I find them, just memmove everything
>>> up (so that the % ends up where the : was, shortening the string by
>>> two characters), then let if fall through for normal processing.
>>> 
>>> 2. AIX forwarding messages
>>> 
>>> AIX defaults to messages in the format
>>> 
>>> <pri> timestamp Message Forwarded From hostname syslogtag msg
>>> 
>>> I want to look for 'Message Forwarded From' starting in the hostname
>>> field, and if I find them, memmove everything up so that the hostname
>>> is in the right place, and again let everything fall through to the
>>> normal parser for handling.
>>> 
>>> I really don't want to have to duplicate the normal parser in each of
>>> these parsers as they are just (almost) trivial cleanups of the log
>>> message before it's handled normally.
>>> 
>>> David Lang
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Thu Jan 13 09:36:13 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Jan 2011 09:36:13 +0100
Subject: [rsyslog] question about the parser interface.
References: <alpine.DEB.2.00.1012132006340.19517@asgard.lang.hm><alpine.DEB.2.00.1012132043470.19517@asgard.lang.hm><4D2EB777.60200@hq.adiscon.com>
	<alpine.DEB.2.00.1101130031590.21529@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9DF@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 13, 2011 9:34 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] question about the parser interface.
> 
> am I correct in thinking that I can have the parser modify it's input
> string and then fall through and have following parsers process the
> modified string?

Yeah, that's right. Saying this, this is actually not a real parser module,
but a message modification module (working at the parser level). This is just
a differentiation in how this module is treated logically: the inner workings
are the same. The trick is to simply modify the message, but return the "I am
not the right parser" state. That will tell the engine to try the other
parsers as well. The other ones do NOT know that the message has been
modified.

Rainer
> 
> If this is possible, then this sort of thing is very simple, if not I
> end
> up duplicating a lot of the default parser with just a small
> modification
> at the beginning.
> 
> David Lang
> 
> On Thu, 13 Jan 2011, Rainer Gerhards wrote:
> 
> > Date: Thu, 13 Jan 2011 09:27:35 +0100
> > From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog-users <rsyslog at lists.adiscon.com>
> > Subject: Re: [rsyslog] question about the parser interface.
> >
> > David,
> >
> > sorry for the delay (I have to admit I did not notice in December
> that you
> > actually needed my help in integrating into the build system -- looks
> like I
> > really needed my vacation ;)).
> >
> > I have now merged the patch into a new branch v5-devel-david and
> integrated
> > the new module into the build system (this is done via a separate
> commit, so
> > it should be fairly obvious what you need to do if there is more need
> :)). I
> > have also tried to compile and that works.
> >
> > However, I have *not at all* looked at the code or tested anything. I
> guess
> > it's best if you can continue working on it. Once you tell me it is
> ready, I
> > can merge it into the main v5-devel branch. It would be great if you
> could
> > provide at least some minimal documentation before we do that (have a
> look at
> > the other modules' docs at ./doc/<modname>.html).
> >
> > Thanks,
> > Rainer
> >
> > On 12/14/2010 05:46 AM, david at lang.hm wrote:
> >> If the message can be modified by the parser, I think this is
> >> appriximatly what is needed for the cisco problem.
> >>
> >> I'm not sure exactly what to do to tie it in to the build system, so
> >> this one isn't even compile tested.
> >>
> >> David Lang
> >>
> >> On Mon, 13 Dec 2010, david at lang.hm wrote:
> >>
> >>> Date: Mon, 13 Dec 2010 20:26:35 -0800 (PST)
> >>> From: david at lang.hm
> >>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>> To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>> Subject: [rsyslog] question about the parser interface.
> >>>
> >>> Is it possible for a parser to just modify the input string and
> then
> >>> let it fall through for another parser to handle the modified
> string?
> >>>
> >>> I have two rather simple parsers I want to write that fall in this
> >>> category
> >>>
> >>> 1. Cisco with name resolution
> >>>
> >>> A cisco without name resolution turned on logs
> >>> <pri> timestamp IPaddr %tag msg
> >>>
> >>> a cisco with name resolution turned on logs
> >>> <pri> timestamp name : %tag msg
> >>>
> >>> I want to detect the bare : in the syslog field followed by the %
> at
> >>> the start of the next tag, and if I find them, just memmove
> everything
> >>> up (so that the % ends up where the : was, shortening the string by
> >>> two characters), then let if fall through for normal processing.
> >>>
> >>> 2. AIX forwarding messages
> >>>
> >>> AIX defaults to messages in the format
> >>>
> >>> <pri> timestamp Message Forwarded From hostname syslogtag msg
> >>>
> >>> I want to look for 'Message Forwarded From' starting in the
> hostname
> >>> field, and if I find them, memmove everything up so that the
> hostname
> >>> is in the right place, and again let everything fall through to the
> >>> normal parser for handling.
> >>>
> >>> I really don't want to have to duplicate the normal parser in each
> of
> >>> these parsers as they are just (almost) trivial cleanups of the log
> >>> message before it's handled normally.
> >>>
> >>> David Lang
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Thu Jan 13 09:39:05 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 13 Jan 2011 00:39:05 -0800 (PST)
Subject: [rsyslog] question about the parser interface.
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD9DF@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1012132006340.19517@asgard.lang.hm><alpine.DEB.2.00.1012132043470.19517@asgard.lang.hm><4D2EB777.60200@hq.adiscon.com>
	<alpine.DEB.2.00.1101130031590.21529@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DD9DF@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101130038000.21529@asgard.lang.hm>

On Thu, 13 Jan 2011, Rainer Gerhards wrote:

>> -----Original Message-----
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>
>> am I correct in thinking that I can have the parser modify it's input
>> string and then fall through and have following parsers process the
>> modified string?
>
> Yeah, that's right. Saying this, this is actually not a real parser module,
> but a message modification module (working at the parser level). This is just
> a differentiation in how this module is treated logically: the inner workings
> are the same. The trick is to simply modify the message, but return the "I am
> not the right parser" state. That will tell the engine to try the other
> parsers as well. The other ones do NOT know that the message has been
> modified.

good, I'll test the cisco one later and write the AIX one (should be 
equally trivial)

David Lang

> Rainer
>>
>> If this is possible, then this sort of thing is very simple, if not I
>> end
>> up duplicating a lot of the default parser with just a small
>> modification
>> at the beginning.
>>
>> David Lang
>>
>> On Thu, 13 Jan 2011, Rainer Gerhards wrote:
>>
>>> Date: Thu, 13 Jan 2011 09:27:35 +0100
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: Re: [rsyslog] question about the parser interface.
>>>
>>> David,
>>>
>>> sorry for the delay (I have to admit I did not notice in December
>> that you
>>> actually needed my help in integrating into the build system -- looks
>> like I
>>> really needed my vacation ;)).
>>>
>>> I have now merged the patch into a new branch v5-devel-david and
>> integrated
>>> the new module into the build system (this is done via a separate
>> commit, so
>>> it should be fairly obvious what you need to do if there is more need
>> :)). I
>>> have also tried to compile and that works.
>>>
>>> However, I have *not at all* looked at the code or tested anything. I
>> guess
>>> it's best if you can continue working on it. Once you tell me it is
>> ready, I
>>> can merge it into the main v5-devel branch. It would be great if you
>> could
>>> provide at least some minimal documentation before we do that (have a
>> look at
>>> the other modules' docs at ./doc/<modname>.html).
>>>
>>> Thanks,
>>> Rainer
>>>
>>> On 12/14/2010 05:46 AM, david at lang.hm wrote:
>>>> If the message can be modified by the parser, I think this is
>>>> appriximatly what is needed for the cisco problem.
>>>>
>>>> I'm not sure exactly what to do to tie it in to the build system, so
>>>> this one isn't even compile tested.
>>>>
>>>> David Lang
>>>>
>>>> On Mon, 13 Dec 2010, david at lang.hm wrote:
>>>>
>>>>> Date: Mon, 13 Dec 2010 20:26:35 -0800 (PST)
>>>>> From: david at lang.hm
>>>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>>>> Subject: [rsyslog] question about the parser interface.
>>>>>
>>>>> Is it possible for a parser to just modify the input string and
>> then
>>>>> let it fall through for another parser to handle the modified
>> string?
>>>>>
>>>>> I have two rather simple parsers I want to write that fall in this
>>>>> category
>>>>>
>>>>> 1. Cisco with name resolution
>>>>>
>>>>> A cisco without name resolution turned on logs
>>>>> <pri> timestamp IPaddr %tag msg
>>>>>
>>>>> a cisco with name resolution turned on logs
>>>>> <pri> timestamp name : %tag msg
>>>>>
>>>>> I want to detect the bare : in the syslog field followed by the %
>> at
>>>>> the start of the next tag, and if I find them, just memmove
>> everything
>>>>> up (so that the % ends up where the : was, shortening the string by
>>>>> two characters), then let if fall through for normal processing.
>>>>>
>>>>> 2. AIX forwarding messages
>>>>>
>>>>> AIX defaults to messages in the format
>>>>>
>>>>> <pri> timestamp Message Forwarded From hostname syslogtag msg
>>>>>
>>>>> I want to look for 'Message Forwarded From' starting in the
>> hostname
>>>>> field, and if I find them, memmove everything up so that the
>> hostname
>>>>> is in the right place, and again let everything fall through to the
>>>>> normal parser for handling.
>>>>>
>>>>> I really don't want to have to duplicate the normal parser in each
>> of
>>>>> these parsers as they are just (almost) trivial cleanups of the log
>>>>> message before it's handled normally.
>>>>>
>>>>> David Lang
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Thu Jan 13 09:40:20 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Jan 2011 09:40:20 +0100
Subject: [rsyslog] question about the parser interface.
References: <alpine.DEB.2.00.1012132006340.19517@asgard.lang.hm><alpine.DEB.2.00.1012132043470.19517@asgard.lang.hm><4D2EB777.60200@hq.adiscon.com><alpine.DEB.2.00.1101130031590.21529@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DD9DF@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101130038000.21529@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9E0@GRFEXC.intern.adiscon.com>

> good, I'll test the cisco one later and write the AIX one (should be
> equally trivial)

It was a design goal to make such things very trival. I hope it turns out to
be true in the first "thrid party" test ;)

Rainer

From rgerhards at hq.adiscon.com  Thu Jan 13 10:43:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Jan 2011 10:43:37 +0100
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
Message-ID: <4D2EC949.4060109@hq.adiscon.com>

I have now also created a new branch for this patch:
   v5-devel-david-imfile

I added the config variable. See the commit log for useful information 
and steps. While I was a bit hesitant to merge this patch soon to the 
official branch (due to the problems I had with imfile), I begin to 
think this is over-cautious. It should really not harm any existing 
code. So please let me know when you have finished your testing of the 
new code, I'll probably merge soon then.

Thanks!
Rainer

On 12/14/2010 04:57 AM, david at lang.hm wrote:
> I discovered UnreadChar and so now mode 2 (indented follow-up lines) has
> a chance of working. again compile tested (and visually code-reviewed by
> someone else), but not executed.
>
> David Lang
>
> On Mon, 13 Dec 2010, david at lang.hm wrote:
>
>> This is a first cut of a modification to imfile to let it read
>> multi-line files.
>>
>> As-is, this should have no effect on a system as it hard-codes the
>> mode to reading single lines (I really don't understand how to set a
>> config variable, but for someone who does, it should be simple to
>> replace the '0' in imfile.c with the value of the config file)
>>
>> With this config option change, it should be possible to real logfiles
>> that have blank lines between multi-line log entries and have those
>> log entries treated as a single line.
>>
>> I also have code in place (but disabled) to try and deal with the more
>> complicated layout where all lines after the first one are indented if
>> they are part of the same log entry. The problem I have is that when I
>> discover that I have finished reading a log entry I have already read
>> the first character of the next log entry. This extra character needs
>> to be put pack into the input buffer, but I don't know if that is
>> possible or not. If this isn't the case, I need a function that will
>> let me peek at the next character in the input buffer and make my
>> decision based on that.
>>
>> This compiles, but I have not tested it anywhere yet. with the
>> hardcoded mode 0 for ('LF termination), there should be no change
>> other than an extra test against a constant for each character read
>> from a file.
>>
>> David Lang


From david at lang.hm  Thu Jan 13 11:02:51 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 13 Jan 2011 02:02:51 -0800 (PST)
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <4D2EC949.4060109@hq.adiscon.com>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>

thanks. I will try to go over both of these tomorrow, but will definantly 
do so no later than this weekend.

David Lang

On Thu, 13 Jan 2011, Rainer Gerhards wrote:

> Date: Thu, 13 Jan 2011 10:43:37 +0100
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> To: david at lang.hm, rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: imfile paragraph patch
> 
> I have now also created a new branch for this patch:
>  v5-devel-david-imfile
>
> I added the config variable. See the commit log for useful information and 
> steps. While I was a bit hesitant to merge this patch soon to the official 
> branch (due to the problems I had with imfile), I begin to think this is 
> over-cautious. It should really not harm any existing code. So please let me 
> know when you have finished your testing of the new code, I'll probably merge 
> soon then.
>
> Thanks!
> Rainer
>
> On 12/14/2010 04:57 AM, david at lang.hm wrote:
>> I discovered UnreadChar and so now mode 2 (indented follow-up lines) has
>> a chance of working. again compile tested (and visually code-reviewed by
>> someone else), but not executed.
>> 
>> David Lang
>> 
>> On Mon, 13 Dec 2010, david at lang.hm wrote:
>> 
>>> This is a first cut of a modification to imfile to let it read
>>> multi-line files.
>>> 
>>> As-is, this should have no effect on a system as it hard-codes the
>>> mode to reading single lines (I really don't understand how to set a
>>> config variable, but for someone who does, it should be simple to
>>> replace the '0' in imfile.c with the value of the config file)
>>> 
>>> With this config option change, it should be possible to real logfiles
>>> that have blank lines between multi-line log entries and have those
>>> log entries treated as a single line.
>>> 
>>> I also have code in place (but disabled) to try and deal with the more
>>> complicated layout where all lines after the first one are indented if
>>> they are part of the same log entry. The problem I have is that when I
>>> discover that I have finished reading a log entry I have already read
>>> the first character of the next log entry. This extra character needs
>>> to be put pack into the input buffer, but I don't know if that is
>>> possible or not. If this isn't the case, I need a function that will
>>> let me peek at the next character in the input buffer and make my
>>> decision based on that.
>>> 
>>> This compiles, but I have not tested it anywhere yet. with the
>>> hardcoded mode 0 for ('LF termination), there should be no change
>>> other than an extra test against a constant for each character read
>>> from a file.
>>> 
>>> David Lang
>
>

From rgerhards at hq.adiscon.com  Thu Jan 13 11:34:13 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Jan 2011 11:34:13 +0100
Subject: [rsyslog] custom parsers / liblognorm list created
References: <20110110174540.GA11336@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9B4@GRFEXC.intern.adiscon.com><20110110180914.GB11909@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9BF@GRFEXC.intern.adiscon.com><20110111150115.GA5622@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9C0@GRFEXC.intern.adiscon.com><20110111153820.GA8396@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9C3@GRFEXC.intern.adiscon.com><20110111154542.GA8942@bundy.vistech.net><9B6E2A8877C38245BFB15CC491A11DA71DD9C6@GRFEXC.intern.adiscon.com>
	<20110112215257.GA30213@bundy.vistech.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DD9E9@GRFEXC.intern.adiscon.com>

> 	I got into writing some parser rules for a Sonicwall (IPS
> device) today.  I've run into a couple of small,  but not impacting
> functionality,  issues.  Rather than clutter this list with them,  I'll
> wait till you get another mailman/mailing list setup for it.
> 
> 	Basically,  just ideas for a couple more parser types and some
> rule writting things....

I have now created the lognorm list, details here:

http://blog.gerhards.net/2011/01/new-mailing-list-for-log-normalization.html

Anyone interested in log normalization please subscribe. We really need some
momentum and exposure to get the whole thing started.

Rainer

From joseph.e.mcdonagh at gmail.com  Mon Jan 17 07:49:58 2011
From: joseph.e.mcdonagh at gmail.com (Joe McDonagh)
Date: Mon, 17 Jan 2011 01:49:58 -0500
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
Message-ID: <4D33E696.4040502@gmail.com>

Right now I currently have compatibility with version 1 on, and I am 
thinking I will build a package for older nodes for version 4, mainly 
because I really need fqdns. Unfortunately when I do a test between two 
version 4 systems, fqdn still doesn't work. Here's some info:

root at syslog:/var/log/hosts/puppet# ps aux | grep rsyslogd | grep -v grep
root     20601  0.0  0.0  29568  1296 ?        S    22:26   0:00 
rsyslogd -c4 -m 0 -t61514 -x -r514

root at puppet:~# ps aux | grep rsyslog
root     30755  0.0  0.0  45844  1284 ?        Sl   22:42   0:00 
rsyslogd -c1 -m 0

Now I am not clear if I need PreserveFQDN on both the node and server, 
so I set it on both for kicks. Here is the node config:

#  /etc/rsyslog.conf Configuration file for rsyslogd.
#
#        For more information see
#        /usr/share/doc/rsyslog/html/rsyslog_conf.html

#
# First some standard logfiles.  Log by facility.
#

$PreserveFQDN on

auth,authpriv.*         /var/log/auth.log
*.*;auth.none;authpriv.none;mail.none;cron.none,daemon.none      
-/var/log/syslog
cron.*            /dev/null
daemon.*       -/var/log/daemon.log
kern.*            -/var/log/kern.log
lpr.*          -/var/log/lpr.log
mail.*            -/var/log/mail.log
user.*            -/var/log/user.log

#
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info         -/var/log/mail.info
mail.warn         -/var/log/mail.warn
mail.err       /var/log/mail.err

#
# Logging for INN news system
#
news.crit         /var/log/news/news.crit
news.err       /var/log/news/news.err
news.notice       -/var/log/news/news.notice

#
# Some `catch-all' logfiles.
#
*.=debug;\
    auth,authpriv.none;\
    news.none;mail.none  -/var/log/debug
*.=info;*.=notice;*.=warn;\
    auth,authpriv.none;\
    cron.none;daemon.none;\
    mail.none,news.none     -/var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg           *

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/

And here is the contents of the files in the .d conf dir:

root at puppet:/var/log# cat /etc/rsyslog.d/*
# Log kernel generated UFW log messages to file
:msg,contains,"[UFW " /var/log/ufw.log

# Uncomment the following to stop logging anything that matches the last 
rule.
# Doing this will stop logging kernel generated UFW log messages to the file
# normally containing kern.* messages (eg, /var/log/kern.log)
#& ~
*.*,cron.none   @@127.0.0.1:61514

Here is the server config:

# Config file for splitting up logs by hostname and related syslog server
# configs

# Show FQDNs
$PreserveFQDN on

# Discard collectd stuff
if $syslogtag contains 'collectd' then ~

# Discard cron stuff
if $syslogtag contains 'CRON' then ~

# Custom log files first since we may discard things like apache2 messages
# later on down.

# Aggregate of corporate website logs
$template DYNcorpsite, "/var/log/custom/corpsite_apache2.log"

if $source != 'localhost' \
    and $HOSTNAME startswith 'www' \
    and $syslogtag contains 'apache2' \
then ?DYNcorpsite

if $source != 'localhost' \
    and $HOSTNAME contains 'updates' \
    and $syslogfacility-text == 'r7license_server' \
then ?DYNr7license_servers

# Aggregate of smtp gateway logs
$template DYNsmtp_gateways, "/var/log/custom/smtp_gateways.log"

if $source != 'localhost' \
    and $HOSTNAME startswith 'smtp' \
    and $syslogfacility-text == 'mail' \
then ?DYNsmtp_gateways

# List of log files without loglevel separation
$template DYNapache2,            "/var/log/hosts/%HOSTNAME%/apache2.log"
$template DYNauth_all,           "/var/log/hosts/%HOSTNAME%/auth.log"
$template DYNcron_all,           "/var/log/hosts/%HOSTNAME%/cron.log"
$template DYNdaemon_all,         "/var/log/hosts/%HOSTNAME%/daemon.log"
$template DYNdhcpd,              "/var/log/hosts/%HOSTNAME%/dhcpd.log"
$template DYNkern_all,           "/var/log/hosts/%HOSTNAME%/kern.log"
$template DYNlpr_all,            "/var/log/hosts/%HOSTNAME%/lpr.log"
$template DYNmail_all,           "/var/log/hosts/%HOSTNAME%/mail.log"
$template DYNnamed,              "/var/log/hosts/%HOSTNAME%/named.log"
$template DYNsshd,               "/var/log/hosts/%HOSTNAME%/sshd.log"
$template DYNsyslog_all,         "/var/log/hosts/%HOSTNAME%/syslog"
$template DYNuser_all,           "/var/log/hosts/%HOSTNAME%/user.log"

# First separate interesting tags then discard to lower
# duplication
if $source != 'localhost' \
    and $syslogtag contains 'apache2' \
then ?DYNapache2

if $syslogtag contains 'apache2' then ~

if $source != 'localhost' \
    and $syslogtag contains 'dhcpd' \
then ?DYNdhcpd

if $syslogtag contains 'dhcpd' then ~

if $source != 'localhost' \
    and $syslogtag contains 'named' \
then ?DYNnamed

if $syslogtag contains 'named' then ~

# Here are regular facility-based separating
if $source != 'localhost' \
    and ( \
          $syslogfacility-text == 'auth' \
          or $syslogfacility-text == 'authpriv' \
        ) \
then ?DYNauth_all

if $source != 'localhost' \
    and $syslogfacility-text == 'cron' \
then ?DYNcron_all

if $source != 'localhost' \
    and $syslogfacility-text == 'daemon' \
then ?DYNdaemon_all

if $source != 'localhost' \
    and $syslogfacility-text == 'kern' \
then ?DYNkern_all

if $source != 'localhost' \
    and $syslogfacility-text == 'lpr' \
then ?DYNlpr_all

if $source != 'localhost' \
    and $syslogfacility-text == 'mail' \
then ?DYNmail_all

if $source != 'localhost' \
    and $syslogtag contains 'sshd' \
then ?DYNsshd

if $source != 'localhost' \
    and $syslogfacility-text != 'authpriv' \
then ?DYNsyslog_all

if $source != 'localhost' \
    and $syslogfacility-text == 'user' \
then ?DYNuser_all

# Logging for the mail system.
$template DYNmail_info,   "/var/log/hosts/%HOSTNAME%/mail.info"
$template DYNmail_warn,   "/var/log/hosts/%HOSTNAME%/mail.warn"
$template DYNmail_err,    "/var/log/hosts/%HOSTNAME%/mail.err"

if $source != 'localhost' \
    and ( \
          $syslogfacility-text == 'mail' \
          and $syslogseverity-text == 'info' \
        ) \
then ?DYNmail_info

if $source != 'localhost' \
    and ( \
          $syslogfacility-text == 'mail' \
          and $syslogseverity-text == 'warn' \
        ) \
then ?DYNmail_warn

if $source != 'localhost' \
    and ( \
          $syslogfacility-text == 'mail' \
          and $syslogseverity-text == 'err' \
        ) \
then ?DYNmail_err

# Catch-all log files
$template DYNdebug,    "/var/log/hosts/%HOSTNAME%/debug"
$template DYNmessages, "/var/log/hosts/%HOSTNAME%/messages"

if $source != 'localhost' \
    and $syslogseverity-text == 'debug' \
then ?DYNdebug

if $source != 'localhost' \
    and ( \
          $syslogseverity-text == 'info' \
          or $syslogseverity-text == 'notice' \
          or $syslogseverity-text == 'warn' \
        ) \
    and ( \
          $syslogfacility-text != 'auth' \
          or $syslogfacility-text != 'authpriv' \
          or $syslogfacility-text != 'cron' \
          or $syslogfacility-text != 'daemon' \
          or $syslogfacility-text != 'mail' \
          or $syslogfacility-text != 'news' \
        ) \
then ?DYNmessages

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/

------

I can't figure this out. The messages still only show the short hostname 
in both the node and server logs. Any ideas?


--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."


From rgerhards at hq.adiscon.com  Mon Jan 17 07:53:56 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Jan 2011 07:53:56 +0100
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
References: <4D33E696.4040502@gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA13@GRFEXC.intern.adiscon.com>

Well, let's start with the basics. What exactly is the problem, e.g. where do
you expect that FQDNs show up, which messages are exactly involved, where are
they generated and which format they have? I know this sounds like obvious,
but I geuss we get close to the answer if we have precise info. 

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Joe McDonagh
> Sent: Monday, January 17, 2011 7:50 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
> 
> Right now I currently have compatibility with version 1 on, and I am
> thinking I will build a package for older nodes for version 4, mainly
> because I really need fqdns. Unfortunately when I do a test between two
> version 4 systems, fqdn still doesn't work. Here's some info:
> 
> root at syslog:/var/log/hosts/puppet# ps aux | grep rsyslogd | grep -v
> grep
> root     20601  0.0  0.0  29568  1296 ?        S    22:26   0:00
> rsyslogd -c4 -m 0 -t61514 -x -r514
> 
> root at puppet:~# ps aux | grep rsyslog
> root     30755  0.0  0.0  45844  1284 ?        Sl   22:42   0:00
> rsyslogd -c1 -m 0
> 
> Now I am not clear if I need PreserveFQDN on both the node and server,
> so I set it on both for kicks. Here is the node config:
> 
> #  /etc/rsyslog.conf Configuration file for rsyslogd.
> #
> #        For more information see
> #        /usr/share/doc/rsyslog/html/rsyslog_conf.html
> 
> #
> # First some standard logfiles.  Log by facility.
> #
> 
> $PreserveFQDN on
> 
> auth,authpriv.*         /var/log/auth.log
> *.*;auth.none;authpriv.none;mail.none;cron.none,daemon.none
> -/var/log/syslog
> cron.*            /dev/null
> daemon.*       -/var/log/daemon.log
> kern.*            -/var/log/kern.log
> lpr.*          -/var/log/lpr.log
> mail.*            -/var/log/mail.log
> user.*            -/var/log/user.log
> 
> #
> # Logging for the mail system.  Split it up so that
> # it is easy to write scripts to parse these files.
> #
> mail.info         -/var/log/mail.info
> mail.warn         -/var/log/mail.warn
> mail.err       /var/log/mail.err
> 
> #
> # Logging for INN news system
> #
> news.crit         /var/log/news/news.crit
> news.err       /var/log/news/news.err
> news.notice       -/var/log/news/news.notice
> 
> #
> # Some `catch-all' logfiles.
> #
> *.=debug;\
>     auth,authpriv.none;\
>     news.none;mail.none  -/var/log/debug
> *.=info;*.=notice;*.=warn;\
>     auth,authpriv.none;\
>     cron.none;daemon.none;\
>     mail.none,news.none     -/var/log/messages
> 
> #
> # Emergencies are sent to everybody logged in.
> #
> *.emerg           *
> 
> # Include all config files in /etc/rsyslog.d/
> $IncludeConfig /etc/rsyslog.d/
> 
> And here is the contents of the files in the .d conf dir:
> 
> root at puppet:/var/log# cat /etc/rsyslog.d/*
> # Log kernel generated UFW log messages to file
> :msg,contains,"[UFW " /var/log/ufw.log
> 
> # Uncomment the following to stop logging anything that matches the
> last
> rule.
> # Doing this will stop logging kernel generated UFW log messages to the
> file
> # normally containing kern.* messages (eg, /var/log/kern.log)
> #& ~
> *.*,cron.none   @@127.0.0.1:61514
> 
> Here is the server config:
> 
> # Config file for splitting up logs by hostname and related syslog
> server
> # configs
> 
> # Show FQDNs
> $PreserveFQDN on
> 
> # Discard collectd stuff
> if $syslogtag contains 'collectd' then ~
> 
> # Discard cron stuff
> if $syslogtag contains 'CRON' then ~
> 
> # Custom log files first since we may discard things like apache2
> messages
> # later on down.
> 
> # Aggregate of corporate website logs
> $template DYNcorpsite, "/var/log/custom/corpsite_apache2.log"
> 
> if $source != 'localhost' \
>     and $HOSTNAME startswith 'www' \
>     and $syslogtag contains 'apache2' \
> then ?DYNcorpsite
> 
> if $source != 'localhost' \
>     and $HOSTNAME contains 'updates' \
>     and $syslogfacility-text == 'r7license_server' \
> then ?DYNr7license_servers
> 
> # Aggregate of smtp gateway logs
> $template DYNsmtp_gateways, "/var/log/custom/smtp_gateways.log"
> 
> if $source != 'localhost' \
>     and $HOSTNAME startswith 'smtp' \
>     and $syslogfacility-text == 'mail' \
> then ?DYNsmtp_gateways
> 
> # List of log files without loglevel separation
> $template DYNapache2,
> "/var/log/hosts/%HOSTNAME%/apache2.log"
> $template DYNauth_all,           "/var/log/hosts/%HOSTNAME%/auth.log"
> $template DYNcron_all,           "/var/log/hosts/%HOSTNAME%/cron.log"
> $template DYNdaemon_all,         "/var/log/hosts/%HOSTNAME%/daemon.log"
> $template DYNdhcpd,              "/var/log/hosts/%HOSTNAME%/dhcpd.log"
> $template DYNkern_all,           "/var/log/hosts/%HOSTNAME%/kern.log"
> $template DYNlpr_all,            "/var/log/hosts/%HOSTNAME%/lpr.log"
> $template DYNmail_all,           "/var/log/hosts/%HOSTNAME%/mail.log"
> $template DYNnamed,              "/var/log/hosts/%HOSTNAME%/named.log"
> $template DYNsshd,               "/var/log/hosts/%HOSTNAME%/sshd.log"
> $template DYNsyslog_all,         "/var/log/hosts/%HOSTNAME%/syslog"
> $template DYNuser_all,           "/var/log/hosts/%HOSTNAME%/user.log"
> 
> # First separate interesting tags then discard to lower
> # duplication
> if $source != 'localhost' \
>     and $syslogtag contains 'apache2' \
> then ?DYNapache2
> 
> if $syslogtag contains 'apache2' then ~
> 
> if $source != 'localhost' \
>     and $syslogtag contains 'dhcpd' \
> then ?DYNdhcpd
> 
> if $syslogtag contains 'dhcpd' then ~
> 
> if $source != 'localhost' \
>     and $syslogtag contains 'named' \
> then ?DYNnamed
> 
> if $syslogtag contains 'named' then ~
> 
> # Here are regular facility-based separating
> if $source != 'localhost' \
>     and ( \
>           $syslogfacility-text == 'auth' \
>           or $syslogfacility-text == 'authpriv' \
>         ) \
> then ?DYNauth_all
> 
> if $source != 'localhost' \
>     and $syslogfacility-text == 'cron' \
> then ?DYNcron_all
> 
> if $source != 'localhost' \
>     and $syslogfacility-text == 'daemon' \
> then ?DYNdaemon_all
> 
> if $source != 'localhost' \
>     and $syslogfacility-text == 'kern' \
> then ?DYNkern_all
> 
> if $source != 'localhost' \
>     and $syslogfacility-text == 'lpr' \
> then ?DYNlpr_all
> 
> if $source != 'localhost' \
>     and $syslogfacility-text == 'mail' \
> then ?DYNmail_all
> 
> if $source != 'localhost' \
>     and $syslogtag contains 'sshd' \
> then ?DYNsshd
> 
> if $source != 'localhost' \
>     and $syslogfacility-text != 'authpriv' \
> then ?DYNsyslog_all
> 
> if $source != 'localhost' \
>     and $syslogfacility-text == 'user' \
> then ?DYNuser_all
> 
> # Logging for the mail system.
> $template DYNmail_info,   "/var/log/hosts/%HOSTNAME%/mail.info"
> $template DYNmail_warn,   "/var/log/hosts/%HOSTNAME%/mail.warn"
> $template DYNmail_err,    "/var/log/hosts/%HOSTNAME%/mail.err"
> 
> if $source != 'localhost' \
>     and ( \
>           $syslogfacility-text == 'mail' \
>           and $syslogseverity-text == 'info' \
>         ) \
> then ?DYNmail_info
> 
> if $source != 'localhost' \
>     and ( \
>           $syslogfacility-text == 'mail' \
>           and $syslogseverity-text == 'warn' \
>         ) \
> then ?DYNmail_warn
> 
> if $source != 'localhost' \
>     and ( \
>           $syslogfacility-text == 'mail' \
>           and $syslogseverity-text == 'err' \
>         ) \
> then ?DYNmail_err
> 
> # Catch-all log files
> $template DYNdebug,    "/var/log/hosts/%HOSTNAME%/debug"
> $template DYNmessages, "/var/log/hosts/%HOSTNAME%/messages"
> 
> if $source != 'localhost' \
>     and $syslogseverity-text == 'debug' \
> then ?DYNdebug
> 
> if $source != 'localhost' \
>     and ( \
>           $syslogseverity-text == 'info' \
>           or $syslogseverity-text == 'notice' \
>           or $syslogseverity-text == 'warn' \
>         ) \
>     and ( \
>           $syslogfacility-text != 'auth' \
>           or $syslogfacility-text != 'authpriv' \
>           or $syslogfacility-text != 'cron' \
>           or $syslogfacility-text != 'daemon' \
>           or $syslogfacility-text != 'mail' \
>           or $syslogfacility-text != 'news' \
>         ) \
> then ?DYNmessages
> 
> # Include all config files in /etc/rsyslog.d/
> $IncludeConfig /etc/rsyslog.d/
> 
> ------
> 
> I can't figure this out. The messages still only show the short
> hostname
> in both the node and server logs. Any ideas?
> 
> 
> --
> Joe McDonagh
> Operations Engineer
> AIM: YoosingYoonickz
> IRC: joe-mac on freenode
> "When the going gets weird, the weird turn pro."
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Mon Jan 17 08:23:43 2011
From: david at lang.hm (david at lang.hm)
Date: Sun, 16 Jan 2011 23:23:43 -0800 (PST)
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <4D33E696.4040502@gmail.com>
References: <4D33E696.4040502@gmail.com>
Message-ID: <alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>

look at /etc/hosts on the client. see if you have the short name or long 
name listed first.

If you have the short name listed first, try switching it to long name 
first. (when looking something up in /etc/hosts by IP, you get the first 
name on the list)

If this doesn't work, then what I suspect is happening is that the sending 
system is putting just it's hostname in the logs when it sends. some 
distros let you put a FQDN in the /etc/hostnames file without problems. If 
your distro lets you do this, try doing that and see if this then changes 
what's getting logged by rsyslog.

The third thing you can try is on the server, change it from using the 
default template that logs %HOSTNAME%, which is the name the client puts 
in the log to %FROMHOST%, which is the name (looked up from the IP) of the 
machine that sent the log packet to the receiving rsyslog

David Lang

On Mon, 17 Jan 2011, Joe McDonagh wrote:

> Date: Mon, 17 Jan 2011 01:49:58 -0500
> From: Joe McDonagh <joseph.e.mcdonagh at gmail.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
> 
> Right now I currently have compatibility with version 1 on, and I am thinking 
> I will build a package for older nodes for version 4, mainly because I really 
> need fqdns. Unfortunately when I do a test between two version 4 systems, 
> fqdn still doesn't work. Here's some info:
>
> root at syslog:/var/log/hosts/puppet# ps aux | grep rsyslogd | grep -v grep
> root     20601  0.0  0.0  29568  1296 ?        S    22:26   0:00 rsyslogd -c4 
> -m 0 -t61514 -x -r514
>
> root at puppet:~# ps aux | grep rsyslog
> root     30755  0.0  0.0  45844  1284 ?        Sl   22:42   0:00 rsyslogd -c1 
> -m 0
>
> Now I am not clear if I need PreserveFQDN on both the node and server, so I 
> set it on both for kicks. Here is the node config:
>
> #  /etc/rsyslog.conf Configuration file for rsyslogd.
> #
> #        For more information see
> #        /usr/share/doc/rsyslog/html/rsyslog_conf.html
>
> #
> # First some standard logfiles.  Log by facility.
> #
>
> $PreserveFQDN on
>
> auth,authpriv.*         /var/log/auth.log
> *.*;auth.none;authpriv.none;mail.none;cron.none,daemon.none 
> -/var/log/syslog
> cron.*            /dev/null
> daemon.*       -/var/log/daemon.log
> kern.*            -/var/log/kern.log
> lpr.*          -/var/log/lpr.log
> mail.*            -/var/log/mail.log
> user.*            -/var/log/user.log
>
> #
> # Logging for the mail system.  Split it up so that
> # it is easy to write scripts to parse these files.
> #
> mail.info         -/var/log/mail.info
> mail.warn         -/var/log/mail.warn
> mail.err       /var/log/mail.err
>
> #
> # Logging for INN news system
> #
> news.crit         /var/log/news/news.crit
> news.err       /var/log/news/news.err
> news.notice       -/var/log/news/news.notice
>
> #
> # Some `catch-all' logfiles.
> #
> *.=debug;\
>   auth,authpriv.none;\
>   news.none;mail.none  -/var/log/debug
> *.=info;*.=notice;*.=warn;\
>   auth,authpriv.none;\
>   cron.none;daemon.none;\
>   mail.none,news.none     -/var/log/messages
>
> #
> # Emergencies are sent to everybody logged in.
> #
> *.emerg           *
>
> # Include all config files in /etc/rsyslog.d/
> $IncludeConfig /etc/rsyslog.d/
>
> And here is the contents of the files in the .d conf dir:
>
> root at puppet:/var/log# cat /etc/rsyslog.d/*
> # Log kernel generated UFW log messages to file
> :msg,contains,"[UFW " /var/log/ufw.log
>
> # Uncomment the following to stop logging anything that matches the last 
> rule.
> # Doing this will stop logging kernel generated UFW log messages to the file
> # normally containing kern.* messages (eg, /var/log/kern.log)
> #& ~
> *.*,cron.none   @@127.0.0.1:61514
>
> Here is the server config:
>
> # Config file for splitting up logs by hostname and related syslog server
> # configs
>
> # Show FQDNs
> $PreserveFQDN on
>
> # Discard collectd stuff
> if $syslogtag contains 'collectd' then ~
>
> # Discard cron stuff
> if $syslogtag contains 'CRON' then ~
>
> # Custom log files first since we may discard things like apache2 messages
> # later on down.
>
> # Aggregate of corporate website logs
> $template DYNcorpsite, "/var/log/custom/corpsite_apache2.log"
>
> if $source != 'localhost' \
>   and $HOSTNAME startswith 'www' \
>   and $syslogtag contains 'apache2' \
> then ?DYNcorpsite
>
> if $source != 'localhost' \
>   and $HOSTNAME contains 'updates' \
>   and $syslogfacility-text == 'r7license_server' \
> then ?DYNr7license_servers
>
> # Aggregate of smtp gateway logs
> $template DYNsmtp_gateways, "/var/log/custom/smtp_gateways.log"
>
> if $source != 'localhost' \
>   and $HOSTNAME startswith 'smtp' \
>   and $syslogfacility-text == 'mail' \
> then ?DYNsmtp_gateways
>
> # List of log files without loglevel separation
> $template DYNapache2,            "/var/log/hosts/%HOSTNAME%/apache2.log"
> $template DYNauth_all,           "/var/log/hosts/%HOSTNAME%/auth.log"
> $template DYNcron_all,           "/var/log/hosts/%HOSTNAME%/cron.log"
> $template DYNdaemon_all,         "/var/log/hosts/%HOSTNAME%/daemon.log"
> $template DYNdhcpd,              "/var/log/hosts/%HOSTNAME%/dhcpd.log"
> $template DYNkern_all,           "/var/log/hosts/%HOSTNAME%/kern.log"
> $template DYNlpr_all,            "/var/log/hosts/%HOSTNAME%/lpr.log"
> $template DYNmail_all,           "/var/log/hosts/%HOSTNAME%/mail.log"
> $template DYNnamed,              "/var/log/hosts/%HOSTNAME%/named.log"
> $template DYNsshd,               "/var/log/hosts/%HOSTNAME%/sshd.log"
> $template DYNsyslog_all,         "/var/log/hosts/%HOSTNAME%/syslog"
> $template DYNuser_all,           "/var/log/hosts/%HOSTNAME%/user.log"
>
> # First separate interesting tags then discard to lower
> # duplication
> if $source != 'localhost' \
>   and $syslogtag contains 'apache2' \
> then ?DYNapache2
>
> if $syslogtag contains 'apache2' then ~
>
> if $source != 'localhost' \
>   and $syslogtag contains 'dhcpd' \
> then ?DYNdhcpd
>
> if $syslogtag contains 'dhcpd' then ~
>
> if $source != 'localhost' \
>   and $syslogtag contains 'named' \
> then ?DYNnamed
>
> if $syslogtag contains 'named' then ~
>
> # Here are regular facility-based separating
> if $source != 'localhost' \
>   and ( \
>         $syslogfacility-text == 'auth' \
>         or $syslogfacility-text == 'authpriv' \
>       ) \
> then ?DYNauth_all
>
> if $source != 'localhost' \
>   and $syslogfacility-text == 'cron' \
> then ?DYNcron_all
>
> if $source != 'localhost' \
>   and $syslogfacility-text == 'daemon' \
> then ?DYNdaemon_all
>
> if $source != 'localhost' \
>   and $syslogfacility-text == 'kern' \
> then ?DYNkern_all
>
> if $source != 'localhost' \
>   and $syslogfacility-text == 'lpr' \
> then ?DYNlpr_all
>
> if $source != 'localhost' \
>   and $syslogfacility-text == 'mail' \
> then ?DYNmail_all
>
> if $source != 'localhost' \
>   and $syslogtag contains 'sshd' \
> then ?DYNsshd
>
> if $source != 'localhost' \
>   and $syslogfacility-text != 'authpriv' \
> then ?DYNsyslog_all
>
> if $source != 'localhost' \
>   and $syslogfacility-text == 'user' \
> then ?DYNuser_all
>
> # Logging for the mail system.
> $template DYNmail_info,   "/var/log/hosts/%HOSTNAME%/mail.info"
> $template DYNmail_warn,   "/var/log/hosts/%HOSTNAME%/mail.warn"
> $template DYNmail_err,    "/var/log/hosts/%HOSTNAME%/mail.err"
>
> if $source != 'localhost' \
>   and ( \
>         $syslogfacility-text == 'mail' \
>         and $syslogseverity-text == 'info' \
>       ) \
> then ?DYNmail_info
>
> if $source != 'localhost' \
>   and ( \
>         $syslogfacility-text == 'mail' \
>         and $syslogseverity-text == 'warn' \
>       ) \
> then ?DYNmail_warn
>
> if $source != 'localhost' \
>   and ( \
>         $syslogfacility-text == 'mail' \
>         and $syslogseverity-text == 'err' \
>       ) \
> then ?DYNmail_err
>
> # Catch-all log files
> $template DYNdebug,    "/var/log/hosts/%HOSTNAME%/debug"
> $template DYNmessages, "/var/log/hosts/%HOSTNAME%/messages"
>
> if $source != 'localhost' \
>   and $syslogseverity-text == 'debug' \
> then ?DYNdebug
>
> if $source != 'localhost' \
>   and ( \
>         $syslogseverity-text == 'info' \
>         or $syslogseverity-text == 'notice' \
>         or $syslogseverity-text == 'warn' \
>       ) \
>   and ( \
>         $syslogfacility-text != 'auth' \
>         or $syslogfacility-text != 'authpriv' \
>         or $syslogfacility-text != 'cron' \
>         or $syslogfacility-text != 'daemon' \
>         or $syslogfacility-text != 'mail' \
>         or $syslogfacility-text != 'news' \
>       ) \
> then ?DYNmessages
>
> # Include all config files in /etc/rsyslog.d/
> $IncludeConfig /etc/rsyslog.d/
>
> ------
>
> I can't figure this out. The messages still only show the short hostname in 
> both the node and server logs. Any ideas?
>
>
> --
> Joe McDonagh
> Operations Engineer
> AIM: YoosingYoonickz
> IRC: joe-mac on freenode
> "When the going gets weird, the weird turn pro."
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Mon Jan 17 09:05:13 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Jan 2011 00:05:13 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
Message-ID: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm>

Rainer,
   I'm testing the cisconames parser, and I've almost got everything 
working (it looks like a total of 2 bugs in the initial code I sent you), 
but I'm running into problems trying to truncate a string.

in the message parser, what would I pass to rsCStrTruncate() to tuncate 2 
characters from the incomeing message?

David Lang

From rgerhards at hq.adiscon.com  Mon Jan 17 09:20:58 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Jan 2011 09:20:58 +0100
Subject: [rsyslog] message manipulation in a parser
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, January 17, 2011 9:05 AM
> To: rsyslog-users
> Subject: [rsyslog] message manipulation in a parser
> 
> Rainer,
>    I'm testing the cisconames parser, and I've almost got everything
> working (it looks like a total of 2 bugs in the initial code I sent
> you),
> but I'm running into problems trying to truncate a string.
> 
> in the message parser, what would I pass to rsCStrTruncate() to tuncate
> 2
> characters from the incomeing message?

I've just checked the code. It's the number of characters to truncate. So you
need to pass in "2".

Code in question:
http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/stringbuf.c;h=f4a9caae
d62d7ad9bca0deaf4728298cc40b5c2d;hb=HEAD#l435

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Mon Jan 17 09:42:33 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 00:42:33 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm>

On Mon, 17 Jan 2011, Rainer Gerhards wrote:

> David Lang wrote:
>> Rainer,
>>    I'm testing the cisconames parser, and I've almost got everything
>> working (it looks like a total of 2 bugs in the initial code I sent
>> you),
>> but I'm running into problems trying to truncate a string.
>>
>> in the message parser, what would I pass to rsCStrTruncate() to tuncate
>> 2
>> characters from the incomeing message?
>
> I've just checked the code. It's the number of characters to truncate. So you
> need to pass in "2".
>
> Code in question:
> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/stringbuf.c;h=f4a9caae
> d62d7ad9bca0deaf4728298cc40b5c2d;hb=HEAD#l435

the problem I'm having is figuring out what to use for the pointer to the 
string. so far everything I've tried is wrong and results in a segfault.

I'm assuming that the right answer is pMsg->something, but I haven't 
figured out what the something is.

David Lang

From rgerhards at hq.adiscon.com  Mon Jan 17 09:46:25 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Jan 2011 09:46:25 +0100
Subject: [rsyslog] message manipulation in a parser
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>

I guess it makes sense when you send me the code and give the line number
where you have the issue. I can't promise, but I'll try to give a good
suggestion in the course of the day.

One important thing you need to think about is that pMsg does NOT necessarily
use cstr objects to represent the entities. For example (IIRC), raw message
is kept in a different buffer, and you cannot use cstr methods on that
buffer. %msg% is actually not stored at all, it is taken from rawmsg. There
is a whole lot of this kind of optimization done inside the message object. I
guess this is what is hitting you.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 17, 2011 9:43 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] message manipulation in a parser
> 
> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> 
> > David Lang wrote:
> >> Rainer,
> >>    I'm testing the cisconames parser, and I've almost got everything
> >> working (it looks like a total of 2 bugs in the initial code I sent
> >> you),
> >> but I'm running into problems trying to truncate a string.
> >>
> >> in the message parser, what would I pass to rsCStrTruncate() to
> tuncate
> >> 2
> >> characters from the incomeing message?
> >
> > I've just checked the code. It's the number of characters to
> truncate. So you
> > need to pass in "2".
> >
> > Code in question:
> >
> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/stringbuf.c;h=f4
> a9caae
> > d62d7ad9bca0deaf4728298cc40b5c2d;hb=HEAD#l435
> 
> the problem I'm having is figuring out what to use for the pointer to
> the
> string. so far everything I've tried is wrong and results in a
> segfault.
> 
> I'm assuming that the right answer is pMsg->something, but I haven't
> figured out what the something is.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Mon Jan 17 09:50:29 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 00:50:29 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm>

On Mon, 17 Jan 2011, Rainer Gerhards wrote:

> I guess it makes sense when you send me the code and give the line number
> where you have the issue. I can't promise, but I'll try to give a good
> suggestion in the course of the day.
>
> One important thing you need to think about is that pMsg does NOT necessarily
> use cstr objects to represent the entities. For example (IIRC), raw message
> is kept in a different buffer, and you cannot use cstr methods on that
> buffer. %msg% is actually not stored at all, it is taken from rawmsg. There
> is a whole lot of this kind of optimization done inside the message object. I
> guess this is what is hitting you.

Ok, that makes sense. so should I just insert a \0 at the appropriate 
place and decrament the iLenRawMsg and iLenMSG values?

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 17, 2011 9:43 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] message manipulation in a parser
>>
>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>
>>> David Lang wrote:
>>>> Rainer,
>>>>    I'm testing the cisconames parser, and I've almost got everything
>>>> working (it looks like a total of 2 bugs in the initial code I sent
>>>> you),
>>>> but I'm running into problems trying to truncate a string.
>>>>
>>>> in the message parser, what would I pass to rsCStrTruncate() to
>> tuncate
>>>> 2
>>>> characters from the incomeing message?
>>>
>>> I've just checked the code. It's the number of characters to
>> truncate. So you
>>> need to pass in "2".
>>>
>>> Code in question:
>>>
>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/stringbuf.c;h=f4
>> a9caae
>>> d62d7ad9bca0deaf4728298cc40b5c2d;hb=HEAD#l435
>>
>> the problem I'm having is figuring out what to use for the pointer to
>> the
>> string. so far everything I've tried is wrong and results in a
>> segfault.
>>
>> I'm assuming that the right answer is pMsg->something, but I haven't
>> figured out what the something is.
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Mon Jan 17 09:53:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Jan 2011 09:53:24 +0100
Subject: [rsyslog] message manipulation in a parser
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 17, 2011 9:50 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] message manipulation in a parser
> 
> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> 
> > I guess it makes sense when you send me the code and give the line
> number
> > where you have the issue. I can't promise, but I'll try to give a
> good
> > suggestion in the course of the day.
> >
> > One important thing you need to think about is that pMsg does NOT
> necessarily
> > use cstr objects to represent the entities. For example (IIRC), raw
> message
> > is kept in a different buffer, and you cannot use cstr methods on
> that
> > buffer. %msg% is actually not stored at all, it is taken from rawmsg.
> There
> > is a whole lot of this kind of optimization done inside the message
> object. I
> > guess this is what is hitting you.
> 
> Ok, that makes sense. so should I just insert a \0 at the appropriate
> place and decrament the iLenRawMsg and iLenMSG values?

Tob e 100% sure, I'd need to check more thoroughly, but I am sure you are on
the right path... But have a look at msg.c. Where the buffer is located
depends on the size of it (it is static inside the object up to a certain
size, and dyn alloced otherwise -- another optimization). The safe bet is to
duplicate the message text and re-set it, but that is obviously much slower
(to gain speed, you need to break layers...).

Rainer

> 
> David Lang
> 
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >> Sent: Monday, January 17, 2011 9:43 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] message manipulation in a parser
> >>
> >> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> >>
> >>> David Lang wrote:
> >>>> Rainer,
> >>>>    I'm testing the cisconames parser, and I've almost got
> everything
> >>>> working (it looks like a total of 2 bugs in the initial code I
> sent
> >>>> you),
> >>>> but I'm running into problems trying to truncate a string.
> >>>>
> >>>> in the message parser, what would I pass to rsCStrTruncate() to
> >> tuncate
> >>>> 2
> >>>> characters from the incomeing message?
> >>>
> >>> I've just checked the code. It's the number of characters to
> >> truncate. So you
> >>> need to pass in "2".
> >>>
> >>> Code in question:
> >>>
> >>
> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/stringbuf.c;h=f4
> >> a9caae
> >>> d62d7ad9bca0deaf4728298cc40b5c2d;hb=HEAD#l435
> >>
> >> the problem I'm having is figuring out what to use for the pointer
> to
> >> the
> >> string. so far everything I've tried is wrong and results in a
> >> segfault.
> >>
> >> I'm assuming that the right answer is pMsg->something, but I haven't
> >> figured out what the something is.
> >>
> >> David Lang
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Mon Jan 17 10:15:12 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 01:15:12 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm>

On Mon, 17 Jan 2011, Rainer Gerhards wrote:

> Date: Mon, 17 Jan 2011 09:53:24 +0100
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>
>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>
>>> I guess it makes sense when you send me the code and give the line 
>>> number where you have the issue. I can't promise, but I'll try to give 
>>> a good suggestion in the course of the day.
>>>
>>> One important thing you need to think about is that pMsg does NOT 
>>> necessarily use cstr objects to represent the entities. For example 
>>> (IIRC), raw message is kept in a different buffer, and you cannot use 
>>> cstr methods on that buffer. %msg% is actually not stored at all, it 
>>> is taken from rawmsg. There is a whole lot of this kind of 
>>> optimization done inside the message object. I guess this is what is 
>>> hitting you.
>>
>> Ok, that makes sense. so should I just insert a \0 at the appropriate
>> place and decrament the iLenRawMsg and iLenMSG values?
>
> Tob e 100% sure, I'd need to check more thoroughly, but I am sure you are on
> the right path... But have a look at msg.c. Where the buffer is located
> depends on the size of it (it is static inside the object up to a certain
> size, and dyn alloced otherwise -- another optimization). The safe bet is to
> duplicate the message text and re-set it, but that is obviously much slower
> (to gain speed, you need to break layers...).

Ok, This is now working, I think I found all the variables to change.

the patch on top of v5-devel-david is

diff --git a/plugins/pmcisconames/pmcisconames.c 
b/plugins/pmcisconames/pmcisconames.c
index 28fe33d..7b76f65 100644
--- a/plugins/pmcisconames/pmcisconames.c
+++ b/plugins/pmcisconames/pmcisconames.c
@@ -41,7 +41,7 @@
  #include "unicode-helper.h"

  MODULE_TYPE_PARSER
-PARSER_NAME("rsyslog.lastline")
+PARSER_NAME("rsyslog.cisconames")

  /* internal structures
   */
@@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
                 ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
         }
         /* bump the message portion up by two characters to overwrite the extra : */
-       memmove(p2parse, p2parse + 2, lenMsg - 2);
+       lenMsg -=2;
+       memmove(p2parse, p2parse + 2, lenMsg);
+       *(p2parse + lenMsg) = '\n';
+       *(p2parse + lenMsg + 1)  = '\0';
+       pMsg->iLenRawMsg -=2;
+       pMsg->iLenMSG -=2;
         /* now, claim to abort so that something else can parse the now modified message */
         DBGPRINTF("pmcisconames detected a mangled Cisco log message message\n");
+       dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg, p2parse);
         ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);

  finalize_it:
@@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
         CHKiRet(objUse(parser, CORE_COMPONENT));
         CHKiRet(objUse(datetime, CORE_COMPONENT));

-       dbgprintf("lastmsg parser init called, compiled with version %s\n", VERSION);
+       dbgprintf("cisconames parser init called, compiled with version %s\n", VERSION);
         bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache value, is set only during rsyslogd option processing */



David Lang

P.S. I think having the file named pmlastmsg but the intermal module name 
being lastline is confusing, they should be related.

From david at lang.hm  Mon Jan 17 19:22:46 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 10:22:46 -0800 (PST)
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
	<alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
Message-ID: <alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>

Attached is  a test file to test various modes of imfile, the results of 
the test, and a consolodated patch going back to 
f7c20920046ebcb94eadadf1ebad97b634a12a2d (your merge of the other imfile 
fixes)

It turned out that there were a lot of problems in my code, so there's not 
much left of my initial patch. This is why I didn't just send in an update 
to the prior patch

It would be good to have the config changes broken out as a separate patch 
(as much for future educational benifits as anything else). I will try and 
figure out how I can manipulate git to do this.

One other problem I had is that I couldn't make the extra parameter work 
with ReadMultiLine, it would segfault every time, since nothing else uses 
ReadLine, I ended up adding the paramter to Readline and eliminating 
ReadMultiLine completely

One thing that surprised me is that it doesn't appear that control 
characters in imfile are escaped the way that network received logs are 
escaped. Did I miss some way to enable this? Initially I thought that 
possibly only \n wasn't escaped, but some of my mistakes generated other 
control characters.

David Lang


  On Thu, 13 Jan 2011, 
david at lang.hm wrote:

> Date: Thu, 13 Jan 2011 02:02:51 -0800 (PST)
> From: david at lang.hm
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Cc: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: [rsyslog] imfile paragraph patch
> 
> thanks. I will try to go over both of these tomorrow, but will definantly do 
> so no later than this weekend.
>
> David Lang
>
> On Thu, 13 Jan 2011, Rainer Gerhards wrote:
>
>> Date: Thu, 13 Jan 2011 10:43:37 +0100
>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>> To: david at lang.hm, rsyslog-users <rsyslog at lists.adiscon.com>
>> Subject: Re: imfile paragraph patch
>> 
>> I have now also created a new branch for this patch:
>>  v5-devel-david-imfile
>> 
>> I added the config variable. See the commit log for useful information and 
>> steps. While I was a bit hesitant to merge this patch soon to the official 
>> branch (due to the problems I had with imfile), I begin to think this is 
>> over-cautious. It should really not harm any existing code. So please let 
>> me know when you have finished your testing of the new code, I'll probably 
>> merge soon then.
>> 
>> Thanks!
>> Rainer
>> 
>> On 12/14/2010 04:57 AM, david at lang.hm wrote:
>>> I discovered UnreadChar and so now mode 2 (indented follow-up lines) has
>>> a chance of working. again compile tested (and visually code-reviewed by
>>> someone else), but not executed.
>>> 
>>> David Lang
>>> 
>>> On Mon, 13 Dec 2010, david at lang.hm wrote:
>>> 
>>>> This is a first cut of a modification to imfile to let it read
>>>> multi-line files.
>>>> 
>>>> As-is, this should have no effect on a system as it hard-codes the
>>>> mode to reading single lines (I really don't understand how to set a
>>>> config variable, but for someone who does, it should be simple to
>>>> replace the '0' in imfile.c with the value of the config file)
>>>> 
>>>> With this config option change, it should be possible to real logfiles
>>>> that have blank lines between multi-line log entries and have those
>>>> log entries treated as a single line.
>>>> 
>>>> I also have code in place (but disabled) to try and deal with the more
>>>> complicated layout where all lines after the first one are indented if
>>>> they are part of the same log entry. The problem I have is that when I
>>>> discover that I have finished reading a log entry I have already read
>>>> the first character of the next log entry. This extra character needs
>>>> to be put pack into the input buffer, but I don't know if that is
>>>> possible or not. If this isn't the case, I need a function that will
>>>> let me peek at the next character in the input buffer and make my
>>>> decision based on that.
>>>> 
>>>> This compiles, but I have not tested it anywhere yet. with the
>>>> hardcoded mode 0 for ('LF termination), there should be no change
>>>> other than an extra test against a constant for each character read
>>>> from a file.
>>>> 
>>>> David Lang
>> 
>> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
-------------- next part --------------
Jan 17 09:55:19 dlang-laptop kernel: imklog 5.7.2, log source = /proc/kmsg started.
Jan 17 09:55:19 dlang-laptop rsyslogd: [origin software="rsyslogd" swVersion="5.7.2" x-pid="14059" x-info="http://www.rsyslog.com"] start
Jan 17 09:55:19 dlang-laptop testfile: message1-1
Jan 17 09:55:19 dlang-laptop testfile: message1-2
Jan 17 09:55:19 dlang-laptop testfile: message1-3
Jan 17 09:55:19 dlang-laptop testfile: message1-4
Jan 17 09:55:19 dlang-laptop testfile: message2-1
Jan 17 09:55:19 dlang-laptop testfile: message2-2
Jan 17 09:55:19 dlang-laptop testfile: message2-3
Jan 17 09:55:19 dlang-laptop testfile: message2-4
Jan 17 09:55:19 dlang-laptop testfile: message3-1
Jan 17 09:55:19 dlang-laptop testfile: message3-2
Jan 17 09:55:19 dlang-laptop testfile: message3-3
Jan 17 09:55:19 dlang-laptop testfile: message3-4
Jan 17 09:55:23 dlang-laptop kernel: Kernel logging (proc) stopped.
Jan 17 09:55:23 dlang-laptop rsyslogd: [origin software="rsyslogd" swVersion="5.7.2" x-pid="14059" x-info="http://www.rsyslog.com"] exiting on signal 2.
Jan 17 09:56:16 dlang-laptop kernel: imklog 5.7.2, log source = /proc/kmsg started.
Jan 17 09:56:16 dlang-laptop rsyslogd: [origin software="rsyslogd" swVersion="5.7.2" x-pid="14068" x-info="http://www.rsyslog.com"] start
Jan 17 09:56:16 dlang-laptop testfile: message1-1
message1-2
 message1-3
 message1-4
Jan 17 09:56:16 dlang-laptop testfile: message2-1
message2-2
 message2-3
 message2-4
Jan 17 09:56:16 dlang-laptop testfile: message3-1
message3-2
 message3-3
 message3-4
Jan 17 09:56:19 dlang-laptop kernel: Kernel logging (proc) stopped.
Jan 17 09:56:19 dlang-laptop rsyslogd: [origin software="rsyslogd" swVersion="5.7.2" x-pid="14068" x-info="http://www.rsyslog.com"] exiting on signal 2.
Jan 17 09:56:44 dlang-laptop kernel: imklog 5.7.2, log source = /proc/kmsg started.
Jan 17 09:56:44 dlang-laptop rsyslogd: [origin software="rsyslogd" swVersion="5.7.2" x-pid="14076" x-info="http://www.rsyslog.com"] start
Jan 17 09:56:44 dlang-laptop testfile: message1-1
Jan 17 09:56:44 dlang-laptop testfile: message1-2
 message1-3
 message1-4
Jan 17 09:56:44 dlang-laptop testfile: message2-1
Jan 17 09:56:44 dlang-laptop testfile: message2-2
 message2-3
 message2-4
Jan 17 09:56:44 dlang-laptop testfile: message3-1
Jan 17 09:56:44 dlang-laptop testfile: message3-2
 message3-3
 message3-4
Jan 17 09:56:45 dlang-laptop kernel: Kernel logging (proc) stopped.
Jan 17 09:56:45 dlang-laptop rsyslogd: [origin software="rsyslogd" swVersion="5.7.2" x-pid="14076" x-info="http://www.rsyslog.com"] exiting on signal 2.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: imfile.patch
Type: text/x-diff
Size: 9836 bytes
Desc: 
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110117/0ec07000/attachment.patch>
-------------- next part --------------
message1-1
message1-2
 message1-3
 message1-4

message2-1
message2-2
 message2-3
 message2-4

message3-1
message3-2
 message3-3
 message3-4



From joseph.e.mcdonagh at gmail.com  Mon Jan 17 20:12:32 2011
From: joseph.e.mcdonagh at gmail.com (Joe McDonagh)
Date: Mon, 17 Jan 2011 14:12:32 -0500
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
References: <4D33E696.4040502@gmail.com>
	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
Message-ID: <4D3494A0.70607@gmail.com>

On 01/17/2011 02:23 AM, david at lang.hm wrote:
> look at /etc/hosts on the client. see if you have the short name or 
> long name listed first.
>
> If you have the short name listed first, try switching it to long name 
> first. (when looking something up in /etc/hosts by IP, you get the 
> first name on the list)
>
> If this doesn't work, then what I suspect is happening is that the 
> sending system is putting just it's hostname in the logs when it 
> sends. some distros let you put a FQDN in the /etc/hostnames file 
> without problems. If your distro lets you do this, try doing that and 
> see if this then changes what's getting logged by rsyslog.
>
> The third thing you can try is on the server, change it from using the 
> default template that logs %HOSTNAME%, which is the name the client 
> puts in the log to %FROMHOST%, which is the name (looked up from the 
> IP) of the machine that sent the log packet to the receiving rsyslog
>
> David Lang
>

Long name first... since I have legacy v1 nodes I'd like to not rush to 
upgrade, maybe I will do fromhost.

I'd love to use FROMHOST, but what happens if there is no reverse 
lookup? Will it evaluate empty or eval to the IP?

--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."


From joseph.e.mcdonagh at gmail.com  Mon Jan 17 20:19:00 2011
From: joseph.e.mcdonagh at gmail.com (Joe McDonagh)
Date: Mon, 17 Jan 2011 14:19:00 -0500
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
References: <4D33E696.4040502@gmail.com>
	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
Message-ID: <4D349624.6000704@gmail.com>

On 01/17/2011 02:23 AM, david at lang.hm wrote:
> look at /etc/hosts on the client. see if you have the short name or 
> long name listed first.
>
> If you have the short name listed first, try switching it to long name 
> first. (when looking something up in /etc/hosts by IP, you get the 
> first name on the list)
>
> If this doesn't work, then what I suspect is happening is that the 
> sending system is putting just it's hostname in the logs when it 
> sends. some distros let you put a FQDN in the /etc/hostnames file 
> without problems. If your distro lets you do this, try doing that and 
> see if this then changes what's getting logged by rsyslog.
>
> The third thing you can try is on the server, change it from using the 
> default template that logs %HOSTNAME%, which is the name the client 
> puts in the log to %FROMHOST%, which is the name (looked up from the 
> IP) of the machine that sent the log packet to the receiving rsyslog
>
> David Lang
>

So, I just realized FROMHOST won't work since I am using SSL via 
stunnel, which makes all messages look like they came from 127.0.0.1.


--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."


From david at lang.hm  Mon Jan 17 20:38:58 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 11:38:58 -0800 (PST)
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <4D3494A0.70607@gmail.com>
References: <4D33E696.4040502@gmail.com>
	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
	<4D3494A0.70607@gmail.com>
Message-ID: <alpine.DEB.2.00.1101171138380.9746@asgard.lang.hm>

On Mon, 17 Jan 2011, Joe McDonagh wrote:

> On 01/17/2011 02:23 AM, david at lang.hm wrote:
>> look at /etc/hosts on the client. see if you have the short name or long 
>> name listed first.
>> 
>> If you have the short name listed first, try switching it to long name 
>> first. (when looking something up in /etc/hosts by IP, you get the first 
>> name on the list)
>> 
>> If this doesn't work, then what I suspect is happening is that the sending 
>> system is putting just it's hostname in the logs when it sends. some 
>> distros let you put a FQDN in the /etc/hostnames file without problems. If 
>> your distro lets you do this, try doing that and see if this then changes 
>> what's getting logged by rsyslog.
>> 
>> The third thing you can try is on the server, change it from using the 
>> default template that logs %HOSTNAME%, which is the name the client puts in 
>> the log to %FROMHOST%, which is the name (looked up from the IP) of the 
>> machine that sent the log packet to the receiving rsyslog
>> 
>> David Lang
>> 
>
> Long name first... since I have legacy v1 nodes I'd like to not rush to 
> upgrade, maybe I will do fromhost.
>
> I'd love to use FROMHOST, but what happens if there is no reverse lookup? 
> Will it evaluate empty or eval to the IP?

it would eval to the IP

David Lang

From david at lang.hm  Mon Jan 17 20:39:56 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 11:39:56 -0800 (PST)
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <4D349624.6000704@gmail.com>
References: <4D33E696.4040502@gmail.com>
	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
	<4D349624.6000704@gmail.com>
Message-ID: <alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>

On Mon, 17 Jan 2011, Joe McDonagh wrote:

> On 01/17/2011 02:23 AM, david at lang.hm wrote:
>> look at /etc/hosts on the client. see if you have the short name or long 
>> name listed first.
>> 
>> If you have the short name listed first, try switching it to long name 
>> first. (when looking something up in /etc/hosts by IP, you get the first 
>> name on the list)
>> 
>> If this doesn't work, then what I suspect is happening is that the sending 
>> system is putting just it's hostname in the logs when it sends. some 
>> distros let you put a FQDN in the /etc/hostnames file without problems. If 
>> your distro lets you do this, try doing that and see if this then changes 
>> what's getting logged by rsyslog.
>> 
>> The third thing you can try is on the server, change it from using the 
>> default template that logs %HOSTNAME%, which is the name the client puts in 
>> the log to %FROMHOST%, which is the name (looked up from the IP) of the 
>> machine that sent the log packet to the receiving rsyslog
>> 
>> David Lang
>> 
>
> So, I just realized FROMHOST won't work since I am using SSL via stunnel, 
> which makes all messages look like they came from 127.0.0.1.

one thing you could do (ugly to maintain) is to change the template on the 
sending machine to hard-code the hostname in the messages.

David Lang

From joseph.e.mcdonagh at gmail.com  Mon Jan 17 20:47:12 2011
From: joseph.e.mcdonagh at gmail.com (Joe McDonagh)
Date: Mon, 17 Jan 2011 14:47:12 -0500
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>
References: <4D33E696.4040502@gmail.com>	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>	<4D349624.6000704@gmail.com>
	<alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>
Message-ID: <4D349CC0.6050900@gmail.com>

On 01/17/2011 02:39 PM, david at lang.hm wrote:
> On Mon, 17 Jan 2011, Joe McDonagh wrote:
>
>> On 01/17/2011 02:23 AM, david at lang.hm wrote:
>>> look at /etc/hosts on the client. see if you have the short name or 
>>> long name listed first.
>>>
>>> If you have the short name listed first, try switching it to long 
>>> name first. (when looking something up in /etc/hosts by IP, you get 
>>> the first name on the list)
>>>
>>> If this doesn't work, then what I suspect is happening is that the 
>>> sending system is putting just it's hostname in the logs when it 
>>> sends. some distros let you put a FQDN in the /etc/hostnames file 
>>> without problems. If your distro lets you do this, try doing that 
>>> and see if this then changes what's getting logged by rsyslog.
>>>
>>> The third thing you can try is on the server, change it from using 
>>> the default template that logs %HOSTNAME%, which is the name the 
>>> client puts in the log to %FROMHOST%, which is the name (looked up 
>>> from the IP) of the machine that sent the log packet to the 
>>> receiving rsyslog
>>>
>>> David Lang
>>>
>>
>> So, I just realized FROMHOST won't work since I am using SSL via 
>> stunnel, which makes all messages look like they came from 127.0.0.1.
>
> one thing you could do (ugly to maintain) is to change the template on 
> the sending machine to hard-code the hostname in the messages.
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
It actually wouldn't be that bad since I could push that out from a 
single source in puppet, how do you edit the default template? I have 
the page open for editing templates but don't see how to override the 
default...

I don't know though I think I'd really rather understand the problem 
since I would guess this would affect someone else. Are you a developer 
for rsyslog? I recognize your name from somewhere...

I could check out the source but it might be faster if someone familiar 
with the code base could tell me what the algorithm for evaluating 
%HOSTNAME% is. I am pretty sure you are right that the evaluation on the 
node is failing. If there is a specific syscall it's using I can 
probably just go on that for more troubleshooting.

--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."


From david at lang.hm  Mon Jan 17 21:15:39 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Jan 2011 12:15:39 -0800 (PST)
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <4D349CC0.6050900@gmail.com>
References: <4D33E696.4040502@gmail.com>
	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>
	<4D349624.6000704@gmail.com>
	<alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>
	<4D349CC0.6050900@gmail.com>
Message-ID: <alpine.DEB.2.00.1101171214130.9746@asgard.lang.hm>

On Mon, 17 Jan 2011, Joe McDonagh wrote:

> On 01/17/2011 02:39 PM, david at lang.hm wrote:
>> On Mon, 17 Jan 2011, Joe McDonagh wrote:
>> 
>>> On 01/17/2011 02:23 AM, david at lang.hm wrote:
>>>> look at /etc/hosts on the client. see if you have the short name or long 
>>>> name listed first.
>>>> 
>>>> If you have the short name listed first, try switching it to long name 
>>>> first. (when looking something up in /etc/hosts by IP, you get the first 
>>>> name on the list)
>>>> 
>>>> If this doesn't work, then what I suspect is happening is that the 
>>>> sending system is putting just it's hostname in the logs when it sends. 
>>>> some distros let you put a FQDN in the /etc/hostnames file without 
>>>> problems. If your distro lets you do this, try doing that and see if this 
>>>> then changes what's getting logged by rsyslog.
>>>> 
>>>> The third thing you can try is on the server, change it from using the 
>>>> default template that logs %HOSTNAME%, which is the name the client puts 
>>>> in the log to %FROMHOST%, which is the name (looked up from the IP) of 
>>>> the machine that sent the log packet to the receiving rsyslog
>>>> 
>>>> David Lang
>>>> 
>>> 
>>> So, I just realized FROMHOST won't work since I am using SSL via stunnel, 
>>> which makes all messages look like they came from 127.0.0.1.
>> 
>> one thing you could do (ugly to maintain) is to change the template on the 
>> sending machine to hard-code the hostname in the messages.
>> 
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> It actually wouldn't be that bad since I could push that out from a single 
> source in puppet, how do you edit the default template? I have the page open 
> for editing templates but don't see how to override the default...
>
> I don't know though I think I'd really rather understand the problem since I 
> would guess this would affect someone else. Are you a developer for rsyslog? 
> I recognize your name from somewhere...
>
> I could check out the source but it might be faster if someone familiar with 
> the code base could tell me what the algorithm for evaluating %HOSTNAME% is. 
> I am pretty sure you are right that the evaluation on the node is failing. If 
> there is a specific syscall it's using I can probably just go on that for 
> more troubleshooting.

take a look at this page and see if it tells you what you want to know. If 
you need more help just ask.

http://www.rsyslog.com/doc/rsyslog_conf_modules.html/rsyslog_conf_templates.html

David Lang

From rgerhards at hq.adiscon.com  Tue Jan 18 13:57:10 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Jan 2011 13:57:10 +0100
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
References: <4D33E696.4040502@gmail.com>	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>	<4D349624.6000704@gmail.com><alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>
	<4D349CC0.6050900@gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA2C@GRFEXC.intern.adiscon.com>

> I could check out the source but it might be faster if someone familiar
> with the code base could tell me what the algorithm for evaluating
> %HOSTNAME% is. I am pretty sure you are right that the evaluation on
> the
> node is failing. If there is a specific syscall it's using I can
> probably just go on that for more troubleshooting.

I still do not know the exact details of your problem, but I assume that you
have some v1 senders pulling data from the local socket and sending it to a
remote end.

Usually I look at v1 only in paid support cases -- it is *really, really*
old. Anyhow, I dug a copy out (not even easy to find as we didn't had git in
these old ages) and had a quick glimpse at it. As it looks, v1 always strips
off the domain part (1.0.5, msg,c, around line 2400). I may be wrong, but
honestly I don't intend to dig any deeper into that ancient code base. So if
sticking with it is very important for you, you should probably read the code
yourself. If it is so urgent that you really need to have someone fix that
problem, I suggest to have a look at our professional support offerings ;)

Rainer

From joseph.e.mcdonagh at gmail.com  Tue Jan 18 18:04:59 2011
From: joseph.e.mcdonagh at gmail.com (Joe McDonagh)
Date: Tue, 18 Jan 2011 12:04:59 -0500
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA2C@GRFEXC.intern.adiscon.com>
References: <4D33E696.4040502@gmail.com>	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>	<4D349624.6000704@gmail.com><alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>	<4D349CC0.6050900@gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA2C@GRFEXC.intern.adiscon.com>
Message-ID: <4D35C83B.4050409@gmail.com>

On 01/18/2011 07:57 AM, Rainer Gerhards wrote:
>> I could check out the source but it might be faster if someone familiar
>> with the code base could tell me what the algorithm for evaluating
>> %HOSTNAME% is. I am pretty sure you are right that the evaluation on
>> the
>> node is failing. If there is a specific syscall it's using I can
>> probably just go on that for more troubleshooting.
> I still do not know the exact details of your problem, but I assume that you
> have some v1 senders pulling data from the local socket and sending it to a
> remote end.
>
> Usually I look at v1 only in paid support cases -- it is *really, really*
> old. Anyhow, I dug a copy out (not even easy to find as we didn't had git in
> these old ages) and had a quick glimpse at it. As it looks, v1 always strips
> off the domain part (1.0.5, msg,c, around line 2400). I may be wrong, but
> honestly I don't intend to dig any deeper into that ancient code base. So if
> sticking with it is very important for you, you should probably read the code
> yourself. If it is so urgent that you really need to have someone fix that
> problem, I suggest to have a look at our professional support offerings ;)
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
Thanks for the effort Rainer, which I kind of thought would be a problem 
however this is still happoening when I force -c4 on node and server... 
here's another question, if i do preservefqdn, will the node show fqdn 
even logging to itself?

-- 
--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."


From rgerhards at hq.adiscon.com  Tue Jan 18 18:10:06 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Jan 2011 18:10:06 +0100
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
References: <4D33E696.4040502@gmail.com>	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>	<4D349624.6000704@gmail.com><alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>	<4D349CC0.6050900@gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DDA2C@GRFEXC.intern.adiscon.com>
	<4D35C83B.4050409@gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA37@GRFEXC.intern.adiscon.com>

> Thanks for the effort Rainer, which I kind of thought would be a
> problem
> however this is still happoening when I force -c4 

What do you mean with "force -c4"?

> on node and server...
> here's another question, if i do preservefqdn, will the node show fqdn
> even logging to itself?

In recent builds, I think it should (I have not checked, but everything would
have a buggish smell to me).

Rainer

From joseph.e.mcdonagh at gmail.com  Tue Jan 18 22:00:11 2011
From: joseph.e.mcdonagh at gmail.com (Joe McDonagh)
Date: Tue, 18 Jan 2011 16:00:11 -0500
Subject: [rsyslog] Cannot for the life of me get preservefqdn to work
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA37@GRFEXC.intern.adiscon.com>
References: <4D33E696.4040502@gmail.com>	<alpine.DEB.2.00.1101162319001.9746@asgard.lang.hm>	<4D349624.6000704@gmail.com><alpine.DEB.2.00.1101171139050.9746@asgard.lang.hm>	<4D349CC0.6050900@gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DDA2C@GRFEXC.intern.adiscon.com>	<4D35C83B.4050409@gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA37@GRFEXC.intern.adiscon.com>
Message-ID: <4D35FF5B.5040702@gmail.com>

On 01/18/2011 12:10 PM, Rainer Gerhards wrote:
>> Thanks for the effort Rainer, which I kind of thought would be a
>> problem
>> however this is still happoening when I force -c4
> What do you mean with "force -c4"?

I mean instead of just running it without explicitly saying the version 
compatibility mode on v4 hosts. Sorry if that was unclear.
>> on node and server...
>> here's another question, if i do preservefqdn, will the node show fqdn
>> even logging to itself?
> In recent builds, I think it should (I have not checked, but everything would
> have a buggish smell to me).
Weird.. the v4 systems are using the default package on ubuntu 10.04. I 
will be coming back to this at some point this week but I have a bunch 
of other stuff to do. I will focus on getting one node logging to itself 
with fqdn and post back to the list. Thanks for your help.
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


-- 
--
Joe McDonagh
Operations Engineer
AIM: YoosingYoonickz
IRC: joe-mac on freenode
"When the going gets weird, the weird turn pro."


From Robert.Jennings at qbelmi.com  Wed Jan 19 03:51:07 2011
From: Robert.Jennings at qbelmi.com (Robert Jennings)
Date: Wed, 19 Jan 2011 13:51:07 +1100
Subject: [rsyslog] rsyslog with heartbeat
Message-ID: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>

I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002) sharing
a virtual IP managed by heartbeat (prd-syslog-000)
 
 
clients are configured to send logs to the virtualIP
 
*.info;mail.none;authpriv.none;cron.none                @@prd-syslog-000
authpriv.*
@@prd-syslog-000
local7.*
@@prd-syslog-000

 
 
 
 
I have found when the primary host goes down and the secondary takes
over, logs are not arriving at the secondary host unless I restart
rsyslog on the client, this was not the behaviour I was expecting. Is
there any default configuration that may be stopping logs from being
sent when the server sitting behind the virtual ip changes?
 
 
The functionality I was trying to achive is as follows:
 

*	clients log to prd-syslog-000 
*	heartbeat controls prd-syslog-000 address (prd-syslog-001
primary, prd-syslog-002 on failover) 
*	prd-syslog-002 is configured to forward logs to prd-syslog-001 
*	during downtime of prd-syslog-001 these logs will sit queued 
*	when prd-syslog-001 comes back up it will take over
prd-syslog-000 and start recieving client logs 
*	prd-syslog-002 will flush its queue back to prd-syslog-001 

 
The reason I am using this setup as opposed to logging to two rsyslog
hosts is to support devices that can only log to one host, and provide
one location with a consistent set of logs instead of two machines with
out of sync logs.
 
running rsyslog-3.22 
 
 
Thanks,
Rob
 
 
 

From david at lang.hm  Wed Jan 19 03:59:34 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Jan 2011 18:59:34 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
Message-ID: <alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>

how long did you wait for the failover to take place? the first thing that 
will have to happen is that the sender will have to notice that the TCP 
connection doesn't work anymore, close it, and open another one.

Since the system doesn't know what messages have actually been sent to the 
server (as opposed to mearly sent to the TCP stack on the sending 
machine), there will be some logs lost at failover time. the longer it 
takes the client to notice the problem, the worse this will be.

version 3.22 is fairly old at this point, Rainer will have to weigh in on 
how long it should take to notice the failure.

to fully avoid the lost messages issue, you would need to use RELP for 
your transport.

I have opted to use UDP and accept that while the boxes are failing over, 
some logs will be lost (you can tune heartbeat for pretty short failover 
times, subsecond if you really push it)

David Lang


  On Wed, 
19 Jan 2011, Robert Jennings wrote:

> I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002) sharing
> a virtual IP managed by heartbeat (prd-syslog-000)
>
>
> clients are configured to send logs to the virtualIP
>
> *.info;mail.none;authpriv.none;cron.none                @@prd-syslog-000
> authpriv.*
> @@prd-syslog-000
> local7.*
> @@prd-syslog-000
>
>
>
>
>
> I have found when the primary host goes down and the secondary takes
> over, logs are not arriving at the secondary host unless I restart
> rsyslog on the client, this was not the behaviour I was expecting. Is
> there any default configuration that may be stopping logs from being
> sent when the server sitting behind the virtual ip changes?
>
>
> The functionality I was trying to achive is as follows:
>
>
> *	clients log to prd-syslog-000
> *	heartbeat controls prd-syslog-000 address (prd-syslog-001
> primary, prd-syslog-002 on failover)
> *	prd-syslog-002 is configured to forward logs to prd-syslog-001
> *	during downtime of prd-syslog-001 these logs will sit queued
> *	when prd-syslog-001 comes back up it will take over
> prd-syslog-000 and start recieving client logs
> *	prd-syslog-002 will flush its queue back to prd-syslog-001
>
>
> The reason I am using this setup as opposed to logging to two rsyslog
> hosts is to support devices that can only log to one host, and provide
> one location with a consistent set of logs instead of two machines with
> out of sync logs.
>
> running rsyslog-3.22
>
>
> Thanks,
> Rob
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From Robert.Jennings at qbelmi.com  Wed Jan 19 04:25:23 2011
From: Robert.Jennings at qbelmi.com (Robert Jennings)
Date: Wed, 19 Jan 2011 14:25:23 +1100
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
Message-ID: <96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>

With TCP it appears the clients remote logging stops indefinitely, in
fact even when a failback occurs it still isnt logging messages to the
primary server until I restart the client rsyslog process.

With UDP behaviour is as expected, with a ~20-30 second failover period
where messages are lost (settings not tweaked in anyway)

I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
interest to try a more recent version.


Which leads me to think perhaps the best solution may be:

Rsyslog clients UDP -> prd-syslog-000
Syslog clients & devices UDP -> prd-syslog-000

Prd-syslog-002 TCP -> prd-syslog-001 

So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
reliably queue and push messages to prd-syslog-001 once it is back
online.


Thanks,
Rob


-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
Sent: Wednesday, 19 January 2011 2:00 PM
To: rsyslog-users
Subject: Re: [rsyslog] rsyslog with heartbeat

how long did you wait for the failover to take place? the first thing
that 
will have to happen is that the sender will have to notice that the TCP 
connection doesn't work anymore, close it, and open another one.

Since the system doesn't know what messages have actually been sent to
the 
server (as opposed to mearly sent to the TCP stack on the sending 
machine), there will be some logs lost at failover time. the longer it 
takes the client to notice the problem, the worse this will be.

version 3.22 is fairly old at this point, Rainer will have to weigh in
on 
how long it should take to notice the failure.

to fully avoid the lost messages issue, you would need to use RELP for 
your transport.

I have opted to use UDP and accept that while the boxes are failing
over, 
some logs will be lost (you can tune heartbeat for pretty short failover

times, subsecond if you really push it)

David Lang


  On Wed, 
19 Jan 2011, Robert Jennings wrote:

> I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
sharing
> a virtual IP managed by heartbeat (prd-syslog-000)
>
>
> clients are configured to send logs to the virtualIP
>
> *.info;mail.none;authpriv.none;cron.none
@@prd-syslog-000
> authpriv.*
> @@prd-syslog-000
> local7.*
> @@prd-syslog-000
>
>
>
>
>
> I have found when the primary host goes down and the secondary takes
> over, logs are not arriving at the secondary host unless I restart
> rsyslog on the client, this was not the behaviour I was expecting. Is
> there any default configuration that may be stopping logs from being
> sent when the server sitting behind the virtual ip changes?
>
>
> The functionality I was trying to achive is as follows:
>
>
> *	clients log to prd-syslog-000
> *	heartbeat controls prd-syslog-000 address (prd-syslog-001
> primary, prd-syslog-002 on failover)
> *	prd-syslog-002 is configured to forward logs to prd-syslog-001
> *	during downtime of prd-syslog-001 these logs will sit queued
> *	when prd-syslog-001 comes back up it will take over
> prd-syslog-000 and start recieving client logs
> *	prd-syslog-002 will flush its queue back to prd-syslog-001
>
>
> The reason I am using this setup as opposed to logging to two rsyslog
> hosts is to support devices that can only log to one host, and provide
> one location with a consistent set of logs instead of two machines
with
> out of sync logs.
>
> running rsyslog-3.22
>
>
> Thanks,
> Rob
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From david at lang.hm  Wed Jan 19 06:55:54 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Jan 2011 21:55:54 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
Message-ID: <alpine.DEB.2.00.1101182151500.9980@asgard.lang.hm>

On Wed, 19 Jan 2011, Robert Jennings wrote:

> With TCP it appears the clients remote logging stops indefinitely, in
> fact even when a failback occurs it still isnt logging messages to the
> primary server until I restart the client rsyslog process.

so it's not properly detecting that the tcp connection has failed.

> With UDP behaviour is as expected, with a ~20-30 second failover period
> where messages are lost (settings not tweaked in anyway)

wow, that's a long time. check the timeout setting in /etc/ha.d/ha.cf, you 
should be able to set it down < 5 seconds.

> I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
> interest to try a more recent version.



>
> Which leads me to think perhaps the best solution may be:
>
> Rsyslog clients UDP -> prd-syslog-000
> Syslog clients & devices UDP -> prd-syslog-000
>
> Prd-syslog-002 TCP -> prd-syslog-001
>
> So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
> reliably queue and push messages to prd-syslog-001 once it is back
> online.

unless you use RELP, you cannot really "reliably queue and push messages 
once it's back online" because the sending box will send some messages to 
the kernel, and will think they have arrived at the destination when the 
kernel is just trying to send them. how much data can be lost this way 
depends on the buffer settings of your system, but that tends to be fairly 
large (>1000 packets, but not all of them would be syslog packets)

David Lang

>
> Thanks,
> Rob
>
>
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, 19 January 2011 2:00 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with heartbeat
>
> how long did you wait for the failover to take place? the first thing
> that
> will have to happen is that the sender will have to notice that the TCP
> connection doesn't work anymore, close it, and open another one.
>
> Since the system doesn't know what messages have actually been sent to
> the
> server (as opposed to mearly sent to the TCP stack on the sending
> machine), there will be some logs lost at failover time. the longer it
> takes the client to notice the problem, the worse this will be.
>
> version 3.22 is fairly old at this point, Rainer will have to weigh in
> on
> how long it should take to notice the failure.
>
> to fully avoid the lost messages issue, you would need to use RELP for
> your transport.
>
> I have opted to use UDP and accept that while the boxes are failing
> over,
> some logs will be lost (you can tune heartbeat for pretty short failover
>
> times, subsecond if you really push it)
>
> David Lang
>
>
>  On Wed,
> 19 Jan 2011, Robert Jennings wrote:
>
>> I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
> sharing
>> a virtual IP managed by heartbeat (prd-syslog-000)
>>
>>
>> clients are configured to send logs to the virtualIP
>>
>> *.info;mail.none;authpriv.none;cron.none
> @@prd-syslog-000
>> authpriv.*
>> @@prd-syslog-000
>> local7.*
>> @@prd-syslog-000
>>
>>
>>
>>
>>
>> I have found when the primary host goes down and the secondary takes
>> over, logs are not arriving at the secondary host unless I restart
>> rsyslog on the client, this was not the behaviour I was expecting. Is
>> there any default configuration that may be stopping logs from being
>> sent when the server sitting behind the virtual ip changes?
>>
>>
>> The functionality I was trying to achive is as follows:
>>
>>
>> *	clients log to prd-syslog-000
>> *	heartbeat controls prd-syslog-000 address (prd-syslog-001
>> primary, prd-syslog-002 on failover)
>> *	prd-syslog-002 is configured to forward logs to prd-syslog-001
>> *	during downtime of prd-syslog-001 these logs will sit queued
>> *	when prd-syslog-001 comes back up it will take over
>> prd-syslog-000 and start recieving client logs
>> *	prd-syslog-002 will flush its queue back to prd-syslog-001
>>
>>
>> The reason I am using this setup as opposed to logging to two rsyslog
>> hosts is to support devices that can only log to one host, and provide
>> one location with a consistent set of logs instead of two machines
> with
>> out of sync logs.
>>
>> running rsyslog-3.22
>>
>>
>> Thanks,
>> Rob
>>
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From dave at fly.srk.fer.hr  Wed Jan 19 12:10:16 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Wed, 19 Jan 2011 12:10:16 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
Message-ID: <20110119111016.GA29659@fly.srk.fer.hr>

Hello.

I have rsyslog 5.6.2 and I just tried to receive messages via TCP. The
sender is syslog-ng, I don't know exactly which version, not under my
control. The transfer mostly works, as far as I can see without strict
testing.

But in rsyslog's log file I see several messages which look like this:

   rsyslogd-2105: Tearing down TCP Session - see previous messages for reason(s)
   [try http://www.rsyslog.com/e/2105 ]

And then there is no previous message which is different than this one and
if I go back far enough there is the startup message. There's nothing
else in the log (produced by the *.* selector at the end of the config file).

I tried to check the URL above, but I'm getting an error page with says:

   Fatal error: Call to undefined function SendEmailToAdiscon() in
   /home/adisconweb/www/kb.monitorware.com/kbeventdb.php on line 217

That's generated by
http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog-rsyslogd-2105.html

I checked the source and the only file with the above message is
plugins/imgssapi/imgssapi.c in the

   if(tcps_sess.DataRcvd(pSess, buf, ret) !  = RS_RET_OK)

block. I didn't even know I was using gssapi plugin. My configuration
related to TCP is:

$ModLoad imtcp
$InputTCPServerBindRuleset indata
$InputTCPServerRun 514

and the rest is setup for the UDP receiver and storing messages. I checked
with Google and rsyslog bug tracker for "Tearing down TCP Session", but
with zero results (except for the source code matches).

I am willing to debug this and submit a patch (which would at least log
the missing previous message), but before I do, I'd just like to ask if
someone has seen this before or has any idea what's going on. I'm a bit
confused with gssapi module being used. My OS is CentOS 5.5, if that's
relevant.

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr

From rgerhards at hq.adiscon.com  Wed Jan 19 12:21:33 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Jan 2011 12:21:33 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
References: <20110119111016.GA29659@fly.srk.fer.hr>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>

This from ./tcpsrv.c, not gssapi. There seems to be some condition triggered
which does not emit a message by itself, maybe removed because it occurred
too frequently.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
> Sent: Wednesday, January 19, 2011 12:10 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Tearing down TCP Session in the log file
> 
> Hello.
> 
> I have rsyslog 5.6.2 and I just tried to receive messages via TCP. The
> sender is syslog-ng, I don't know exactly which version, not under my
> control. The transfer mostly works, as far as I can see without strict
> testing.
> 
> But in rsyslog's log file I see several messages which look like this:
> 
>    rsyslogd-2105: Tearing down TCP Session - see previous messages for
> reason(s)
>    [try http://www.rsyslog.com/e/2105 ]
> 
> And then there is no previous message which is different than this one
> and
> if I go back far enough there is the startup message. There's nothing
> else in the log (produced by the *.* selector at the end of the config
> file).
> 
> I tried to check the URL above, but I'm getting an error page with
> says:
> 
>    Fatal error: Call to undefined function SendEmailToAdiscon() in
>    /home/adisconweb/www/kb.monitorware.com/kbeventdb.php on line 217
> 
> That's generated by
> http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog-rsyslogd-
> 2105.html
> 
> I checked the source and the only file with the above message is
> plugins/imgssapi/imgssapi.c in the
> 
>    if(tcps_sess.DataRcvd(pSess, buf, ret) !  = RS_RET_OK)
> 
> block. I didn't even know I was using gssapi plugin. My configuration
> related to TCP is:
> 
> $ModLoad imtcp
> $InputTCPServerBindRuleset indata
> $InputTCPServerRun 514
> 
> and the rest is setup for the UDP receiver and storing messages. I
> checked
> with Google and rsyslog bug tracker for "Tearing down TCP Session", but
> with zero results (except for the source code matches).
> 
> I am willing to debug this and submit a patch (which would at least log
> the missing previous message), but before I do, I'd just like to ask if
> someone has seen this before or has any idea what's going on. I'm a bit
> confused with gssapi module being used. My OS is CentOS 5.5, if that's
> relevant.
> 
> --
>  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
> (_  \ /  _)   ceremonial.
>      |
>      |        dave at fly.srk.fer.hr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From dave at fly.srk.fer.hr  Wed Jan 19 12:46:07 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Wed, 19 Jan 2011 12:46:07 +0100
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
Message-ID: <20110119114607.GB29659@fly.srk.fer.hr>

Robert Jennings wrote:
> With TCP it appears the clients remote logging stops indefinitely, in
> fact even when a failback occurs it still isnt logging messages to the
> primary server until I restart the client rsyslog process.

How does your failover procedure work exactly? If it's crashing the OS
(or turning off power), then there's nothing which can send TCP close
packet, so your clients won't know they should close the connection.

If that's the case, there should be retransmitting TCP timeout on the
client, which is usually 13-15 minutes by default. The exact number
depends on the OS and the network MTU. Did you wait that long?

> With UDP behaviour is as expected, with a ~20-30 second failover period
> where messages are lost (settings not tweaked in anyway)
> 
> I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
> interest to try a more recent version.
> 
> 
> Which leads me to think perhaps the best solution may be:
> 
> Rsyslog clients UDP -> prd-syslog-000
> Syslog clients & devices UDP -> prd-syslog-000
> 
> Prd-syslog-002 TCP -> prd-syslog-001 
> 
> So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
> reliably queue and push messages to prd-syslog-001 once it is back
> online.
> 
> 
> Thanks,
> Rob
> 
> 
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, 19 January 2011 2:00 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with heartbeat
> 
> how long did you wait for the failover to take place? the first thing
> that 
> will have to happen is that the sender will have to notice that the TCP 
> connection doesn't work anymore, close it, and open another one.
> 
> Since the system doesn't know what messages have actually been sent to
> the 
> server (as opposed to mearly sent to the TCP stack on the sending 
> machine), there will be some logs lost at failover time. the longer it 
> takes the client to notice the problem, the worse this will be.
> 
> version 3.22 is fairly old at this point, Rainer will have to weigh in
> on 
> how long it should take to notice the failure.
> 
> to fully avoid the lost messages issue, you would need to use RELP for 
> your transport.
> 
> I have opted to use UDP and accept that while the boxes are failing
> over, 
> some logs will be lost (you can tune heartbeat for pretty short failover
> 
> times, subsecond if you really push it)
> 
> David Lang
> 
> 
>   On Wed, 
> 19 Jan 2011, Robert Jennings wrote:
> 
> > I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
> sharing
> > a virtual IP managed by heartbeat (prd-syslog-000)
> >
> >
> > clients are configured to send logs to the virtualIP
> >
> > *.info;mail.none;authpriv.none;cron.none
> @@prd-syslog-000
> > authpriv.*
> > @@prd-syslog-000
> > local7.*
> > @@prd-syslog-000
> >
> >
> >
> >
> >
> > I have found when the primary host goes down and the secondary takes
> > over, logs are not arriving at the secondary host unless I restart
> > rsyslog on the client, this was not the behaviour I was expecting. Is
> > there any default configuration that may be stopping logs from being
> > sent when the server sitting behind the virtual ip changes?
> >
> >
> > The functionality I was trying to achive is as follows:
> >
> >
> > *	clients log to prd-syslog-000
> > *	heartbeat controls prd-syslog-000 address (prd-syslog-001
> > primary, prd-syslog-002 on failover)
> > *	prd-syslog-002 is configured to forward logs to prd-syslog-001
> > *	during downtime of prd-syslog-001 these logs will sit queued
> > *	when prd-syslog-001 comes back up it will take over
> > prd-syslog-000 and start recieving client logs
> > *	prd-syslog-002 will flush its queue back to prd-syslog-001
> >
> >
> > The reason I am using this setup as opposed to logging to two rsyslog
> > hosts is to support devices that can only log to one host, and provide
> > one location with a consistent set of logs instead of two machines
> with
> > out of sync logs.
> >
> > running rsyslog-3.22
> >
> >
> > Thanks,
> > Rob
> >
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr

From Luis.Fernando.Munoz.Mejias at cern.ch  Wed Jan 19 12:43:53 2011
From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?utf-8?q?Mu=C3=B1oz_Mej=C3=ADas?=)
Date: Wed, 19 Jan 2011 12:43:53 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>
References: <20110119111016.GA29659@fly.srk.fer.hr>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>
Message-ID: <201101191243.53743.Luis.Fernando.Munoz.Mejias@cern.ch>

El Wednesday, 19 de January de 2011 12:21, Rainer Gerhards escribi?:
> This from ./tcpsrv.c, not gssapi. There seems to be some condition
> triggered which does not emit a message by itself, maybe removed because it
> occurred too frequently.

Can the failing function print the strerror(errno) before returning?

If that's not meaningful because other threads may have messed up errno,
please change the error message so that it doesn't mislead to
non-existing log messages.

Thanks.
-- 
Luis Fernando Mu?oz Mej?as
Luis.Fernando.Munoz.Mejias at cern.ch


From rgerhards at hq.adiscon.com  Wed Jan 19 12:51:59 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Jan 2011 12:51:59 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
References: <20110119111016.GA29659@fly.srk.fer.hr><9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>
	<201101191243.53743.Luis.Fernando.Munoz.Mejias@cern.ch>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA4B@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Luis Fernando Mu?oz Mej?as
> Sent: Wednesday, January 19, 2011 12:44 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Tearing down TCP Session in the log file
> 
> El Wednesday, 19 de January de 2011 12:21, Rainer Gerhards escribi?:
> > This from ./tcpsrv.c, not gssapi. There seems to be some condition
> > triggered which does not emit a message by itself, maybe removed
> because it
> > occurred too frequently.
> 
> Can the failing function print the strerror(errno) before returning?
> 
> If that's not meaningful because other threads may have messed up
> errno,
> please change the error message so that it doesn't mislead to
> non-existing log messages.

It depends on what's the exact cause (and reason, if any) why the previous
message is not logged.  "too frequent occurrence" was just a wild guess from
me... (sorry for not being explicit enough).

Rainer

From dave at fly.srk.fer.hr  Wed Jan 19 14:09:26 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Wed, 19 Jan 2011 14:09:26 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>
References: <20110119111016.GA29659@fly.srk.fer.hr>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>
Message-ID: <20110119130926.GC29659@fly.srk.fer.hr>

You're right. I managed to skip the file name returned by grep. Thanks.

I was digging a bit and it turns out that doEnqSingleObj() in queue.c has
only DBGOPRINT directives for these messages:

  - enqueueMsg: queue FULL - waiting to drain
  - enqueueMsg: cond timeout, dropping message!

The second one is followed by ABORT_FINALIZE and it will return an error
which can lead to the situation I described. That's the only such
condition in the code I found and my disk queues do get filled up from
time to time, while the testing period lasts.

I have 23 such messages per day, but only one day of testing. I think it
should be logged to give an administrator an indication about what's going
on.

The patch is attached, but I'm not sure if it's good. I'd very much like
to know which queue is having problems, so I'm trying to print its name. I
don't know what's set in objData.pszName nor whether a casual
administrator can link it to the stuff that goes in the config file, so
the first choice is file prefix for disk queues, then objData.pszName and
then "unnamed queue". I'd be grateful if you can suggest a better
alternative.

   
Rainer Gerhards wrote:
> This from ./tcpsrv.c, not gssapi. There seems to be some condition triggered
> which does not emit a message by itself, maybe removed because it occurred
> too frequently.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
> > Sent: Wednesday, January 19, 2011 12:10 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Tearing down TCP Session in the log file
> > 
> > Hello.
> > 
> > I have rsyslog 5.6.2 and I just tried to receive messages via TCP. The
> > sender is syslog-ng, I don't know exactly which version, not under my
> > control. The transfer mostly works, as far as I can see without strict
> > testing.
> > 
> > But in rsyslog's log file I see several messages which look like this:
> > 
> >    rsyslogd-2105: Tearing down TCP Session - see previous messages for
> > reason(s)
> >    [try http://www.rsyslog.com/e/2105 ]
> > 
> > And then there is no previous message which is different than this one
> > and
> > if I go back far enough there is the startup message. There's nothing
> > else in the log (produced by the *.* selector at the end of the config
> > file).
> > 
> > I tried to check the URL above, but I'm getting an error page with
> > says:
> > 
> >    Fatal error: Call to undefined function SendEmailToAdiscon() in
> >    /home/adisconweb/www/kb.monitorware.com/kbeventdb.php on line 217
> > 
> > That's generated by
> > http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog-rsyslogd-
> > 2105.html
> > 
> > I checked the source and the only file with the above message is
> > plugins/imgssapi/imgssapi.c in the
> > 
> >    if(tcps_sess.DataRcvd(pSess, buf, ret) !  = RS_RET_OK)
> > 
> > block. I didn't even know I was using gssapi plugin. My configuration
> > related to TCP is:
> > 
> > $ModLoad imtcp
> > $InputTCPServerBindRuleset indata
> > $InputTCPServerRun 514
> > 
> > and the rest is setup for the UDP receiver and storing messages. I
> > checked
> > with Google and rsyslog bug tracker for "Tearing down TCP Session", but
> > with zero results (except for the source code matches).
> > 
> > I am willing to debug this and submit a patch (which would at least log
> > the missing previous message), but before I do, I'd just like to ask if
> > someone has seen this before or has any idea what's going on. I'm a bit
> > confused with gssapi module being used. My OS is CentOS 5.5, if that's
> > relevant.
> > 
> > --
> >  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
> > (_  \ /  _)   ceremonial.
> >      |
> >      |        dave at fly.srk.fer.hr
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rsyslog-logerr-on-queue-full.diff
Type: text/x-diff
Size: 791 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110119/20b71d41/attachment.diff>

From rgerhards at hq.adiscon.com  Wed Jan 19 15:20:51 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Jan 2011 15:20:51 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
References: <20110119111016.GA29659@fly.srk.fer.hr><9B6E2A8877C38245BFB15CC491A11DA71DDA4A@GRFEXC.intern.adiscon.com>
	<20110119130926.GC29659@fly.srk.fer.hr>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA52@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
> Sent: Wednesday, January 19, 2011 2:09 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Tearing down TCP Session in the log file
> 
> You're right. I managed to skip the file name returned by grep. Thanks.
> 
> I was digging a bit and it turns out that doEnqSingleObj() in queue.c
> has
> only DBGOPRINT directives for these messages:
> 
>   - enqueueMsg: queue FULL - waiting to drain
>   - enqueueMsg: cond timeout, dropping message!
> 
> The second one is followed by ABORT_FINALIZE and it will return an
> error
> which can lead to the situation I described. That's the only such
> condition in the code I found and my disk queues do get filled up from
> time to time, while the testing period lasts.
> 
> I have 23 such messages per day, but only one day of testing. I think
> it
> should be logged to give an administrator an indication about what's
> going
> on.

Ah! Logging at this point is a bit problematic, because if the queue is full,
there is no space left for logging. Probably the best approach is to report
the exact error and *hope* that the queue has drained enough so that there is
space left to actually log that.

Please note that the disk queue subsystem is pretty slow, especially after
the last changes. I really needs to be redone, but this is such a big effort
that I currently can not do it without a sponsor :(

> 
> The patch is attached, but I'm not sure if it's good. I'd very much
> like
> to know which queue is having problems, so I'm trying to print its
> name. I
> don't know what's set in objData.pszName nor whether a casual
> administrator can link it to the stuff that goes in the config file, so
> the first choice is file prefix for disk queues, then objData.pszName
> and
> then "unnamed queue". I'd be grateful if you can suggest a better
> alternative.
> 

I'll have a look at the patch in detail. It probably is worth trying to log
an error message, especially if this problem occurs in an action queue.

I am very grateful for all your help, and sorry that I am extremely busy with
a couple of other things during all of our conversations. Hopefully it will
get better in the not so distant future (when I hope to be able to re-focus
on the next iteration of rsyslog tuning).

Rainer
> 
> Rainer Gerhards wrote:
> > This from ./tcpsrv.c, not gssapi. There seems to be some condition
> triggered
> > which does not emit a message by itself, maybe removed because it
> occurred
> > too frequently.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
> > > Sent: Wednesday, January 19, 2011 12:10 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] Tearing down TCP Session in the log file
> > >
> > > Hello.
> > >
> > > I have rsyslog 5.6.2 and I just tried to receive messages via TCP.
> The
> > > sender is syslog-ng, I don't know exactly which version, not under
> my
> > > control. The transfer mostly works, as far as I can see without
> strict
> > > testing.
> > >
> > > But in rsyslog's log file I see several messages which look like
> this:
> > >
> > >    rsyslogd-2105: Tearing down TCP Session - see previous messages
> for
> > > reason(s)
> > >    [try http://www.rsyslog.com/e/2105 ]
> > >
> > > And then there is no previous message which is different than this
> one
> > > and
> > > if I go back far enough there is the startup message. There's
> nothing
> > > else in the log (produced by the *.* selector at the end of the
> config
> > > file).
> > >
> > > I tried to check the URL above, but I'm getting an error page with
> > > says:
> > >
> > >    Fatal error: Call to undefined function SendEmailToAdiscon() in
> > >    /home/adisconweb/www/kb.monitorware.com/kbeventdb.php on line
> 217
> > >
> > > That's generated by
> > > http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog-
> rsyslogd-
> > > 2105.html
> > >
> > > I checked the source and the only file with the above message is
> > > plugins/imgssapi/imgssapi.c in the
> > >
> > >    if(tcps_sess.DataRcvd(pSess, buf, ret) !  = RS_RET_OK)
> > >
> > > block. I didn't even know I was using gssapi plugin. My
> configuration
> > > related to TCP is:
> > >
> > > $ModLoad imtcp
> > > $InputTCPServerBindRuleset indata
> > > $InputTCPServerRun 514
> > >
> > > and the rest is setup for the UDP receiver and storing messages. I
> > > checked
> > > with Google and rsyslog bug tracker for "Tearing down TCP Session",
> but
> > > with zero results (except for the source code matches).
> > >
> > > I am willing to debug this and submit a patch (which would at least
> log
> > > the missing previous message), but before I do, I'd just like to
> ask if
> > > someone has seen this before or has any idea what's going on. I'm a
> bit
> > > confused with gssapi module being used. My OS is CentOS 5.5, if
> that's
> > > relevant.
> > >
> > > --
> > >  .-.   .-.    Yes, I am an agent of Satan, but my duties are
> largely
> > > (_  \ /  _)   ceremonial.
> > >      |
> > >      |        dave at fly.srk.fer.hr
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> --
>  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
> (_  \ /  _)   ceremonial.
>      |
>      |        dave at fly.srk.fer.hr

From dave at fly.srk.fer.hr  Wed Jan 19 16:19:20 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Wed, 19 Jan 2011 16:19:20 +0100
Subject: [rsyslog] Tearing down TCP Session in the log file
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA52@GRFEXC.intern.adiscon.com>
References: <20110119130926.GC29659@fly.srk.fer.hr>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA52@GRFEXC.intern.adiscon.com>
Message-ID: <20110119151920.GD29659@fly.srk.fer.hr>

Rainer Gerhards wrote:
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar

> > The second one is followed by ABORT_FINALIZE and it will return an
> > error
> > which can lead to the situation I described. That's the only such
> > condition in the code I found and my disk queues do get filled up from
> > time to time, while the testing period lasts.
> > 
> > I have 23 such messages per day, but only one day of testing. I think
> > it
> > should be logged to give an administrator an indication about what's
> > going
> > on.

> Ah! Logging at this point is a bit problematic, because if the queue is full,
> there is no space left for logging.

That depends, I hope. I have set a separate OS partition for rsyslog
queues. All other partitions on the system will be empty enough, and my
log file for errors resides on one of them.

The disk assisted action queues are in their own ruleset and the action
queue for error messages is in the default ruleset, purely in memory. This
works reasonably fine, otherwise the TCP module wouldn't be able to log
its error messages.

But yes, there are situations when the message wouldn't be written
anywhere.

> Probably the best approach is to report
> the exact error and *hope* that the queue has drained enough so that there is
> space left to actually log that.

That's in case that the queue which is full and the queue for the error
message are the same queue, right?

> Please note that the disk queue subsystem is pretty slow, especially after
> the last changes. I really needs to be redone, but this is such a big effort
> that I currently can not do it without a sponsor :(

I've noticed that it takes a lot of time to empty 14 GB of disk queue. :-)
Rsyslog barely manages to keep up with the incoming messages while doing
that on my system, but it manages.

I am more concerned with fail-safe operations, though. Once, while the
partition with the queue files was full, rsyslog crashed. I have a
monitoring program which restarts it immediately, and it crashed again and
again. I cleaned up the mess by deleting everything on disk and rsyslog
started working again. I didn't have the time to investigate the problem
at that moment. My guess was that one of the files on disk was truncated
because there was no space left and that (part of) the rsyslog code
doesn't have checks for this possibility.

That was on a test system which was intentionally being left to fill up
the partition to see what happens. It shouldn't fill up in production.
I hope. :-)

And the partition has been filled up a number of times since then, but the
crashing never happened again.

> I'll have a look at the patch in detail. It probably is worth trying to log
> an error message, especially if this problem occurs in an action queue.

That's where it was, I think.

> I am very grateful for all your help, and sorry that I am extremely busy with
> a couple of other things during all of our conversations. Hopefully it will
> get better in the not so distant future (when I hope to be able to re-focus
> on the next iteration of rsyslog tuning).

I'm very grateful for your help, which happens to be terse, but it
contains all the information that I need. You're one of the most responsive
maintainers, as far as my experience goes, even though you don't have much
time at the moment. It's a pleasure working with you.

> 
> Rainer
> > 
> > Rainer Gerhards wrote:
> > > This from ./tcpsrv.c, not gssapi. There seems to be some condition
> > triggered
> > > which does not emit a message by itself, maybe removed because it
> > occurred
> > > too frequently.
> > >
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
> > > > Sent: Wednesday, January 19, 2011 12:10 PM
> > > > To: rsyslog at lists.adiscon.com
> > > > Subject: [rsyslog] Tearing down TCP Session in the log file
> > > >
> > > > Hello.
> > > >
> > > > I have rsyslog 5.6.2 and I just tried to receive messages via TCP.
> > The
> > > > sender is syslog-ng, I don't know exactly which version, not under
> > my
> > > > control. The transfer mostly works, as far as I can see without
> > strict
> > > > testing.
> > > >
> > > > But in rsyslog's log file I see several messages which look like
> > this:
> > > >
> > > >    rsyslogd-2105: Tearing down TCP Session - see previous messages
> > for
> > > > reason(s)
> > > >    [try http://www.rsyslog.com/e/2105 ]
> > > >
> > > > And then there is no previous message which is different than this
> > one
> > > > and
> > > > if I go back far enough there is the startup message. There's
> > nothing
> > > > else in the log (produced by the *.* selector at the end of the
> > config
> > > > file).
> > > >
> > > > I tried to check the URL above, but I'm getting an error page with
> > > > says:
> > > >
> > > >    Fatal error: Call to undefined function SendEmailToAdiscon() in
> > > >    /home/adisconweb/www/kb.monitorware.com/kbeventdb.php on line
> > 217
> > > >
> > > > That's generated by
> > > > http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog-
> > rsyslogd-
> > > > 2105.html
> > > >
> > > > I checked the source and the only file with the above message is
> > > > plugins/imgssapi/imgssapi.c in the
> > > >
> > > >    if(tcps_sess.DataRcvd(pSess, buf, ret) !  = RS_RET_OK)
> > > >
> > > > block. I didn't even know I was using gssapi plugin. My
> > configuration
> > > > related to TCP is:
> > > >
> > > > $ModLoad imtcp
> > > > $InputTCPServerBindRuleset indata
> > > > $InputTCPServerRun 514
> > > >
> > > > and the rest is setup for the UDP receiver and storing messages. I
> > > > checked
> > > > with Google and rsyslog bug tracker for "Tearing down TCP Session",
> > but
> > > > with zero results (except for the source code matches).
> > > >
> > > > I am willing to debug this and submit a patch (which would at least
> > log
> > > > the missing previous message), but before I do, I'd just like to
> > ask if
> > > > someone has seen this before or has any idea what's going on. I'm a
> > bit
> > > > confused with gssapi module being used. My OS is CentOS 5.5, if
> > that's
> > > > relevant.
> > > >
> > > > --
> > > >  .-.   .-.    Yes, I am an agent of Satan, but my duties are
> > largely
> > > > (_  \ /  _)   ceremonial.
> > > >      |
> > > >      |        dave at fly.srk.fer.hr
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > 
> > --
> >  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
> > (_  \ /  _)   ceremonial.
> >      |
> >      |        dave at fly.srk.fer.hr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr

From sean at conman.org  Wed Jan 19 17:07:35 2011
From: sean at conman.org (Sean Conner)
Date: Wed, 19 Jan 2011 11:07:35 -0500
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
Message-ID: <20110119160735.GA3783@brevard.conman.org>

It was thus said that the Great Robert Jennings once stated:
> With TCP it appears the clients remote logging stops indefinitely, in
> fact even when a failback occurs it still isnt logging messages to the
> primary server until I restart the client rsyslog process.
> 
> With UDP behaviour is as expected, with a ~20-30 second failover period
> where messages are lost (settings not tweaked in anyway)

  Does the virtual IP address have its own MAC address?  It could be an ARP
issue.

  Also, if you are using UDP, have you thought maybe of setting the rsyslogd
on each box to listen to a multicast address and send the traffic out to
said multicast address?  That way, both machines will get the traffic.  This
will only work if all the systems are on the same network segment though
(I'm doing this on my home network with a syslog daemon specifically
programmed to accept multicast addresses).

  -spc (Just a thought ... )



From raffy at loggly.com  Wed Jan 19 22:59:29 2011
From: raffy at loggly.com (Raffael Marty)
Date: Wed, 19 Jan 2011 13:59:29 -0800
Subject: [rsyslog] ImFile Question
Message-ID: <58061627-FB17-411F-8A1D-4306A0DBBB01@loggly.com>

I have the following config:

/etc/rsyslog.d/loggly-dal-nginx-access-prod.conf
$ModLoad imfile 
$InputFileName /var/log/nginx/access.log
$InputFileTag nginx-access:
$InputFileStateFile stat-nginx-access
$InputRunFileMonitor
*.*    @@logs.loggly.com:11111

/etc/rsyslog.d/loggly-dal-nginx-error-prod.conf
$ModLoad imfile 
$InputFileName /var/log/nginx/error.log
$InputFileTag nginx-error:
$InputFileStateFile stat-nginx-error
$InputRunFileMonitor
*.*    @@logs.loggly.com:22222

Two files to monitor. Each file I want to send to a different TCP port on logs.loggly.com. Problem is that both logs show up on both ports. How do I stop that behavior? Do I just add the "& ~" at the end of each block?

Thanks!

  Raffael

--
Raffael Marty                          Founder and COO @ Loggly
@zrlram                                          about.me/raffy





From ariel.p at hostdime.com  Wed Jan 19 23:16:09 2011
From: ariel.p at hostdime.com (Ariel P.)
Date: Wed, 19 Jan 2011 17:16:09 -0500
Subject: [rsyslog] ImFile Question
In-Reply-To: <58061627-FB17-411F-8A1D-4306A0DBBB01@loggly.com>
References: <58061627-FB17-411F-8A1D-4306A0DBBB01@loggly.com>
Message-ID: <4D3762A9.4030209@hostdime.com>

The official documentation [ 
http://www.rsyslog.com/doc/rsconf1_includeconfig.html ] says:
"This directive allows to include other files into the main 
configuration file. As soon as an IncludeConfig directive is found, the 
contents of the new file is processed. IncludeConfigs can be nested. 
Please note that from a logical point of view the files are merged. 
Thus, if the include modifies some parameters (e.g. 
$DynaFileChacheSize), these new parameters are in place for the 
"calling" configuration file when the include is completed. To avoid any 
side effects, do a $ResetConfigVariables after the $IncludeConfig. It 
may also be a good idea to do a $ResetConfigVariables right at the start 
of the include, so that the module knows exactly what it does. Of 
course, one might specifically NOT do this to inherit parameters from 
the main file. As always, use it as it best fits..."

Basically, all the files in [/etc/rsyslog.d] are cat'd together and 
parsed into the runtime configuration.

Your configuration is telling rsyslog to send all messages to two 
different servers.
If you add "& ~", only the first server will get everything, and the 
second will get nothing.
You should look into $InputFileFacility and $InputFileSeverity 
[http://www.rsyslog.com/doc/imfile.html] to create the selector line.

Ariel


On 2011-01-19 16:59, Raffael Marty wrote:
> I have the following config:
>
> /etc/rsyslog.d/loggly-dal-nginx-access-prod.conf
> $ModLoad imfile
> $InputFileName /var/log/nginx/access.log
> $InputFileTag nginx-access:
> $InputFileStateFile stat-nginx-access
> $InputRunFileMonitor
> *.*    @@logs.loggly.com:11111
>
> /etc/rsyslog.d/loggly-dal-nginx-error-prod.conf
> $ModLoad imfile
> $InputFileName /var/log/nginx/error.log
> $InputFileTag nginx-error:
> $InputFileStateFile stat-nginx-error
> $InputRunFileMonitor
> *.*    @@logs.loggly.com:22222
>
> Two files to monitor. Each file I want to send to a different TCP port on logs.loggly.com. Problem is that both logs show up on both ports. How do I stop that behavior? Do I just add the "&  ~" at the end of each block?
>
> Thanks!
>
>    Raffael
>
> --
> Raffael Marty                          Founder and COO @ Loggly
> @zrlram                                          about.me/raffy
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From paul.ruiz at gmail.com  Wed Jan 19 23:17:32 2011
From: paul.ruiz at gmail.com (Paul Ruiz)
Date: Wed, 19 Jan 2011 14:17:32 -0800
Subject: [rsyslog] ImFile Question
In-Reply-To: <58061627-FB17-411F-8A1D-4306A0DBBB01@loggly.com>
References: <58061627-FB17-411F-8A1D-4306A0DBBB01@loggly.com>
Message-ID: <AANLkTimcozVv8hYA_mXZYzdQF445iG9sZnnDbqdKL2H_@mail.gmail.com>

You can use the & ~ to accomplish thisYou'll need to add an if $programname
==

if $programname == 'NginxAccess' then @@syslog1:10518;commonForwardFormat
& ~


On Wed, Jan 19, 2011 at 1:59 PM, Raffael Marty <raffy at loggly.com> wrote:

> I have the following config:
>
> /etc/rsyslog.d/loggly-dal-nginx-access-prod.conf
> $ModLoad imfile
> $InputFileName /var/log/nginx/access.log
> $InputFileTag nginx-access:
> $InputFileStateFile stat-nginx-access
> $InputRunFileMonitor
> *.*    @@logs.loggly.com:11111
>
> /etc/rsyslog.d/loggly-dal-nginx-error-prod.conf
> $ModLoad imfile
> $InputFileName /var/log/nginx/error.log
> $InputFileTag nginx-error:
> $InputFileStateFile stat-nginx-error
> $InputRunFileMonitor
> *.*    @@logs.loggly.com:22222
>
> Two files to monitor. Each file I want to send to a different TCP port on
> logs.loggly.com. Problem is that both logs show up on both ports. How do I
> stop that behavior? Do I just add the "& ~" at the end of each block?
>
> Thanks!
>
>  Raffael
>
> --
> Raffael Marty                          Founder and COO @ Loggly
> @zrlram                                          about.me/raffy
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From Robert.Jennings at qbelmi.com  Thu Jan 20 00:49:36 2011
From: Robert.Jennings at qbelmi.com (Robert Jennings)
Date: Thu, 20 Jan 2011 10:49:36 +1100
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <20110119114607.GB29659@fly.srk.fer.hr>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net><alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm><96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
	<20110119114607.GB29659@fly.srk.fer.hr>
Message-ID: <96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>

 

-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
Sent: Wednesday, 19 January 2011 10:46 PM
To: rsyslog-users
Subject: Re: [rsyslog] rsyslog with heartbeat

Robert Jennings wrote:
> With TCP it appears the clients remote logging stops indefinitely, in
> fact even when a failback occurs it still isnt logging messages to the
> primary server until I restart the client rsyslog process.

How does your failover procedure work exactly? If it's crashing the OS
(or turning off power), then there's nothing which can send TCP close
packet, so your clients won't know they should close the connection.

If that's the case, there should be retransmitting TCP timeout on the
client, which is usually 13-15 minutes by default. The exact number
depends on the OS and the network MTU. Did you wait that long?





The failover simulates a network link going down, so effectively the network cable is just yanked.

I have switched to using UDP which is working fine, I never waited a full 15minutes but that is an unacceotable period of downtime. 

Is it known how RELP would behave in a heartbeat failover scenario? I have yet to test it as im running the stock rhel5 version of rsyslog 3.22





> With UDP behaviour is as expected, with a ~20-30 second failover period
> where messages are lost (settings not tweaked in anyway)
> 
> I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
> interest to try a more recent version.
> 
> 
> Which leads me to think perhaps the best solution may be:
> 
> Rsyslog clients UDP -> prd-syslog-000
> Syslog clients & devices UDP -> prd-syslog-000
> 
> Prd-syslog-002 TCP -> prd-syslog-001 
> 
> So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
> reliably queue and push messages to prd-syslog-001 once it is back
> online.
> 
> 
> Thanks,
> Rob
> 
> 
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, 19 January 2011 2:00 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog with heartbeat
> 
> how long did you wait for the failover to take place? the first thing
> that 
> will have to happen is that the sender will have to notice that the TCP 
> connection doesn't work anymore, close it, and open another one.
> 
> Since the system doesn't know what messages have actually been sent to
> the 
> server (as opposed to mearly sent to the TCP stack on the sending 
> machine), there will be some logs lost at failover time. the longer it 
> takes the client to notice the problem, the worse this will be.
> 
> version 3.22 is fairly old at this point, Rainer will have to weigh in
> on 
> how long it should take to notice the failure.
> 
> to fully avoid the lost messages issue, you would need to use RELP for 
> your transport.
> 
> I have opted to use UDP and accept that while the boxes are failing
> over, 
> some logs will be lost (you can tune heartbeat for pretty short failover
> 
> times, subsecond if you really push it)
> 
> David Lang
> 
> 
>   On Wed, 
> 19 Jan 2011, Robert Jennings wrote:
> 
> > I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
> sharing
> > a virtual IP managed by heartbeat (prd-syslog-000)
> >
> >
> > clients are configured to send logs to the virtualIP
> >
> > *.info;mail.none;authpriv.none;cron.none
> @@prd-syslog-000
> > authpriv.*
> > @@prd-syslog-000
> > local7.*
> > @@prd-syslog-000
> >
> >
> >
> >
> >
> > I have found when the primary host goes down and the secondary takes
> > over, logs are not arriving at the secondary host unless I restart
> > rsyslog on the client, this was not the behaviour I was expecting. Is
> > there any default configuration that may be stopping logs from being
> > sent when the server sitting behind the virtual ip changes?
> >
> >
> > The functionality I was trying to achive is as follows:
> >
> >
> > *	clients log to prd-syslog-000
> > *	heartbeat controls prd-syslog-000 address (prd-syslog-001
> > primary, prd-syslog-002 on failover)
> > *	prd-syslog-002 is configured to forward logs to prd-syslog-001
> > *	during downtime of prd-syslog-001 these logs will sit queued
> > *	when prd-syslog-001 comes back up it will take over
> > prd-syslog-000 and start recieving client logs
> > *	prd-syslog-002 will flush its queue back to prd-syslog-001
> >
> >
> > The reason I am using this setup as opposed to logging to two rsyslog
> > hosts is to support devices that can only log to one host, and provide
> > one location with a consistent set of logs instead of two machines
> with
> > out of sync logs.
> >
> > running rsyslog-3.22
> >
> >
> > Thanks,
> > Rob
> >
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From david at lang.hm  Thu Jan 20 01:03:09 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 19 Jan 2011 16:03:09 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <20110119114607.GB29659@fly.srk.fer.hr>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
	<20110119114607.GB29659@fly.srk.fer.hr>
Message-ID: <alpine.DEB.2.00.1101191601310.9980@asgard.lang.hm>

On Wed, 19 Jan 2011, Dra?en Ka?ar wrote:

> Robert Jennings wrote:
>> With TCP it appears the clients remote logging stops indefinitely, in
>> fact even when a failback occurs it still isnt logging messages to the
>> primary server until I restart the client rsyslog process.
>
> How does your failover procedure work exactly? If it's crashing the OS
> (or turning off power), then there's nothing which can send TCP close
> packet, so your clients won't know they should close the connection.

as far as the client sees, the IP address changes it's MAC address. any 
packets sent to the same IP address, but as part of the old connection, 
will be refused as the server will have no record of that connection.

this should happen on the first packet after the IP moves to the new 
machine.

This is the same thing that would happen if the server were to reboot.

David Lang


> If that's the case, there should be retransmitting TCP timeout on the
> client, which is usually 13-15 minutes by default. The exact number
> depends on the OS and the network MTU. Did you wait that long?
>
>> With UDP behaviour is as expected, with a ~20-30 second failover period
>> where messages are lost (settings not tweaked in anyway)
>>
>> I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
>> interest to try a more recent version.
>>
>>
>> Which leads me to think perhaps the best solution may be:
>>
>> Rsyslog clients UDP -> prd-syslog-000
>> Syslog clients & devices UDP -> prd-syslog-000
>>
>> Prd-syslog-002 TCP -> prd-syslog-001
>>
>> So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
>> reliably queue and push messages to prd-syslog-001 once it is back
>> online.
>>
>>
>> Thanks,
>> Rob
>>
>>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Wednesday, 19 January 2011 2:00 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog with heartbeat
>>
>> how long did you wait for the failover to take place? the first thing
>> that
>> will have to happen is that the sender will have to notice that the TCP
>> connection doesn't work anymore, close it, and open another one.
>>
>> Since the system doesn't know what messages have actually been sent to
>> the
>> server (as opposed to mearly sent to the TCP stack on the sending
>> machine), there will be some logs lost at failover time. the longer it
>> takes the client to notice the problem, the worse this will be.
>>
>> version 3.22 is fairly old at this point, Rainer will have to weigh in
>> on
>> how long it should take to notice the failure.
>>
>> to fully avoid the lost messages issue, you would need to use RELP for
>> your transport.
>>
>> I have opted to use UDP and accept that while the boxes are failing
>> over,
>> some logs will be lost (you can tune heartbeat for pretty short failover
>>
>> times, subsecond if you really push it)
>>
>> David Lang
>>
>>
>>   On Wed,
>> 19 Jan 2011, Robert Jennings wrote:
>>
>>> I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
>> sharing
>>> a virtual IP managed by heartbeat (prd-syslog-000)
>>>
>>>
>>> clients are configured to send logs to the virtualIP
>>>
>>> *.info;mail.none;authpriv.none;cron.none
>> @@prd-syslog-000
>>> authpriv.*
>>> @@prd-syslog-000
>>> local7.*
>>> @@prd-syslog-000
>>>
>>>
>>>
>>>
>>>
>>> I have found when the primary host goes down and the secondary takes
>>> over, logs are not arriving at the secondary host unless I restart
>>> rsyslog on the client, this was not the behaviour I was expecting. Is
>>> there any default configuration that may be stopping logs from being
>>> sent when the server sitting behind the virtual ip changes?
>>>
>>>
>>> The functionality I was trying to achive is as follows:
>>>
>>>
>>> *	clients log to prd-syslog-000
>>> *	heartbeat controls prd-syslog-000 address (prd-syslog-001
>>> primary, prd-syslog-002 on failover)
>>> *	prd-syslog-002 is configured to forward logs to prd-syslog-001
>>> *	during downtime of prd-syslog-001 these logs will sit queued
>>> *	when prd-syslog-001 comes back up it will take over
>>> prd-syslog-000 and start recieving client logs
>>> *	prd-syslog-002 will flush its queue back to prd-syslog-001
>>>
>>>
>>> The reason I am using this setup as opposed to logging to two rsyslog
>>> hosts is to support devices that can only log to one host, and provide
>>> one location with a consistent set of logs instead of two machines
>> with
>>> out of sync logs.
>>>
>>> running rsyslog-3.22
>>>
>>>
>>> Thanks,
>>> Rob
>>>
>>>
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
>

From david at lang.hm  Thu Jan 20 01:05:12 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 19 Jan 2011 16:05:12 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <20110119160735.GA3783@brevard.conman.org>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
	<20110119160735.GA3783@brevard.conman.org>
Message-ID: <alpine.DEB.2.00.1101191603370.9980@asgard.lang.hm>

On Wed, 19 Jan 2011, Sean Conner wrote:

> It was thus said that the Great Robert Jennings once stated:
>> With TCP it appears the clients remote logging stops indefinitely, in
>> fact even when a failback occurs it still isnt logging messages to the
>> primary server until I restart the client rsyslog process.
>>
>> With UDP behaviour is as expected, with a ~20-30 second failover period
>> where messages are lost (settings not tweaked in anyway)
>
>  Does the virtual IP address have its own MAC address?  It could be an ARP
> issue.
>
>  Also, if you are using UDP, have you thought maybe of setting the rsyslogd
> on each box to listen to a multicast address and send the traffic out to
> said multicast address?  That way, both machines will get the traffic.  This
> will only work if all the systems are on the same network segment though
> (I'm doing this on my home network with a syslog daemon specifically
> programmed to accept multicast addresses).

I actually do that (a multicast MAC address rather than a multicast IP 
address, but the same concept), the problem is reconciling the logs on the 
two boxes. if one goes down from 1:00-1:10 and the other is down from 
2:00-2:10, neither has a complete set of logs

David Lang

From david at lang.hm  Thu Jan 20 02:12:34 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 19 Jan 2011 17:12:34 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <alpine.DEB.2.00.1101191601310.9980@asgard.lang.hm>
References: <96066344EB62A74EBE0ACD4A52721ABA0201DB7C@PMI-AU-EXCH.auspac.net>
	<alpine.DEB.2.00.1101181853470.5754@asgard.lang.hm>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB7F@PMI-AU-EXCH.auspac.net>
	<20110119114607.GB29659@fly.srk.fer.hr>
	<alpine.DEB.2.00.1101191601310.9980@asgard.lang.hm>
Message-ID: <alpine.DEB.2.00.1101191705290.9980@asgard.lang.hm>

On Wed, 19 Jan 2011, david at lang.hm wrote:

> On Wed, 19 Jan 2011, Dra?en Ka?ar wrote:
>
>> Robert Jennings wrote:
>>> With TCP it appears the clients remote logging stops indefinitely, in
>>> fact even when a failback occurs it still isnt logging messages to the
>>> primary server until I restart the client rsyslog process.
>> 
>> How does your failover procedure work exactly? If it's crashing the OS
>> (or turning off power), then there's nothing which can send TCP close
>> packet, so your clients won't know they should close the connection.
>
> as far as the client sees, the IP address changes it's MAC address. any 
> packets sent to the same IP address, but as part of the old connection, will 
> be refused as the server will have no record of that connection.

this should generatea reset packet from the server. it could be that there 
are firewall rules that are blocking this (not accepting packets that 
aren't either the start of a connection or part of an existing connection)

David Lang

> this should happen on the first packet after the IP moves to the new machine.
>
> This is the same thing that would happen if the server were to reboot.
>
> David Lang
>
>
>> If that's the case, there should be retransmitting TCP timeout on the
>> client, which is usually 13-15 minutes by default. The exact number
>> depends on the OS and the network MTU. Did you wait that long?
>> 
>>> With UDP behaviour is as expected, with a ~20-30 second failover period
>>> where messages are lost (settings not tweaked in anyway)
>>> 
>>> I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
>>> interest to try a more recent version.
>>> 
>>> 
>>> Which leads me to think perhaps the best solution may be:
>>> 
>>> Rsyslog clients UDP -> prd-syslog-000
>>> Syslog clients & devices UDP -> prd-syslog-000
>>> 
>>> Prd-syslog-002 TCP -> prd-syslog-001
>>> 
>>> So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
>>> reliably queue and push messages to prd-syslog-001 once it is back
>>> online.
>>> 
>>> 
>>> Thanks,
>>> Rob
>>> 
>>> 
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Wednesday, 19 January 2011 2:00 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] rsyslog with heartbeat
>>> 
>>> how long did you wait for the failover to take place? the first thing
>>> that
>>> will have to happen is that the sender will have to notice that the TCP
>>> connection doesn't work anymore, close it, and open another one.
>>> 
>>> Since the system doesn't know what messages have actually been sent to
>>> the
>>> server (as opposed to mearly sent to the TCP stack on the sending
>>> machine), there will be some logs lost at failover time. the longer it
>>> takes the client to notice the problem, the worse this will be.
>>> 
>>> version 3.22 is fairly old at this point, Rainer will have to weigh in
>>> on
>>> how long it should take to notice the failure.
>>> 
>>> to fully avoid the lost messages issue, you would need to use RELP for
>>> your transport.
>>> 
>>> I have opted to use UDP and accept that while the boxes are failing
>>> over,
>>> some logs will be lost (you can tune heartbeat for pretty short failover
>>> 
>>> times, subsecond if you really push it)
>>> 
>>> David Lang
>>> 
>>>
>>>   On Wed,
>>> 19 Jan 2011, Robert Jennings wrote:
>>> 
>>>> I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
>>> sharing
>>>> a virtual IP managed by heartbeat (prd-syslog-000)
>>>> 
>>>> 
>>>> clients are configured to send logs to the virtualIP
>>>> 
>>>> *.info;mail.none;authpriv.none;cron.none
>>> @@prd-syslog-000
>>>> authpriv.*
>>>> @@prd-syslog-000
>>>> local7.*
>>>> @@prd-syslog-000
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> I have found when the primary host goes down and the secondary takes
>>>> over, logs are not arriving at the secondary host unless I restart
>>>> rsyslog on the client, this was not the behaviour I was expecting. Is
>>>> there any default configuration that may be stopping logs from being
>>>> sent when the server sitting behind the virtual ip changes?
>>>> 
>>>> 
>>>> The functionality I was trying to achive is as follows:
>>>> 
>>>> 
>>>> *	clients log to prd-syslog-000
>>>> *	heartbeat controls prd-syslog-000 address (prd-syslog-001
>>>> primary, prd-syslog-002 on failover)
>>>> *	prd-syslog-002 is configured to forward logs to prd-syslog-001
>>>> *	during downtime of prd-syslog-001 these logs will sit queued
>>>> *	when prd-syslog-001 comes back up it will take over
>>>> prd-syslog-000 and start recieving client logs
>>>> *	prd-syslog-002 will flush its queue back to prd-syslog-001
>>>> 
>>>> 
>>>> The reason I am using this setup as opposed to logging to two rsyslog
>>>> hosts is to support devices that can only log to one host, and provide
>>>> one location with a consistent set of logs instead of two machines
>>> with
>>>> out of sync logs.
>>>> 
>>>> running rsyslog-3.22
>>>> 
>>>> 
>>>> Thanks,
>>>> Rob
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>> 
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> 
>> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From andersns at basefarm.no  Thu Jan 20 10:07:51 2011
From: andersns at basefarm.no (Anders Synstad)
Date: Thu, 20 Jan 2011 10:07:51 +0100
Subject: [rsyslog] Problem with "corrupt" log message
Message-ID: <4D37FB67.60406@basefarm.no>

Hi,

I just noticed the following in my logfile:

2011-01-19T15:21:43.246003+01:00 HOSTA2011-01-19T15:21:43.247517+01:00 
HOSTB <164>Jan 19 2011 15:21:43: RAWMESSAGE


It seems like the log message from 2 hosts has been merged in the middle 
of the first logmessage.

The relevant configuration looks like:
#
# Load modules
#
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support (previously done by 
rklogd)
$ModLoad imudp    # Standard input module for UDP
$ModLoad imtcp    # Standard input module for TCP

#
# Logfile templates
#
$template t-network,"/var/log/network.%$myhostname%.log"
$template bf-default,"%timegenerated:::date-rfc3339% %fromhost% 
%rawmsg:::drop-last-lf%\n"

# Ruleset: network
$Ruleset network-udp-10514
$RulesetCreateMainQueue on
*.*             -?t-network;bf-default

$RuleSet network-tcp-10514
$RulesetCreateMainQueue on
*.*             -?t-network;bf-default


Any ideas what would cause the log message to be corrupted like that?



Regards,
Anders Synstad
Basefarm AS

From rgerhards at hq.adiscon.com  Thu Jan 20 10:26:15 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Jan 2011 10:26:15 +0100
Subject: [rsyslog] Problem with "corrupt" log message
References: <4D37FB67.60406@basefarm.no>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA5C@GRFEXC.intern.adiscon.com>

Puuh, that's (very) hard to guess. Obviously it could be some program bug in
the receiver, but it could also be a problem at the sender side. Do you run
the latest builds of rsyslog? IF the problem is reproducible, a network trace
or debug log would be useful, but I guess I am not telling you anything new
;)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Anders Synstad
> Sent: Thursday, January 20, 2011 10:08 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Problem with "corrupt" log message
> 
> Hi,
> 
> I just noticed the following in my logfile:
> 
> 2011-01-19T15:21:43.246003+01:00 HOSTA2011-01-19T15:21:43.247517+01:00
> HOSTB <164>Jan 19 2011 15:21:43: RAWMESSAGE
> 
> 
> It seems like the log message from 2 hosts has been merged in the
> middle
> of the first logmessage.
> 
> The relevant configuration looks like:
> #
> # Load modules
> #
> $ModLoad imuxsock # provides support for local system logging
> $ModLoad imklog   # provides kernel logging support (previously done by
> rklogd)
> $ModLoad imudp    # Standard input module for UDP
> $ModLoad imtcp    # Standard input module for TCP
> 
> #
> # Logfile templates
> #
> $template t-network,"/var/log/network.%$myhostname%.log"
> $template bf-default,"%timegenerated:::date-rfc3339% %fromhost%
> %rawmsg:::drop-last-lf%\n"
> 
> # Ruleset: network
> $Ruleset network-udp-10514
> $RulesetCreateMainQueue on
> *.*             -?t-network;bf-default
> 
> $RuleSet network-tcp-10514
> $RulesetCreateMainQueue on
> *.*             -?t-network;bf-default
> 
> 
> Any ideas what would cause the log message to be corrupted like that?
> 
> 
> 
> Regards,
> Anders Synstad
> Basefarm AS
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Thu Jan 20 11:21:50 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 20 Jan 2011 02:21:50 -0800 (PST)
Subject: [rsyslog] Problem with "corrupt" log message
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA5C@GRFEXC.intern.adiscon.com>
References: <4D37FB67.60406@basefarm.no>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA5C@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101200219590.9980@asgard.lang.hm>

I have seen this on some older (6 month or so) builds, but only when there 
are multiple selectors that write to the same file (the same selectors 
send the logs further upstream, and the upstream logs are clean)

I know there have been a lot of improvements in this area, so I was 
planning to upgrade before reporting it

David Lang

On Thu, 20 Jan 2011, Rainer 
Gerhards wrote:

> Date: Thu, 20 Jan 2011 10:26:15 +0100
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: [rsyslog] Problem with "corrupt" log message
> 
> Puuh, that's (very) hard to guess. Obviously it could be some program bug in
> the receiver, but it could also be a problem at the sender side. Do you run
> the latest builds of rsyslog? IF the problem is reproducible, a network trace
> or debug log would be useful, but I guess I am not telling you anything new
> ;)
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Anders Synstad
>> Sent: Thursday, January 20, 2011 10:08 AM
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] Problem with "corrupt" log message
>>
>> Hi,
>>
>> I just noticed the following in my logfile:
>>
>> 2011-01-19T15:21:43.246003+01:00 HOSTA2011-01-19T15:21:43.247517+01:00
>> HOSTB <164>Jan 19 2011 15:21:43: RAWMESSAGE
>>
>>
>> It seems like the log message from 2 hosts has been merged in the
>> middle
>> of the first logmessage.
>>
>> The relevant configuration looks like:
>> #
>> # Load modules
>> #
>> $ModLoad imuxsock # provides support for local system logging
>> $ModLoad imklog   # provides kernel logging support (previously done by
>> rklogd)
>> $ModLoad imudp    # Standard input module for UDP
>> $ModLoad imtcp    # Standard input module for TCP
>>
>> #
>> # Logfile templates
>> #
>> $template t-network,"/var/log/network.%$myhostname%.log"
>> $template bf-default,"%timegenerated:::date-rfc3339% %fromhost%
>> %rawmsg:::drop-last-lf%\n"
>>
>> # Ruleset: network
>> $Ruleset network-udp-10514
>> $RulesetCreateMainQueue on
>> *.*             -?t-network;bf-default
>>
>> $RuleSet network-tcp-10514
>> $RulesetCreateMainQueue on
>> *.*             -?t-network;bf-default
>>
>>
>> Any ideas what would cause the log message to be corrupted like that?
>>
>>
>>
>> Regards,
>> Anders Synstad
>> Basefarm AS
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From andersns at basefarm.no  Thu Jan 20 11:54:02 2011
From: andersns at basefarm.no (Anders Synstad)
Date: Thu, 20 Jan 2011 11:54:02 +0100
Subject: [rsyslog] Problem with "corrupt" log message
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDA5C@GRFEXC.intern.adiscon.com>
References: <4D37FB67.60406@basefarm.no>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA5C@GRFEXC.intern.adiscon.com>
Message-ID: <4D38144A.4000006@basefarm.no>

You guess right ;)

I'm running latest Rsyslog 5.6.2. I pretty much came to the same 
conclusions as you.

I doubt I can reproduce it. Out of several gigabytes of logs each day, 
this is the first time I've seen it.


BR
Anders Synstad
Basefarm AS

On 01/20/2011 10:26 AM, Rainer Gerhards wrote:
> Puuh, that's (very) hard to guess. Obviously it could be some program bug in
> the receiver, but it could also be a problem at the sender side. Do you run
> the latest builds of rsyslog? IF the problem is reproducible, a network trace
> or debug log would be useful, but I guess I am not telling you anything new
> ;)
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Anders Synstad
>> Sent: Thursday, January 20, 2011 10:08 AM
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] Problem with "corrupt" log message
>>
>> Hi,
>>
>> I just noticed the following in my logfile:
>>
>> 2011-01-19T15:21:43.246003+01:00 HOSTA2011-01-19T15:21:43.247517+01:00
>> HOSTB<164>Jan 19 2011 15:21:43: RAWMESSAGE
>>
>>
>> It seems like the log message from 2 hosts has been merged in the
>> middle
>> of the first logmessage.
>>
>> The relevant configuration looks like:
>> #
>> # Load modules
>> #
>> $ModLoad imuxsock # provides support for local system logging
>> $ModLoad imklog   # provides kernel logging support (previously done by
>> rklogd)
>> $ModLoad imudp    # Standard input module for UDP
>> $ModLoad imtcp    # Standard input module for TCP
>>
>> #
>> # Logfile templates
>> #
>> $template t-network,"/var/log/network.%$myhostname%.log"
>> $template bf-default,"%timegenerated:::date-rfc3339% %fromhost%
>> %rawmsg:::drop-last-lf%\n"
>>
>> # Ruleset: network
>> $Ruleset network-udp-10514
>> $RulesetCreateMainQueue on
>> *.*             -?t-network;bf-default
>>
>> $RuleSet network-tcp-10514
>> $RulesetCreateMainQueue on
>> *.*             -?t-network;bf-default
>>
>>
>> Any ideas what would cause the log message to be corrupted like that?
>>
>>
>>
>> Regards,
>> Anders Synstad
>> Basefarm AS
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dave at fly.srk.fer.hr  Thu Jan 20 12:54:18 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Thu, 20 Jan 2011 12:54:18 +0100
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>
References: <20110119114607.GB29659@fly.srk.fer.hr>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>
Message-ID: <20110120115418.GA4688@fly.srk.fer.hr>

Robert Jennings wrote:
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar

> > How does your failover procedure work exactly? If it's crashing the OS
> > (or turning off power), then there's nothing which can send TCP close
> > packet, so your clients won't know they should close the connection.
> > 
> > If that's the case, there should be retransmitting TCP timeout on the
> > client, which is usually 13-15 minutes by default. The exact number
> > depends on the OS and the network MTU. Did you wait that long?
> 
> The failover simulates a network link going down, so effectively the
> network cable is just yanked.

In that case TCP FIN packets can't get to the client, so you get the
behavior you're describing.

> I have switched to using UDP which is working fine, I never waited a
> full 15minutes but that is an unacceotable period of downtime. 

Sure, but that timeout is configurable on the OS level. Write timeouts
should be configurable on the application level (in this case that's the
client rsyslog you're using), but I don't know if your version has that
feature. Check the documentation.

And there's an option of using TCP keepalive (which also has configurable
timeouts), so client can check for the availability of server. I do not
know if rsyslog can set keepalive option on its outgoing TCP connections,
though. But that can be easily added. It can be added even if the
executable you're using doesn't support it.

> Is it known how RELP would behave in a heartbeat failover scenario? I
> have yet to test it as im running the stock rhel5 version of rsyslog
> 3.22

It should be able to detect that the remote server is unavailable when it
tries to send the first message under those conditions. I haven't tried
it, though.

> 
> 
> 
> 
> 
> > With UDP behaviour is as expected, with a ~20-30 second failover period
> > where messages are lost (settings not tweaked in anyway)
> > 
> > I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
> > interest to try a more recent version.
> > 
> > 
> > Which leads me to think perhaps the best solution may be:
> > 
> > Rsyslog clients UDP -> prd-syslog-000
> > Syslog clients & devices UDP -> prd-syslog-000
> > 
> > Prd-syslog-002 TCP -> prd-syslog-001 
> > 
> > So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
> > reliably queue and push messages to prd-syslog-001 once it is back
> > online.
> > 
> > 
> > Thanks,
> > Rob
> > 
> > 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com
> > [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Wednesday, 19 January 2011 2:00 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog with heartbeat
> > 
> > how long did you wait for the failover to take place? the first thing
> > that 
> > will have to happen is that the sender will have to notice that the TCP 
> > connection doesn't work anymore, close it, and open another one.
> > 
> > Since the system doesn't know what messages have actually been sent to
> > the 
> > server (as opposed to mearly sent to the TCP stack on the sending 
> > machine), there will be some logs lost at failover time. the longer it 
> > takes the client to notice the problem, the worse this will be.
> > 
> > version 3.22 is fairly old at this point, Rainer will have to weigh in
> > on 
> > how long it should take to notice the failure.
> > 
> > to fully avoid the lost messages issue, you would need to use RELP for 
> > your transport.
> > 
> > I have opted to use UDP and accept that while the boxes are failing
> > over, 
> > some logs will be lost (you can tune heartbeat for pretty short failover
> > 
> > times, subsecond if you really push it)
> > 
> > David Lang
> > 
> > 
> >   On Wed, 
> > 19 Jan 2011, Robert Jennings wrote:
> > 
> > > I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
> > sharing
> > > a virtual IP managed by heartbeat (prd-syslog-000)
> > >
> > >
> > > clients are configured to send logs to the virtualIP
> > >
> > > *.info;mail.none;authpriv.none;cron.none
> > @@prd-syslog-000
> > > authpriv.*
> > > @@prd-syslog-000
> > > local7.*
> > > @@prd-syslog-000
> > >
> > >
> > >
> > >
> > >
> > > I have found when the primary host goes down and the secondary takes
> > > over, logs are not arriving at the secondary host unless I restart
> > > rsyslog on the client, this was not the behaviour I was expecting. Is
> > > there any default configuration that may be stopping logs from being
> > > sent when the server sitting behind the virtual ip changes?
> > >
> > >
> > > The functionality I was trying to achive is as follows:
> > >
> > >
> > > *	clients log to prd-syslog-000
> > > *	heartbeat controls prd-syslog-000 address (prd-syslog-001
> > > primary, prd-syslog-002 on failover)
> > > *	prd-syslog-002 is configured to forward logs to prd-syslog-001
> > > *	during downtime of prd-syslog-001 these logs will sit queued
> > > *	when prd-syslog-001 comes back up it will take over
> > > prd-syslog-000 and start recieving client logs
> > > *	prd-syslog-002 will flush its queue back to prd-syslog-001
> > >
> > >
> > > The reason I am using this setup as opposed to logging to two rsyslog
> > > hosts is to support devices that can only log to one host, and provide
> > > one location with a consistent set of logs instead of two machines
> > with
> > > out of sync logs.
> > >
> > > running rsyslog-3.22
> > >
> > >
> > > Thanks,
> > > Rob
> > >
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> -- 
>  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
> (_  \ /  _)   ceremonial.
>      |
>      |        dave at fly.srk.fer.hr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr

From rgerhards at hq.adiscon.com  Thu Jan 20 13:21:20 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Jan 2011 13:21:20 +0100
Subject: [rsyslog] Problem with "corrupt" log message
References: <4D37FB67.60406@basefarm.no><9B6E2A8877C38245BFB15CC491A11DA71DDA5C@GRFEXC.intern.adiscon.com>
	<4D38144A.4000006@basefarm.no>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDA60@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Anders Synstad
> Sent: Thursday, January 20, 2011 11:54 AM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Problem with "corrupt" log message
> 
> You guess right ;)
> 
> I'm running latest Rsyslog 5.6.2. I pretty much came to the same
> conclusions as you.
> 
> I doubt I can reproduce it. Out of several gigabytes of logs each day,
> this is the first time I've seen it.

Yup... Maybe this helps: I got some reports from folks who have a bit of a
problem with 5.6.2 and a very good report came in yesterday. I am about to
look at it. It could be that all of this has a common reason. But other than
that, I do not have any advise right now...

RAiner
> 
> 
> BR
> Anders Synstad
> Basefarm AS
> 
> On 01/20/2011 10:26 AM, Rainer Gerhards wrote:
> > Puuh, that's (very) hard to guess. Obviously it could be some program
> bug in
> > the receiver, but it could also be a problem at the sender side. Do
> you run
> > the latest builds of rsyslog? IF the problem is reproducible, a
> network trace
> > or debug log would be useful, but I guess I am not telling you
> anything new
> > ;)
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Anders Synstad
> >> Sent: Thursday, January 20, 2011 10:08 AM
> >> To: rsyslog at lists.adiscon.com
> >> Subject: [rsyslog] Problem with "corrupt" log message
> >>
> >> Hi,
> >>
> >> I just noticed the following in my logfile:
> >>
> >> 2011-01-19T15:21:43.246003+01:00 HOSTA2011-01-
> 19T15:21:43.247517+01:00
> >> HOSTB<164>Jan 19 2011 15:21:43: RAWMESSAGE
> >>
> >>
> >> It seems like the log message from 2 hosts has been merged in the
> >> middle
> >> of the first logmessage.
> >>
> >> The relevant configuration looks like:
> >> #
> >> # Load modules
> >> #
> >> $ModLoad imuxsock # provides support for local system logging
> >> $ModLoad imklog   # provides kernel logging support (previously done
> by
> >> rklogd)
> >> $ModLoad imudp    # Standard input module for UDP
> >> $ModLoad imtcp    # Standard input module for TCP
> >>
> >> #
> >> # Logfile templates
> >> #
> >> $template t-network,"/var/log/network.%$myhostname%.log"
> >> $template bf-default,"%timegenerated:::date-rfc3339% %fromhost%
> >> %rawmsg:::drop-last-lf%\n"
> >>
> >> # Ruleset: network
> >> $Ruleset network-udp-10514
> >> $RulesetCreateMainQueue on
> >> *.*             -?t-network;bf-default
> >>
> >> $RuleSet network-tcp-10514
> >> $RulesetCreateMainQueue on
> >> *.*             -?t-network;bf-default
> >>
> >>
> >> Any ideas what would cause the log message to be corrupted like
> that?
> >>
> >>
> >>
> >> Regards,
> >> Anders Synstad
> >> Basefarm AS
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Thu Jan 20 18:28:15 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 20 Jan 2011 09:28:15 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <20110120115418.GA4688@fly.srk.fer.hr>
References: <20110119114607.GB29659@fly.srk.fer.hr>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>
	<20110120115418.GA4688@fly.srk.fer.hr>
Message-ID: <alpine.DEB.2.00.1101200927150.9980@asgard.lang.hm>

On Thu, 20 Jan 2011, Dra?en Ka?ar wrote:

> Robert Jennings wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Dra?en Kacar
>
>>> How does your failover procedure work exactly? If it's crashing the OS
>>> (or turning off power), then there's nothing which can send TCP close
>>> packet, so your clients won't know they should close the connection.
>>>
>>> If that's the case, there should be retransmitting TCP timeout on the
>>> client, which is usually 13-15 minutes by default. The exact number
>>> depends on the OS and the network MTU. Did you wait that long?
>>
>> The failover simulates a network link going down, so effectively the
>> network cable is just yanked.
>
> In that case TCP FIN packets can't get to the client, so you get the
> behavior you're describing.

FIN packets don't get there, but once the new box is up, it should be 
sending a RESET packet

David Lang

>> I have switched to using UDP which is working fine, I never waited a
>> full 15minutes but that is an unacceotable period of downtime.
>
> Sure, but that timeout is configurable on the OS level. Write timeouts
> should be configurable on the application level (in this case that's the
> client rsyslog you're using), but I don't know if your version has that
> feature. Check the documentation.
>
> And there's an option of using TCP keepalive (which also has configurable
> timeouts), so client can check for the availability of server. I do not
> know if rsyslog can set keepalive option on its outgoing TCP connections,
> though. But that can be easily added. It can be added even if the
> executable you're using doesn't support it.
>
>> Is it known how RELP would behave in a heartbeat failover scenario? I
>> have yet to test it as im running the stock rhel5 version of rsyslog
>> 3.22
>
> It should be able to detect that the remote server is unavailable when it
> tries to send the first message under those conditions. I haven't tried
> it, though.
>
>>
>>
>>
>>
>>
>>> With UDP behaviour is as expected, with a ~20-30 second failover period
>>> where messages are lost (settings not tweaked in anyway)
>>>
>>> I am using 3.22 simply as it is shipped with RHEL5/CENTOS5, but would be
>>> interest to try a more recent version.
>>>
>>>
>>> Which leads me to think perhaps the best solution may be:
>>>
>>> Rsyslog clients UDP -> prd-syslog-000
>>> Syslog clients & devices UDP -> prd-syslog-000
>>>
>>> Prd-syslog-002 TCP -> prd-syslog-001
>>>
>>> So prd-syslog-002 is only ever hit when prd-syslog-001 is down, and can
>>> reliably queue and push messages to prd-syslog-001 once it is back
>>> online.
>>>
>>>
>>> Thanks,
>>> Rob
>>>
>>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Wednesday, 19 January 2011 2:00 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] rsyslog with heartbeat
>>>
>>> how long did you wait for the failover to take place? the first thing
>>> that
>>> will have to happen is that the sender will have to notice that the TCP
>>> connection doesn't work anymore, close it, and open another one.
>>>
>>> Since the system doesn't know what messages have actually been sent to
>>> the
>>> server (as opposed to mearly sent to the TCP stack on the sending
>>> machine), there will be some logs lost at failover time. the longer it
>>> takes the client to notice the problem, the worse this will be.
>>>
>>> version 3.22 is fairly old at this point, Rainer will have to weigh in
>>> on
>>> how long it should take to notice the failure.
>>>
>>> to fully avoid the lost messages issue, you would need to use RELP for
>>> your transport.
>>>
>>> I have opted to use UDP and accept that while the boxes are failing
>>> over,
>>> some logs will be lost (you can tune heartbeat for pretty short failover
>>>
>>> times, subsecond if you really push it)
>>>
>>> David Lang
>>>
>>>
>>>   On Wed,
>>> 19 Jan 2011, Robert Jennings wrote:
>>>
>>>> I am running 2 rsyslog servers (prd-syslog-001, prd-syslog-002)
>>> sharing
>>>> a virtual IP managed by heartbeat (prd-syslog-000)
>>>>
>>>>
>>>> clients are configured to send logs to the virtualIP
>>>>
>>>> *.info;mail.none;authpriv.none;cron.none
>>> @@prd-syslog-000
>>>> authpriv.*
>>>> @@prd-syslog-000
>>>> local7.*
>>>> @@prd-syslog-000
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I have found when the primary host goes down and the secondary takes
>>>> over, logs are not arriving at the secondary host unless I restart
>>>> rsyslog on the client, this was not the behaviour I was expecting. Is
>>>> there any default configuration that may be stopping logs from being
>>>> sent when the server sitting behind the virtual ip changes?
>>>>
>>>>
>>>> The functionality I was trying to achive is as follows:
>>>>
>>>>
>>>> *	clients log to prd-syslog-000
>>>> *	heartbeat controls prd-syslog-000 address (prd-syslog-001
>>>> primary, prd-syslog-002 on failover)
>>>> *	prd-syslog-002 is configured to forward logs to prd-syslog-001
>>>> *	during downtime of prd-syslog-001 these logs will sit queued
>>>> *	when prd-syslog-001 comes back up it will take over
>>>> prd-syslog-000 and start recieving client logs
>>>> *	prd-syslog-002 will flush its queue back to prd-syslog-001
>>>>
>>>>
>>>> The reason I am using this setup as opposed to logging to two rsyslog
>>>> hosts is to support devices that can only log to one host, and provide
>>>> one location with a consistent set of logs instead of two machines
>>> with
>>>> out of sync logs.
>>>>
>>>> running rsyslog-3.22
>>>>
>>>>
>>>> Thanks,
>>>> Rob
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>
>> --
>>  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
>> (_  \ /  _)   ceremonial.
>>      |
>>      |        dave at fly.srk.fer.hr
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
>

From dave at fly.srk.fer.hr  Thu Jan 20 20:12:15 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Thu, 20 Jan 2011 20:12:15 +0100
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <alpine.DEB.2.00.1101200927150.9980@asgard.lang.hm>
References: <20110119114607.GB29659@fly.srk.fer.hr>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>
	<20110120115418.GA4688@fly.srk.fer.hr>
	<alpine.DEB.2.00.1101200927150.9980@asgard.lang.hm>
Message-ID: <20110120191215.GA22425@fly.srk.fer.hr>

david at lang.hm wrote:
> On Thu, 20 Jan 2011, Dra?en Ka?ar wrote:

>> In that case TCP FIN packets can't get to the client, so you get the
>> behavior you're describing.
>
> FIN packets don't get there, but once the new box is up, it should be  
> sending a RESET packet

You're right. So it's firewall causing trouble most likely.

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr

From mmonitz at gmail.com  Tue Jan 25 07:33:10 2011
From: mmonitz at gmail.com (matan monitz)
Date: Tue, 25 Jan 2011 08:33:10 +0200
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <20110120191215.GA22425@fly.srk.fer.hr>
References: <20110119114607.GB29659@fly.srk.fer.hr>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>
	<20110120115418.GA4688@fly.srk.fer.hr>
	<alpine.DEB.2.00.1101200927150.9980@asgard.lang.hm>
	<20110120191215.GA22425@fly.srk.fer.hr>
Message-ID: <AANLkTi=Aqvc36HsTJTSwmbFmACCAB+FjPXV+o1YnTZfD@mail.gmail.com>

is the rsyslog process still running before you restart the rsyslog client?
in what way do you create the vip?

2011/1/20 Dra?en Ka?ar <dave at fly.srk.fer.hr>

> david at lang.hm wrote:
> > On Thu, 20 Jan 2011, Dra?en Ka?ar wrote:
>
> >> In that case TCP FIN packets can't get to the client, so you get the
> >> behavior you're describing.
> >
> > FIN packets don't get there, but once the new box is up, it should be
> > sending a RESET packet
>
> You're right. So it's firewall causing trouble most likely.
>
> --
>  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
> (_  \ /  _)   ceremonial.
>     |
>     |        dave at fly.srk.fer.hr
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From david at lang.hm  Tue Jan 25 09:09:31 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 25 Jan 2011 00:09:31 -0800 (PST)
Subject: [rsyslog] rsyslog with heartbeat
In-Reply-To: <AANLkTi=Aqvc36HsTJTSwmbFmACCAB+FjPXV+o1YnTZfD@mail.gmail.com>
References: <20110119114607.GB29659@fly.srk.fer.hr>
	<96066344EB62A74EBE0ACD4A52721ABA0201DB81@PMI-AU-EXCH.auspac.net>
	<20110120115418.GA4688@fly.srk.fer.hr>
	<alpine.DEB.2.00.1101200927150.9980@asgard.lang.hm>
	<20110120191215.GA22425@fly.srk.fer.hr>
	<AANLkTi=Aqvc36HsTJTSwmbFmACCAB+FjPXV+o1YnTZfD@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1101250006130.29659@asgard.lang.hm>

In my case I have rsyslog running all the time.

I still use the v1 style config file (haresources), with that mode I just 
put the vip IP address on in the haresources file.

David Lang


On Tue, 25 Jan 2011, matan monitz wrote:

> is the rsyslog process still running before you restart the rsyslog client?
> in what way do you create the vip?
>
> 2011/1/20 Dra?en Ka?ar <dave at fly.srk.fer.hr>
>
>> david at lang.hm wrote:
>> > On Thu, 20 Jan 2011, Dra?en Ka?ar wrote:
>>
>> >> In that case TCP FIN packets can't get to the client, so you get the
>> >> behavior you're describing.
>> >
>> > FIN packets don't get there, but once the new box is up, it should be
>> > sending a RESET packet
>>
>> You're right. So it's firewall causing trouble most likely.
>>
>> --
>>  .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
>> (_  \ /  _)   ceremonial.
>>     |
>>     |        dave at fly.srk.fer.hr
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Tue Jan 25 14:14:46 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 25 Jan 2011 14:14:46 +0100
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm>
Message-ID: <4D3ECCC6.5070705@hq.adiscon.com>

David,

I now integrated the patch. I think I will also move it into the main 
branch soon.

Rainer

On 01/17/2011 10:15 AM, david at lang.hm wrote:
> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>
>> Date: Mon, 17 Jan 2011 09:53:24 +0100
>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>
>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>>
>>>> I guess it makes sense when you send me the code and give the line
>>>> number where you have the issue. I can't promise, but I'll try to
>>>> give a good suggestion in the course of the day.
>>>>
>>>> One important thing you need to think about is that pMsg does NOT
>>>> necessarily use cstr objects to represent the entities. For example
>>>> (IIRC), raw message is kept in a different buffer, and you cannot
>>>> use cstr methods on that buffer. %msg% is actually not stored at
>>>> all, it is taken from rawmsg. There is a whole lot of this kind of
>>>> optimization done inside the message object. I guess this is what is
>>>> hitting you.
>>>
>>> Ok, that makes sense. so should I just insert a \0 at the appropriate
>>> place and decrament the iLenRawMsg and iLenMSG values?
>>
>> Tob e 100% sure, I'd need to check more thoroughly, but I am sure you
>> are on
>> the right path... But have a look at msg.c. Where the buffer is located
>> depends on the size of it (it is static inside the object up to a certain
>> size, and dyn alloced otherwise -- another optimization). The safe bet
>> is to
>> duplicate the message text and re-set it, but that is obviously much
>> slower
>> (to gain speed, you need to break layers...).
>
> Ok, This is now working, I think I found all the variables to change.
>
> the patch on top of v5-devel-david is
>
> diff --git a/plugins/pmcisconames/pmcisconames.c
> b/plugins/pmcisconames/pmcisconames.c
> index 28fe33d..7b76f65 100644
> --- a/plugins/pmcisconames/pmcisconames.c
> +++ b/plugins/pmcisconames/pmcisconames.c
> @@ -41,7 +41,7 @@
> #include "unicode-helper.h"
>
> MODULE_TYPE_PARSER
> -PARSER_NAME("rsyslog.lastline")
> +PARSER_NAME("rsyslog.cisconames")
>
> /* internal structures
> */
> @@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
> }
> /* bump the message portion up by two characters to overwrite the extra
> : */
> - memmove(p2parse, p2parse + 2, lenMsg - 2);
> + lenMsg -=2;
> + memmove(p2parse, p2parse + 2, lenMsg);
> + *(p2parse + lenMsg) = '\n';
> + *(p2parse + lenMsg + 1) = '\0';
> + pMsg->iLenRawMsg -=2;
> + pMsg->iLenMSG -=2;
> /* now, claim to abort so that something else can parse the now modified
> message */
> DBGPRINTF("pmcisconames detected a mangled Cisco log message message\n");
> + dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg, p2parse);
> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>
> finalize_it:
> @@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
> CHKiRet(objUse(parser, CORE_COMPONENT));
> CHKiRet(objUse(datetime, CORE_COMPONENT));
>
> - dbgprintf("lastmsg parser init called, compiled with version %s\n",
> VERSION);
> + dbgprintf("cisconames parser init called, compiled with version %s\n",
> VERSION);
> bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache value, is
> set only during rsyslogd option processing */
>
>
>
> David Lang
>
> P.S. I think having the file named pmlastmsg but the intermal module
> name being lastline is confusing, they should be related.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 25 14:37:00 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 25 Jan 2011 14:37:00 +0100
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
	<alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
	<alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>
Message-ID: <4D3ED1FC.2010305@hq.adiscon.com>

On 01/17/2011 07:22 PM, david at lang.hm wrote:
> Attached is a test file to test various modes of imfile, the results of
> the test, and a consolodated patch going back to
> f7c20920046ebcb94eadadf1ebad97b634a12a2d (your merge of the other imfile
> fixes)

I have merged this as well now. Please re-check, but I think it is OK. I 
will merge this into v5-devel tomorrow if I don't hear anything against 
that :)

>
> It turned out that there were a lot of problems in my code, so there's
> not much left of my initial patch. This is why I didn't just send in an
> update to the prior patch
>
> It would be good to have the config changes broken out as a separate
> patch (as much for future educational benifits as anything else). I will
> try and figure out how I can manipulate git to do this.
>
> One other problem I had is that I couldn't make the extra parameter work
> with ReadMultiLine, it would segfault every time, since nothing else
> uses ReadLine, I ended up adding the paramter to Readline and
> eliminating ReadMultiLine completely

I think this is a good solution. If we need to change that again, we can 
always do that.

> One thing that surprised me is that it doesn't appear that control
> characters in imfile are escaped the way that network received logs are
> escaped. Did I miss some way to enable this? Initially I thought that
> possibly only \n wasn't escaped, but some of my mistakes generated other
> control characters.

I guess this issue simply never came up. The way imfile works, it needs 
to do the sanitazion itself. Because that would happen in the parsing 
step, but imfile data can not be run through the parser.

Please also have a look at this forum post:
http://kb.monitorware.com/viewtopic.php?f=37&t=10622

Not sure how serious this approach is, but it sounds interesting.

Rainer
>
> David Lang
>
>
> On Thu, 13 Jan 2011, david at lang.hm wrote:
>
>> Date: Thu, 13 Jan 2011 02:02:51 -0800 (PST)
>> From: david at lang.hm
>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>> To: Rainer Gerhards <rgerhards at hq.adiscon.com>
>> Cc: rsyslog-users <rsyslog at lists.adiscon.com>
>> Subject: Re: [rsyslog] imfile paragraph patch
>>
>> thanks. I will try to go over both of these tomorrow, but will
>> definantly do so no later than this weekend.
>>
>> David Lang
>>
>> On Thu, 13 Jan 2011, Rainer Gerhards wrote:
>>
>>> Date: Thu, 13 Jan 2011 10:43:37 +0100
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> To: david at lang.hm, rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: Re: imfile paragraph patch
>>>
>>> I have now also created a new branch for this patch:
>>> v5-devel-david-imfile
>>>
>>> I added the config variable. See the commit log for useful
>>> information and steps. While I was a bit hesitant to merge this patch
>>> soon to the official branch (due to the problems I had with imfile),
>>> I begin to think this is over-cautious. It should really not harm any
>>> existing code. So please let me know when you have finished your
>>> testing of the new code, I'll probably merge soon then.
>>>
>>> Thanks!
>>> Rainer
>>>
>>> On 12/14/2010 04:57 AM, david at lang.hm wrote:
>>>> I discovered UnreadChar and so now mode 2 (indented follow-up lines)
>>>> has
>>>> a chance of working. again compile tested (and visually
>>>> code-reviewed by
>>>> someone else), but not executed.
>>>>
>>>> David Lang
>>>>
>>>> On Mon, 13 Dec 2010, david at lang.hm wrote:
>>>>
>>>>> This is a first cut of a modification to imfile to let it read
>>>>> multi-line files.
>>>>>
>>>>> As-is, this should have no effect on a system as it hard-codes the
>>>>> mode to reading single lines (I really don't understand how to set a
>>>>> config variable, but for someone who does, it should be simple to
>>>>> replace the '0' in imfile.c with the value of the config file)
>>>>>
>>>>> With this config option change, it should be possible to real logfiles
>>>>> that have blank lines between multi-line log entries and have those
>>>>> log entries treated as a single line.
>>>>>
>>>>> I also have code in place (but disabled) to try and deal with the more
>>>>> complicated layout where all lines after the first one are indented if
>>>>> they are part of the same log entry. The problem I have is that when I
>>>>> discover that I have finished reading a log entry I have already read
>>>>> the first character of the next log entry. This extra character needs
>>>>> to be put pack into the input buffer, but I don't know if that is
>>>>> possible or not. If this isn't the case, I need a function that will
>>>>> let me peek at the next character in the input buffer and make my
>>>>> decision based on that.
>>>>>
>>>>> This compiles, but I have not tested it anywhere yet. with the
>>>>> hardcoded mode 0 for ('LF termination), there should be no change
>>>>> other than an extra test against a constant for each character read
>>>>> from a file.
>>>>>
>>>>> David Lang
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>


From mmonitz at gmail.com  Tue Jan 25 16:18:28 2011
From: mmonitz at gmail.com (matan monitz)
Date: Tue, 25 Jan 2011 17:18:28 +0200
Subject: [rsyslog] rsyslog Text File Input Module losing seek?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD6D8@GRFEXC.intern.adiscon.com>
References: <AANLkTinOSgNDNAew1qxQot79rFexwb53W=-ppHWKvnK2@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com>
	<AANLkTin6Uqk2Cg3XyusLcRi3B8k34QEyudTO4LdyydR-@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com>
	<AANLkTin6qv2juvziBQfGgkP-c9_mpE4fpUR7Vi9T9Kw-@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com>
	<AANLkTimHcSVjaeJnyGsXvc9c1-LcPDn=kExh54xd5OA=@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD6D8@GRFEXC.intern.adiscon.com>
Message-ID: <AANLkTikXb2=PdZ0YSHanAf7-ZONUegq+R7h4SEcPNFC_@mail.gmail.com>

sorry for the bump
we had just upgraded to 5.6.2 and it is still happening
this is now happening almost hourly and not at the same time as file
rotation
we will try catching it while in debug mode tomorrow morning
any other ideas on the subject for now?

On Wed, Nov 10, 2010 at 7:47 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com>wrote:

> Could you run it in debug mode and provide me a log covering a situation
> where the problem occurs? I may be able to gain some information from that.
>
> Rainer
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > Sent: Wednesday, November 10, 2010 6:46 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> >
> > hello rainer
> > unfortunately we will not be able to apply a fix if provided,
> > if an upgrade will eventually happen it will likely be to Version 5.
> >
> > i have reviewed the references you provided and i don't believe they
> > are
> > relevant to our case
> > out files rotate at 10MB which is miles off from 2GB.
> > also, as i mentioned,i think this is not related to state files because
> > we
> > made sure they are being created and that rsyslog is not crashing
> > i agree there is not much point in providing a fix at this point,
> > however i
> > would like to hear any other guesses you may have regarding the cause
> > of the
> > situation so that i may figure out a way to work around it outside of
> > rsyslog
> > also i will be happy to provide technical data and logs as instructed
> > if you
> > need another test case
> >
> > thanks in advance
> >
> >
> >
> >
> > On Wed, Nov 10, 2010 at 11:39 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com>wrote:
> >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > Sent: Monday, November 08, 2010 7:08 PM
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> > > >
> > > > hi rainer
> > > > unfortunately this is not a server i'm directly in charge of so we
> > > > can't
> > > > really upgrade for while
> > > > (they will only upgrade as part of the planned upgrade for the
> > whole
> > > > server)
> > >
> > > Does that also mean you would not be able to apply a fix if I I
> > provide one
> > > for 3.22.2 (the current v3-stable)?
> > >
> > > > can you please elaborate on the cause of the problem so that maybe
> > i
> > > > can
> > > > figure out a temporary workaround?
> > >
> > > There definitely is a problem when the file size is larger than 2GB.
> > The
> > > rest
> > > may be related to dirty shutdown when the state file is deleted. See
> > >
> > >
> > >
> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536
> > b3dc4a
> > >
> > 28b94f80a361b41d<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=4
> > 0814bb0307f9d8536b3dc4a%0A28b94f80a361b41d>
> > >
> > >
> > >
> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2
> > 124ba7
> > >
> > d830652e9b1ead96<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9
> > 0933057bc2f014fd2124ba7%0Ad830652e9b1ead96>
> > >
> > > I have not yet receive feedback that makes me sure it's 100% fixed,
> > so I'd
> > > actually be quite interested in another test case. However, quite
> > honestly,
> > > if we can't run a better version on your box, there is no point for
> > me in
> > > further looking at that case (because it can't be solved anyway...).
> > >
> > > Rainer
> > >
> > > > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards
> > > > <rgerhards at hq.adiscon.com>wrote:
> > > >
> > > > > > -----Original Message-----
> > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > Sent: Monday, November 08, 2010 3:51 PM
> > > > > > To: rsyslog-users
> > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing
> > seek?
> > > > > >
> > > > > > hello rainer
> > > > > > we are using  3.22.1-3 from rpm on rhel 5
> > > > >
> > > > > That's definitely too old. I maintain it when I am at it, but the
> > fix
> > > > went
> > > > > into v4+ so far. I suggest that you pull the latest v4-stable
> > from
> > > > git. If
> > > > > git doesn't work for you, let me know and I'll mail you a private
> > > > tarball.
> > > > > I
> > > > > am just right now very busy (probably tomorrow as well, as I am
> > > > writing a
> > > > > lot
> > > > > of new code in support for CEE).
> > > > >
> > > > > Rainer
> > > > >
> > > > > >
> > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards
> > > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > > >
> > > > > > > which version do you use? I have recently addressed such a
> > > > problem...
> > > > > > >
> > > > > > > Rainer
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > > Sent: Monday, November 08, 2010 2:45 PM
> > > > > > > > To: rsyslog at lists.adiscon.com
> > > > > > > > Subject: [rsyslog] rsyslog Text File Input Module losing
> > seek?
> > > > > > > >
> > > > > > > > Hello all
> > > > > > > > we have a bind server writing and rotating logs on file
> > system
> > > > > > > > (rotating
> > > > > > > > when log file reaches 10Mb).
> > > > > > > > we also have rsyslog configured with the text file input
> > module
> > > > > > reading
> > > > > > > > the
> > > > > > > > file and sending it over UDP to another rsyslog server.
> > > > > > > > several times during the day the rsyslog on the bind
> > machine
> > > > starts
> > > > > > > > reading
> > > > > > > > the file from the top, causing a sudden high rate of
> > duplicate
> > > > and
> > > > > > old
> > > > > > > > events.
> > > > > > > > in an attempt to identify the problem:
> > > > > > > > 1.verified this using tcpdump to make sure the and saw the
> > old
> > > > logs
> > > > > > > > coming
> > > > > > > > down the wire to the rsyslog server
> > > > > > > > 2.made sure state files are correctly configured and are
> > being
> > > > > > created
> > > > > > > > when
> > > > > > > > needed
> > > > > > > > 3.made sure the rsyslog is not crashing and restarting by
> > > > > > monitoring
> > > > > > > > the pid
> > > > > > > > googeling hasn't helped as well
> > > > > > > >
> > > > > > > > any ideas?
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > >
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Tue Jan 25 18:28:28 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 25 Jan 2011 18:28:28 +0100
Subject: [rsyslog] rsyslog Text File Input Module losing seek?
References: <AANLkTinOSgNDNAew1qxQot79rFexwb53W=-ppHWKvnK2@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com><AANLkTin6Uqk2Cg3XyusLcRi3B8k34QEyudTO4LdyydR-@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com><AANLkTin6qv2juvziBQfGgkP-c9_mpE4fpUR7Vi9T9Kw-@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com><AANLkTimHcSVjaeJnyGsXvc9c1-LcPDn=kExh54xd5OA=@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD6D8@GRFEXC.intern.adiscon.com>
	<AANLkTikXb2=PdZ0YSHanAf7-ZONUegq+R7h4SEcPNFC_@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAA7@GRFEXC.intern.adiscon.com>

I guess I need to merge and release a new v5-stable. Fixes are already in
git, but as it seems I missed v5-stable (-devel and all others have it).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of matan monitz
> Sent: Tuesday, January 25, 2011 4:18 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> 
> sorry for the bump
> we had just upgraded to 5.6.2 and it is still happening
> this is now happening almost hourly and not at the same time as file
> rotation
> we will try catching it while in debug mode tomorrow morning
> any other ideas on the subject for now?
> 
> On Wed, Nov 10, 2010 at 7:47 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com>wrote:
> 
> > Could you run it in debug mode and provide me a log covering a
> situation
> > where the problem occurs? I may be able to gain some information from
> that.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > Sent: Wednesday, November 10, 2010 6:46 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> > >
> > > hello rainer
> > > unfortunately we will not be able to apply a fix if provided,
> > > if an upgrade will eventually happen it will likely be to Version
> 5.
> > >
> > > i have reviewed the references you provided and i don't believe
> they
> > > are
> > > relevant to our case
> > > out files rotate at 10MB which is miles off from 2GB.
> > > also, as i mentioned,i think this is not related to state files
> because
> > > we
> > > made sure they are being created and that rsyslog is not crashing
> > > i agree there is not much point in providing a fix at this point,
> > > however i
> > > would like to hear any other guesses you may have regarding the
> cause
> > > of the
> > > situation so that i may figure out a way to work around it outside
> of
> > > rsyslog
> > > also i will be happy to provide technical data and logs as
> instructed
> > > if you
> > > need another test case
> > >
> > > thanks in advance
> > >
> > >
> > >
> > >
> > > On Wed, Nov 10, 2010 at 11:39 AM, Rainer Gerhards
> > > <rgerhards at hq.adiscon.com>wrote:
> > >
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > Sent: Monday, November 08, 2010 7:08 PM
> > > > > To: rsyslog-users
> > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing
> seek?
> > > > >
> > > > > hi rainer
> > > > > unfortunately this is not a server i'm directly in charge of so
> we
> > > > > can't
> > > > > really upgrade for while
> > > > > (they will only upgrade as part of the planned upgrade for the
> > > whole
> > > > > server)
> > > >
> > > > Does that also mean you would not be able to apply a fix if I I
> > > provide one
> > > > for 3.22.2 (the current v3-stable)?
> > > >
> > > > > can you please elaborate on the cause of the problem so that
> maybe
> > > i
> > > > > can
> > > > > figure out a temporary workaround?
> > > >
> > > > There definitely is a problem when the file size is larger than
> 2GB.
> > > The
> > > > rest
> > > > may be related to dirty shutdown when the state file is deleted.
> See
> > > >
> > > >
> > > >
> > >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536
> > > b3dc4a
> > > >
> > >
> 28b94f80a361b41d<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=4
> > > 0814bb0307f9d8536b3dc4a%0A28b94f80a361b41d>
> > > >
> > > >
> > > >
> > >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2
> > > 124ba7
> > > >
> > >
> d830652e9b1ead96<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9
> > > 0933057bc2f014fd2124ba7%0Ad830652e9b1ead96>
> > > >
> > > > I have not yet receive feedback that makes me sure it's 100%
> fixed,
> > > so I'd
> > > > actually be quite interested in another test case. However, quite
> > > honestly,
> > > > if we can't run a better version on your box, there is no point
> for
> > > me in
> > > > further looking at that case (because it can't be solved
> anyway...).
> > > >
> > > > Rainer
> > > >
> > > > > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards
> > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > Sent: Monday, November 08, 2010 3:51 PM
> > > > > > > To: rsyslog-users
> > > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module
> losing
> > > seek?
> > > > > > >
> > > > > > > hello rainer
> > > > > > > we are using  3.22.1-3 from rpm on rhel 5
> > > > > >
> > > > > > That's definitely too old. I maintain it when I am at it, but
> the
> > > fix
> > > > > went
> > > > > > into v4+ so far. I suggest that you pull the latest v4-stable
> > > from
> > > > > git. If
> > > > > > git doesn't work for you, let me know and I'll mail you a
> private
> > > > > tarball.
> > > > > > I
> > > > > > am just right now very busy (probably tomorrow as well, as I
> am
> > > > > writing a
> > > > > > lot
> > > > > > of new code in support for CEE).
> > > > > >
> > > > > > Rainer
> > > > > >
> > > > > > >
> > > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards
> > > > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > > > >
> > > > > > > > which version do you use? I have recently addressed such
> a
> > > > > problem...
> > > > > > > >
> > > > > > > > Rainer
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > > > Sent: Monday, November 08, 2010 2:45 PM
> > > > > > > > > To: rsyslog at lists.adiscon.com
> > > > > > > > > Subject: [rsyslog] rsyslog Text File Input Module
> losing
> > > seek?
> > > > > > > > >
> > > > > > > > > Hello all
> > > > > > > > > we have a bind server writing and rotating logs on file
> > > system
> > > > > > > > > (rotating
> > > > > > > > > when log file reaches 10Mb).
> > > > > > > > > we also have rsyslog configured with the text file
> input
> > > module
> > > > > > > reading
> > > > > > > > > the
> > > > > > > > > file and sending it over UDP to another rsyslog server.
> > > > > > > > > several times during the day the rsyslog on the bind
> > > machine
> > > > > starts
> > > > > > > > > reading
> > > > > > > > > the file from the top, causing a sudden high rate of
> > > duplicate
> > > > > and
> > > > > > > old
> > > > > > > > > events.
> > > > > > > > > in an attempt to identify the problem:
> > > > > > > > > 1.verified this using tcpdump to make sure the and saw
> the
> > > old
> > > > > logs
> > > > > > > > > coming
> > > > > > > > > down the wire to the rsyslog server
> > > > > > > > > 2.made sure state files are correctly configured and
> are
> > > being
> > > > > > > created
> > > > > > > > > when
> > > > > > > > > needed
> > > > > > > > > 3.made sure the rsyslog is not crashing and restarting
> by
> > > > > > > monitoring
> > > > > > > > > the pid
> > > > > > > > > googeling hasn't helped as well
> > > > > > > > >
> > > > > > > > > any ideas?
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > >
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Tue Jan 25 20:28:23 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 25 Jan 2011 11:28:23 -0800 (PST)
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <4D3ED1FC.2010305@hq.adiscon.com>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
	<alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
	<alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>
	<4D3ED1FC.2010305@hq.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101251126280.17579@asgard.lang.hm>

On Tue, 25 Jan 2011, Rainer Gerhards wrote:

> On 01/17/2011 07:22 PM, david at lang.hm wrote:
>> Attached is a test file to test various modes of imfile, the results of
>> the test, and a consolodated patch going back to
>> f7c20920046ebcb94eadadf1ebad97b634a12a2d (your merge of the other imfile
>> fixes)
>
> I have merged this as well now. Please re-check, but I think it is OK. I will 
> merge this into v5-devel tomorrow if I don't hear anything against that :)

I will test this today. did you include the test file and make a unit test 
for this?

>> One thing that surprised me is that it doesn't appear that control
>> characters in imfile are escaped the way that network received logs are
>> escaped. Did I miss some way to enable this? Initially I thought that
>> possibly only \n wasn't escaped, but some of my mistakes generated other
>> control characters.
>
> I guess this issue simply never came up. The way imfile works, it needs to do 
> the sanitazion itself. Because that would happen in the parsing step, but 
> imfile data can not be run through the parser.

Ok, I'll do some digging and find the routine that's called to do the 
sanitizing and see if we can just add a call to the existing code.

> Please also have a look at this forum post:
> http://kb.monitorware.com/viewtopic.php?f=37&t=10622
>
> Not sure how serious this approach is, but it sounds interesting.

I will look at this this afternoon

David Lang

> Rainer
>> 
>> David Lang
>> 
>> 
>> On Thu, 13 Jan 2011, david at lang.hm wrote:
>> 
>>> Date: Thu, 13 Jan 2011 02:02:51 -0800 (PST)
>>> From: david at lang.hm
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Cc: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: Re: [rsyslog] imfile paragraph patch
>>> 
>>> thanks. I will try to go over both of these tomorrow, but will
>>> definantly do so no later than this weekend.
>>> 
>>> David Lang
>>> 
>>> On Thu, 13 Jan 2011, Rainer Gerhards wrote:
>>> 
>>>> Date: Thu, 13 Jan 2011 10:43:37 +0100
>>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>>> To: david at lang.hm, rsyslog-users <rsyslog at lists.adiscon.com>
>>>> Subject: Re: imfile paragraph patch
>>>> 
>>>> I have now also created a new branch for this patch:
>>>> v5-devel-david-imfile
>>>> 
>>>> I added the config variable. See the commit log for useful
>>>> information and steps. While I was a bit hesitant to merge this patch
>>>> soon to the official branch (due to the problems I had with imfile),
>>>> I begin to think this is over-cautious. It should really not harm any
>>>> existing code. So please let me know when you have finished your
>>>> testing of the new code, I'll probably merge soon then.
>>>> 
>>>> Thanks!
>>>> Rainer
>>>> 
>>>> On 12/14/2010 04:57 AM, david at lang.hm wrote:
>>>>> I discovered UnreadChar and so now mode 2 (indented follow-up lines)
>>>>> has
>>>>> a chance of working. again compile tested (and visually
>>>>> code-reviewed by
>>>>> someone else), but not executed.
>>>>> 
>>>>> David Lang
>>>>> 
>>>>> On Mon, 13 Dec 2010, david at lang.hm wrote:
>>>>> 
>>>>>> This is a first cut of a modification to imfile to let it read
>>>>>> multi-line files.
>>>>>> 
>>>>>> As-is, this should have no effect on a system as it hard-codes the
>>>>>> mode to reading single lines (I really don't understand how to set a
>>>>>> config variable, but for someone who does, it should be simple to
>>>>>> replace the '0' in imfile.c with the value of the config file)
>>>>>> 
>>>>>> With this config option change, it should be possible to real logfiles
>>>>>> that have blank lines between multi-line log entries and have those
>>>>>> log entries treated as a single line.
>>>>>> 
>>>>>> I also have code in place (but disabled) to try and deal with the more
>>>>>> complicated layout where all lines after the first one are indented if
>>>>>> they are part of the same log entry. The problem I have is that when I
>>>>>> discover that I have finished reading a log entry I have already read
>>>>>> the first character of the next log entry. This extra character needs
>>>>>> to be put pack into the input buffer, but I don't know if that is
>>>>>> possible or not. If this isn't the case, I need a function that will
>>>>>> let me peek at the next character in the input buffer and make my
>>>>>> decision based on that.
>>>>>> 
>>>>>> This compiles, but I have not tested it anywhere yet. with the
>>>>>> hardcoded mode 0 for ('LF termination), there should be no change
>>>>>> other than an extra test against a constant for each character read
>>>>>> from a file.
>>>>>> 
>>>>>> David Lang
>>>> 
>>>> 
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>> 
>
>

From david at lang.hm  Tue Jan 25 20:29:27 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 25 Jan 2011 11:29:27 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <4D3ECCC6.5070705@hq.adiscon.com>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm>
	<4D3ECCC6.5070705@hq.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101251129000.17579@asgard.lang.hm>

Thanks for adding these in, I need to compile a new build to roll out to 
production this week and it's nice to have these upstream.

David Lang

On Tue, 25 Jan 2011, Rainer Gerhards wrote:

> Date: Tue, 25 Jan 2011 14:14:46 +0100
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: [rsyslog] message manipulation in a parser
> 
> David,
>
> I now integrated the patch. I think I will also move it into the main branch 
> soon.
>
> Rainer
>
> On 01/17/2011 10:15 AM, david at lang.hm wrote:
>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>> 
>>> Date: Mon, 17 Jan 2011 09:53:24 +0100
>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>> 
>>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>>> 
>>>>> I guess it makes sense when you send me the code and give the line
>>>>> number where you have the issue. I can't promise, but I'll try to
>>>>> give a good suggestion in the course of the day.
>>>>> 
>>>>> One important thing you need to think about is that pMsg does NOT
>>>>> necessarily use cstr objects to represent the entities. For example
>>>>> (IIRC), raw message is kept in a different buffer, and you cannot
>>>>> use cstr methods on that buffer. %msg% is actually not stored at
>>>>> all, it is taken from rawmsg. There is a whole lot of this kind of
>>>>> optimization done inside the message object. I guess this is what is
>>>>> hitting you.
>>>> 
>>>> Ok, that makes sense. so should I just insert a \0 at the appropriate
>>>> place and decrament the iLenRawMsg and iLenMSG values?
>>> 
>>> Tob e 100% sure, I'd need to check more thoroughly, but I am sure you
>>> are on
>>> the right path... But have a look at msg.c. Where the buffer is located
>>> depends on the size of it (it is static inside the object up to a certain
>>> size, and dyn alloced otherwise -- another optimization). The safe bet
>>> is to
>>> duplicate the message text and re-set it, but that is obviously much
>>> slower
>>> (to gain speed, you need to break layers...).
>> 
>> Ok, This is now working, I think I found all the variables to change.
>> 
>> the patch on top of v5-devel-david is
>> 
>> diff --git a/plugins/pmcisconames/pmcisconames.c
>> b/plugins/pmcisconames/pmcisconames.c
>> index 28fe33d..7b76f65 100644
>> --- a/plugins/pmcisconames/pmcisconames.c
>> +++ b/plugins/pmcisconames/pmcisconames.c
>> @@ -41,7 +41,7 @@
>> #include "unicode-helper.h"
>> 
>> MODULE_TYPE_PARSER
>> -PARSER_NAME("rsyslog.lastline")
>> +PARSER_NAME("rsyslog.cisconames")
>> 
>> /* internal structures
>> */
>> @@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>> }
>> /* bump the message portion up by two characters to overwrite the extra
>> : */
>> - memmove(p2parse, p2parse + 2, lenMsg - 2);
>> + lenMsg -=2;
>> + memmove(p2parse, p2parse + 2, lenMsg);
>> + *(p2parse + lenMsg) = '\n';
>> + *(p2parse + lenMsg + 1) = '\0';
>> + pMsg->iLenRawMsg -=2;
>> + pMsg->iLenMSG -=2;
>> /* now, claim to abort so that something else can parse the now modified
>> message */
>> DBGPRINTF("pmcisconames detected a mangled Cisco log message message\n");
>> + dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg, p2parse);
>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>> 
>> finalize_it:
>> @@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
>> CHKiRet(objUse(parser, CORE_COMPONENT));
>> CHKiRet(objUse(datetime, CORE_COMPONENT));
>> 
>> - dbgprintf("lastmsg parser init called, compiled with version %s\n",
>> VERSION);
>> + dbgprintf("cisconames parser init called, compiled with version %s\n",
>> VERSION);
>> bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache value, is
>> set only during rsyslogd option processing */
>> 
>> 
>> 
>> David Lang
>> 
>> P.S. I think having the file named pmlastmsg but the intermal module
>> name being lastline is confusing, they should be related.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Wed Jan 26 07:23:43 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Jan 2011 07:23:43 +0100
Subject: [rsyslog] imfile paragraph patch
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
	<alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
	<alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>
	<4D3ED1FC.2010305@hq.adiscon.com>
	<alpine.DEB.2.00.1101251126280.17579@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAA8@GRFEXC.intern.adiscon.com>



> -----Original Message-----
> From: david at lang.hm [mailto:david at lang.hm]
> Sent: Tuesday, January 25, 2011 8:28 PM
> To: Rainer Gerhards
> Cc: rsyslog-users
> Subject: Re: [rsyslog] imfile paragraph patch
> 
> On Tue, 25 Jan 2011, Rainer Gerhards wrote:
> 
> > On 01/17/2011 07:22 PM, david at lang.hm wrote:
> >> Attached is a test file to test various modes of imfile, the results
> of
> >> the test, and a consolodated patch going back to
> >> f7c20920046ebcb94eadadf1ebad97b634a12a2d (your merge of the other
> imfile
> >> fixes)
> >
> > I have merged this as well now. Please re-check, but I think it is
> OK. I will
> > merge this into v5-devel tomorrow if I don't hear anything against
> that :)
> 
> I will test this today. did you include the test file and make a unit
> test
> for this?

I have to admit I did not get that you wanted me to do that ;) Will see that
I add a testcase later today.

> 
> >> One thing that surprised me is that it doesn't appear that control
> >> characters in imfile are escaped the way that network received logs
> are
> >> escaped. Did I miss some way to enable this? Initially I thought
> that
> >> possibly only \n wasn't escaped, but some of my mistakes generated
> other
> >> control characters.
> >
> > I guess this issue simply never came up. The way imfile works, it
> needs to do
> > the sanitazion itself. Because that would happen in the parsing step,
> but
> > imfile data can not be run through the parser.
> 
> Ok, I'll do some digging and find the routine that's called to do the
> sanitizing and see if we can just add a call to the existing code.

I probably makes sense to expose this function as a callable API (especially
as it needs some update to be more Unicode-friendly). Will put this on the
list as well.

Side-Note: I will also see that I merge some fixes from older releases in
today. This can be a bit more time-consuming due to interface changes. 

Rainer



From david at lang.hm  Wed Jan 26 08:01:56 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 25 Jan 2011 23:01:56 -0800 (PST)
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDAA8@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
	<alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
	<alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>
	<4D3ED1FC.2010305@hq.adiscon.com>
	<alpine.DEB.2.00.1101251126280.17579@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAA8@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101252259020.29659@asgard.lang.hm>

On Wed, 26 Jan 2011, Rainer Gerhards wrote:

>> -----Original Message-----
>> From: david at lang.hm [mailto:david at lang.hm]
>>
>> On Tue, 25 Jan 2011, Rainer Gerhards wrote:
>>
>>> On 01/17/2011 07:22 PM, david at lang.hm wrote:
>>>> Attached is a test file to test various modes of imfile, the results
>> of
>>>> the test, and a consolodated patch going back to
>>>> f7c20920046ebcb94eadadf1ebad97b634a12a2d (your merge of the other
>> imfile
>>>> fixes)
>>>
>>> I have merged this as well now. Please re-check, but I think it is
>> OK. I will
>>> merge this into v5-devel tomorrow if I don't hear anything against
>> that :)
>>
>> I will test this today. did you include the test file and make a unit
>> test
>> for this?
>
> I have to admit I did not get that you wanted me to do that ;) Will see that
> I add a testcase later today.

no problem, I never said anything explicitly, but I had sent you a test 
file and the expected output. Since you are so aggressive about having 
test cases for everything, I assumed that you would want one for this as 
well.

I just compiled it and will test it in the next few min.

>>
>>>> One thing that surprised me is that it doesn't appear that control
>>>> characters in imfile are escaped the way that network received logs
>> are
>>>> escaped. Did I miss some way to enable this? Initially I thought
>> that
>>>> possibly only \n wasn't escaped, but some of my mistakes generated
>> other
>>>> control characters.
>>>
>>> I guess this issue simply never came up. The way imfile works, it
>> needs to do
>>> the sanitazion itself. Because that would happen in the parsing step,
>> but
>>> imfile data can not be run through the parser.
>>
>> Ok, I'll do some digging and find the routine that's called to do the
>> sanitizing and see if we can just add a call to the existing code.
>
> I probably makes sense to expose this function as a callable API (especially
> as it needs some update to be more Unicode-friendly). Will put this on the
> list as well.

Ok, in that case I will wait on investigating this

> Side-Note: I will also see that I merge some fixes from older releases in
> today. This can be a bit more time-consuming due to interface changes.

sounds good. I will probably generate a AIX fixup parser today or tomorrow 
as well (very similar to the cisco one, it detects and removes a chunk of 
text from the log before 'failing' through to the default parser)

David Lang

From david at lang.hm  Wed Jan 26 08:08:13 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 25 Jan 2011 23:08:13 -0800 (PST)
Subject: [rsyslog] imfile paragraph patch
In-Reply-To: <alpine.DEB.2.00.1101252259020.29659@asgard.lang.hm>
References: <alpine.DEB.2.00.1012131921500.19517@asgard.lang.hm>
	<alpine.DEB.2.00.1012131952130.19517@asgard.lang.hm>
	<4D2EC949.4060109@hq.adiscon.com>
	<alpine.DEB.2.00.1101130202170.21529@asgard.lang.hm>
	<alpine.DEB.2.00.1101171010520.9746@asgard.lang.hm>
	<4D3ED1FC.2010305@hq.adiscon.com>
	<alpine.DEB.2.00.1101251126280.17579@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAA8@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101252259020.29659@asgard.lang.hm>
Message-ID: <alpine.DEB.2.00.1101252307160.29659@asgard.lang.hm>

On Tue, 25 Jan 2011, david at lang.hm wrote:

> On Wed, 26 Jan 2011, Rainer Gerhards wrote:
>
>>> -----Original Message-----
>>> From: david at lang.hm [mailto:david at lang.hm]
>>> 
>>> On Tue, 25 Jan 2011, Rainer Gerhards wrote:
>>> 
>>>> On 01/17/2011 07:22 PM, david at lang.hm wrote:
>>>>> Attached is a test file to test various modes of imfile, the results
>>> of
>>>>> the test, and a consolodated patch going back to
>>>>> f7c20920046ebcb94eadadf1ebad97b634a12a2d (your merge of the other
>>> imfile
>>>>> fixes)
>>>> 
>>>> I have merged this as well now. Please re-check, but I think it is
>>> OK. I will
>>>> merge this into v5-devel tomorrow if I don't hear anything against
>>> that :)
>>> 
>>> I will test this today. did you include the test file and make a unit
>>> test
>>> for this?
>> 
>> I have to admit I did not get that you wanted me to do that ;) Will see 
>> that
>> I add a testcase later today.
>
> no problem, I never said anything explicitly, but I had sent you a test file 
> and the expected output. Since you are so aggressive about having test cases 
> for everything, I assumed that you would want one for this as well.
>
> I just compiled it and will test it in the next few min.

testing confirmed, I tested with all three options, plus with the option 
commented out to get the default.

David Lang

>>> 
>>>>> One thing that surprised me is that it doesn't appear that control
>>>>> characters in imfile are escaped the way that network received logs
>>> are
>>>>> escaped. Did I miss some way to enable this? Initially I thought
>>> that
>>>>> possibly only \n wasn't escaped, but some of my mistakes generated
>>> other
>>>>> control characters.
>>>> 
>>>> I guess this issue simply never came up. The way imfile works, it
>>> needs to do
>>>> the sanitazion itself. Because that would happen in the parsing step,
>>> but
>>>> imfile data can not be run through the parser.
>>> 
>>> Ok, I'll do some digging and find the routine that's called to do the
>>> sanitizing and see if we can just add a call to the existing code.
>> 
>> I probably makes sense to expose this function as a callable API 
>> (especially
>> as it needs some update to be more Unicode-friendly). Will put this on the
>> list as well.
>
> Ok, in that case I will wait on investigating this
>
>> Side-Note: I will also see that I merge some fixes from older releases in
>> today. This can be a bit more time-consuming due to interface changes.
>
> sounds good. I will probably generate a AIX fixup parser today or tomorrow as 
> well (very similar to the cisco one, it detects and removes a chunk of text 
> from the log before 'failing' through to the default parser)
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From david at lang.hm  Wed Jan 26 10:03:41 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 26 Jan 2011 01:03:41 -0800 (PST)
Subject: [rsyslog] pmaixforwardedfrom parser
Message-ID: <alpine.DEB.2.00.1101260058480.29659@asgard.lang.hm>

this is another cleanup parser to fix logs sent by broken clients

AIX by default sends messages like this:

Jan 25 23:09:48 Message forwarded from w50: syslog: /usr/sbin/ifconfig -a

this parser takes these messages, and messages like this:

Jan 25 23:09:48 Message forwarded from w50 syslog: /usr/sbin/ifconfig -a

and converts them to:

Jan 25 23:09:48 w50 syslog: /usr/sbin/ifconfig -a

this allows the normal parser to handle them correctly.

I'm not absolutly sure that the second form of message is sent by 
anything, but it was only a couple lines to handle this, and it was easier 
to handle this than to run the risk that this would hit and I would never 
find the ':' that I was looking for after the hostname (and also far 
easier than detecting that the ':' after the hostname was missing and not 
remove the 'Message forwarded from ' text)

David Lang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pmaixforwardedfrom.patch
Type: text/x-diff
Size: 8210 bytes
Desc: 
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110126/0165dc44/attachment.patch>

From david at lang.hm  Wed Jan 26 10:05:44 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 26 Jan 2011 01:05:44 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <4D3ECCC6.5070705@hq.adiscon.com>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm>
	<4D3ECCC6.5070705@hq.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101260105130.29659@asgard.lang.hm>

I tested the code in your branch and verified that it works.

David Lang

On Tue, 25 Jan 2011, Rainer Gerhards wrote:

> David,
>
> I now integrated the patch. I think I will also move it into the main branch 
> soon.
>
> Rainer
>
> On 01/17/2011 10:15 AM, david at lang.hm wrote:
>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>> 
>>> Date: Mon, 17 Jan 2011 09:53:24 +0100
>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>> 
>>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>>> 
>>>>> I guess it makes sense when you send me the code and give the line
>>>>> number where you have the issue. I can't promise, but I'll try to
>>>>> give a good suggestion in the course of the day.
>>>>> 
>>>>> One important thing you need to think about is that pMsg does NOT
>>>>> necessarily use cstr objects to represent the entities. For example
>>>>> (IIRC), raw message is kept in a different buffer, and you cannot
>>>>> use cstr methods on that buffer. %msg% is actually not stored at
>>>>> all, it is taken from rawmsg. There is a whole lot of this kind of
>>>>> optimization done inside the message object. I guess this is what is
>>>>> hitting you.
>>>> 
>>>> Ok, that makes sense. so should I just insert a \0 at the appropriate
>>>> place and decrament the iLenRawMsg and iLenMSG values?
>>> 
>>> Tob e 100% sure, I'd need to check more thoroughly, but I am sure you
>>> are on
>>> the right path... But have a look at msg.c. Where the buffer is located
>>> depends on the size of it (it is static inside the object up to a certain
>>> size, and dyn alloced otherwise -- another optimization). The safe bet
>>> is to
>>> duplicate the message text and re-set it, but that is obviously much
>>> slower
>>> (to gain speed, you need to break layers...).
>> 
>> Ok, This is now working, I think I found all the variables to change.
>> 
>> the patch on top of v5-devel-david is
>> 
>> diff --git a/plugins/pmcisconames/pmcisconames.c
>> b/plugins/pmcisconames/pmcisconames.c
>> index 28fe33d..7b76f65 100644
>> --- a/plugins/pmcisconames/pmcisconames.c
>> +++ b/plugins/pmcisconames/pmcisconames.c
>> @@ -41,7 +41,7 @@
>> #include "unicode-helper.h"
>> 
>> MODULE_TYPE_PARSER
>> -PARSER_NAME("rsyslog.lastline")
>> +PARSER_NAME("rsyslog.cisconames")
>> 
>> /* internal structures
>> */
>> @@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>> }
>> /* bump the message portion up by two characters to overwrite the extra
>> : */
>> - memmove(p2parse, p2parse + 2, lenMsg - 2);
>> + lenMsg -=2;
>> + memmove(p2parse, p2parse + 2, lenMsg);
>> + *(p2parse + lenMsg) = '\n';
>> + *(p2parse + lenMsg + 1) = '\0';
>> + pMsg->iLenRawMsg -=2;
>> + pMsg->iLenMSG -=2;
>> /* now, claim to abort so that something else can parse the now modified
>> message */
>> DBGPRINTF("pmcisconames detected a mangled Cisco log message message\n");
>> + dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg, p2parse);
>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>> 
>> finalize_it:
>> @@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
>> CHKiRet(objUse(parser, CORE_COMPONENT));
>> CHKiRet(objUse(datetime, CORE_COMPONENT));
>> 
>> - dbgprintf("lastmsg parser init called, compiled with version %s\n",
>> VERSION);
>> + dbgprintf("cisconames parser init called, compiled with version %s\n",
>> VERSION);
>> bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache value, is
>> set only during rsyslogd option processing */
>> 
>> 
>> 
>> David Lang
>> 
>> P.S. I think having the file named pmlastmsg but the intermal module
>> name being lastline is confusing, they should be related.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Wed Jan 26 10:25:13 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Jan 2011 10:25:13 +0100
Subject: [rsyslog] message manipulation in a parser
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm><4D3ECCC6.5070705@hq.adiscon.com>
	<alpine.DEB.2.00.1101260105130.29659@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAAA@GRFEXC.intern.adiscon.com>

Excellent, thx. I'll see that I merge both into the main branch today. Right
now my main machine is running a performance benchmark, so I need to do this
later today.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 26, 2011 10:06 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] message manipulation in a parser
> 
> I tested the code in your branch and verified that it works.
> 
> David Lang
> 
> On Tue, 25 Jan 2011, Rainer Gerhards wrote:
> 
> > David,
> >
> > I now integrated the patch. I think I will also move it into the main
> branch
> > soon.
> >
> > Rainer
> >
> > On 01/17/2011 10:15 AM, david at lang.hm wrote:
> >> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> >>
> >>> Date: Mon, 17 Jan 2011 09:53:24 +0100
> >>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >>>>
> >>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> >>>>
> >>>>> I guess it makes sense when you send me the code and give the
> line
> >>>>> number where you have the issue. I can't promise, but I'll try to
> >>>>> give a good suggestion in the course of the day.
> >>>>>
> >>>>> One important thing you need to think about is that pMsg does NOT
> >>>>> necessarily use cstr objects to represent the entities. For
> example
> >>>>> (IIRC), raw message is kept in a different buffer, and you cannot
> >>>>> use cstr methods on that buffer. %msg% is actually not stored at
> >>>>> all, it is taken from rawmsg. There is a whole lot of this kind
> of
> >>>>> optimization done inside the message object. I guess this is what
> is
> >>>>> hitting you.
> >>>>
> >>>> Ok, that makes sense. so should I just insert a \0 at the
> appropriate
> >>>> place and decrament the iLenRawMsg and iLenMSG values?
> >>>
> >>> Tob e 100% sure, I'd need to check more thoroughly, but I am sure
> you
> >>> are on
> >>> the right path... But have a look at msg.c. Where the buffer is
> located
> >>> depends on the size of it (it is static inside the object up to a
> certain
> >>> size, and dyn alloced otherwise -- another optimization). The safe
> bet
> >>> is to
> >>> duplicate the message text and re-set it, but that is obviously
> much
> >>> slower
> >>> (to gain speed, you need to break layers...).
> >>
> >> Ok, This is now working, I think I found all the variables to
> change.
> >>
> >> the patch on top of v5-devel-david is
> >>
> >> diff --git a/plugins/pmcisconames/pmcisconames.c
> >> b/plugins/pmcisconames/pmcisconames.c
> >> index 28fe33d..7b76f65 100644
> >> --- a/plugins/pmcisconames/pmcisconames.c
> >> +++ b/plugins/pmcisconames/pmcisconames.c
> >> @@ -41,7 +41,7 @@
> >> #include "unicode-helper.h"
> >>
> >> MODULE_TYPE_PARSER
> >> -PARSER_NAME("rsyslog.lastline")
> >> +PARSER_NAME("rsyslog.cisconames")
> >>
> >> /* internal structures
> >> */
> >> @@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
> >> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
> >> }
> >> /* bump the message portion up by two characters to overwrite the
> extra
> >> : */
> >> - memmove(p2parse, p2parse + 2, lenMsg - 2);
> >> + lenMsg -=2;
> >> + memmove(p2parse, p2parse + 2, lenMsg);
> >> + *(p2parse + lenMsg) = '\n';
> >> + *(p2parse + lenMsg + 1) = '\0';
> >> + pMsg->iLenRawMsg -=2;
> >> + pMsg->iLenMSG -=2;
> >> /* now, claim to abort so that something else can parse the now
> modified
> >> message */
> >> DBGPRINTF("pmcisconames detected a mangled Cisco log message
> message\n");
> >> + dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg,
> p2parse);
> >> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
> >>
> >> finalize_it:
> >> @@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
> >> CHKiRet(objUse(parser, CORE_COMPONENT));
> >> CHKiRet(objUse(datetime, CORE_COMPONENT));
> >>
> >> - dbgprintf("lastmsg parser init called, compiled with version
> %s\n",
> >> VERSION);
> >> + dbgprintf("cisconames parser init called, compiled with version
> %s\n",
> >> VERSION);
> >> bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache
> value, is
> >> set only during rsyslogd option processing */
> >>
> >>
> >>
> >> David Lang
> >>
> >> P.S. I think having the file named pmlastmsg but the intermal module
> >> name being lastline is confusing, they should be related.
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Wed Jan 26 10:38:54 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 26 Jan 2011 01:38:54 -0800 (PST)
Subject: [rsyslog] message manipulation in a parser
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDAAA@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm><4D3ECCC6.5070705@hq.adiscon.com>
	<alpine.DEB.2.00.1101260105130.29659@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAAA@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1101260130131.29659@asgard.lang.hm>

the message parser is turning out to be extremely easy to use in this 
mode.

I was able to re-write the cisconames fixup into the aixforwardedfrom 
fixup in about 2 hours, and the completed patch (including tieing it in to 
the build system) is only 288 lines.

It may very well be worth taking a bit of time to look at the default 
rfc3164 parser and see how much it could be simplified by pulling some of 
the odd corner cases out to fixup parsers like this.

it's not worth doing for flaws that show up frequently (having the code in 
the rfx3164 parser is going to be faster than going in to a separate 
parser, not to mention the config complexity), but some of the more 
strange cases may be worth pulling out, not as much for performance 
(although it will probably help a smidge there), but more so in terms of 
simplifying the code and documenting what generates strange messages.

Also, I saw your tweet about imttcp and was wondering, with a small number 
of threads, was it enough faster that it could be useful for the second 
tier of a multi-tier logging infrastructure, where you have your first 
tier collecting from a lot of machines to a small number of machines, and 
then this first tier is sending the logs (at a fairly high volume per 
connection) to the next tier up? this is the opposite of the case you 
were testing (which as I understand it was lots of source machines, each 
with a fairly low log volume)

it may not be worth the overhead of maintaining a separate input module, 
but I thought I'd mention the possibility.

David Lang


  On Wed, 26 Jan 2011, Rainer Gerhards 
wrote:

> Excellent, thx. I'll see that I merge both into the main branch today. Right
> now my main machine is running a performance benchmark, so I need to do this
> later today.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Wednesday, January 26, 2011 10:06 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] message manipulation in a parser
>>
>> I tested the code in your branch and verified that it works.
>>
>> David Lang
>>
>> On Tue, 25 Jan 2011, Rainer Gerhards wrote:
>>
>>> David,
>>>
>>> I now integrated the patch. I think I will also move it into the main
>> branch
>>> soon.
>>>
>>> Rainer
>>>
>>> On 01/17/2011 10:15 AM, david at lang.hm wrote:
>>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>>>
>>>>> Date: Mon, 17 Jan 2011 09:53:24 +0100
>>>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>>>>
>>>>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
>>>>>>
>>>>>>> I guess it makes sense when you send me the code and give the
>> line
>>>>>>> number where you have the issue. I can't promise, but I'll try to
>>>>>>> give a good suggestion in the course of the day.
>>>>>>>
>>>>>>> One important thing you need to think about is that pMsg does NOT
>>>>>>> necessarily use cstr objects to represent the entities. For
>> example
>>>>>>> (IIRC), raw message is kept in a different buffer, and you cannot
>>>>>>> use cstr methods on that buffer. %msg% is actually not stored at
>>>>>>> all, it is taken from rawmsg. There is a whole lot of this kind
>> of
>>>>>>> optimization done inside the message object. I guess this is what
>> is
>>>>>>> hitting you.
>>>>>>
>>>>>> Ok, that makes sense. so should I just insert a \0 at the
>> appropriate
>>>>>> place and decrament the iLenRawMsg and iLenMSG values?
>>>>>
>>>>> Tob e 100% sure, I'd need to check more thoroughly, but I am sure
>> you
>>>>> are on
>>>>> the right path... But have a look at msg.c. Where the buffer is
>> located
>>>>> depends on the size of it (it is static inside the object up to a
>> certain
>>>>> size, and dyn alloced otherwise -- another optimization). The safe
>> bet
>>>>> is to
>>>>> duplicate the message text and re-set it, but that is obviously
>> much
>>>>> slower
>>>>> (to gain speed, you need to break layers...).
>>>>
>>>> Ok, This is now working, I think I found all the variables to
>> change.
>>>>
>>>> the patch on top of v5-devel-david is
>>>>
>>>> diff --git a/plugins/pmcisconames/pmcisconames.c
>>>> b/plugins/pmcisconames/pmcisconames.c
>>>> index 28fe33d..7b76f65 100644
>>>> --- a/plugins/pmcisconames/pmcisconames.c
>>>> +++ b/plugins/pmcisconames/pmcisconames.c
>>>> @@ -41,7 +41,7 @@
>>>> #include "unicode-helper.h"
>>>>
>>>> MODULE_TYPE_PARSER
>>>> -PARSER_NAME("rsyslog.lastline")
>>>> +PARSER_NAME("rsyslog.cisconames")
>>>>
>>>> /* internal structures
>>>> */
>>>> @@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
>>>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>>>> }
>>>> /* bump the message portion up by two characters to overwrite the
>> extra
>>>> : */
>>>> - memmove(p2parse, p2parse + 2, lenMsg - 2);
>>>> + lenMsg -=2;
>>>> + memmove(p2parse, p2parse + 2, lenMsg);
>>>> + *(p2parse + lenMsg) = '\n';
>>>> + *(p2parse + lenMsg + 1) = '\0';
>>>> + pMsg->iLenRawMsg -=2;
>>>> + pMsg->iLenMSG -=2;
>>>> /* now, claim to abort so that something else can parse the now
>> modified
>>>> message */
>>>> DBGPRINTF("pmcisconames detected a mangled Cisco log message
>> message\n");
>>>> + dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg,
>> p2parse);
>>>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
>>>>
>>>> finalize_it:
>>>> @@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
>>>> CHKiRet(objUse(parser, CORE_COMPONENT));
>>>> CHKiRet(objUse(datetime, CORE_COMPONENT));
>>>>
>>>> - dbgprintf("lastmsg parser init called, compiled with version
>> %s\n",
>>>> VERSION);
>>>> + dbgprintf("cisconames parser init called, compiled with version
>> %s\n",
>>>> VERSION);
>>>> bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache
>> value, is
>>>> set only during rsyslogd option processing */
>>>>
>>>>
>>>>
>>>> David Lang
>>>>
>>>> P.S. I think having the file named pmlastmsg but the intermal module
>>>> name being lastline is confusing, they should be related.
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Wed Jan 26 11:19:28 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Jan 2011 11:19:28 +0100
Subject: [rsyslog] message manipulation in a parser
References: <alpine.DEB.2.00.1101162358330.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA14@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170039110.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA16@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170048580.9746@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDA17@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1101170111250.9746@asgard.lang.hm><4D3ECCC6.5070705@hq.adiscon.com><alpine.DEB.2.00.1101260105130.29659@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71DDAAA@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1101260130131.29659@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAAE@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 26, 2011 10:39 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] message manipulation in a parser
> 
> the message parser is turning out to be extremely easy to use in this
> mode.
> 
> I was able to re-write the cisconames fixup into the aixforwardedfrom
> fixup in about 2 hours, and the completed patch (including tieing it in
> to
> the build system) is only 288 lines.

I am glad it worked out to be easy -- this is actually how it was designed :)

> 
> It may very well be worth taking a bit of time to look at the default
> rfc3164 parser and see how much it could be simplified by pulling some
> of
> the odd corner cases out to fixup parsers like this.
> 
> it's not worth doing for flaws that show up frequently (having the code
> in
> the rfx3164 parser is going to be faster than going in to a separate
> parser, not to mention the config complexity), but some of the more
> strange cases may be worth pulling out, not as much for performance
> (although it will probably help a smidge there), but more so in terms
> of
> simplifying the code and documenting what generates strange messages.

I think the legacy parser is a good compromise, and that's one reason why I
did not tear it apart. It contains cases which frequently occur, at least I
think so. Also the guesswork should not cost too much performance.

> Also, I saw your tweet about imttcp and was wondering, with a small
> number
> of threads, was it enough faster that it could be useful for the second
> tier of a multi-tier logging infrastructure, where you have your first
> tier collecting from a lot of machines to a small number of machines,
> and
> then this first tier is sending the logs (at a fairly high volume per
> connection) to the next tier up? this is the opposite of the case you
> were testing (which as I understand it was lots of source machines,
> each
> with a fairly low log volume)

Possibly, 5% to 10%, but with a very low number of connections. I left the
code inside the tree. It is not bug free, but enough to try it out. Anyhow,
this was just a re-verification of my concepts. I will now see that I change
imtcp so that it can utilize a small number of worker threads. That should
bring the same effect, but scale better. It requires some steps before I can
actually do that, but I am progressing.

Rainer

> it may not be worth the overhead of maintaining a separate input
> module,
> but I thought I'd mention the possibility.
> 
> David Lang
> 
> 
>   On Wed, 26 Jan 2011, Rainer Gerhards
> wrote:
> 
> > Excellent, thx. I'll see that I merge both into the main branch
> today. Right
> > now my main machine is running a performance benchmark, so I need to
> do this
> > later today.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >> Sent: Wednesday, January 26, 2011 10:06 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] message manipulation in a parser
> >>
> >> I tested the code in your branch and verified that it works.
> >>
> >> David Lang
> >>
> >> On Tue, 25 Jan 2011, Rainer Gerhards wrote:
> >>
> >>> David,
> >>>
> >>> I now integrated the patch. I think I will also move it into the
> main
> >> branch
> >>> soon.
> >>>
> >>> Rainer
> >>>
> >>> On 01/17/2011 10:15 AM, david at lang.hm wrote:
> >>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> >>>>
> >>>>> Date: Mon, 17 Jan 2011 09:53:24 +0100
> >>>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >>>>>>
> >>>>>> On Mon, 17 Jan 2011, Rainer Gerhards wrote:
> >>>>>>
> >>>>>>> I guess it makes sense when you send me the code and give the
> >> line
> >>>>>>> number where you have the issue. I can't promise, but I'll try
> to
> >>>>>>> give a good suggestion in the course of the day.
> >>>>>>>
> >>>>>>> One important thing you need to think about is that pMsg does
> NOT
> >>>>>>> necessarily use cstr objects to represent the entities. For
> >> example
> >>>>>>> (IIRC), raw message is kept in a different buffer, and you
> cannot
> >>>>>>> use cstr methods on that buffer. %msg% is actually not stored
> at
> >>>>>>> all, it is taken from rawmsg. There is a whole lot of this kind
> >> of
> >>>>>>> optimization done inside the message object. I guess this is
> what
> >> is
> >>>>>>> hitting you.
> >>>>>>
> >>>>>> Ok, that makes sense. so should I just insert a \0 at the
> >> appropriate
> >>>>>> place and decrament the iLenRawMsg and iLenMSG values?
> >>>>>
> >>>>> Tob e 100% sure, I'd need to check more thoroughly, but I am sure
> >> you
> >>>>> are on
> >>>>> the right path... But have a look at msg.c. Where the buffer is
> >> located
> >>>>> depends on the size of it (it is static inside the object up to a
> >> certain
> >>>>> size, and dyn alloced otherwise -- another optimization). The
> safe
> >> bet
> >>>>> is to
> >>>>> duplicate the message text and re-set it, but that is obviously
> >> much
> >>>>> slower
> >>>>> (to gain speed, you need to break layers...).
> >>>>
> >>>> Ok, This is now working, I think I found all the variables to
> >> change.
> >>>>
> >>>> the patch on top of v5-devel-david is
> >>>>
> >>>> diff --git a/plugins/pmcisconames/pmcisconames.c
> >>>> b/plugins/pmcisconames/pmcisconames.c
> >>>> index 28fe33d..7b76f65 100644
> >>>> --- a/plugins/pmcisconames/pmcisconames.c
> >>>> +++ b/plugins/pmcisconames/pmcisconames.c
> >>>> @@ -41,7 +41,7 @@
> >>>> #include "unicode-helper.h"
> >>>>
> >>>> MODULE_TYPE_PARSER
> >>>> -PARSER_NAME("rsyslog.lastline")
> >>>> +PARSER_NAME("rsyslog.cisconames")
> >>>>
> >>>> /* internal structures
> >>>> */
> >>>> @@ -108,9 +108,15 @@ dbgprintf("not a cisco name mangled log!\n");
> >>>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
> >>>> }
> >>>> /* bump the message portion up by two characters to overwrite the
> >> extra
> >>>> : */
> >>>> - memmove(p2parse, p2parse + 2, lenMsg - 2);
> >>>> + lenMsg -=2;
> >>>> + memmove(p2parse, p2parse + 2, lenMsg);
> >>>> + *(p2parse + lenMsg) = '\n';
> >>>> + *(p2parse + lenMsg + 1) = '\0';
> >>>> + pMsg->iLenRawMsg -=2;
> >>>> + pMsg->iLenMSG -=2;
> >>>> /* now, claim to abort so that something else can parse the now
> >> modified
> >>>> message */
> >>>> DBGPRINTF("pmcisconames detected a mangled Cisco log message
> >> message\n");
> >>>> + dbgprintf("pmcisconames: new mesage: [%d]'%s'\n", lenMsg,
> >> p2parse);
> >>>> ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE);
> >>>>
> >>>> finalize_it:
> >>>> @@ -143,7 +149,7 @@ CODEmodInit_QueryRegCFSLineHdlr
> >>>> CHKiRet(objUse(parser, CORE_COMPONENT));
> >>>> CHKiRet(objUse(datetime, CORE_COMPONENT));
> >>>>
> >>>> - dbgprintf("lastmsg parser init called, compiled with version
> >> %s\n",
> >>>> VERSION);
> >>>> + dbgprintf("cisconames parser init called, compiled with version
> >> %s\n",
> >>>> VERSION);
> >>>> bParseHOSTNAMEandTAG = glbl.GetParseHOSTNAMEandTAG(); /* cache
> >> value, is
> >>>> set only during rsyslogd option processing */
> >>>>
> >>>>
> >>>>
> >>>> David Lang
> >>>>
> >>>> P.S. I think having the file named pmlastmsg but the intermal
> module
> >>>> name being lastline is confusing, they should be related.
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mmonitz at gmail.com  Wed Jan 26 13:44:59 2011
From: mmonitz at gmail.com (matan monitz)
Date: Wed, 26 Jan 2011 14:44:59 +0200
Subject: [rsyslog] rsyslog Text File Input Module losing seek?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDAA7@GRFEXC.intern.adiscon.com>
References: <AANLkTinOSgNDNAew1qxQot79rFexwb53W=-ppHWKvnK2@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com>
	<AANLkTin6Uqk2Cg3XyusLcRi3B8k34QEyudTO4LdyydR-@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com>
	<AANLkTin6qv2juvziBQfGgkP-c9_mpE4fpUR7Vi9T9Kw-@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com>
	<AANLkTimHcSVjaeJnyGsXvc9c1-LcPDn=kExh54xd5OA=@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD6D8@GRFEXC.intern.adiscon.com>
	<AANLkTikXb2=PdZ0YSHanAf7-ZONUegq+R7h4SEcPNFC_@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAA7@GRFEXC.intern.adiscon.com>
Message-ID: <AANLkTikKGe-698EYp9Y9uGY0aoZ+LPiEhf7+hAuqoQ+8@mail.gmail.com>

hi Rainer
is this a known issue? i could not find it documented anywhere

On Tue, Jan 25, 2011 at 7:28 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com>wrote:

> I guess I need to merge and release a new v5-stable. Fixes are already in
> git, but as it seems I missed v5-stable (-devel and all others have it).
>
> Rainer
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > Sent: Tuesday, January 25, 2011 4:18 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> >
> > sorry for the bump
> > we had just upgraded to 5.6.2 and it is still happening
> > this is now happening almost hourly and not at the same time as file
> > rotation
> > we will try catching it while in debug mode tomorrow morning
> > any other ideas on the subject for now?
> >
> > On Wed, Nov 10, 2010 at 7:47 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com>wrote:
> >
> > > Could you run it in debug mode and provide me a log covering a
> > situation
> > > where the problem occurs? I may be able to gain some information from
> > that.
> > >
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > Sent: Wednesday, November 10, 2010 6:46 PM
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> > > >
> > > > hello rainer
> > > > unfortunately we will not be able to apply a fix if provided,
> > > > if an upgrade will eventually happen it will likely be to Version
> > 5.
> > > >
> > > > i have reviewed the references you provided and i don't believe
> > they
> > > > are
> > > > relevant to our case
> > > > out files rotate at 10MB which is miles off from 2GB.
> > > > also, as i mentioned,i think this is not related to state files
> > because
> > > > we
> > > > made sure they are being created and that rsyslog is not crashing
> > > > i agree there is not much point in providing a fix at this point,
> > > > however i
> > > > would like to hear any other guesses you may have regarding the
> > cause
> > > > of the
> > > > situation so that i may figure out a way to work around it outside
> > of
> > > > rsyslog
> > > > also i will be happy to provide technical data and logs as
> > instructed
> > > > if you
> > > > need another test case
> > > >
> > > > thanks in advance
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Nov 10, 2010 at 11:39 AM, Rainer Gerhards
> > > > <rgerhards at hq.adiscon.com>wrote:
> > > >
> > > > > > -----Original Message-----
> > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > Sent: Monday, November 08, 2010 7:08 PM
> > > > > > To: rsyslog-users
> > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing
> > seek?
> > > > > >
> > > > > > hi rainer
> > > > > > unfortunately this is not a server i'm directly in charge of so
> > we
> > > > > > can't
> > > > > > really upgrade for while
> > > > > > (they will only upgrade as part of the planned upgrade for the
> > > > whole
> > > > > > server)
> > > > >
> > > > > Does that also mean you would not be able to apply a fix if I I
> > > > provide one
> > > > > for 3.22.2 (the current v3-stable)?
> > > > >
> > > > > > can you please elaborate on the cause of the problem so that
> > maybe
> > > > i
> > > > > > can
> > > > > > figure out a temporary workaround?
> > > > >
> > > > > There definitely is a problem when the file size is larger than
> > 2GB.
> > > > The
> > > > > rest
> > > > > may be related to dirty shutdown when the state file is deleted.
> > See
> > > > >
> > > > >
> > > > >
> > > >
> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536
> > > > b3dc4a
> > > > >
> > > >
> > 28b94f80a361b41d<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=4
> > > > 0814bb0307f9d8536b3dc4a%0A28b94f80a361b41d>
> > > > >
> > > > >
> > > > >
> > > >
> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2
> > > > 124ba7
> > > > >
> > > >
> > d830652e9b1ead96<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9
> > > > 0933057bc2f014fd2124ba7%0Ad830652e9b1ead96>
> > > > >
> > > > > I have not yet receive feedback that makes me sure it's 100%
> > fixed,
> > > > so I'd
> > > > > actually be quite interested in another test case. However, quite
> > > > honestly,
> > > > > if we can't run a better version on your box, there is no point
> > for
> > > > me in
> > > > > further looking at that case (because it can't be solved
> > anyway...).
> > > > >
> > > > > Rainer
> > > > >
> > > > > > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards
> > > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > > Sent: Monday, November 08, 2010 3:51 PM
> > > > > > > > To: rsyslog-users
> > > > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module
> > losing
> > > > seek?
> > > > > > > >
> > > > > > > > hello rainer
> > > > > > > > we are using  3.22.1-3 from rpm on rhel 5
> > > > > > >
> > > > > > > That's definitely too old. I maintain it when I am at it, but
> > the
> > > > fix
> > > > > > went
> > > > > > > into v4+ so far. I suggest that you pull the latest v4-stable
> > > > from
> > > > > > git. If
> > > > > > > git doesn't work for you, let me know and I'll mail you a
> > private
> > > > > > tarball.
> > > > > > > I
> > > > > > > am just right now very busy (probably tomorrow as well, as I
> > am
> > > > > > writing a
> > > > > > > lot
> > > > > > > of new code in support for CEE).
> > > > > > >
> > > > > > > Rainer
> > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards
> > > > > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > > > > >
> > > > > > > > > which version do you use? I have recently addressed such
> > a
> > > > > > problem...
> > > > > > > > >
> > > > > > > > > Rainer
> > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > [mailto:rsyslog-
> > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > > > > Sent: Monday, November 08, 2010 2:45 PM
> > > > > > > > > > To: rsyslog at lists.adiscon.com
> > > > > > > > > > Subject: [rsyslog] rsyslog Text File Input Module
> > losing
> > > > seek?
> > > > > > > > > >
> > > > > > > > > > Hello all
> > > > > > > > > > we have a bind server writing and rotating logs on file
> > > > system
> > > > > > > > > > (rotating
> > > > > > > > > > when log file reaches 10Mb).
> > > > > > > > > > we also have rsyslog configured with the text file
> > input
> > > > module
> > > > > > > > reading
> > > > > > > > > > the
> > > > > > > > > > file and sending it over UDP to another rsyslog server.
> > > > > > > > > > several times during the day the rsyslog on the bind
> > > > machine
> > > > > > starts
> > > > > > > > > > reading
> > > > > > > > > > the file from the top, causing a sudden high rate of
> > > > duplicate
> > > > > > and
> > > > > > > > old
> > > > > > > > > > events.
> > > > > > > > > > in an attempt to identify the problem:
> > > > > > > > > > 1.verified this using tcpdump to make sure the and saw
> > the
> > > > old
> > > > > > logs
> > > > > > > > > > coming
> > > > > > > > > > down the wire to the rsyslog server
> > > > > > > > > > 2.made sure state files are correctly configured and
> > are
> > > > being
> > > > > > > > created
> > > > > > > > > > when
> > > > > > > > > > needed
> > > > > > > > > > 3.made sure the rsyslog is not crashing and restarting
> > by
> > > > > > > > monitoring
> > > > > > > > > > the pid
> > > > > > > > > > googeling hasn't helped as well
> > > > > > > > > >
> > > > > > > > > > any ideas?
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > >
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Wed Jan 26 14:02:29 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Jan 2011 14:02:29 +0100
Subject: [rsyslog] rsyslog Text File Input Module losing seek?
References: <AANLkTinOSgNDNAew1qxQot79rFexwb53W=-ppHWKvnK2@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD694@GRFEXC.intern.adiscon.com><AANLkTin6Uqk2Cg3XyusLcRi3B8k34QEyudTO4LdyydR-@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD69A@GRFEXC.intern.adiscon.com><AANLkTin6qv2juvziBQfGgkP-c9_mpE4fpUR7Vi9T9Kw-@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD6CA@GRFEXC.intern.adiscon.com><AANLkTimHcSVjaeJnyGsXvc9c1-LcPDn=kExh54xd5OA=@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DD6D8@GRFEXC.intern.adiscon.com><AANLkTikXb2=PdZ0YSHanAf7-ZONUegq+R7h4SEcPNFC_@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71DDAA7@GRFEXC.intern.adiscon.com>
	<AANLkTikKGe-698EYp9Y9uGY0aoZ+LPiEhf7+hAuqoQ+8@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAB0@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of matan monitz
> Sent: Wednesday, January 26, 2011 1:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> 
> hi Rainer
> is this a known issue? i could not find it documented anywhere

yes it is -- probably it was never submitted via bugzilla or some other
persistent ressource. I get lot's of report "on the fly" and have given up
try to document them all. But the changelog is complete.

I have a new 5.6.3 ready for release, but the web server has some problems
(being worked on). I'll see that I mail you a copy.

Rainer

> 
> On Tue, Jan 25, 2011 at 7:28 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com>wrote:
> 
> > I guess I need to merge and release a new v5-stable. Fixes are
> already in
> > git, but as it seems I missed v5-stable (-devel and all others have
> it).
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > Sent: Tuesday, January 25, 2011 4:18 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing seek?
> > >
> > > sorry for the bump
> > > we had just upgraded to 5.6.2 and it is still happening
> > > this is now happening almost hourly and not at the same time as
> file
> > > rotation
> > > we will try catching it while in debug mode tomorrow morning
> > > any other ideas on the subject for now?
> > >
> > > On Wed, Nov 10, 2010 at 7:47 PM, Rainer Gerhards
> > > <rgerhards at hq.adiscon.com>wrote:
> > >
> > > > Could you run it in debug mode and provide me a log covering a
> > > situation
> > > > where the problem occurs? I may be able to gain some information
> from
> > > that.
> > > >
> > > > Rainer
> > > >
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > Sent: Wednesday, November 10, 2010 6:46 PM
> > > > > To: rsyslog-users
> > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module losing
> seek?
> > > > >
> > > > > hello rainer
> > > > > unfortunately we will not be able to apply a fix if provided,
> > > > > if an upgrade will eventually happen it will likely be to
> Version
> > > 5.
> > > > >
> > > > > i have reviewed the references you provided and i don't believe
> > > they
> > > > > are
> > > > > relevant to our case
> > > > > out files rotate at 10MB which is miles off from 2GB.
> > > > > also, as i mentioned,i think this is not related to state files
> > > because
> > > > > we
> > > > > made sure they are being created and that rsyslog is not
> crashing
> > > > > i agree there is not much point in providing a fix at this
> point,
> > > > > however i
> > > > > would like to hear any other guesses you may have regarding the
> > > cause
> > > > > of the
> > > > > situation so that i may figure out a way to work around it
> outside
> > > of
> > > > > rsyslog
> > > > > also i will be happy to provide technical data and logs as
> > > instructed
> > > > > if you
> > > > > need another test case
> > > > >
> > > > > thanks in advance
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Nov 10, 2010 at 11:39 AM, Rainer Gerhards
> > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > Sent: Monday, November 08, 2010 7:08 PM
> > > > > > > To: rsyslog-users
> > > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module
> losing
> > > seek?
> > > > > > >
> > > > > > > hi rainer
> > > > > > > unfortunately this is not a server i'm directly in charge
> of so
> > > we
> > > > > > > can't
> > > > > > > really upgrade for while
> > > > > > > (they will only upgrade as part of the planned upgrade for
> the
> > > > > whole
> > > > > > > server)
> > > > > >
> > > > > > Does that also mean you would not be able to apply a fix if I
> I
> > > > > provide one
> > > > > > for 3.22.2 (the current v3-stable)?
> > > > > >
> > > > > > > can you please elaborate on the cause of the problem so
> that
> > > maybe
> > > > > i
> > > > > > > can
> > > > > > > figure out a temporary workaround?
> > > > > >
> > > > > > There definitely is a problem when the file size is larger
> than
> > > 2GB.
> > > > > The
> > > > > > rest
> > > > > > may be related to dirty shutdown when the state file is
> deleted.
> > > See
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=40814bb0307f9d8536
> > > > > b3dc4a
> > > > > >
> > > > >
> > >
> 28b94f80a361b41d<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=4
> > > > > 0814bb0307f9d8536b3dc4a%0A28b94f80a361b41d>
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=90933057bc2f014fd2
> > > > > 124ba7
> > > > > >
> > > > >
> > >
> d830652e9b1ead96<http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9
> > > > > 0933057bc2f014fd2124ba7%0Ad830652e9b1ead96>
> > > > > >
> > > > > > I have not yet receive feedback that makes me sure it's 100%
> > > fixed,
> > > > > so I'd
> > > > > > actually be quite interested in another test case. However,
> quite
> > > > > honestly,
> > > > > > if we can't run a better version on your box, there is no
> point
> > > for
> > > > > me in
> > > > > > further looking at that case (because it can't be solved
> > > anyway...).
> > > > > >
> > > > > > Rainer
> > > > > >
> > > > > > > On Mon, Nov 8, 2010 at 6:39 PM, Rainer Gerhards
> > > > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan monitz
> > > > > > > > > Sent: Monday, November 08, 2010 3:51 PM
> > > > > > > > > To: rsyslog-users
> > > > > > > > > Subject: Re: [rsyslog] rsyslog Text File Input Module
> > > losing
> > > > > seek?
> > > > > > > > >
> > > > > > > > > hello rainer
> > > > > > > > > we are using  3.22.1-3 from rpm on rhel 5
> > > > > > > >
> > > > > > > > That's definitely too old. I maintain it when I am at it,
> but
> > > the
> > > > > fix
> > > > > > > went
> > > > > > > > into v4+ so far. I suggest that you pull the latest v4-
> stable
> > > > > from
> > > > > > > git. If
> > > > > > > > git doesn't work for you, let me know and I'll mail you a
> > > private
> > > > > > > tarball.
> > > > > > > > I
> > > > > > > > am just right now very busy (probably tomorrow as well,
> as I
> > > am
> > > > > > > writing a
> > > > > > > > lot
> > > > > > > > of new code in support for CEE).
> > > > > > > >
> > > > > > > > Rainer
> > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Nov 8, 2010 at 3:46 PM, Rainer Gerhards
> > > > > > > > > <rgerhards at hq.adiscon.com>wrote:
> > > > > > > > >
> > > > > > > > > > which version do you use? I have recently addressed
> such
> > > a
> > > > > > > problem...
> > > > > > > > > >
> > > > > > > > > > Rainer
> > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > [mailto:rsyslog-
> > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of matan
> monitz
> > > > > > > > > > > Sent: Monday, November 08, 2010 2:45 PM
> > > > > > > > > > > To: rsyslog at lists.adiscon.com
> > > > > > > > > > > Subject: [rsyslog] rsyslog Text File Input Module
> > > losing
> > > > > seek?
> > > > > > > > > > >
> > > > > > > > > > > Hello all
> > > > > > > > > > > we have a bind server writing and rotating logs on
> file
> > > > > system
> > > > > > > > > > > (rotating
> > > > > > > > > > > when log file reaches 10Mb).
> > > > > > > > > > > we also have rsyslog configured with the text file
> > > input
> > > > > module
> > > > > > > > > reading
> > > > > > > > > > > the
> > > > > > > > > > > file and sending it over UDP to another rsyslog
> server.
> > > > > > > > > > > several times during the day the rsyslog on the
> bind
> > > > > machine
> > > > > > > starts
> > > > > > > > > > > reading
> > > > > > > > > > > the file from the top, causing a sudden high rate
> of
> > > > > duplicate
> > > > > > > and
> > > > > > > > > old
> > > > > > > > > > > events.
> > > > > > > > > > > in an attempt to identify the problem:
> > > > > > > > > > > 1.verified this using tcpdump to make sure the and
> saw
> > > the
> > > > > old
> > > > > > > logs
> > > > > > > > > > > coming
> > > > > > > > > > > down the wire to the rsyslog server
> > > > > > > > > > > 2.made sure state files are correctly configured
> and
> > > are
> > > > > being
> > > > > > > > > created
> > > > > > > > > > > when
> > > > > > > > > > > needed
> > > > > > > > > > > 3.made sure the rsyslog is not crashing and
> restarting
> > > by
> > > > > > > > > monitoring
> > > > > > > > > > > the pid
> > > > > > > > > > > googeling hasn't helped as well
> > > > > > > > > > >
> > > > > > > > > > > any ideas?
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > >
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From dirk.schulz at kinzesberg.de  Thu Jan 27 13:23:28 2011
From: dirk.schulz at kinzesberg.de (Dirk)
Date: Thu, 27 Jan 2011 13:23:28 +0100
Subject: [rsyslog] Rsyslog and FIN_WAIT1
Message-ID: <4D4163C0.4060001@kinzesberg.de>

  Hi folks,

we are running Rsyslog 3.18.x on SLES10 and have stumbled upon a weird 
phenomenon.

Our client rsyslog sent messages to a central rsyslog using tcp. They 
are configured to buffer locally if the central is not available.

Now, if the tcp connection does not leave FIN_WAIT1 state (may be the 
TCP ACK does not reach the client for network reasons) the client 
rsyslog stops all activity (writing to local logs, importing local logs, 
etc.) and does not write anything into the queue buffer files. In fact 
it seems dead until rebooted.

To my understanding this is not the exptected/intended behaviour, so the 
question is: Is this a known bug of this version? Is there a version 
that can handle frozen FIN_WAIT1 states (I know they should not appear 
at all but that is a different issue)?

Any hint to docs and understanding is greatly appreciated.

Dirk

From dave at fly.srk.fer.hr  Thu Jan 27 17:54:58 2011
From: dave at fly.srk.fer.hr (=?iso-8859-2?Q?Dra=BEen_Ka=E8ar?=)
Date: Thu, 27 Jan 2011 17:54:58 +0100
Subject: [rsyslog] Rsyslog and FIN_WAIT1
In-Reply-To: <4D4163C0.4060001@kinzesberg.de>
References: <4D4163C0.4060001@kinzesberg.de>
Message-ID: <20110127165458.GA3286@fly.srk.fer.hr>

Dirk wrote:

> Now, if the tcp connection does not leave FIN_WAIT1 state (may be the  
> TCP ACK does not reach the client for network reasons) the client  
> rsyslog stops all activity (writing to local logs, importing local logs,  
> etc.) and does not write anything into the queue buffer files. In fact  
> it seems dead until rebooted.

Can you run strace on the client when this happens? My guess is that it will
be hanging in the close() syscall.

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave at fly.srk.fer.hr

From friedl at hq.adiscon.com  Fri Jan 28 11:20:00 2011
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 28 Jan 2011 11:20:00 +0100
Subject: [rsyslog] website problems - work in progress
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAD4@GRFEXC.intern.adiscon.com>

Dear Mailing-list,

 

as some of you may have noticed, we had some problems with our web server. 

 

I wrote a post on this issue on the website. You can find it here:

http://www.rsyslog.com/website-downtime/

 

Best regards,

 

Florian Riedl

 

--

Support

=======

 

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

 

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html
<http://www.rsyslog.com/doc-how2help.html>  .

 


From tbergfeld at hq.adiscon.com  Fri Jan 28 11:35:48 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Fri, 28 Jan 2011 11:35:48 +0100
Subject: [rsyslog] rsyslog 5.6.3 (v5-stable) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAD5@GRFEXC.intern.adiscon.com>

Hi all,

We have just released rsyslog 5.6.3, the new v5-stable.
This release includes some bug fixes but no new functionality.

Please see the ChangeLog for more details.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-6-3-v5-stable/

Download:

http://www.rsyslog.com/rsyslog-5-6-3-v5-stable/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support

=======
 

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.


Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From dirk.schulz at kinzesberg.de  Fri Jan 28 14:36:28 2011
From: dirk.schulz at kinzesberg.de (Dirk)
Date: Fri, 28 Jan 2011 14:36:28 +0100
Subject: [rsyslog] Missing input file unnerving rsyslogd - was: Re: Rsyslog
 and FIN_WAIT1
In-Reply-To: <20110127165458.GA3286@fly.srk.fer.hr>
References: <4D4163C0.4060001@kinzesberg.de>
	<20110127165458.GA3286@fly.srk.fer.hr>
Message-ID: <4D42C65C.9030009@kinzesberg.de>

  Hi all,

I have tracked the problem down: if rsyslog is configured to read 
messages from a log file and the file does not exist, it does not
- refrain from starting
- place relevant messages somewhere
or anything sensible. It starts the weird behaviour I described even up 
to TCP connections hanging in FIN_WAIT1 for ever.

Is this a known bug of 3.18.3? This is the RPM from SLES11.

Is this solved in a later release?

Or do I misunderstand something completely?

Dirk

Am 27.01.11 17:54, schrieb Dra?en Ka?ar:
> Dirk wrote:
>
>> Now, if the tcp connection does not leave FIN_WAIT1 state (may be the
>> TCP ACK does not reach the client for network reasons) the client
>> rsyslog stops all activity (writing to local logs, importing local logs,
>> etc.) and does not write anything into the queue buffer files. In fact
>> it seems dead until rebooted.
> Can you run strace on the client when this happens? My guess is that it will
> be hanging in the close() syscall.
>


From rgerhards at hq.adiscon.com  Fri Jan 28 14:41:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Jan 2011 14:41:37 +0100
Subject: [rsyslog] Missing input file unnerving rsyslogd - was: Re:
	Rsyslog and FIN_WAIT1
References: <4D4163C0.4060001@kinzesberg.de><20110127165458.GA3286@fly.srk.fer.hr>
	<4D42C65C.9030009@kinzesberg.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDADD@GRFEXC.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dirk
> Sent: Friday, January 28, 2011 2:36 PM
> To: rsyslog-users
> Subject: [rsyslog] Missing input file unnerving rsyslogd - was: Re:
> Rsyslog and FIN_WAIT1
> 
>   Hi all,
> 
> I have tracked the problem down: if rsyslog is configured to read
> messages from a log file and the file does not exist, it does not
> - refrain from starting
> - place relevant messages somewhere
> or anything sensible. It starts the weird behaviour I described even up
> to TCP connections hanging in FIN_WAIT1 for ever.
> 
> Is this a known bug of 3.18.3? This is the RPM from SLES11.

sounds so

> 
> Is this solved in a later release?

very probably, as I did a couple of changes to imfile recently. Note that v3
is no longer officially supported, so the fixes went into v4 and above.

Rainer
> 
> Or do I misunderstand something completely?
> 
> Dirk
> 
> Am 27.01.11 17:54, schrieb Dra?en Ka?ar:
> > Dirk wrote:
> >
> >> Now, if the tcp connection does not leave FIN_WAIT1 state (may be
> the
> >> TCP ACK does not reach the client for network reasons) the client
> >> rsyslog stops all activity (writing to local logs, importing local
> logs,
> >> etc.) and does not write anything into the queue buffer files. In
> fact
> >> it seems dead until rebooted.
> > Can you run strace on the client when this happens? My guess is that
> it will
> > be hanging in the close() syscall.
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Fri Jan 28 14:52:54 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Jan 2011 14:52:54 +0100
Subject: [rsyslog] Missing input file unnerving rsyslogd - was:
	Re:Rsyslog and FIN_WAIT1
References: <4D4163C0.4060001@kinzesberg.de><20110127165458.GA3286@fly.srk.fer.hr><4D42C65C.9030009@kinzesberg.de>
	<9B6E2A8877C38245BFB15CC491A11DA71DDADD@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDADE@GRFEXC.intern.adiscon.com>

> > Is this a known bug of 3.18.3? This is the RPM from SLES11.
> 
> sounds so

I did not read the word "known". So: no, it is not a known bug, but it
probably is a bug of 3.18.3.

Sorry for any confusion created...

Rainer

From dirk.schulz at kinzesberg.de  Fri Jan 28 14:54:37 2011
From: dirk.schulz at kinzesberg.de (Dirk)
Date: Fri, 28 Jan 2011 14:54:37 +0100
Subject: [rsyslog] Missing input file unnerving rsyslogd -
 was:	Re:Rsyslog and FIN_WAIT1
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDADE@GRFEXC.intern.adiscon.com>
References: <4D4163C0.4060001@kinzesberg.de><20110127165458.GA3286@fly.srk.fer.hr><4D42C65C.9030009@kinzesberg.de>	<9B6E2A8877C38245BFB15CC491A11DA71DDADD@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DDADE@GRFEXC.intern.adiscon.com>
Message-ID: <4D42CA9D.5080106@kinzesberg.de>

  Thanks for the information.

Dirk

Am 28.01.11 14:52, schrieb Rainer Gerhards:
>>> Is this a known bug of 3.18.3? This is the RPM from SLES11.
>> sounds so
> I did not read the word "known". So: no, it is not a known bug, but it
> probably is a bug of 3.18.3.
>
> Sorry for any confusion created...
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dirk.schulz at kinzesberg.de  Fri Jan 28 14:59:50 2011
From: dirk.schulz at kinzesberg.de (Dirk)
Date: Fri, 28 Jan 2011 14:59:50 +0100
Subject: [rsyslog] Limit of number of output files?
Message-ID: <4D42CBD6.50007@kinzesberg.de>

  We are setting up Rsyslogd 3.18.3 as central logserver in a large 
environment.

At the moment there are around 1.800 output log files in the plan, and 
more to come.

So questions concerning the number of output files start to come into mind:
Is there a limit depending on kernel parameters or else?
Is there a limit that is fixed absolutely?

If there is limits we cannot expand: Can Rsyslod 3 be run in several 
instances per Server?

Thanks for any hint or help.

Dirk

From rgerhards at hq.adiscon.com  Fri Jan 28 15:03:09 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Jan 2011 15:03:09 +0100
Subject: [rsyslog] Limit of number of output files?
References: <4D42CBD6.50007@kinzesberg.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAE0@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dirk
> Sent: Friday, January 28, 2011 3:00 PM
> To: rsyslog-users
> Subject: [rsyslog] Limit of number of output files?
> 
>   We are setting up Rsyslogd 3.18.3 as central logserver in a large
> environment.
> 
> At the moment there are around 1.800 output log files in the plan, and
> more to come.
> 
> So questions concerning the number of output files start to come into
> mind:
> Is there a limit depending on kernel parameters or else?
> Is there a limit that is fixed absolutely?
I don't think so, but you need the right config parameters. I am not sure if
v3 can adjust the max # of file handles. If not, you need to do this form
external.

> 
> If there is limits we cannot expand: Can Rsyslod 3 be run in several
> instances per Server?

definitely

Rainer
> 
> Thanks for any hint or help.
> 
> Dirk
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From Luis.Fernando.Munoz.Mejias at cern.ch  Fri Jan 28 15:10:08 2011
From: Luis.Fernando.Munoz.Mejias at cern.ch (Luis Fernando =?iso-8859-1?q?Mu=F1oz_Mej=EDas?=)
Date: Fri, 28 Jan 2011 15:10:08 +0100
Subject: [rsyslog] Limit of number of output files?
In-Reply-To: <4D42CBD6.50007@kinzesberg.de>
References: <4D42CBD6.50007@kinzesberg.de>
Message-ID: <201101281510.08398.Luis.Fernando.Munoz.Mejias@cern.ch>

Dirk,

> At the moment there are around 1.800 output log files in the plan, and
> more to come.
>
We generate our log files in a /var/log/.../$HOSTNAME$ way, with ~5K
hosts, and havent needed to customise the interactive limits. It works
fine from rsyslog 3.18 to 5.6.2. On 3.18 I had bad problems with log
rotations, though.

OTOH, be sure to disable DNS (if you receive messages from old sysklogd
it may not be possible): that's the worst slowdown in our case.

Cheers.
-- 
Luis Fernando Mu?oz Mej?as
Luis.Fernando.Munoz.Mejias at cern.ch


From dirk.schulz at kinzesberg.de  Fri Jan 28 15:26:59 2011
From: dirk.schulz at kinzesberg.de (Dirk)
Date: Fri, 28 Jan 2011 15:26:59 +0100
Subject: [rsyslog] Limit of number of output files?
In-Reply-To: <201101281510.08398.Luis.Fernando.Munoz.Mejias@cern.ch>
References: <4D42CBD6.50007@kinzesberg.de>
	<201101281510.08398.Luis.Fernando.Munoz.Mejias@cern.ch>
Message-ID: <4D42D233.9080700@kinzesberg.de>

  Fernando,

thanks a lot.

Dirk

Am 28.01.11 15:10, schrieb Luis Fernando Mu?oz Mej?as:
> Dirk,
>
>> At the moment there are around 1.800 output log files in the plan, and
>> more to come.
>>
> We generate our log files in a /var/log/.../$HOSTNAME$ way, with ~5K
> hosts, and havent needed to customise the interactive limits. It works
> fine from rsyslog 3.18 to 5.6.2. On 3.18 I had bad problems with log
> rotations, though.
>
> OTOH, be sure to disable DNS (if you receive messages from old sysklogd
> it may not be possible): that's the worst slowdown in our case.
>
> Cheers.


From dirk.schulz at kinzesberg.de  Fri Jan 28 16:36:14 2011
From: dirk.schulz at kinzesberg.de (Dirk)
Date: Fri, 28 Jan 2011 16:36:14 +0100
Subject: [rsyslog] Rsyslog4 or 5 on SLES10?
Message-ID: <4D42E26E.6090709@kinzesberg.de>

  Has anybody out there successfully built rsyslog 4 or 5 on SLES10? If 
yes, can you point me to any docs or give hints?

Any help is appreciated.

Dirk

From ariel.p at hostdime.com  Sun Jan 30 22:22:00 2011
From: ariel.p at hostdime.com (Ariel P.)
Date: Sun, 30 Jan 2011 16:22:00 -0500
Subject: [rsyslog] Feedback on ommysql requested
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DD93D@GRFEXC.intern.adiscon.com>
References: <20101202155126.GZ19162@aart.is.rice.edu>	<4D0BE506.8090000@hostdime.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DD93D@GRFEXC.intern.adiscon.com>
Message-ID: <4D45D678.7090303@hostdime.com>

Hi all,

Any word on the merging of this patch into the main code base, given 
that no further opposing arguments were given in the mailing list?

Ariel P.
Server Security Analyst
HostDime.com, Inc.


On 2010-12-22 13:07, Rainer Gerhards wrote:
> Hi all,
>
> thanks to Ariel for the hard work. I, too, think that all concerns are
> addressed. I intend to merge the patch when I am back in January if I don't
> here any good argument why I should not ;)
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Ariel P.
>> Sent: Friday, December 17, 2010 11:33 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] Feedback on ommysql requested
>>
>> I have created a new version of my patch, which creates two
>> configuration settings for ommysql.
>> The first setting sets the path for the 'my.cnf' file. If this setting
>> is not specified in the configuration file, the mysql library is not
>> informed of a file path (current behavior)
>> The second setting is used to change the section of the 'my.cnf' file
>> to
>> use, which defaults to "client" if not specified in the configuration
>> file (this is mysql standard behavior).
>>
>> Please see [ http://bugzilla.adiscon.com/show_bug.cgi?id=213#c3 ] for
>> details.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From up at 3.am  Mon Jan 31 15:54:22 2011
From: up at 3.am (up at 3.am)
Date: Mon, 31 Jan 2011 09:54:22 -0500
Subject: [rsyslog] Configuring which file(s) to log to
Message-ID: <db78a071442590a3ac3d9028f0e40591.squirrel@ssl.pil.net>

I am noob to rsyslog, but from the documentation I've been able to find, I
was able to get remote logging for the standard syslog stuff set up fine,
messages, mail, auth/secure, etc.

However, I've been trying to get a couple of non-standard stuff going with
mixed results.  First, there is fail2ban.  I configured /etc/rsyslog.conf
as follows:

# fail2ban log
local6.*	                                /var/log/fail2ban.log

and fail2ban.conf as follows:

loglevel = 3
logtarget = SYSLOG
syslog-facility = 22
syslog-target = /var/log/fail2ban.log

fail2ban logs remotely, but instead of to the file specified above (I
touched it), it logs to messages.  What did I miss?

Similarly, I created a cisco log and generated logging on the cisco.  In
this case, it does log to the file I created, but it duplicates it to
messages as well.

Clues appreciated!

From rgerhards at hq.adiscon.com  Mon Jan 31 16:51:41 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 31 Jan 2011 16:51:41 +0100
Subject: [rsyslog] Configuring which file(s) to log to
References: <db78a071442590a3ac3d9028f0e40591.squirrel@ssl.pil.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAFB@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of up at 3.am
> Sent: Monday, January 31, 2011 3:54 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Configuring which file(s) to log to
> 
> I am noob to rsyslog, but from the documentation I've been able to
> find, I
> was able to get remote logging for the standard syslog stuff set up
> fine,
> messages, mail, auth/secure, etc.
> 
> However, I've been trying to get a couple of non-standard stuff going
> with
> mixed results.  First, there is fail2ban.  I configured
> /etc/rsyslog.conf
> as follows:
> 
> # fail2ban log
> local6.*	                                /var/log/fail2ban.log
> 
> and fail2ban.conf as follows:
> 
> loglevel = 3
> logtarget = SYSLOG
> syslog-facility = 22
> syslog-target = /var/log/fail2ban.log
> 
> fail2ban logs remotely, but instead of to the file specified above (I
> touched it), it logs to messages.  What did I miss?

Unfortunately, I don't know fail2ban. But some insight may be possible if you
add

*.* /var/log/debugfile;RSYSLOG_DebugFormat

Then the debugfile will show us some details.

> 
> Similarly, I created a cisco log and generated logging on the cisco.
> In
> this case, it does log to the file I created, but it duplicates it to
> messages as well.

you need to discard the message if it shall not be written to a second file.
See here:

http://www.rsyslog.com/storing-messages-from-a-remote-system-into-a-specific-
file/

HTH
Rainer
> 
> Clues appreciated!
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Mon Jan 31 17:00:51 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 31 Jan 2011 17:00:51 +0100
Subject: [rsyslog] Feedback on ommysql requested
References: <20101202155126.GZ19162@aart.is.rice.edu>	<4D0BE506.8090000@hostdime.com><9B6E2A8877C38245BFB15CC491A11DA71DD93D@GRFEXC.intern.adiscon.com>
	<4D45D678.7090303@hostdime.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71DDAFC@GRFEXC.intern.adiscon.com>

sorry, I had forgotten the merge after my vacation. Now merged. It would be
good if you could also provide a patch for the doc file.

Thanks again!
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ariel P.
> Sent: Sunday, January 30, 2011 10:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Feedback on ommysql requested
> 
> Hi all,
> 
> Any word on the merging of this patch into the main code base, given
> that no further opposing arguments were given in the mailing list?
> 
> Ariel P.
> Server Security Analyst
> HostDime.com, Inc.
> 
> 
> On 2010-12-22 13:07, Rainer Gerhards wrote:
> > Hi all,
> >
> > thanks to Ariel for the hard work. I, too, think that all concerns
> are
> > addressed. I intend to merge the patch when I am back in January if I
> don't
> > here any good argument why I should not ;)
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Ariel P.
> >> Sent: Friday, December 17, 2010 11:33 PM
> >> To: rsyslog at lists.adiscon.com
> >> Subject: Re: [rsyslog] Feedback on ommysql requested
> >>
> >> I have created a new version of my patch, which creates two
> >> configuration settings for ommysql.
> >> The first setting sets the path for the 'my.cnf' file. If this
> setting
> >> is not specified in the configuration file, the mysql library is not
> >> informed of a file path (current behavior)
> >> The second setting is used to change the section of the 'my.cnf'
> file
> >> to
> >> use, which defaults to "client" if not specified in the
> configuration
> >> file (this is mysql standard behavior).
> >>
> >> Please see [ http://bugzilla.adiscon.com/show_bug.cgi?id=213#c3 ]
> for
> >> details.
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From up at 3.am  Mon Jan 31 19:34:37 2011
From: up at 3.am (up at 3.am)
Date: Mon, 31 Jan 2011 13:34:37 -0500
Subject: [rsyslog] Configuring which file(s) to log to
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDAFB@GRFEXC.intern.adiscon.com>
References: <db78a071442590a3ac3d9028f0e40591.squirrel@ssl.pil.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAFB@GRFEXC.intern.adiscon.com>
Message-ID: <35a8ce1d92f83dfb8d5916085f96c674.squirrel@ssl.pil.net>


>> Similarly, I created a cisco log and generated logging on the cisco.
>> In
>> this case, it does log to the file I created, but it duplicates it to
>> messages as well.
>
> you need to discard the message if it shall not be written to a second
> file.
> See here:
>
> http://www.rsyslog.com/storing-messages-from-a-remote-system-into-a-specific-file/

Thanks for your response, Rainer.  It looks like that addresses messages
that are logged both remotely and locally, not ones that are logged
remotely to both messages and the log I specified in the config.  It also
appears to require TCP.  That may be the problem...cisco routers won't
work using TCP to remote log to rsyslog (you can specify it, but it's
broken), so I had to use UDP for it.  I already have this:

$ModLoad imudp.so
$UDPServerRun 514
$AllowedSender UDP, 127.0.0.1, our.cisco.ip

In the config.  I tried using just TCP, but nothing works then.  Is it
possible that the localhost entry is causing the dup?



From up at 3.am  Mon Jan 31 20:08:04 2011
From: up at 3.am (up at 3.am)
Date: Mon, 31 Jan 2011 14:08:04 -0500
Subject: [rsyslog] Configuring which file(s) to log to
In-Reply-To: <35a8ce1d92f83dfb8d5916085f96c674.squirrel@ssl.pil.net>
References: <db78a071442590a3ac3d9028f0e40591.squirrel@ssl.pil.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAFB@GRFEXC.intern.adiscon.com>
	<35a8ce1d92f83dfb8d5916085f96c674.squirrel@ssl.pil.net>
Message-ID: <8fdb97e29e72199890fb732cfb55a62c.squirrel@ssl.pil.net>

>
>>> Similarly, I created a cisco log and generated logging on the cisco.
>>> In
>>> this case, it does log to the file I created, but it duplicates it to
>>> messages as well.
>>
>> you need to discard the message if it shall not be written to a second
>> file.
>> See here:
>>
>> http://www.rsyslog.com/storing-messages-from-a-remote-system-into-a-specific-file/
>
> Thanks for your response, Rainer.  It looks like that addresses messages
> that are logged both remotely and locally, not ones that are logged
> remotely to both messages and the log I specified in the config.  It also
> appears to require TCP.  That may be the problem...cisco routers won't
> work using TCP to remote log to rsyslog (you can specify it, but it's
> broken), so I had to use UDP for it.  I already have this:
>
> $ModLoad imudp.so
> $UDPServerRun 514
> $AllowedSender UDP, 127.0.0.1, our.cisco.ip
>
> In the config.  I tried using just TCP, but nothing works then.  Is it
> possible that the localhost entry is causing the dup?

Disregard all this...like I said, I'm noob to rsyslog.  I had the default
setting for messages:

*.info;mail.none;authpriv.none;cron.none     /var/log/messages

I simply added ;local7.none above and that fixed it, of course.


From up at 3.am  Mon Jan 31 20:42:52 2011
From: up at 3.am (up at 3.am)
Date: Mon, 31 Jan 2011 14:42:52 -0500
Subject: [rsyslog] Configuring which file(s) to log to
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDAFB@GRFEXC.intern.adiscon.com>
References: <db78a071442590a3ac3d9028f0e40591.squirrel@ssl.pil.net>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAFB@GRFEXC.intern.adiscon.com>
Message-ID: <5bb0266ef541aacd252b03480e5c9d2e.squirrel@ssl.pil.net>

>> First, there is fail2ban.  I configured
>> /etc/rsyslog.conf
>> as follows:
>>
>> # fail2ban log
>> local6.*	                                /var/log/fail2ban.log
>>
>> and fail2ban.conf as follows:
>>
>> loglevel = 3
>> logtarget = SYSLOG
>> syslog-facility = 22
>> syslog-target = /var/log/fail2ban.log
>>
>> fail2ban logs remotely, but instead of to the file specified above (I
>> touched it), it logs to messages.  What did I miss?
>
> Unfortunately, I don't know fail2ban. But some insight may be possible if
> you
> add
>
> *.* /var/log/debugfile;RSYSLOG_DebugFormat
>
> Then the debugfile will show us some details.

I could see nothing of use there.  However, I did the same thing with this
facility (local6) as I did with the one I was using for Cisco (local7),
and interestingly, rsyslog kept on logging fail2ban messages, so I can
only assume that even though I have the syslog-facility numeric code for
local6 defined above (I also tried just typing "local6" with no
difference), fail2ban is logging to a different facility.

Unless somebody here can see something obvious I'm doing wrong, I guess
this is a question for the fail2ban lists.

Thanks!




From ariel.p at hostdime.com  Mon Jan 31 22:33:11 2011
From: ariel.p at hostdime.com (Ariel P.)
Date: Mon, 31 Jan 2011 16:33:11 -0500
Subject: [rsyslog] Feedback on ommysql requested
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71DDAFC@GRFEXC.intern.adiscon.com>
References: <20101202155126.GZ19162@aart.is.rice.edu>	<4D0BE506.8090000@hostdime.com><9B6E2A8877C38245BFB15CC491A11DA71DD93D@GRFEXC.intern.adiscon.com>	<4D45D678.7090303@hostdime.com>
	<9B6E2A8877C38245BFB15CC491A11DA71DDAFC@GRFEXC.intern.adiscon.com>
Message-ID: <4D472A97.4010300@hostdime.com>

The patch for doc/ommysql.html has been submitted via the bug tracker.

Ariel P.
Server Security Analyst
HostDime.com, Inc.

On 2011-01-31 11:00, Rainer Gerhards wrote:
> sorry, I had forgotten the merge after my vacation. Now merged. It would be
> good if you could also provide a patch for the doc file.
>
> Thanks again!
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Ariel P.
>> Sent: Sunday, January 30, 2011 10:22 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Feedback on ommysql requested
>>
>> Hi all,
>>
>> Any word on the merging of this patch into the main code base, given
>> that no further opposing arguments were given in the mailing list?
>>
>> Ariel P.
>> Server Security Analyst
>> HostDime.com, Inc.
>>
>>
>> On 2010-12-22 13:07, Rainer Gerhards wrote:
>>> Hi all,
>>>
>>> thanks to Ariel for the hard work. I, too, think that all concerns
>> are
>>> addressed. I intend to merge the patch when I am back in January if I
>> don't
>>> here any good argument why I should not ;)
>>>
>>> Rainer
>>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of Ariel P.
>>>> Sent: Friday, December 17, 2010 11:33 PM
>>>> To: rsyslog at lists.adiscon.com
>>>> Subject: Re: [rsyslog] Feedback on ommysql requested
>>>>
>>>> I have created a new version of my patch, which creates two
>>>> configuration settings for ommysql.
>>>> The first setting sets the path for the 'my.cnf' file. If this
>> setting
>>>> is not specified in the configuration file, the mysql library is not
>>>> informed of a file path (current behavior)
>>>> The second setting is used to change the section of the 'my.cnf'
>> file
>>>> to
>>>> use, which defaults to "client" if not specified in the
>> configuration
>>>> file (this is mysql standard behavior).
>>>>
>>>> Please see [ http://bugzilla.adiscon.com/show_bug.cgi?id=213#c3 ]
>> for
>>>> details.
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com